2024-05-25T05:49:56.703 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-25T05:49:56.703 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353 -- /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/orig.config.yaml 2024-05-25T05:49:56.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-25T05:49:56.867 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7725353' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 17c123e524b17eb0cf38bdb0a0ca89d80b36c131 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4121 sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 sleep_before_teardown: 0 subset: 40/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 17c123e524b17eb0cf38bdb0a0ca89d80b36c131 targets: smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6n8sm3yHHuoSHQamMybOQsPLN9myIA88yhq1ODV6wFma8c89TSJ8y40Ie+2VL0z56eQNV8hPpNyoImp8a9CcA= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiJ5KPj3BYlD+P1xfWH1+drXFGDXWjcTv/a08yN9Bwy1NT0xm6f9O8K0uEKOpv6ZKa3+x+ttBMrPou96jqClXU= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-24_21:08:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-25T05:49:56.867 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa; will attempt to use it 2024-05-25T05:49:56.868 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks 2024-05-25T05:49:56.868 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-25T05:49:56.870 INFO:teuthology.task.internal:Checking packages... 2024-05-25T05:49:56.890 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ea0684c08cbcf83f4af9930de2267b3563ac5f78' 2024-05-25T05:49:56.890 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T05:49:56.890 INFO:teuthology.packaging:ref: None 2024-05-25T05:49:56.890 INFO:teuthology.packaging:tag: None 2024-05-25T05:49:56.890 INFO:teuthology.packaging:branch: squid 2024-05-25T05:49:56.890 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T05:49:56.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-25T05:49:57.083 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2847.g17c123e5 2024-05-25T05:49:57.085 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-25T05:49:57.092 INFO:teuthology.task.internal:no buildpackages task found 2024-05-25T05:49:57.092 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-25T05:49:57.100 INFO:teuthology.task.internal:Saving configuration 2024-05-25T05:49:57.124 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-25T05:49:57.131 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-25T05:49:57.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-25 05:44:48.109025', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6n8sm3yHHuoSHQamMybOQsPLN9myIA88yhq1ODV6wFma8c89TSJ8y40Ie+2VL0z56eQNV8hPpNyoImp8a9CcA='} 2024-05-25T05:49:57.170 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-25 05:44:48.110965', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiJ5KPj3BYlD+P1xfWH1+drXFGDXWjcTv/a08yN9Bwy1NT0xm6f9O8K0uEKOpv6ZKa3+x+ttBMrPou96jqClXU='} 2024-05-25T05:49:57.170 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-25T05:49:57.177 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-25T05:49:57.177 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-25T05:49:57.177 INFO:teuthology.run_tasks:Running task console_log... 2024-05-25T05:49:57.237 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff38f2e7ac0>, signals=[15]) 2024-05-25T05:49:57.237 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-25T05:49:57.243 INFO:teuthology.task.internal:Opening connections... 2024-05-25T05:49:57.243 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T05:49:57.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:49:57.322 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-05-25T05:49:57.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:49:57.396 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-25T05:49:57.404 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-05-25T05:49:57.422 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-05-25T05:49:57.423 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-25T05:49:57.480 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-05-25T05:49:57.481 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-25T05:49:57.481 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-25T05:49:57.481 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-25T05:49:57.481 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-05-25T05:49:57.511 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-05-25T05:49:57.528 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-05-25T05:49:57.529 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-05-25T05:49:57.586 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-05-25T05:49:57.586 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-05-25T05:49:57.586 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-05-25T05:49:57.586 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-05-25T05:49:57.586 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-25T05:49:57.587 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-25T05:49:57.587 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-05-25T05:49:57.610 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-25T05:49:57.620 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-25T05:49:57.647 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-25T05:49:57.647 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-05-25T05:49:57.650 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-05-25T05:49:57.666 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-25T05:49:57.673 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-25T05:49:57.673 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-05-25T05:49:57.707 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-05-25T05:49:57.739 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-25T05:49:57.795 INFO:teuthology.run_tasks:Running task kernel... 2024-05-25T05:49:57.809 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-25T05:49:57.809 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-25T05:49:57.810 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-25T05:49:57.810 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-25T05:49:57.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T05:49:57.826 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-05-25T05:49:57.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T05:49:57.827 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-05-25T05:49:57.884 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-437.el9.x86_64 2024-05-25T05:49:57.884 INFO:teuthology.task.kernel:Running kernel on smithi160: 5.14.0-437.el9.x86_64 2024-05-25T05:49:57.884 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2024-05-25T05:49:57.885 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-437.el9.x86_64 2024-05-25T05:49:57.885 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.14.0-437.el9.x86_64 2024-05-25T05:49:57.886 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2024-05-25T05:50:00.536 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.1 MB 00:00 2024-05-25T05:50:03.928 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.1 MB 00:04 2024-05-25T05:50:04.703 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-05-25T05:50:07.606 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-05-25T05:50:10.403 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-05-25T05:50:12.654 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 17 kB 00:01 2024-05-25T05:50:13.699 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-25T05:50:14.174 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-25T05:50:15.331 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 17 kB 00:00 2024-05-25T05:50:16.891 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-25T05:50:19.380 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-25T05:50:21.270 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-25T05:50:21.270 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-25T05:50:21.365 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-25T05:50:21.381 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout:Install 4 Packages 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 78 M 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 123 M 2024-05-25T05:50:21.382 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:50:22.040 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-05-25T05:50:22.731 INFO:teuthology.orchestra.run.smithi160.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 534 kB/s | 576 kB 00:01 2024-05-25T05:50:23.406 INFO:teuthology.orchestra.run.smithi160.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 8.8 MB/s | 16 MB 00:01 2024-05-25T05:50:23.980 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-25T05:50:23.980 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-25T05:50:23.990 INFO:teuthology.orchestra.run.smithi160.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 15 MB/s | 34 MB 00:02 2024-05-25T05:50:24.074 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-25T05:50:24.085 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout:Install 4 Packages 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 78 M 2024-05-25T05:50:24.086 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 123 M 2024-05-25T05:50:24.087 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:50:24.475 INFO:teuthology.orchestra.run.smithi203.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 2.1 MB/s | 576 kB 00:00 2024-05-25T05:50:25.807 INFO:teuthology.orchestra.run.smithi160.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 9.1 MB/s | 28 MB 00:03 2024-05-25T05:50:25.807 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:50:25.807 INFO:teuthology.orchestra.run.smithi160.stdout:Total 18 MB/s | 78 MB 00:04 2024-05-25T05:50:26.491 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:50:26.821 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:50:26.821 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:50:27.259 INFO:teuthology.orchestra.run.smithi203.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 5.1 MB/s | 16 MB 00:03 2024-05-25T05:50:27.727 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:50:27.727 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:50:28.784 INFO:teuthology.orchestra.run.smithi203.stdout:(3/4): kernel-modules-core-5.14.0-447.el9.x86_6 6.5 MB/s | 28 MB 00:04 2024-05-25T05:50:29.268 INFO:teuthology.orchestra.run.smithi203.stdout:(4/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 6.7 MB/s | 34 MB 00:05 2024-05-25T05:50:29.268 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:50:29.268 INFO:teuthology.orchestra.run.smithi203.stdout:Total 15 MB/s | 78 MB 00:05 2024-05-25T05:50:29.957 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:50:30.126 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:50:30.290 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:50:30.290 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:50:30.819 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-25T05:50:30.906 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-25T05:50:31.236 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:50:31.236 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:50:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-25T05:50:33.635 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:50:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-25T05:50:34.333 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-25T05:50:34.420 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-25T05:50:37.081 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-25T05:50:37.224 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-25T05:50:38.469 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-25T05:50:38.586 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:50:41.907 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-25T05:50:42.060 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:50:43.341 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:50:46.788 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:12.863 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:12.901 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-25T05:51:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-25T05:51:13.266 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-25T05:51:14.061 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-25T05:51:14.062 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:51:14.062 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:51:14.429 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-05-25T05:51:14.504 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-25T05:51:14.505 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-25T05:51:14.505 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-25T05:51:14.505 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-25T05:51:14.505 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-25T05:51:14.505 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-25T05:51:14.505 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi160.front.sepia.ceph.com, path=None, version=distro) 2024-05-25T05:51:14.506 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-05-25T05:51:14.555 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-25T05:51:14.556 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-25T05:51:14.556 DEBUG:teuthology.orchestra.run.smithi160:> sudo rpm -qi grub2-tools 2024-05-25T05:51:14.586 INFO:teuthology.orchestra.run.smithi160.stdout:Name : grub2-tools 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Epoch : 1 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Version : 2.06 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Release : 68.el9 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: x86_64 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Group : Unspecified 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Size : 8264254 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:License : GPLv3+ 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-25T05:51:14.587 INFO:teuthology.orchestra.run.smithi160.stdout:Packager : builder@centos.org 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:Vendor : CentOS 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:Summary : Support tools for GRUB. 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:Description : 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:hardware devices. 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:51:14.588 INFO:teuthology.orchestra.run.smithi160.stdout:This subpackage provides tools for support of all platforms. 2024-05-25T05:51:14.589 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-25T05:51:14.589 INFO:teuthology.task.kernel:Updating grub on smithi160 to boot 5.14.0-447.el9.x86_64 2024-05-25T05:51:14.589 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-25T05:51:15.211 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2024-05-25T05:51:16.065 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:16.102 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:16.474 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:16.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-25T05:51:16.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-25T05:51:16.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-25T05:51:16.602 INFO:teuthology.orchestra.run.smithi160.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-25T05:51:16.635 INFO:teuthology.orchestra.run.smithi160.stderr:done 2024-05-25T05:51:16.637 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-05-25T05:51:16.652 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.4VKGrEoJ3j 2024-05-25T05:51:16.652 DEBUG:teuthology.orchestra.run.smithi160:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4VKGrEoJ3j 2024-05-25T05:51:16.718 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.4VKGrEoJ3j 2024-05-25T05:51:16.848 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.4VKGrEoJ3j is 6KB 2024-05-25T05:51:16.896 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.4VKGrEoJ3j 2024-05-25T05:51:16.910 DEBUG:teuthology.orchestra.run.smithi160:> sudo /bin/ls /boot/loader/entries || true 2024-05-25T05:51:16.972 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-25T05:51:16.972 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-25T05:51:16.972 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-25T05:51:16.972 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-25T05:51:16.973 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.122 DEBUG:teuthology.orchestra.run.smithi160:> sudo shutdown -r now 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:51:17.297 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:51:17.658 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-05-25T05:51:17.728 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.729 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-25T05:51:17.729 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-25T05:51:17.729 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-25T05:51:17.729 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-25T05:51:17.729 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-25T05:51:17.729 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=None, version=distro) 2024-05-25T05:51:17.729 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-05-25T05:51:17.778 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-25T05:51:17.779 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-25T05:51:17.779 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -qi grub2-tools 2024-05-25T05:51:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:Name : grub2-tools 2024-05-25T05:51:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:Epoch : 1 2024-05-25T05:51:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:Version : 2.06 2024-05-25T05:51:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:Release : 68.el9 2024-05-25T05:51:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: x86_64 2024-05-25T05:51:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Group : Unspecified 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Size : 8264254 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:License : GPLv3+ 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Packager : builder@centos.org 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Vendor : CentOS 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Summary : Support tools for GRUB. 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout:Description : 2024-05-25T05:51:17.810 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:51:17.811 INFO:teuthology.orchestra.run.smithi203.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-25T05:51:17.811 INFO:teuthology.orchestra.run.smithi203.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-25T05:51:17.811 INFO:teuthology.orchestra.run.smithi203.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-25T05:51:17.811 INFO:teuthology.orchestra.run.smithi203.stdout:hardware devices. 2024-05-25T05:51:17.811 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:51:17.811 INFO:teuthology.orchestra.run.smithi203.stdout:This subpackage provides tools for support of all platforms. 2024-05-25T05:51:17.812 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-25T05:51:17.812 INFO:teuthology.task.kernel:Updating grub on smithi203 to boot 5.14.0-447.el9.x86_64 2024-05-25T05:51:17.812 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-25T05:51:18.392 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2024-05-25T05:51:19.718 INFO:teuthology.orchestra.run.smithi203.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-25T05:51:19.749 INFO:teuthology.orchestra.run.smithi203.stderr:done 2024-05-25T05:51:19.751 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-05-25T05:51:19.773 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.72ctcnHB1i 2024-05-25T05:51:19.774 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.72ctcnHB1i 2024-05-25T05:51:19.839 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.72ctcnHB1i 2024-05-25T05:51:19.979 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.72ctcnHB1i is 6KB 2024-05-25T05:51:19.998 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.72ctcnHB1i 2024-05-25T05:51:20.012 DEBUG:teuthology.orchestra.run.smithi203:> sudo /bin/ls /boot/loader/entries || true 2024-05-25T05:51:20.073 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-25T05:51:20.073 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-25T05:51:20.074 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-25T05:51:20.074 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-25T05:51:20.075 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-25T05:51:20.234 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2024-05-25T05:51:47.152 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-25T05:51:47.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-25T05:51:47.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:51:50.237 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-25T05:51:50.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:51:50.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:52:21.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-05-25T05:52:24.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-05-25T05:52:30.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-25T05:52:30.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:52:33.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:52:33.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:52:52.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-05-25T05:53:04.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:53:04.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:53:07.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-05-25T05:53:22.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:53:22.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:53:22.772 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-05-25T05:53:23.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:53:23.127 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-447.el9.x86_64"... 2024-05-25T05:53:23.127 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-05-25T05:53:23.143 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-447.el9.x86_64 2024-05-25T05:53:23.144 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-25T05:53:23.144 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-25T05:53:23.144 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-25T05:53:24.145 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-25T05:53:24.146 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-25T05:53:24.208 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2024-05-25T05:53:24.250 DEBUG:teuthology.parallel:result is None 2024-05-25T05:53:30.798 DEBUG:teuthology.orchestra.remote:timed out 2024-05-25T05:53:42.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-25T05:53:42.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:53:43.181 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-25T05:53:43.534 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-25T05:53:43.534 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-447.el9.x86_64"... 2024-05-25T05:53:43.534 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-05-25T05:53:43.549 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-447.el9.x86_64 2024-05-25T05:53:43.550 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-25T05:53:43.550 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-25T05:53:43.550 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-25T05:53:44.551 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-25T05:53:44.551 DEBUG:teuthology.orchestra.run.smithi160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-25T05:53:44.913 INFO:teuthology.orchestra.run.smithi160.stdout:ttyS1 2024-05-25T05:53:44.942 DEBUG:teuthology.parallel:result is None 2024-05-25T05:53:44.943 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-25T05:53:44.950 INFO:teuthology.task.internal:Creating test directory... 2024-05-25T05:53:44.950 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-25T05:53:44.953 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-25T05:53:44.972 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-25T05:53:44.981 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-25T05:53:44.991 INFO:teuthology.task.internal:Creating archive directory... 2024-05-25T05:53:44.991 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-25T05:53:45.012 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-25T05:53:45.139 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-25T05:53:45.146 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-25T05:53:45.146 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-25T05:53:45.148 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-25T05:53:45.184 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-25T05:53:45.185 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-25T05:53:45.195 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-25T05:53:45.196 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-25T05:53:45.198 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-25T05:53:45.205 INFO:teuthology.task.internal:Configuring sudo... 2024-05-25T05:53:45.205 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-25T05:53:45.240 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-25T05:53:45.269 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-25T05:53:45.277 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-25T05:53:45.278 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-25T05:53:45.308 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-25T05:53:45.325 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-25T05:53:45.407 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-25T05:53:45.457 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-25T05:53:45.538 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-25T05:53:45.581 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T05:53:45.581 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-25T05:53:45.647 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-25T05:53:45.691 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-25T05:53:45.744 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-25T05:53:45.827 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-25T05:53:45.871 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T05:53:45.871 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-25T05:53:45.939 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-05-25T05:53:45.942 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-05-25T05:53:45.977 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-25T05:53:46.026 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-25T05:53:46.436 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-25T05:53:46.444 INFO:teuthology.task.internal:Starting timer... 2024-05-25T05:53:46.444 INFO:teuthology.run_tasks:Running task pcp... 2024-05-25T05:53:46.458 INFO:teuthology.run_tasks:Running task selinux... 2024-05-25T05:53:46.466 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-25T05:53:46.469 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-05-25T05:53:46.518 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-05-25T05:53:46.520 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-05-25T05:53:46.569 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-05-25T05:53:46.571 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-25T05:53:46.571 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-05-25T05:53:46.601 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-05-25T05:53:46.602 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-05-25T05:53:46.650 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-05-25T05:53:46.650 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-05-25T05:53:46.651 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-25T05:53:46.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T05:53:46.679 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-25T05:53:46.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T05:53:46.708 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-25T05:53:46.708 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-05-25T05:53:46.747 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-05-25T05:53:46.776 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-25T05:53:46.786 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-25T05:53:46.789 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-25T05:53:46.914 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-25T05:53:46.924 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-25T05:53:46.925 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi160.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-25T05:58:04.294 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-05-25T05:58:04.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-25T05:58:04.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:58:04.373 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-25T05:58:04.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-25T05:58:04.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:58:04.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-25T05:58:04.542 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-05-25T05:58:04.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-25T05:58:04.622 INFO:teuthology.run_tasks:Running task clock... 2024-05-25T05:58:04.633 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-25T05:58:04.633 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-25T05:58:04.634 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-25T05:58:04.637 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-25T05:58:04.637 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-25T05:58:04.666 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-25T05:58:04.682 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-25T05:58:04.691 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-25T05:58:04.706 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-25T05:58:04.707 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-05-25T05:58:04.718 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-05-25T05:58:04.730 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-05-25T05:58:04.733 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-25T05:58:04.741 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-05-25T05:58:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-25T05:58:04.756 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-25T05:58:04.770 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-25T05:58:04.803 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-25T05:58:04.806 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-25T05:58:04.806 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-25T05:58:04.807 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.807 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.807 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.807 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.829 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-05-25T05:58:04.833 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-25T05:58:04.833 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-05-25T05:58:04.833 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.833 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.833 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.833 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T05:58:04.834 INFO:teuthology.run_tasks:Running task install... 2024-05-25T05:58:04.842 DEBUG:teuthology.task.install:project ceph 2024-05-25T05:58:04.842 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ea0684c08cbcf83f4af9930de2267b3563ac5f78'}} 2024-05-25T05:58:04.842 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'ea0684c08cbcf83f4af9930de2267b3563ac5f78'} 2024-05-25T05:58:04.842 INFO:teuthology.task.install:Using flavor: default 2024-05-25T05:58:04.848 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-25T05:58:04.848 INFO:teuthology.task.install:extra packages: [] 2024-05-25T05:58:04.848 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ea0684c08cbcf83f4af9930de2267b3563ac5f78', 'tag': None, 'wait_for_package': False} 2024-05-25T05:58:04.848 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T05:58:04.849 INFO:teuthology.packaging:ref: None 2024-05-25T05:58:04.849 INFO:teuthology.packaging:tag: None 2024-05-25T05:58:04.849 INFO:teuthology.packaging:branch: quincy 2024-05-25T05:58:04.849 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T05:58:04.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-25T05:58:04.851 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ea0684c08cbcf83f4af9930de2267b3563ac5f78', 'tag': None, 'wait_for_package': False} 2024-05-25T05:58:04.851 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T05:58:04.851 INFO:teuthology.packaging:ref: None 2024-05-25T05:58:04.851 INFO:teuthology.packaging:tag: None 2024-05-25T05:58:04.851 INFO:teuthology.packaging:branch: quincy 2024-05-25T05:58:04.851 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T05:58:04.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-25T05:58:05.079 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/ 2024-05-25T05:58:05.080 INFO:teuthology.task.install.rpm:Package version is 17.2.7-958.g7b01852c 2024-05-25T05:58:05.149 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/ 2024-05-25T05:58:05.150 INFO:teuthology.task.install.rpm:Package version is 17.2.7-958.g7b01852c 2024-05-25T05:58:05.194 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-25T05:58:05.195 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T05:58:05.196 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-25T05:58:05.222 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-25T05:58:05.222 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T05:58:05.222 INFO:teuthology.packaging:ref: None 2024-05-25T05:58:05.222 INFO:teuthology.packaging:tag: None 2024-05-25T05:58:05.222 INFO:teuthology.packaging:branch: quincy 2024-05-25T05:58:05.222 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T05:58:05.222 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-25T05:58:05.259 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/7b01852c69b416ba2d4f5e12d093f476d16043d8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-25T05:58:05.259 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T05:58:05.259 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-25T05:58:05.287 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-25T05:58:05.287 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T05:58:05.287 INFO:teuthology.packaging:ref: None 2024-05-25T05:58:05.287 INFO:teuthology.packaging:tag: None 2024-05-25T05:58:05.287 INFO:teuthology.packaging:branch: quincy 2024-05-25T05:58:05.287 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T05:58:05.287 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-25T05:58:05.288 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-25T05:58:05.355 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-25T05:58:05.365 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-25T05:58:05.429 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-05-25T05:58:05.430 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-05-25T05:58:05.431 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-25T05:58:05.496 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-05-25T05:58:05.498 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-05-25T05:58:05.660 INFO:teuthology.orchestra.run.smithi160.stdout:45 files removed 2024-05-25T05:58:05.690 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw 2024-05-25T05:58:05.725 INFO:teuthology.orchestra.run.smithi203.stdout:45 files removed 2024-05-25T05:58:05.752 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw 2024-05-25T05:58:06.332 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 212 kB/s | 77 kB 00:00 2024-05-25T05:58:06.494 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 168 kB/s | 77 kB 00:00 2024-05-25T05:58:06.691 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-05-25T05:58:06.912 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-25T05:58:07.087 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-05-25T05:58:07.274 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 5.6 kB/s | 1.8 kB 00:00 2024-05-25T05:58:07.574 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.1 MB 00:00 2024-05-25T05:58:09.434 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-25T05:58:10.129 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-25T05:58:13.493 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-25T05:58:13.991 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-05-25T05:58:15.094 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 17 kB 00:00 2024-05-25T05:58:18.262 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:03 2024-05-25T05:58:19.548 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-05-25T05:58:21.874 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 17 kB 00:01 2024-05-25T05:58:23.512 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-25T05:58:23.953 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-25T05:58:25.753 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:58:25.756 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:58:25.756 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:58:25.756 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:58:25.756 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:58:25.756 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:17.2.7-958.g7b01852c.el9 ceph 14 M 2024-05-25T05:58:25.756 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 3.1 M 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 2.9 M 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:17.2.7-958.g7b01852c.el9 ceph 5.0 M 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:17.2.7-958.g7b01852c.el9 ceph 22 M 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:17.2.7-958.g7b01852c.el9 ceph 25 k 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 637 k 2024-05-25T05:58:25.757 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 457 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 4.9 M 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:17.2.7-958.g7b01852c.el9 ceph 45 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:17.2.7-958.g7b01852c.el9 ceph 118 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:17.2.7-958.g7b01852c.el9 ceph 156 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-25T05:58:25.758 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:17.2.7-958.g7b01852c.el9 ceph 320 k 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:17.2.7-958.g7b01852c.el9 ceph 293 k 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:17.2.7-958.g7b01852c.el9 ceph 99 k 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout:Install 22 Packages 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 2 Packages 2024-05-25T05:58:25.759 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:25.761 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 58 M 2024-05-25T05:58:25.761 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:58:26.730 INFO:teuthology.orchestra.run.smithi160.stdout:(1/24): ceph-base-17.2.7-958.g7b01852c.el9.x86_ 6.0 MB/s | 5.0 MB 00:00 2024-05-25T05:58:26.788 INFO:teuthology.orchestra.run.smithi160.stdout:(2/24): ceph-selinux-17.2.7-958.g7b01852c.el9.x 433 kB/s | 25 kB 00:00 2024-05-25T05:58:26.922 INFO:teuthology.orchestra.run.smithi160.stdout:(3/24): libcephfs2-17.2.7-958.g7b01852c.el9.x86 4.7 MB/s | 637 kB 00:00 2024-05-25T05:58:27.014 INFO:teuthology.orchestra.run.smithi160.stdout:(4/24): libradosstriper1-17.2.7-958.g7b01852c.e 4.9 MB/s | 457 kB 00:00 2024-05-25T05:58:27.164 INFO:teuthology.orchestra.run.smithi160.stdout:(5/24): ceph-radosgw-17.2.7-958.g7b01852c.el9.x 11 MB/s | 14 MB 00:01 2024-05-25T05:58:27.223 INFO:teuthology.orchestra.run.smithi160.stdout:(6/24): python3-ceph-argparse-17.2.7-958.g7b018 778 kB/s | 45 kB 00:00 2024-05-25T05:58:27.281 INFO:teuthology.orchestra.run.smithi160.stdout:(7/24): python3-ceph-common-17.2.7-958.g7b01852 2.0 MB/s | 118 kB 00:00 2024-05-25T05:58:27.340 INFO:teuthology.orchestra.run.smithi160.stdout:(8/24): python3-cephfs-17.2.7-958.g7b01852c.el9 2.6 MB/s | 156 kB 00:00 2024-05-25T05:58:27.432 INFO:teuthology.orchestra.run.smithi160.stdout:(9/24): python3-rados-17.2.7-958.g7b01852c.el9. 3.4 MB/s | 320 kB 00:00 2024-05-25T05:58:27.499 INFO:teuthology.orchestra.run.smithi160.stdout:(10/24): librgw2-17.2.7-958.g7b01852c.el9.x86_6 10 MB/s | 4.9 MB 00:00 2024-05-25T05:58:27.524 INFO:teuthology.orchestra.run.smithi160.stdout:(11/24): python3-rbd-17.2.7-958.g7b01852c.el9.x 3.1 MB/s | 293 kB 00:00 2024-05-25T05:58:27.549 INFO:teuthology.orchestra.run.smithi160.stdout:(12/24): python3-rgw-17.2.7-958.g7b01852c.el9.x 1.9 MB/s | 99 kB 00:00 2024-05-25T05:58:27.858 INFO:teuthology.orchestra.run.smithi160.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 147 kB/s | 45 kB 00:00 2024-05-25T05:58:27.883 INFO:teuthology.orchestra.run.smithi160.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 446 kB/s | 160 kB 00:00 2024-05-25T05:58:28.109 INFO:teuthology.orchestra.run.smithi160.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-05-25T05:58:28.134 INFO:teuthology.orchestra.run.smithi160.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-05-25T05:58:28.268 INFO:teuthology.orchestra.run.smithi160.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 266 kB/s | 42 kB 00:00 2024-05-25T05:58:28.468 INFO:teuthology.orchestra.run.smithi160.stdout:(18/24): ceph-common-17.2.7-958.g7b01852c.el9.x 8.7 MB/s | 22 MB 00:02 2024-05-25T05:58:28.960 INFO:teuthology.orchestra.run.smithi160.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 70 kB/s | 49 kB 00:00 2024-05-25T05:58:29.052 INFO:teuthology.orchestra.run.smithi160.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 121 kB/s | 67 kB 00:00 2024-05-25T05:58:29.127 INFO:teuthology.orchestra.run.smithi160.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 311 kB/s | 309 kB 00:00 2024-05-25T05:58:29.151 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-25T05:58:29.344 INFO:teuthology.orchestra.run.smithi160.stdout:(22/24): librados2-17.2.7-958.g7b01852c.el9.x86 11 MB/s | 3.1 MB 00:00 2024-05-25T05:58:29.477 INFO:teuthology.orchestra.run.smithi160.stdout:(23/24): librbd1-17.2.7-958.g7b01852c.el9.x86_6 8.4 MB/s | 2.9 MB 00:00 2024-05-25T05:58:30.944 INFO:teuthology.orchestra.run.smithi160.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 816 kB/s | 1.6 MB 00:01 2024-05-25T05:58:30.950 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:58:30.950 INFO:teuthology.orchestra.run.smithi160.stdout:Total 11 MB/s | 58 MB 00:05 2024-05-25T05:58:31.014 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:58:31.047 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:58:31.047 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:58:31.433 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:58:31.448 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:17.2.7-958.g7b01852c.el9 ceph 14 M 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 3.1 M 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 2.9 M 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:17.2.7-958.g7b01852c.el9 ceph 5.0 M 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:17.2.7-958.g7b01852c.el9 ceph 22 M 2024-05-25T05:58:31.449 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:17.2.7-958.g7b01852c.el9 ceph 25 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 637 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 457 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:17.2.7-958.g7b01852c.el9 ceph 4.9 M 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:17.2.7-958.g7b01852c.el9 ceph 45 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:17.2.7-958.g7b01852c.el9 ceph 118 k 2024-05-25T05:58:31.450 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:17.2.7-958.g7b01852c.el9 ceph 156 k 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:17.2.7-958.g7b01852c.el9 ceph 320 k 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:17.2.7-958.g7b01852c.el9 ceph 293 k 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:17.2.7-958.g7b01852c.el9 ceph 99 k 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout:Install 22 Packages 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 2 Packages 2024-05-25T05:58:31.451 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:31.452 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 58 M 2024-05-25T05:58:31.452 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:58:31.637 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:58:31.637 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:58:32.235 INFO:teuthology.orchestra.run.smithi203.stdout:(1/24): ceph-base-17.2.7-958.g7b01852c.el9.x86_ 7.7 MB/s | 5.0 MB 00:00 2024-05-25T05:58:32.312 INFO:teuthology.orchestra.run.smithi203.stdout:(2/24): ceph-selinux-17.2.7-958.g7b01852c.el9.x 330 kB/s | 25 kB 00:00 2024-05-25T05:58:32.427 INFO:teuthology.orchestra.run.smithi203.stdout:(3/24): libcephfs2-17.2.7-958.g7b01852c.el9.x86 5.4 MB/s | 637 kB 00:00 2024-05-25T05:58:32.511 INFO:teuthology.orchestra.run.smithi203.stdout:(4/24): libradosstriper1-17.2.7-958.g7b01852c.e 5.4 MB/s | 457 kB 00:00 2024-05-25T05:58:32.820 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:58:33.120 INFO:teuthology.orchestra.run.smithi203.stdout:(5/24): librgw2-17.2.7-958.g7b01852c.el9.x86_64 8.1 MB/s | 4.9 MB 00:00 2024-05-25T05:58:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-05-25T05:58:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:17.2.7-958.g7b01852c.el9.x86_64 2/26 2024-05-25T05:58:33.279 INFO:teuthology.orchestra.run.smithi203.stdout:(6/24): ceph-radosgw-17.2.7-958.g7b01852c.el9.x 8.3 MB/s | 14 MB 00:01 2024-05-25T05:58:33.304 INFO:teuthology.orchestra.run.smithi203.stdout:(7/24): python3-ceph-argparse-17.2.7-958.g7b018 246 kB/s | 45 kB 00:00 2024-05-25T05:58:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:(8/24): python3-ceph-common-17.2.7-958.g7b01852 2.3 MB/s | 118 kB 00:00 2024-05-25T05:58:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:17.2.7-958.g7b01852c.el9.x86_64 2/26 2024-05-25T05:58:33.354 INFO:teuthology.orchestra.run.smithi203.stdout:(9/24): python3-cephfs-17.2.7-958.g7b01852c.el9 3.0 MB/s | 156 kB 00:00 2024-05-25T05:58:33.388 INFO:teuthology.orchestra.run.smithi203.stdout:(10/24): python3-rados-17.2.7-958.g7b01852c.el9 5.3 MB/s | 320 kB 00:00 2024-05-25T05:58:33.414 INFO:teuthology.orchestra.run.smithi203.stdout:(11/24): python3-rbd-17.2.7-958.g7b01852c.el9.x 4.9 MB/s | 293 kB 00:00 2024-05-25T05:58:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 3/26 2024-05-25T05:58:33.439 INFO:teuthology.orchestra.run.smithi203.stdout:(12/24): python3-rgw-17.2.7-958.g7b01852c.el9.x 1.9 MB/s | 99 kB 00:00 2024-05-25T05:58:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 4/26 2024-05-25T05:58:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 4/26 2024-05-25T05:58:33.539 INFO:teuthology.orchestra.run.smithi203.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-05-25T05:58:33.564 INFO:teuthology.orchestra.run.smithi203.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-25T05:58:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-05-25T05:58:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-05-25T05:58:33.648 INFO:teuthology.orchestra.run.smithi203.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-25T05:58:33.674 INFO:teuthology.orchestra.run.smithi203.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-25T05:58:33.724 INFO:teuthology.orchestra.run.smithi203.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 562 kB/s | 42 kB 00:00 2024-05-25T05:58:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-05-25T05:58:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 8/26 2024-05-25T05:58:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 8/26 2024-05-25T05:58:34.133 INFO:teuthology.orchestra.run.smithi203.stdout:(18/24): ceph-common-17.2.7-958.g7b01852c.el9.x 8.8 MB/s | 22 MB 00:02 2024-05-25T05:58:34.158 INFO:teuthology.orchestra.run.smithi203.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 112 kB/s | 49 kB 00:00 2024-05-25T05:58:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-05-25T05:58:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:17.2.7-958.g7b01852c.el9.x 10/26 2024-05-25T05:58:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 11/26 2024-05-25T05:58:34.359 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 12/26 2024-05-25T05:58:34.386 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 13/26 2024-05-25T05:58:34.453 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 13/26 2024-05-25T05:58:34.493 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-05-25T05:58:34.528 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-05-25T05:58:34.561 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-05-25T05:58:34.616 INFO:teuthology.orchestra.run.smithi203.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 139 kB/s | 67 kB 00:00 2024-05-25T05:58:34.792 INFO:teuthology.orchestra.run.smithi203.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 276 kB/s | 309 kB 00:01 2024-05-25T05:58:34.856 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-05-25T05:58:34.899 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 18/26 2024-05-25T05:58:34.900 INFO:teuthology.orchestra.run.smithi203.stdout:(22/24): librados2-17.2.7-958.g7b01852c.el9.x86 11 MB/s | 3.1 MB 00:00 2024-05-25T05:58:34.972 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 18/26 2024-05-25T05:58:35.046 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 19/26 2024-05-25T05:58:35.077 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:17.2.7-958.g7b01852c.el9.x86 20/26 2024-05-25T05:58:35.092 INFO:teuthology.orchestra.run.smithi203.stdout:(23/24): librbd1-17.2.7-958.g7b01852c.el9.x86_6 9.8 MB/s | 2.9 MB 00:00 2024-05-25T05:58:37.034 INFO:teuthology.orchestra.run.smithi203.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 563 kB/s | 1.6 MB 00:02 2024-05-25T05:58:37.036 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:58:37.037 INFO:teuthology.orchestra.run.smithi203.stdout:Total 10 MB/s | 58 MB 00:05 2024-05-25T05:58:37.099 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:58:37.133 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:58:37.133 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:58:37.485 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 21/26 2024-05-25T05:58:37.523 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 21/26 2024-05-25T05:58:37.717 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:58:37.718 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:58:38.059 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 21/26 2024-05-25T05:58:38.094 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 22/26 2024-05-25T05:58:38.174 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 22/26 2024-05-25T05:58:38.174 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-25T05:58:38.174 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-25T05:58:38.174 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:38.203 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 23/26 2024-05-25T05:58:38.886 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:58:39.275 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-05-25T05:58:39.320 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:17.2.7-958.g7b01852c.el9.x86_64 2/26 2024-05-25T05:58:39.411 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:17.2.7-958.g7b01852c.el9.x86_64 2/26 2024-05-25T05:58:39.499 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 3/26 2024-05-25T05:58:39.522 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 4/26 2024-05-25T05:58:39.629 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 4/26 2024-05-25T05:58:39.673 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-05-25T05:58:39.701 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-05-25T05:58:40.137 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-05-25T05:58:40.158 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 8/26 2024-05-25T05:58:40.230 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 8/26 2024-05-25T05:58:40.291 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-05-25T05:58:40.334 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:17.2.7-958.g7b01852c.el9.x 10/26 2024-05-25T05:58:40.371 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 11/26 2024-05-25T05:58:40.432 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 12/26 2024-05-25T05:58:40.452 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 13/26 2024-05-25T05:58:40.510 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 13/26 2024-05-25T05:58:40.551 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-05-25T05:58:40.586 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-05-25T05:58:40.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-05-25T05:58:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-05-25T05:58:40.974 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 18/26 2024-05-25T05:58:41.071 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 18/26 2024-05-25T05:58:41.145 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 19/26 2024-05-25T05:58:41.177 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:17.2.7-958.g7b01852c.el9.x86 20/26 2024-05-25T05:58:43.652 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 21/26 2024-05-25T05:58:43.691 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 21/26 2024-05-25T05:58:44.217 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 21/26 2024-05-25T05:58:44.262 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 22/26 2024-05-25T05:58:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 22/26 2024-05-25T05:58:44.366 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-25T05:58:44.366 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-25T05:58:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:44.397 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 23/26 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 23/26 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-05-25T05:58:47.403 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:48.698 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 24/26 2024-05-25T05:58:48.767 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 24/26 2024-05-25T05:58:48.768 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:58:48.768 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-25T05:58:48.768 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-25T05:58:48.768 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-25T05:58:48.768 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:48.769 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-25T05:58:48.859 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-25T05:58:48.859 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-25T05:58:50.052 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 1/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 2/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 3/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 4/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 5/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 6/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 7/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:17.2.7-958.g7b01852c.el9.x 8/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:17.2.7-958.g7b01852c.el9.x86 9/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 10/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 11/26 2024-05-25T05:58:50.053 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 12/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 13/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:17.2.7-958.g7b01852c.el9.x86_64 23/26 2024-05-25T05:58:50.054 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-05-25T05:58:50.055 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 25/26 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.682 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-25T05:58:50.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.684 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.684 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:50.684 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-25T05:58:50.684 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:50.684 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:58:51.025 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-test 2024-05-25T05:58:51.565 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:28 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:58:51.687 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:58:51.687 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:58:51.687 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:58:51.687 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:58:51.688 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:58:51.688 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:17.2.7-958.g7b01852c.el9 ceph 48 M 2024-05-25T05:58:51.688 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:58:51.688 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:17.2.7-958.g7b01852c.el9 ceph 163 k 2024-05-25T05:58:51.688 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-25T05:58:51.689 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-25T05:58:51.689 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:51.689 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:58:51.689 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:58:51.689 INFO:teuthology.orchestra.run.smithi160.stdout:Install 4 Packages 2024-05-25T05:58:51.689 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:58:51.690 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 48 M 2024-05-25T05:58:51.690 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 192 M 2024-05-25T05:58:51.690 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:58:52.038 INFO:teuthology.orchestra.run.smithi160.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-05-25T05:58:52.063 INFO:teuthology.orchestra.run.smithi160.stdout:(2/4): libcephsqlite-17.2.7-958.g7b01852c.el9.x 624 kB/s | 163 kB 00:00 2024-05-25T05:58:52.088 INFO:teuthology.orchestra.run.smithi160.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-25T05:58:54.166 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 23/26 2024-05-25T05:58:54.166 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-05-25T05:58:54.166 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-05-25T05:58:54.167 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-05-25T05:58:54.167 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-05-25T05:58:54.167 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-05-25T05:58:54.167 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-05-25T05:58:54.167 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-05-25T05:58:54.167 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:54.280 INFO:teuthology.orchestra.run.smithi160.stdout:(4/4): ceph-test-17.2.7-958.g7b01852c.el9.x86_6 19 MB/s | 48 MB 00:02 2024-05-25T05:58:54.282 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:58:54.282 INFO:teuthology.orchestra.run.smithi160.stdout:Total 19 MB/s | 48 MB 00:02 2024-05-25T05:58:54.310 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:58:54.330 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:58:54.330 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:58:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:58:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:58:55.213 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:58:55.304 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-25T05:58:55.353 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-25T05:58:55.382 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 3/4 2024-05-25T05:58:55.471 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 24/26 2024-05-25T05:58:55.549 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 24/26 2024-05-25T05:58:55.550 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:58:55.550 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-25T05:58:55.550 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-25T05:58:55.550 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-25T05:58:55.550 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:55.551 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-25T05:58:55.640 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-25T05:58:55.641 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 1/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 2/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 3/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 4/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 5/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 6/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 7/26 2024-05-25T05:58:56.935 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:17.2.7-958.g7b01852c.el9.x 8/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:17.2.7-958.g7b01852c.el9.x86 9/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 10/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 11/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 12/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 13/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-05-25T05:58:56.936 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-05-25T05:58:56.937 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-05-25T05:58:56.937 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:17.2.7-958.g7b01852c.el9.x86_64 23/26 2024-05-25T05:58:56.937 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-05-25T05:58:56.937 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 25/26 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.544 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.545 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:57.546 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:58:57.877 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-test 2024-05-25T05:58:58.417 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:29 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:58:58.538 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:17.2.7-958.g7b01852c.el9 ceph 48 M 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:17.2.7-958.g7b01852c.el9 ceph 163 k 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-25T05:58:58.539 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:58.540 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:58:58.540 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:58:58.540 INFO:teuthology.orchestra.run.smithi203.stdout:Install 4 Packages 2024-05-25T05:58:58.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:58:58.540 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 48 M 2024-05-25T05:58:58.540 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 192 M 2024-05-25T05:58:58.541 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:58:58.897 INFO:teuthology.orchestra.run.smithi203.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-05-25T05:58:58.923 INFO:teuthology.orchestra.run.smithi203.stdout:(2/4): libcephsqlite-17.2.7-958.g7b01852c.el9.x 587 kB/s | 163 kB 00:00 2024-05-25T05:58:58.948 INFO:teuthology.orchestra.run.smithi203.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-25T05:58:59.835 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 3/4 2024-05-25T05:58:59.914 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 4/4 2024-05-25T05:59:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:(4/4): ceph-test-17.2.7-958.g7b01852c.el9.x86_6 20 MB/s | 48 MB 00:02 2024-05-25T05:59:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:Total 20 MB/s | 48 MB 00:02 2024-05-25T05:59:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:01.047 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:01.048 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:01.410 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:01.410 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:01.935 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:02.012 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-25T05:59:02.064 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-25T05:59:02.074 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 4/4 2024-05-25T05:59:02.074 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 1/4 2024-05-25T05:59:02.074 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 2/4 2024-05-25T05:59:02.074 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-25T05:59:02.083 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 3/4 2024-05-25T05:59:02.349 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:02.350 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:02.630 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph 2024-05-25T05:59:03.168 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:40 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:03.292 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:03.294 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:03.294 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:03.294 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:03.294 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:03.294 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:17.2.7-958.g7b01852c.el9 ceph 6.5 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:17.2.7-958.g7b01852c.el9 ceph 2.1 M 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:17.2.7-958.g7b01852c.el9 ceph 1.4 M 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 240 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:17.2.7-958.g7b01852c.el9 ceph 3.9 M 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:17.2.7-958.g7b01852c.el9 ceph 15 M 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-25T05:59:03.295 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-25T05:59:03.296 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout:Install 33 Packages 2024-05-25T05:59:03.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:03.298 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 27 M 2024-05-25T05:59:03.298 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 96 M 2024-05-25T05:59:03.298 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:03.897 INFO:teuthology.orchestra.run.smithi160.stdout:(1/33): ceph-17.2.7-958.g7b01852c.el9.x86_64.rp 37 kB/s | 6.5 kB 00:00 2024-05-25T05:59:04.097 INFO:teuthology.orchestra.run.smithi160.stdout:(2/33): ceph-mds-17.2.7-958.g7b01852c.el9.x86_6 5.5 MB/s | 2.1 MB 00:00 2024-05-25T05:59:04.898 INFO:teuthology.orchestra.run.smithi160.stdout:(3/33): ceph-osd-17.2.7-958.g7b01852c.el9.x86_6 19 MB/s | 15 MB 00:00 2024-05-25T05:59:04.965 INFO:teuthology.orchestra.run.smithi160.stdout:(4/33): ceph-mgr-17.2.7-958.g7b01852c.el9.x86_6 1.1 MB/s | 1.4 MB 00:01 2024-05-25T05:59:04.990 INFO:teuthology.orchestra.run.smithi160.stdout:(5/33): ceph-mgr-modules-core-17.2.7-958.g7b018 2.6 MB/s | 240 kB 00:00 2024-05-25T05:59:05.049 INFO:teuthology.orchestra.run.smithi160.stdout:(6/33): ceph-mon-17.2.7-958.g7b01852c.el9.x86_6 3.4 MB/s | 3.9 MB 00:01 2024-05-25T05:59:05.766 INFO:teuthology.orchestra.run.smithi160.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 149 kB/s | 106 kB 00:00 2024-05-25T05:59:05.791 INFO:teuthology.orchestra.run.smithi160.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 307 kB/s | 253 kB 00:00 2024-05-25T05:59:05.991 INFO:teuthology.orchestra.run.smithi160.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 1.2 MB/s | 1.2 MB 00:01 2024-05-25T05:59:06.495 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 3/4 2024-05-25T05:59:06.508 INFO:teuthology.orchestra.run.smithi160.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 182 kB/s | 135 kB 00:00 2024-05-25T05:59:06.533 INFO:teuthology.orchestra.run.smithi160.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 168 kB/s | 125 kB 00:00 2024-05-25T05:59:06.590 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 4/4 2024-05-25T05:59:06.633 INFO:teuthology.orchestra.run.smithi160.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-05-25T05:59:06.675 INFO:teuthology.orchestra.run.smithi160.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-25T05:59:06.709 INFO:teuthology.orchestra.run.smithi160.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-05-25T05:59:06.734 INFO:teuthology.orchestra.run.smithi160.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-05-25T05:59:06.834 INFO:teuthology.orchestra.run.smithi160.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 271 kB/s | 215 kB 00:00 2024-05-25T05:59:07.060 INFO:teuthology.orchestra.run.smithi160.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 124 kB/s | 43 kB 00:00 2024-05-25T05:59:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 420 kB/s | 172 kB 00:00 2024-05-25T05:59:07.169 INFO:teuthology.orchestra.run.smithi160.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 98 kB/s | 11 kB 00:00 2024-05-25T05:59:07.235 INFO:teuthology.orchestra.run.smithi160.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 192 kB/s | 18 kB 00:00 2024-05-25T05:59:07.261 INFO:teuthology.orchestra.run.smithi160.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 252 kB/s | 23 kB 00:00 2024-05-25T05:59:07.327 INFO:teuthology.orchestra.run.smithi160.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-05-25T05:59:07.353 INFO:teuthology.orchestra.run.smithi160.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 692 kB/s | 358 kB 00:00 2024-05-25T05:59:07.378 INFO:teuthology.orchestra.run.smithi160.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 168 kB/s | 20 kB 00:00 2024-05-25T05:59:07.420 INFO:teuthology.orchestra.run.smithi160.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 503 kB/s | 46 kB 00:00 2024-05-25T05:59:07.445 INFO:teuthology.orchestra.run.smithi160.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 857 kB/s | 79 kB 00:00 2024-05-25T05:59:07.512 INFO:teuthology.orchestra.run.smithi160.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-05-25T05:59:07.529 INFO:teuthology.orchestra.run.smithi160.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-05-25T05:59:07.554 INFO:teuthology.orchestra.run.smithi160.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-05-25T05:59:07.605 INFO:teuthology.orchestra.run.smithi160.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 388 kB/s | 36 kB 00:00 2024-05-25T05:59:07.630 INFO:teuthology.orchestra.run.smithi160.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-25T05:59:07.655 INFO:teuthology.orchestra.run.smithi160.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-05-25T05:59:07.697 INFO:teuthology.orchestra.run.smithi160.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-05-25T05:59:07.700 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:07.700 INFO:teuthology.orchestra.run.smithi160.stdout:Total 6.2 MB/s | 27 MB 00:04 2024-05-25T05:59:07.822 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:07.859 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:07.859 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:08.237 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:08.237 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:08.940 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 4/4 2024-05-25T05:59:08.940 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 1/4 2024-05-25T05:59:08.940 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 2/4 2024-05-25T05:59:08.940 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-25T05:59:09.195 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-25T05:59:09.195 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:09.195 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:09.195 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:09.196 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:09.196 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-25T05:59:09.196 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-25T05:59:09.196 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:09.196 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:09.387 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:09.449 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph 2024-05-25T05:59:09.458 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-25T05:59:09.491 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-25T05:59:09.523 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-25T05:59:09.618 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-25T05:59:09.695 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-25T05:59:09.743 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-25T05:59:09.778 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-25T05:59:09.807 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-25T05:59:09.842 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-25T05:59:09.875 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-25T05:59:09.916 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-25T05:59:09.959 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-25T05:59:09.993 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-25T05:59:10.017 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:41 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:10.031 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-25T05:59:10.114 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-25T05:59:10.140 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:10.142 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:10.142 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:17.2.7-958.g7b01852c.el9 ceph 6.5 k 2024-05-25T05:59:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:17.2.7-958.g7b01852c.el9 ceph 2.1 M 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:17.2.7-958.g7b01852c.el9 ceph 1.4 M 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 240 k 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:17.2.7-958.g7b01852c.el9 ceph 3.9 M 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:17.2.7-958.g7b01852c.el9 ceph 15 M 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-25T05:59:10.143 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-25T05:59:10.144 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout:Install 33 Packages 2024-05-25T05:59:10.145 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:10.146 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 27 M 2024-05-25T05:59:10.146 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 96 M 2024-05-25T05:59:10.146 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:10.261 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-25T05:59:10.348 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-25T05:59:10.402 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-25T05:59:10.439 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-25T05:59:10.497 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-25T05:59:10.552 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-25T05:59:10.753 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-25T05:59:10.804 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-25T05:59:10.857 INFO:teuthology.orchestra.run.smithi203.stdout:(1/33): ceph-17.2.7-958.g7b01852c.el9.x86_64.rp 29 kB/s | 6.5 kB 00:00 2024-05-25T05:59:10.882 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-25T05:59:10.970 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-25T05:59:10.999 INFO:teuthology.orchestra.run.smithi203.stdout:(2/33): ceph-mds-17.2.7-958.g7b01852c.el9.x86_6 5.6 MB/s | 2.1 MB 00:00 2024-05-25T05:59:11.010 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-25T05:59:11.129 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-25T05:59:11.273 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-958.g7b01852c.el9.n 28/33 2024-05-25T05:59:11.312 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 29/33 2024-05-25T05:59:11.338 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 29/33 2024-05-25T05:59:11.338 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:11.338 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-25T05:59:11.338 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-25T05:59:11.338 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-25T05:59:11.338 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:11.382 INFO:teuthology.orchestra.run.smithi203.stdout:(3/33): ceph-mgr-17.2.7-958.g7b01852c.el9.x86_6 1.9 MB/s | 1.4 MB 00:00 2024-05-25T05:59:11.457 INFO:teuthology.orchestra.run.smithi203.stdout:(4/33): ceph-mgr-modules-core-17.2.7-958.g7b018 3.1 MB/s | 240 kB 00:00 2024-05-25T05:59:11.766 INFO:teuthology.orchestra.run.smithi203.stdout:(5/33): ceph-osd-17.2.7-958.g7b01852c.el9.x86_6 20 MB/s | 15 MB 00:00 2024-05-25T05:59:11.791 INFO:teuthology.orchestra.run.smithi203.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 759 kB/s | 253 kB 00:00 2024-05-25T05:59:11.867 INFO:teuthology.orchestra.run.smithi203.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-25T05:59:11.942 INFO:teuthology.orchestra.run.smithi203.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-05-25T05:59:12.034 INFO:teuthology.orchestra.run.smithi203.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.3 MB/s | 125 kB 00:00 2024-05-25T05:59:12.167 INFO:teuthology.orchestra.run.smithi203.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.6 MB/s | 215 kB 00:00 2024-05-25T05:59:12.381 INFO:teuthology.orchestra.run.smithi203.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 2.0 MB/s | 1.2 MB 00:00 2024-05-25T05:59:12.618 INFO:teuthology.orchestra.run.smithi203.stdout:(12/33): ceph-mon-17.2.7-958.g7b01852c.el9.x86_ 2.2 MB/s | 3.9 MB 00:01 2024-05-25T05:59:12.672 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 30/33 2024-05-25T05:59:12.702 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 30/33 2024-05-25T05:59:12.702 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:12.702 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-25T05:59:12.702 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-25T05:59:12.703 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-25T05:59:12.703 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:13.106 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 31/33 2024-05-25T05:59:13.131 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 31/33 2024-05-25T05:59:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-25T05:59:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-25T05:59:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-25T05:59:13.131 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:13.160 INFO:teuthology.orchestra.run.smithi203.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 173 kB/s | 172 kB 00:00 2024-05-25T05:59:13.185 INFO:teuthology.orchestra.run.smithi203.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 43 kB/s | 35 kB 00:00 2024-05-25T05:59:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 32/33 2024-05-25T05:59:13.372 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 32/33 2024-05-25T05:59:13.372 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:13.372 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-25T05:59:13.373 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-25T05:59:13.373 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-25T05:59:13.373 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:13.438 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:17.2.7-958.g7b01852c.el9.x86_64 33/33 2024-05-25T05:59:13.452 INFO:teuthology.orchestra.run.smithi203.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 61 kB/s | 51 kB 00:00 2024-05-25T05:59:13.477 INFO:teuthology.orchestra.run.smithi203.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 149 kB/s | 43 kB 00:00 2024-05-25T05:59:13.686 INFO:teuthology.orchestra.run.smithi203.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 735 kB/s | 172 kB 00:00 2024-05-25T05:59:13.770 INFO:teuthology.orchestra.run.smithi203.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-05-25T05:59:13.786 INFO:teuthology.orchestra.run.smithi203.stdout:(19/33): python3-toml-0.10.2-6.el9.noarch.rpm 66 kB/s | 42 kB 00:00 2024-05-25T05:59:13.845 INFO:teuthology.orchestra.run.smithi203.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2024-05-25T05:59:13.929 INFO:teuthology.orchestra.run.smithi203.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 795 kB/s | 358 kB 00:00 2024-05-25T05:59:13.954 INFO:teuthology.orchestra.run.smithi203.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-05-25T05:59:13.988 INFO:teuthology.orchestra.run.smithi203.stdout:(23/33): python3-jaraco-collections-3.0.0-8.el9 115 kB/s | 23 kB 00:00 2024-05-25T05:59:14.013 INFO:teuthology.orchestra.run.smithi203.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 234 kB/s | 20 kB 00:00 2024-05-25T05:59:14.038 INFO:teuthology.orchestra.run.smithi203.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 552 kB/s | 46 kB 00:00 2024-05-25T05:59:14.122 INFO:teuthology.orchestra.run.smithi203.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 197 kB/s | 16 kB 00:00 2024-05-25T05:59:14.164 INFO:teuthology.orchestra.run.smithi203.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-25T05:59:14.206 INFO:teuthology.orchestra.run.smithi203.stdout:(28/33): python3-more-itertools-8.12.0-2.el9.no 362 kB/s | 79 kB 00:00 2024-05-25T05:59:14.231 INFO:teuthology.orchestra.run.smithi203.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 826 kB/s | 90 kB 00:00 2024-05-25T05:59:14.256 INFO:teuthology.orchestra.run.smithi203.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 388 kB/s | 36 kB 00:00 2024-05-25T05:59:14.340 INFO:teuthology.orchestra.run.smithi203.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 240 kB/s | 20 kB 00:00 2024-05-25T05:59:14.365 INFO:teuthology.orchestra.run.smithi203.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-05-25T05:59:14.407 INFO:teuthology.orchestra.run.smithi203.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-05-25T05:59:14.409 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:14.410 INFO:teuthology.orchestra.run.smithi203.stdout:Total 6.4 MB/s | 27 MB 00:04 2024-05-25T05:59:14.530 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:14.568 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:14.568 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:14.937 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:14.937 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-2:17.2.7-958.g7b01852c.el9.x86_64 33/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:17.2.7-958.g7b01852c.el9.x86_64 1/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 2/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 3/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 4/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 5/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-958.g7b01852c.el9.n 6/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-25T05:59:14.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-25T05:59:14.980 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-25T05:59:14.981 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:16.017 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-25T05:59:16.018 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-25T05:59:16.019 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-25T05:59:16.020 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-25T05:59:16.020 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-25T05:59:16.020 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-25T05:59:16.020 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:16.020 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:16.080 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:16.126 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-25T05:59:16.158 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-25T05:59:16.191 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-25T05:59:16.293 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-25T05:59:16.346 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-base 2024-05-25T05:59:16.410 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-25T05:59:16.469 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-25T05:59:16.513 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-25T05:59:16.542 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-25T05:59:16.577 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-25T05:59:16.610 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-25T05:59:16.651 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-25T05:59:16.685 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-25T05:59:16.720 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-25T05:59:16.758 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-25T05:59:16.832 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-25T05:59:16.879 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:53 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:16.959 INFO:teuthology.orchestra.run.smithi160.stdout:Package ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:16.981 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-25T05:59:17.001 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:17.002 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T05:59:17.002 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:17.058 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-25T05:59:17.070 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install cephadm 2024-05-25T05:59:17.112 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-25T05:59:17.150 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-25T05:59:17.207 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-25T05:59:17.263 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-25T05:59:17.465 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-25T05:59:17.512 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-25T05:59:17.590 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-25T05:59:17.598 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:54 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:17.680 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-25T05:59:17.719 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:17.720 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-25T05:59:17.720 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 82 k 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 82 k 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 372 k 2024-05-25T05:59:17.721 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:17.845 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-25T05:59:17.998 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-958.g7b01852c.el9.n 28/33 2024-05-25T05:59:18.030 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 29/33 2024-05-25T05:59:18.038 INFO:teuthology.orchestra.run.smithi160.stdout:cephadm-17.2.7-958.g7b01852c.el9.noarch.rpm 259 kB/s | 82 kB 00:00 2024-05-25T05:59:18.039 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:18.039 INFO:teuthology.orchestra.run.smithi160.stdout:Total 257 kB/s | 82 kB 00:00 2024-05-25T05:59:18.039 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:18.044 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:18.045 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:18.046 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:18.047 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:18.055 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 29/33 2024-05-25T05:59:18.055 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:18.056 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-25T05:59:18.056 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-25T05:59:18.056 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-25T05:59:18.056 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:18.117 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:18.554 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:18.650 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:18.970 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:19.231 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:19.231 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:19.231 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:19.231 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:19.231 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:19.231 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:19.381 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 30/33 2024-05-25T05:59:19.409 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 30/33 2024-05-25T05:59:19.410 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:19.410 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-25T05:59:19.410 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-25T05:59:19.410 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-25T05:59:19.410 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:19.432 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-immutable-object-cache 2024-05-25T05:59:19.799 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 31/33 2024-05-25T05:59:19.824 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 31/33 2024-05-25T05:59:19.824 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:19.825 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-25T05:59:19.825 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-25T05:59:19.825 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-25T05:59:19.825 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:56 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:20.025 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 32/33 2024-05-25T05:59:20.049 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 32/33 2024-05-25T05:59:20.049 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:20.049 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-25T05:59:20.049 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-25T05:59:20.049 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-25T05:59:20.049 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:20.105 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-958.g7b01852c.el9 ceph 137 k 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T05:59:20.106 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:20.107 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 137 k 2024-05-25T05:59:20.107 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 379 k 2024-05-25T05:59:20.107 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:20.113 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:17.2.7-958.g7b01852c.el9.x86_64 33/33 2024-05-25T05:59:20.344 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-immutable-object-cache-17.2.7-958.g7b01852 581 kB/s | 137 kB 00:00 2024-05-25T05:59:20.344 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:20.345 INFO:teuthology.orchestra.run.smithi160.stdout:Total 577 kB/s | 137 kB 00:00 2024-05-25T05:59:20.345 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:20.352 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:20.352 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:20.408 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:20.408 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:20.546 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:20.620 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.e 1/1 2024-05-25T05:59:20.645 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.e 1/1 2024-05-25T05:59:20.645 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:20.646 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-25T05:59:20.646 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:21.539 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.e 1/1 2024-05-25T05:59:21.539 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:21.539 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:21.539 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:21.539 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:21.539 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-2:17.2.7-958.g7b01852c.el9.x86_64 33/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:17.2.7-958.g7b01852c.el9.x86_64 1/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 2/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 3/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 4/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 5/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-958.g7b01852c.el9.n 6/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-25T05:59:21.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-25T05:59:21.646 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-25T05:59:21.647 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-25T05:59:21.647 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-25T05:59:21.647 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-25T05:59:21.647 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-25T05:59:21.647 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-25T05:59:21.648 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-25T05:59:21.763 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr 2024-05-25T05:59:22.297 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:59 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:22.379 INFO:teuthology.orchestra.run.smithi160.stdout:Package ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:22.424 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:22.425 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T05:59:22.425 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:22.488 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-dashboard 2024-05-25T05:59:22.516 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-25T05:59:22.517 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-25T05:59:22.518 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:22.519 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:22.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-base 2024-05-25T05:59:23.015 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:00 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:23.148 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 3.4 M 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 24 k 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 15 k 2024-05-25T05:59:23.149 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout:Install 7 Packages 2024-05-25T05:59:23.150 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:23.151 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 3.7 M 2024-05-25T05:59:23.151 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 80 M 2024-05-25T05:59:23.151 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:23.390 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:54 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:23.470 INFO:teuthology.orchestra.run.smithi203.stdout:Package ceph-base-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:23.512 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:23.513 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T05:59:23.513 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:23.546 INFO:teuthology.orchestra.run.smithi160.stdout:(1/7): ceph-prometheus-alerts-17.2.7-958.g7b018 74 kB/s | 15 kB 00:00 2024-05-25T05:59:23.595 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install cephadm 2024-05-25T05:59:23.638 INFO:teuthology.orchestra.run.smithi160.stdout:(2/7): ceph-grafana-dashboards-17.2.7-958.g7b01 84 kB/s | 24 kB 00:00 2024-05-25T05:59:23.655 INFO:teuthology.orchestra.run.smithi160.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-05-25T05:59:23.730 INFO:teuthology.orchestra.run.smithi160.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 445 kB/s | 41 kB 00:00 2024-05-25T05:59:23.789 INFO:teuthology.orchestra.run.smithi160.stdout:(5/7): ceph-mgr-dashboard-17.2.7-958.g7b01852c. 7.6 MB/s | 3.4 MB 00:00 2024-05-25T05:59:23.814 INFO:teuthology.orchestra.run.smithi160.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 193 kB/s | 31 kB 00:00 2024-05-25T05:59:23.856 INFO:teuthology.orchestra.run.smithi160.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-05-25T05:59:23.858 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:23.858 INFO:teuthology.orchestra.run.smithi160.stdout:Total 5.2 MB/s | 3.7 MB 00:00 2024-05-25T05:59:23.858 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:23.866 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:23.866 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:23.928 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:23.928 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:24.121 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:55 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:24.233 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:24.242 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 82 k 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T05:59:24.243 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:24.244 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 82 k 2024-05-25T05:59:24.244 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 372 k 2024-05-25T05:59:24.244 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:24.290 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-25T05:59:24.325 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-25T05:59:24.372 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-25T05:59:24.403 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-25T05:59:24.424 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-958.g7b01852c.el9.no 5/7 2024-05-25T05:59:24.565 INFO:teuthology.orchestra.run.smithi203.stdout:cephadm-17.2.7-958.g7b01852c.el9.noarch.rpm 256 kB/s | 82 kB 00:00 2024-05-25T05:59:24.565 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:24.566 INFO:teuthology.orchestra.run.smithi203.stdout:Total 254 kB/s | 82 kB 00:00 2024-05-25T05:59:24.566 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:24.571 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:24.571 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:24.573 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:24.573 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:24.640 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:25.090 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:25.186 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:25.418 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-958.g7b01852c.el9.n 6/7 2024-05-25T05:59:25.452 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 7/7 2024-05-25T05:59:25.524 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:25.749 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/1 2024-05-25T05:59:25.749 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:25.749 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:25.750 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:25.750 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:25.750 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:25.978 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-immutable-object-cache 2024-05-25T05:59:26.398 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 7/7 2024-05-25T05:59:26.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-958.g7b01852c.el9.n 1/7 2024-05-25T05:59:26.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 2/7 2024-05-25T05:59:26.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-958.g7b01852c.el9.no 3/7 2024-05-25T05:59:26.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-25T05:59:26.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-25T05:59:26.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-25T05:59:26.517 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:57 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:26.638 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-25T05:59:26.638 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:26.639 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:26.642 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repo Size 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-958.g7b01852c.el9 ceph 137 k 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:26.643 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:26.644 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T05:59:26.644 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:26.644 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 137 k 2024-05-25T05:59:26.644 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 379 k 2024-05-25T05:59:26.644 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:26.848 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-25T05:59:26.980 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-immutable-object-cache-17.2.7-958.g7b01852 409 kB/s | 137 kB 00:00 2024-05-25T05:59:26.981 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:26.981 INFO:teuthology.orchestra.run.smithi203.stdout:Total 407 kB/s | 137 kB 00:00 2024-05-25T05:59:26.981 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:26.989 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:26.989 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:27.044 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:27.044 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:27.194 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:27.265 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.e 1/1 2024-05-25T05:59:27.290 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.e 1/1 2024-05-25T05:59:27.290 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T05:59:27.290 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-25T05:59:27.290 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:27.377 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:04 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:27.503 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout:====================================================================================== 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout:====================================================================================== 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 7.4 M 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:59:27.504 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout:====================================================================================== 2024-05-25T05:59:27.505 INFO:teuthology.orchestra.run.smithi160.stdout:Install 12 Packages 2024-05-25T05:59:27.506 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:27.506 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 38 M 2024-05-25T05:59:27.506 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 209 M 2024-05-25T05:59:27.506 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:28.230 INFO:teuthology.orchestra.run.smithi160.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-95 13 MB/s | 7.4 MB 00:00 2024-05-25T05:59:28.261 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.e 1/1 2024-05-25T05:59:28.261 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:28.262 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:28.262 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:28.262 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:28.262 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:28.491 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-mgr 2024-05-25T05:59:28.522 INFO:teuthology.orchestra.run.smithi160.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 227 kB/s | 195 kB 00:00 2024-05-25T05:59:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 780 kB/s | 803 kB 00:01 2024-05-25T05:59:28.814 INFO:teuthology.orchestra.run.smithi160.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 56 kB/s | 33 kB 00:00 2024-05-25T05:59:29.035 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:00 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:29.116 INFO:teuthology.orchestra.run.smithi203.stdout:Package ceph-mgr-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:29.159 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:29.160 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T05:59:29.161 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:29.231 INFO:teuthology.orchestra.run.smithi160.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 34 kB/s | 18 kB 00:00 2024-05-25T05:59:29.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-mgr-dashboard 2024-05-25T05:59:29.398 INFO:teuthology.orchestra.run.smithi160.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 60 kB/s | 35 kB 00:00 2024-05-25T05:59:29.665 INFO:teuthology.orchestra.run.smithi160.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-05-25T05:59:29.766 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:00 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:29.888 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 3.4 M 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 24 k 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 15 k 2024-05-25T05:59:29.889 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout:Install 7 Packages 2024-05-25T05:59:29.890 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:29.891 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 3.7 M 2024-05-25T05:59:29.894 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 80 M 2024-05-25T05:59:29.894 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:30.248 INFO:teuthology.orchestra.run.smithi160.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 287 kB/s | 244 kB 00:00 2024-05-25T05:59:30.442 INFO:teuthology.orchestra.run.smithi203.stdout:(1/7): ceph-grafana-dashboards-17.2.7-958.g7b01 125 kB/s | 24 kB 00:00 2024-05-25T05:59:30.467 INFO:teuthology.orchestra.run.smithi203.stdout:(2/7): ceph-prometheus-alerts-17.2.7-958.g7b018 66 kB/s | 15 kB 00:00 2024-05-25T05:59:30.499 INFO:teuthology.orchestra.run.smithi160.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.7 MB/s | 4.7 MB 00:01 2024-05-25T05:59:30.584 INFO:teuthology.orchestra.run.smithi203.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 64 kB/s | 9.0 kB 00:00 2024-05-25T05:59:30.634 INFO:teuthology.orchestra.run.smithi203.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 245 kB/s | 41 kB 00:00 2024-05-25T05:59:30.676 INFO:teuthology.orchestra.run.smithi203.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 336 kB/s | 31 kB 00:00 2024-05-25T05:59:30.760 INFO:teuthology.orchestra.run.smithi203.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-05-25T05:59:30.827 INFO:teuthology.orchestra.run.smithi203.stdout:(7/7): ceph-mgr-dashboard-17.2.7-958.g7b01852c. 5.8 MB/s | 3.4 MB 00:00 2024-05-25T05:59:30.829 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:30.829 INFO:teuthology.orchestra.run.smithi203.stdout:Total 3.9 MB/s | 3.7 MB 00:00 2024-05-25T05:59:30.829 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:30.837 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:30.838 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:30.899 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:30.899 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:30.932 INFO:teuthology.orchestra.run.smithi160.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-25T05:59:31.107 INFO:teuthology.orchestra.run.smithi160.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 292 kB/s | 250 kB 00:00 2024-05-25T05:59:31.196 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:31.262 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-25T05:59:31.296 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-25T05:59:31.342 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-25T05:59:31.374 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-25T05:59:31.403 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-958.g7b01852c.el9.no 5/7 2024-05-25T05:59:32.064 INFO:teuthology.orchestra.run.smithi160.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-05-25T05:59:32.066 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:32.066 INFO:teuthology.orchestra.run.smithi160.stdout:Total 8.3 MB/s | 38 MB 00:04 2024-05-25T05:59:32.400 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-958.g7b01852c.el9.n 6/7 2024-05-25T05:59:32.420 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:32.448 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:32.448 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:32.448 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 7/7 2024-05-25T05:59:32.952 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:32.953 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 7/7 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-958.g7b01852c.el9.n 1/7 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 2/7 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-958.g7b01852c.el9.no 3/7 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-25T05:59:33.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-25T05:59:33.717 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-25T05:59:33.718 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-25T05:59:33.718 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-25T05:59:33.718 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:33.718 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-25T05:59:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-25T05:59:33.936 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-25T05:59:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-25T05:59:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-25T05:59:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-25T05:59:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-25T05:59:34.367 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-25T05:59:34.466 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:05 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:34.590 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:34.591 INFO:teuthology.orchestra.run.smithi203.stdout:====================================================================================== 2024-05-25T05:59:34.591 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:34.591 INFO:teuthology.orchestra.run.smithi203.stdout:====================================================================================== 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 7.4 M 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-25T05:59:34.592 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout:====================================================================================== 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout:Install 12 Packages 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:34.593 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 38 M 2024-05-25T05:59:34.594 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 209 M 2024-05-25T05:59:34.594 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:35.040 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-25T05:59:35.153 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-25T05:59:35.601 INFO:teuthology.orchestra.run.smithi203.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-95 12 MB/s | 7.4 MB 00:00 2024-05-25T05:59:35.685 INFO:teuthology.orchestra.run.smithi203.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-05-25T05:59:35.885 INFO:teuthology.orchestra.run.smithi203.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-05-25T05:59:35.911 INFO:teuthology.orchestra.run.smithi203.stdout:(4/12): libquadmath-11.4.1-3.el9.x86_64.rpm 213 kB/s | 195 kB 00:00 2024-05-25T05:59:35.927 INFO:teuthology.orchestra.run.smithi203.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 434 kB/s | 18 kB 00:00 2024-05-25T05:59:35.994 INFO:teuthology.orchestra.run.smithi203.stdout:(6/12): libgfortran-11.4.1-3.el9.x86_64.rpm 803 kB/s | 803 kB 00:00 2024-05-25T05:59:36.020 INFO:teuthology.orchestra.run.smithi203.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-05-25T05:59:36.095 INFO:teuthology.orchestra.run.smithi203.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2024-05-25T05:59:36.145 INFO:teuthology.orchestra.run.smithi203.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-05-25T05:59:36.196 INFO:teuthology.orchestra.run.smithi203.stdout:(10/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-05-25T05:59:36.296 INFO:teuthology.orchestra.run.smithi203.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-05-25T05:59:36.423 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-25T05:59:36.538 INFO:teuthology.orchestra.run.smithi203.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-05-25T05:59:36.539 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:36.539 INFO:teuthology.orchestra.run.smithi203.stdout:Total 20 MB/s | 38 MB 00:01 2024-05-25T05:59:36.894 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:36.922 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:36.922 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:37.443 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:37.444 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:37.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-25T05:59:37.495 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-958.g7b0185 12/12 2024-05-25T05:59:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:38.337 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-25T05:59:38.378 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-25T05:59:38.479 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-25T05:59:38.521 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-25T05:59:38.703 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-25T05:59:38.736 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-25T05:59:38.824 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-25T05:59:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-25T05:59:39.600 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-25T05:59:39.935 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-958.g7b0185 12/12 2024-05-25T05:59:39.935 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-958.g7b0185 1/12 2024-05-25T05:59:39.935 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-25T05:59:39.935 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-25T05:59:39.935 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-25T05:59:39.936 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-25T05:59:40.370 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-25T05:59:40.370 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:40.370 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-25T05:59:40.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-25T05:59:40.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-25T05:59:40.372 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:40.372 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:40.721 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-rook 2024-05-25T05:59:40.849 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-25T05:59:41.276 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:18 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:41.406 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:41.407 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:41.407 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:41.407 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:41.407 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:41.407 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 47 k 2024-05-25T05:59:41.407 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-25T05:59:41.408 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-25T05:59:41.409 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:41.409 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:41.409 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:41.409 INFO:teuthology.orchestra.run.smithi160.stdout:Install 13 Packages 2024-05-25T05:59:41.409 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:41.409 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 2.2 M 2024-05-25T05:59:41.410 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 26 M 2024-05-25T05:59:41.410 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:41.886 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-25T05:59:41.919 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-958.g7b0185 12/12 2024-05-25T05:59:42.053 INFO:teuthology.orchestra.run.smithi160.stdout:(1/13): ceph-mgr-rook-17.2.7-958.g7b01852c.el9. 266 kB/s | 47 kB 00:00 2024-05-25T05:59:42.128 INFO:teuthology.orchestra.run.smithi160.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 101 kB/s | 26 kB 00:00 2024-05-25T05:59:42.145 INFO:teuthology.orchestra.run.smithi160.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 71 kB/s | 19 kB 00:00 2024-05-25T05:59:42.470 INFO:teuthology.orchestra.run.smithi160.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 464 kB/s | 159 kB 00:00 2024-05-25T05:59:42.562 INFO:teuthology.orchestra.run.smithi160.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 668 kB/s | 279 kB 00:00 2024-05-25T05:59:42.587 INFO:teuthology.orchestra.run.smithi160.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 460 kB/s | 54 kB 00:00 2024-05-25T05:59:42.604 INFO:teuthology.orchestra.run.smithi160.stdout:(7/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-05-25T05:59:42.705 INFO:teuthology.orchestra.run.smithi160.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2024-05-25T05:59:42.730 INFO:teuthology.orchestra.run.smithi160.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2024-05-25T05:59:42.822 INFO:teuthology.orchestra.run.smithi160.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 946 kB/s | 206 kB 00:00 2024-05-25T05:59:42.847 INFO:teuthology.orchestra.run.smithi160.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-05-25T05:59:42.889 INFO:teuthology.orchestra.run.smithi160.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-05-25T05:59:42.964 INFO:teuthology.orchestra.run.smithi160.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.9 MB/s | 1.0 MB 00:00 2024-05-25T05:59:42.965 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:42.965 INFO:teuthology.orchestra.run.smithi160.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-05-25T05:59:43.030 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:43.045 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:43.045 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:43.129 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:43.131 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:43.630 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:43.801 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-25T05:59:43.858 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-25T05:59:43.921 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-25T05:59:43.958 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-25T05:59:43.999 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-25T05:59:44.102 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-25T05:59:44.221 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-25T05:59:44.276 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-25T05:59:44.495 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-958.g7b0185 12/12 2024-05-25T05:59:44.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-958.g7b0185 1/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-25T05:59:44.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-25T05:59:44.862 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-25T05:59:44.862 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-25T05:59:44.863 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-25T05:59:44.864 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:44.864 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:45.001 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-25T05:59:45.047 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-25T05:59:45.081 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-25T05:59:45.126 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-25T05:59:45.150 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 13/13 2024-05-25T05:59:45.210 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-mgr-rook 2024-05-25T05:59:45.750 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:16 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:45.871 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 47 k 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-25T05:59:45.873 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout:Install 13 Packages 2024-05-25T05:59:45.874 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:45.875 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 2.2 M 2024-05-25T05:59:45.875 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 26 M 2024-05-25T05:59:45.875 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:45.927 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 13/13 2024-05-25T05:59:45.927 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 1/13 2024-05-25T05:59:45.927 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-25T05:59:45.927 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-25T05:59:45.927 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-25T05:59:45.927 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-25T05:59:45.928 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-25T05:59:46.153 INFO:teuthology.orchestra.run.smithi203.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 286 kB/s | 26 kB 00:00 2024-05-25T05:59:46.178 INFO:teuthology.orchestra.run.smithi203.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 166 kB/s | 19 kB 00:00 2024-05-25T05:59:46.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-25T05:59:46.246 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:46.246 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:46.246 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:46.246 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:46.247 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:46.261 INFO:teuthology.orchestra.run.smithi203.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-05-25T05:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout:(4/13): ceph-mgr-rook-17.2.7-958.g7b01852c.el9. 212 kB/s | 47 kB 00:00 2024-05-25T05:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-25T05:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-05-25T05:59:46.404 INFO:teuthology.orchestra.run.smithi203.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 458 kB/s | 54 kB 00:00 2024-05-25T05:59:46.497 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-cephadm 2024-05-25T05:59:46.555 INFO:teuthology.orchestra.run.smithi203.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 69 kB/s | 15 kB 00:00 2024-05-25T05:59:46.580 INFO:teuthology.orchestra.run.smithi203.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 120 kB/s | 32 kB 00:00 2024-05-25T05:59:46.705 INFO:teuthology.orchestra.run.smithi203.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-05-25T05:59:46.731 INFO:teuthology.orchestra.run.smithi203.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 631 kB/s | 206 kB 00:00 2024-05-25T05:59:46.789 INFO:teuthology.orchestra.run.smithi203.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-05-25T05:59:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2024-05-25T05:59:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-05-25T05:59:46.890 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:46.904 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:46.904 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:46.991 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:46.991 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:47.024 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:24 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:47.147 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:47.148 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:47.148 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:47.148 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:47.148 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:47.148 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 123 k 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout:Install 6 Packages 2024-05-25T05:59:47.149 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:47.150 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 7.0 M 2024-05-25T05:59:47.150 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 33 M 2024-05-25T05:59:47.150 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:47.496 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:47.663 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-25T05:59:47.698 INFO:teuthology.orchestra.run.smithi160.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-05-25T05:59:47.716 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-25T05:59:47.757 INFO:teuthology.orchestra.run.smithi160.stdout:(2/6): ceph-mgr-cephadm-17.2.7-958.g7b01852c.el 577 kB/s | 123 kB 00:00 2024-05-25T05:59:47.771 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-25T05:59:47.808 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-25T05:59:47.849 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-25T05:59:47.944 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-25T05:59:47.965 INFO:teuthology.orchestra.run.smithi160.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-25T05:59:48.061 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-25T05:59:48.109 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-25T05:59:48.224 INFO:teuthology.orchestra.run.smithi160.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 124 kB/s | 58 kB 00:00 2024-05-25T05:59:48.366 INFO:teuthology.orchestra.run.smithi160.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 821 kB/s | 548 kB 00:00 2024-05-25T05:59:48.390 INFO:teuthology.orchestra.run.smithi160.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 120 kB/s | 51 kB 00:00 2024-05-25T05:59:48.394 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:48.394 INFO:teuthology.orchestra.run.smithi160.stdout:Total 5.6 MB/s | 7.0 MB 00:01 2024-05-25T05:59:48.465 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:48.476 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:48.477 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:48.573 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:48.851 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-25T05:59:48.876 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:48.905 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-25T05:59:48.939 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-25T05:59:48.985 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-25T05:59:49.009 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 13/13 2024-05-25T05:59:49.039 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-25T05:59:49.092 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-25T05:59:49.612 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-25T05:59:49.694 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-25T05:59:49.750 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-25T05:59:49.771 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 6/6 2024-05-25T05:59:49.772 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 13/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 1/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-25T05:59:49.773 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-25T05:59:50.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-25T05:59:50.093 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:50.094 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:50.332 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-mgr-cephadm 2024-05-25T05:59:50.536 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 6/6 2024-05-25T05:59:50.536 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/6 2024-05-25T05:59:50.536 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-25T05:59:50.536 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-25T05:59:50.536 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-25T05:59:50.536 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-25T05:59:50.803 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-25T05:59:50.804 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:50.804 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:50.881 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:21 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:51.004 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:51.005 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:51.005 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:51.005 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:51.005 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:51.005 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:17.2.7-958.g7b01852c.el9 ceph-noarch 123 k 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout:Install 6 Packages 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 7.0 M 2024-05-25T05:59:51.006 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 33 M 2024-05-25T05:59:51.007 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:51.046 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-fuse 2024-05-25T05:59:51.570 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:28 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:51.641 INFO:teuthology.orchestra.run.smithi203.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-05-25T05:59:51.691 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:17.2.7-958.g7b01852c.el9 ceph 812 k 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:51.692 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:51.693 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:51.693 INFO:teuthology.orchestra.run.smithi160.stdout:Install 2 Packages 2024-05-25T05:59:51.693 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:51.693 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 893 k 2024-05-25T05:59:51.693 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 2.6 M 2024-05-25T05:59:51.693 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:51.817 INFO:teuthology.orchestra.run.smithi203.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-25T05:59:51.842 INFO:teuthology.orchestra.run.smithi203.stdout:(3/6): ceph-mgr-cephadm-17.2.7-958.g7b01852c.el 337 kB/s | 123 kB 00:00 2024-05-25T05:59:51.984 INFO:teuthology.orchestra.run.smithi160.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 464 kB/s | 80 kB 00:00 2024-05-25T05:59:52.117 INFO:teuthology.orchestra.run.smithi203.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 193 kB/s | 58 kB 00:00 2024-05-25T05:59:52.117 INFO:teuthology.orchestra.run.smithi160.stdout:(2/2): ceph-fuse-17.2.7-958.g7b01852c.el9.x86_6 2.6 MB/s | 812 kB 00:00 2024-05-25T05:59:52.118 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:52.118 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.0 MB/s | 893 kB 00:00 2024-05-25T05:59:52.130 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:52.134 INFO:teuthology.orchestra.run.smithi203.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 175 kB/s | 51 kB 00:00 2024-05-25T05:59:52.151 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:52.152 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:52.175 INFO:teuthology.orchestra.run.smithi203.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-05-25T05:59:52.178 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:52.179 INFO:teuthology.orchestra.run.smithi203.stdout:Total 5.9 MB/s | 7.0 MB 00:01 2024-05-25T05:59:52.228 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:52.229 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:52.249 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:52.260 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:52.261 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:52.422 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:52.555 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-25T05:59:52.642 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 2/2 2024-05-25T05:59:52.656 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:52.819 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-25T05:59:52.870 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-25T05:59:53.274 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 2/2 2024-05-25T05:59:53.274 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/2 2024-05-25T05:59:53.393 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-25T05:59:53.481 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-25T05:59:53.537 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-25T05:59:53.541 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-25T05:59:53.541 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:53.541 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:53.541 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-25T05:59:53.541 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:53.541 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:53.565 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 6/6 2024-05-25T05:59:53.762 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install librados-devel 2024-05-25T05:59:54.306 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:31 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:54.342 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 6/6 2024-05-25T05:59:54.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 1/6 2024-05-25T05:59:54.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-25T05:59:54.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-25T05:59:54.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-25T05:59:54.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-25T05:59:54.433 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:54.433 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:54.433 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:17.2.7-958.g7b01852c.el9 ceph 126 k 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T05:59:54.434 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:54.435 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 126 k 2024-05-25T05:59:54.435 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 448 k 2024-05-25T05:59:54.435 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:54.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-25T05:59:54.604 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:17.2.7-958.g7b01852c.el9.noarch 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:54.605 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:54.639 INFO:teuthology.orchestra.run.smithi160.stdout:librados-devel-17.2.7-958.g7b01852c.el9.x86_64. 619 kB/s | 126 kB 00:00 2024-05-25T05:59:54.639 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:54.639 INFO:teuthology.orchestra.run.smithi160.stdout:Total 614 kB/s | 126 kB 00:00 2024-05-25T05:59:54.640 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:54.645 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:54.645 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:54.709 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:54.709 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:54.855 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:54.874 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-fuse 2024-05-25T05:59:54.953 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:55.365 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:55.404 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:26 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:55.527 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:55.527 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:55.527 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-25T05:59:55.527 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:55.527 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:17.2.7-958.g7b01852c.el9 ceph 812 k 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout:Install 2 Packages 2024-05-25T05:59:55.528 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:55.529 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 893 k 2024-05-25T05:59:55.529 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 2.6 M 2024-05-25T05:59:55.529 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:55.613 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:55.613 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:55.824 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install libcephfs2 2024-05-25T05:59:56.012 INFO:teuthology.orchestra.run.smithi203.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 381 kB/s | 80 kB 00:00 2024-05-25T05:59:56.272 INFO:teuthology.orchestra.run.smithi203.stdout:(2/2): ceph-fuse-17.2.7-958.g7b01852c.el9.x86_6 1.7 MB/s | 812 kB 00:00 2024-05-25T05:59:56.272 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:56.272 INFO:teuthology.orchestra.run.smithi203.stdout:Total 1.2 MB/s | 893 kB 00:00 2024-05-25T05:59:56.283 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:56.306 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:56.306 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:56.370 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:33 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:56.383 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:56.383 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:56.452 INFO:teuthology.orchestra.run.smithi160.stdout:Package libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:56.496 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:56.497 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T05:59:56.498 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:56.558 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install libcephfs-devel 2024-05-25T05:59:56.573 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:56.708 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-25T05:59:56.787 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 2/2 2024-05-25T05:59:57.091 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:34 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:57.213 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:57.213 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:57.213 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-05-25T05:59:57.213 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:17.2.7-958.g7b01852c.el9 ceph 25 k 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 25 k 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 110 k 2024-05-25T05:59:57.214 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T05:59:57.437 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 2/2 2024-05-25T05:59:57.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/2 2024-05-25T05:59:57.542 INFO:teuthology.orchestra.run.smithi160.stdout:libcephfs-devel-17.2.7-958.g7b01852c.el9.x86_64 76 kB/s | 25 kB 00:00 2024-05-25T05:59:57.542 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:57.543 INFO:teuthology.orchestra.run.smithi160.stdout:Total 76 kB/s | 25 kB 00:00 2024-05-25T05:59:57.543 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T05:59:57.546 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T05:59:57.546 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T05:59:57.548 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T05:59:57.548 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T05:59:57.631 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T05:59:57.713 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-25T05:59:57.713 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:57.713 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:57.714 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-25T05:59:57.714 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:57.714 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:57.771 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:57.979 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install librados-devel 2024-05-25T05:59:58.116 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:58.324 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:58.324 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:58.324 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T05:59:58.324 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:58.324 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T05:59:58.324 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:58.506 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install librados2 2024-05-25T05:59:58.527 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:29 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T05:59:58.655 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repo Size 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:17.2.7-958.g7b01852c.el9 ceph 126 k 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T05:59:58.656 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T05:59:58.657 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T05:59:58.657 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:58.657 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 126 k 2024-05-25T05:59:58.657 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 448 k 2024-05-25T05:59:58.657 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T05:59:58.994 INFO:teuthology.orchestra.run.smithi203.stdout:librados-devel-17.2.7-958.g7b01852c.el9.x86_64. 374 kB/s | 126 kB 00:00 2024-05-25T05:59:58.995 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T05:59:58.995 INFO:teuthology.orchestra.run.smithi203.stdout:Total 372 kB/s | 126 kB 00:00 2024-05-25T05:59:58.995 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T05:59:59.001 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T05:59:59.001 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T05:59:59.034 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:36 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:59.064 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T05:59:59.064 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T05:59:59.116 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:59.158 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:59.159 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T05:59:59.159 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:59.211 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T05:59:59.226 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install librbd1 2024-05-25T05:59:59.309 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:59.705 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:59.755 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:36 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T05:59:59.835 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T05:59:59.876 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T05:59:59.877 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T05:59:59.877 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T05:59:59.929 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T05:59:59.929 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:59.929 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T05:59:59.930 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T05:59:59.930 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T05:59:59.930 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T05:59:59.940 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-rados 2024-05-25T06:00:00.173 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install libcephfs2 2024-05-25T06:00:00.475 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:37 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:00.556 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:00.599 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:00.600 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T06:00:00.600 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:00.669 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-rgw 2024-05-25T06:00:00.723 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:31 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:00.807 INFO:teuthology.orchestra.run.smithi203.stdout:Package libcephfs2-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:00.850 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:00.851 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:00.851 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:00.967 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install libcephfs-devel 2024-05-25T06:00:01.207 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:38 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:01.287 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:01.329 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:01.330 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T06:00:01.330 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:01.390 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-cephfs 2024-05-25T06:00:01.506 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:32 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:01.628 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:01.628 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:01.628 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repo Size 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:17.2.7-958.g7b01852c.el9 ceph 25 k 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 25 k 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 110 k 2024-05-25T06:00:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T06:00:01.791 INFO:teuthology.orchestra.run.smithi203.stdout:libcephfs-devel-17.2.7-958.g7b01852c.el9.x86_64 154 kB/s | 25 kB 00:00 2024-05-25T06:00:01.792 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:01.792 INFO:teuthology.orchestra.run.smithi203.stdout:Total 152 kB/s | 25 kB 00:00 2024-05-25T06:00:01.792 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T06:00:01.796 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T06:00:01.796 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T06:00:01.797 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T06:00:01.798 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T06:00:01.871 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T06:00:01.920 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:38 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:01.988 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:02.001 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:02.042 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:02.043 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T06:00:02.043 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:02.114 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-rbd 2024-05-25T06:00:02.370 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:02.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:02.587 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:02.587 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T06:00:02.587 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:02.587 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:02.588 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:02.648 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:39 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:02.730 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:02.773 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:02.774 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-05-25T06:00:02.774 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:02.824 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install librados2 2024-05-25T06:00:02.842 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-fuse 2024-05-25T06:00:03.358 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:34 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:03.375 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:40 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:03.439 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:03.479 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:03.480 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:03.480 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:03.497 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:03.497 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:03.497 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-05-25T06:00:03.497 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:17.2.7-958.g7b01852c.el9 ceph 82 k 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T06:00:03.498 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:03.499 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 82 k 2024-05-25T06:00:03.499 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 222 k 2024-05-25T06:00:03.499 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T06:00:03.561 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install librbd1 2024-05-25T06:00:03.703 INFO:teuthology.orchestra.run.smithi160.stdout:rbd-fuse-17.2.7-958.g7b01852c.el9.x86_64.rpm 399 kB/s | 82 kB 00:00 2024-05-25T06:00:03.704 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:03.704 INFO:teuthology.orchestra.run.smithi160.stdout:Total 397 kB/s | 82 kB 00:00 2024-05-25T06:00:03.705 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T06:00:03.711 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T06:00:03.711 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T06:00:03.774 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T06:00:03.774 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T06:00:03.921 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T06:00:04.017 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:04.103 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:35 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:04.194 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:04.235 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:04.236 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:04.236 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:04.321 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install python3-rados 2024-05-25T06:00:04.397 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:04.650 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:04.651 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:04.651 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T06:00:04.651 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:04.651 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:04.651 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:04.858 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:35 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:04.878 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-mirror 2024-05-25T06:00:04.939 INFO:teuthology.orchestra.run.smithi203.stdout:Package python3-rados-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:04.981 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:04.982 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:04.983 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:05.060 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install python3-rgw 2024-05-25T06:00:05.428 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:42 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:05.554 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:05.555 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:05.555 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-05-25T06:00:05.555 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:05.555 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T06:00:05.555 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:17.2.7-958.g7b01852c.el9 ceph 2.9 M 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 2.9 M 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 12 M 2024-05-25T06:00:05.556 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T06:00:05.593 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:36 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:05.674 INFO:teuthology.orchestra.run.smithi203.stdout:Package python3-rgw-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:05.717 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:05.718 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:05.718 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:05.802 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install python3-cephfs 2024-05-25T06:00:06.335 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:37 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:06.416 INFO:teuthology.orchestra.run.smithi203.stdout:Package python3-cephfs-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:06.457 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:06.458 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:06.458 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:06.546 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install python3-rbd 2024-05-25T06:00:06.765 INFO:teuthology.orchestra.run.smithi160.stdout:rbd-mirror-17.2.7-958.g7b01852c.el9.x86_64.rpm 2.4 MB/s | 2.9 MB 00:01 2024-05-25T06:00:06.765 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:06.766 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.4 MB/s | 2.9 MB 00:01 2024-05-25T06:00:06.766 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T06:00:06.777 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T06:00:06.777 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T06:00:06.858 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T06:00:06.858 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T06:00:07.082 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:38 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:07.163 INFO:teuthology.orchestra.run.smithi203.stdout:Package python3-rbd-2:17.2.7-958.g7b01852c.el9.x86_64 is already installed. 2024-05-25T06:00:07.205 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:07.206 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-05-25T06:00:07.206 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:07.277 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T06:00:07.283 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install rbd-fuse 2024-05-25T06:00:07.357 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:07.383 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:07.383 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T06:00:07.383 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-25T06:00:07.383 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-25T06:00:07.383 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-25T06:00:07.383 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:07.819 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:38 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:07.940 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:17.2.7-958.g7b01852c.el9 ceph 82 k 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:07.941 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T06:00:07.942 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:07.942 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T06:00:07.942 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:07.942 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 82 k 2024-05-25T06:00:07.942 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 222 k 2024-05-25T06:00:07.942 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T06:00:08.162 INFO:teuthology.orchestra.run.smithi203.stdout:rbd-fuse-17.2.7-958.g7b01852c.el9.x86_64.rpm 372 kB/s | 82 kB 00:00 2024-05-25T06:00:08.163 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:08.163 INFO:teuthology.orchestra.run.smithi203.stdout:Total 370 kB/s | 82 kB 00:00 2024-05-25T06:00:08.163 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T06:00:08.170 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T06:00:08.170 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T06:00:08.232 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T06:00:08.232 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T06:00:08.375 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T06:00:08.403 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:08.403 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:08.403 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T06:00:08.403 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:08.403 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:08.403 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:08.476 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:08.615 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-nbd 2024-05-25T06:00:08.870 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:09.128 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:09.129 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:09.129 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T06:00:09.129 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:09.129 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:09.129 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:09.158 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:46 ago on Sat 25 May 2024 05:58:23 AM UTC. 2024-05-25T06:00:09.283 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:17.2.7-958.g7b01852c.el9 ceph 165 k 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-25T06:00:09.284 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-25T06:00:09.285 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-05-25T06:00:09.285 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:09.285 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 165 k 2024-05-25T06:00:09.285 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 477 k 2024-05-25T06:00:09.285 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-25T06:00:09.365 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install rbd-mirror 2024-05-25T06:00:09.643 INFO:teuthology.orchestra.run.smithi160.stdout:rbd-nbd-17.2.7-958.g7b01852c.el9.x86_64.rpm 462 kB/s | 165 kB 00:00 2024-05-25T06:00:09.644 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:09.644 INFO:teuthology.orchestra.run.smithi160.stdout:Total 460 kB/s | 165 kB 00:00 2024-05-25T06:00:09.644 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-25T06:00:09.651 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-25T06:00:09.652 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-25T06:00:09.715 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-25T06:00:09.715 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-25T06:00:09.861 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-25T06:00:09.904 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:40 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:09.956 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:10.029 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repo Size 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:17.2.7-958.g7b01852c.el9 ceph 2.9 M 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:10.030 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T06:00:10.031 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:10.031 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 2.9 M 2024-05-25T06:00:10.031 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 12 M 2024-05-25T06:00:10.031 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T06:00:10.374 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:10.625 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:10.625 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:10.625 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-25T06:00:10.625 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:10.625 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:00:10.626 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-25T06:00:10.642 INFO:teuthology.orchestra.run.smithi203.stdout:rbd-mirror-17.2.7-958.g7b01852c.el9.x86_64.rpm 4.8 MB/s | 2.9 MB 00:00 2024-05-25T06:00:10.643 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:10.644 INFO:teuthology.orchestra.run.smithi203.stdout:Total 4.8 MB/s | 2.9 MB 00:00 2024-05-25T06:00:10.644 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T06:00:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T06:00:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T06:00:10.739 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T06:00:10.739 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T06:00:10.841 DEBUG:teuthology.parallel:result is None 2024-05-25T06:00:11.160 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T06:00:11.226 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:11.252 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:11.252 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-25T06:00:11.252 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-25T06:00:11.252 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-25T06:00:11.252 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-25T06:00:11.252 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:12.250 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:12.250 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:12.250 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T06:00:12.250 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:12.250 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:12.250 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:12.509 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install rbd-nbd 2024-05-25T06:00:13.038 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:44 ago on Sat 25 May 2024 05:58:29 AM UTC. 2024-05-25T06:00:13.162 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:17.2.7-958.g7b01852c.el9 ceph 165 k 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:Install 1 Package 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:13.163 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 165 k 2024-05-25T06:00:13.164 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 477 k 2024-05-25T06:00:13.164 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-25T06:00:13.389 INFO:teuthology.orchestra.run.smithi203.stdout:rbd-nbd-17.2.7-958.g7b01852c.el9.x86_64.rpm 733 kB/s | 165 kB 00:00 2024-05-25T06:00:13.390 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:00:13.390 INFO:teuthology.orchestra.run.smithi203.stdout:Total 727 kB/s | 165 kB 00:00 2024-05-25T06:00:13.390 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-25T06:00:13.397 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-25T06:00:13.398 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-25T06:00:13.459 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-25T06:00:13.460 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-25T06:00:13.609 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-25T06:00:13.719 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:14.117 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:14.371 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 1/1 2024-05-25T06:00:14.371 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:14.372 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-25T06:00:14.372 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:17.2.7-958.g7b01852c.el9.x86_64 2024-05-25T06:00:14.372 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:00:14.372 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-25T06:00:14.593 DEBUG:teuthology.parallel:result is None 2024-05-25T06:00:14.594 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T06:00:14.594 INFO:teuthology.packaging:ref: None 2024-05-25T06:00:14.594 INFO:teuthology.packaging:tag: None 2024-05-25T06:00:14.594 INFO:teuthology.packaging:branch: quincy 2024-05-25T06:00:14.594 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:00:14.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-25T06:00:14.909 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-25T06:00:14.935 INFO:teuthology.orchestra.run.smithi160.stdout:17.2.7-958.g7b01852c.el9 2024-05-25T06:00:14.936 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-958.g7b01852c.el9 2024-05-25T06:00:14.936 INFO:teuthology.task.install:The correct ceph version 17.2.7-958.g7b01852c is installed. 2024-05-25T06:00:14.938 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-25T06:00:14.938 INFO:teuthology.packaging:ref: None 2024-05-25T06:00:14.938 INFO:teuthology.packaging:tag: None 2024-05-25T06:00:14.938 INFO:teuthology.packaging:branch: quincy 2024-05-25T06:00:14.938 INFO:teuthology.packaging:sha1: ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:00:14.939 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-25T06:00:15.249 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-25T06:00:15.272 INFO:teuthology.orchestra.run.smithi203.stdout:17.2.7-958.g7b01852c.el9 2024-05-25T06:00:15.272 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-958.g7b01852c.el9 2024-05-25T06:00:15.272 INFO:teuthology.task.install:The correct ceph version 17.2.7-958.g7b01852c is installed. 2024-05-25T06:00:15.274 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-25T06:00:15.274 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:00:15.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-25T06:00:15.306 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:00:15.306 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-25T06:00:15.339 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-25T06:00:15.339 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:00:15.339 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-05-25T06:00:15.378 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-25T06:00:15.446 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:00:15.446 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-05-25T06:00:15.471 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-25T06:00:15.536 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-25T06:00:15.536 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:00:15.537 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-25T06:00:15.563 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-25T06:00:15.632 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:00:15.632 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-25T06:00:15.656 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-25T06:00:15.722 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-25T06:00:15.722 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:00:15.722 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-05-25T06:00:15.750 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-25T06:00:15.817 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:00:15.817 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-05-25T06:00:15.841 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-25T06:00:15.906 INFO:teuthology.run_tasks:Running task print... 2024-05-25T06:00:15.915 INFO:teuthology.task.print:**** done install task... 2024-05-25T06:00:15.915 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-25T06:00:16.002 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'ea0684c08cbcf83f4af9930de2267b3563ac5f78'} 2024-05-25T06:00:16.002 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-05-25T06:00:16.003 INFO:tasks.cephadm:Cluster fsid is 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:00:16.003 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-25T06:00:16.003 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-25T06:00:16.003 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi160': '172.21.15.160', 'mon.smithi203': '172.21.15.203'} 2024-05-25T06:00:16.003 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-25T06:00:16.003 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-05-25T06:00:16.031 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-05-25T06:00:16.056 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-05-25T06:00:16.057 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-25T06:00:16.338 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 May 25 06:00 /home/ubuntu/cephtest/cephadm 2024-05-25T06:00:16.338 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-25T06:00:16.633 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 May 25 06:00 /home/ubuntu/cephtest/cephadm 2024-05-25T06:00:16.634 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-25T06:00:16.653 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-25T06:00:16.678 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-05-25T06:00:16.678 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-25T06:00:16.697 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-25T06:00:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-25T06:00:17.633 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-25T06:01:14.600 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-05-25T06:01:14.601 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)", 2024-05-25T06:01:14.601 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "baf1547092e7c72c2f7d4e5f13d7eaa7f754aad671d1b31b838412ad770f1454", 2024-05-25T06:01:14.601 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-05-25T06:01:14.601 INFO:teuthology.orchestra.run.smithi203.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45" 2024-05-25T06:01:14.601 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-05-25T06:01:14.601 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-05-25T06:01:19.475 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:01:19.475 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)", 2024-05-25T06:01:19.476 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "baf1547092e7c72c2f7d4e5f13d7eaa7f754aad671d1b31b838412ad770f1454", 2024-05-25T06:01:19.476 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-05-25T06:01:19.476 INFO:teuthology.orchestra.run.smithi160.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45" 2024-05-25T06:01:19.476 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-05-25T06:01:19.476 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:01:19.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-05-25T06:01:19.548 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-05-25T06:01:19.582 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-05-25T06:01:19.606 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-05-25T06:01:19.651 INFO:tasks.cephadm:Writing seed config... 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-25T06:01:19.653 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-25T06:01:19.654 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-25T06:01:19.655 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-25T06:01:19.656 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:01:19.656 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-25T06:01:19.672 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0ed6924e-1a5c-11ef-bc9b-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-25T06:01:19.672 DEBUG:teuthology.orchestra.run.smithi160:mon.smithi160> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service 2024-05-25T06:01:19.714 INFO:tasks.cephadm:Bootstrapping... 2024-05-25T06:01:19.715 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.160 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:01:19.918 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-25T06:01:19.918 INFO:teuthology.orchestra.run.smithi160.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '0ed6924e-1a5c-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.160', '--skip-admin-label'] 2024-05-25T06:01:19.943 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 5.0.2 2024-05-25T06:01:19.944 INFO:teuthology.orchestra.run.smithi160.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-25T06:01:19.944 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying podman|docker is present... 2024-05-25T06:01:19.967 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 5.0.2 2024-05-25T06:01:19.968 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying lvm2 is present... 2024-05-25T06:01:19.968 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying time synchronization is in place... 2024-05-25T06:01:19.976 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-25T06:01:19.976 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-25T06:01:19.982 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-25T06:01:19.983 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-05-25T06:01:19.990 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout enabled 2024-05-25T06:01:19.996 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout active 2024-05-25T06:01:19.997 INFO:teuthology.orchestra.run.smithi160.stdout:Unit chronyd.service is enabled and running 2024-05-25T06:01:19.997 INFO:teuthology.orchestra.run.smithi160.stdout:Repeating the final host check... 2024-05-25T06:01:20.020 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 5.0.2 2024-05-25T06:01:20.020 INFO:teuthology.orchestra.run.smithi160.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-25T06:01:20.020 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl is present 2024-05-25T06:01:20.020 INFO:teuthology.orchestra.run.smithi160.stdout:lvcreate is present 2024-05-25T06:01:20.027 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-25T06:01:20.027 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-25T06:01:20.034 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-25T06:01:20.034 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-05-25T06:01:20.041 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout enabled 2024-05-25T06:01:20.048 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout active 2024-05-25T06:01:20.048 INFO:teuthology.orchestra.run.smithi160.stdout:Unit chronyd.service is enabled and running 2024-05-25T06:01:20.048 INFO:teuthology.orchestra.run.smithi160.stdout:Host looks OK 2024-05-25T06:01:20.048 INFO:teuthology.orchestra.run.smithi160.stdout:Cluster fsid: 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:20.049 INFO:teuthology.orchestra.run.smithi160.stdout:Acquiring lock 140266463871424 on /run/cephadm/0ed6924e-1a5c-11ef-bc9b-c7b262605968.lock 2024-05-25T06:01:20.049 INFO:teuthology.orchestra.run.smithi160.stdout:Lock 140266463871424 acquired on /run/cephadm/0ed6924e-1a5c-11ef-bc9b-c7b262605968.lock 2024-05-25T06:01:20.049 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying IP 172.21.15.160 port 3300 ... 2024-05-25T06:01:20.049 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying IP 172.21.15.160 port 6789 ... 2024-05-25T06:01:20.049 INFO:teuthology.orchestra.run.smithi160.stdout:Base mon IP(s) is [172.21.15.160:3300, 172.21.15.160:6789], mon addrv is [v2:172.21.15.160:3300,v1:172.21.15.160:6789] 2024-05-25T06:01:20.053 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.160 metric 100 2024-05-25T06:01:20.053 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.160 metric 100 2024-05-25T06:01:20.056 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-25T06:01:20.056 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-25T06:01:20.056 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-05-25T06:01:20.059 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-25T06:01:20.059 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-25T06:01:20.059 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-25T06:01:20.059 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-25T06:01:20.059 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2024-05-25T06:01:20.059 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-25T06:01:20.060 INFO:teuthology.orchestra.run.smithi160.stdout:Mon IP `172.21.15.160` is in CIDR network `172.21.0.0/20` 2024-05-25T06:01:20.060 INFO:teuthology.orchestra.run.smithi160.stdout:Mon IP `172.21.15.160` is in CIDR network `172.21.0.0/20` 2024-05-25T06:01:20.060 INFO:teuthology.orchestra.run.smithi160.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-25T06:01:20.060 INFO:teuthology.orchestra.run.smithi160.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-25T06:01:20.061 INFO:teuthology.orchestra.run.smithi160.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-25T06:01:20.509 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout baf1547092e7c72c2f7d4e5f13d7eaa7f754aad671d1b31b838412ad770f1454 2024-05-25T06:01:20.509 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-25T06:01:20.510 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Getting image source signatures 2024-05-25T06:01:20.510 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying blob sha256:c2442f8e6c0925c1d5d63292489878b3e42aa4216c701b9bac32ab7c70ee8f61 2024-05-25T06:01:20.510 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-25T06:01:20.510 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying config sha256:baf1547092e7c72c2f7d4e5f13d7eaa7f754aad671d1b31b838412ad770f1454 2024-05-25T06:01:20.510 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-25T06:01:21.396 INFO:teuthology.orchestra.run.smithi160.stdout:ceph: stdout ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable) 2024-05-25T06:01:21.396 INFO:teuthology.orchestra.run.smithi160.stdout:Ceph version: ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable) 2024-05-25T06:01:21.397 INFO:teuthology.orchestra.run.smithi160.stdout:Extracting ceph user uid/gid from container image... 2024-05-25T06:01:22.294 INFO:teuthology.orchestra.run.smithi160.stdout:stat: stdout 167 167 2024-05-25T06:01:22.294 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial keys... 2024-05-25T06:01:23.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQCyflFmLu5uJBAA2tEF3am23MEVLUaJ7jFKpw== 2024-05-25T06:01:24.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQCzflFm4B63HxAAyMoNWWniDImkNTxeabf2OA== 2024-05-25T06:01:25.069 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQC0flFmEt0VHRAASmX9005q3AdW9x9g+ghOVw== 2024-05-25T06:01:25.070 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial monmap... 2024-05-25T06:01:25.942 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-25T06:01:25.942 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool for smithi160 [v2:172.21.15.160:3300,v1:172.21.15.160:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:setting min_mon_release = octopus 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: set fsid to 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:01:25.943 INFO:teuthology.orchestra.run.smithi160.stdout:Creating mon... 2024-05-25T06:01:26.882 INFO:teuthology.orchestra.run.smithi160.stdout:create mon.smithi160 on 2024-05-25T06:01:27.079 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-25T06:01:27.236 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-25T06:01:27.383 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968.target. 2024-05-25T06:01:27.383 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968.target. 2024-05-25T06:01:27.580 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160 2024-05-25T06:01:27.580 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service: Unit ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service not loaded. 2024-05-25T06:01:27.759 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968.target.wants/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service → /etc/systemd/system/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@.service. 2024-05-25T06:01:28.337 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-25T06:01:28.338 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-05-25T06:01:28.345 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-25T06:01:28.345 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-05-25T06:01:28.345 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-05-25T06:01:28.346 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to enable service . firewalld.service is not available 2024-05-25T06:01:28.346 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mon to start... 2024-05-25T06:01:28.346 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mon... 2024-05-25T06:01:28.376 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:28 smithi160 systemd[1]: Started Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout cluster: 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout id: 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout services: 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi160 (age 0.559919s) 2024-05-25T06:01:29.323 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout data: 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout pgs: 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfab085700 1 Processor -- start 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfab085700 1 -- start start 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfab085700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfab085700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4105980 con 0x7fbfa4076c10 2024-05-25T06:01:29.324 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfa8e21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfa8e21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51152/0 (socket says 172.21.15.160:51152) 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.747+0000 7fbfa8e21700 1 -- 172.21.15.160:0/891204992 learned_addr learned my addr 172.21.15.160:0/891204992 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.748+0000 7fbfa8e21700 1 -- 172.21.15.160:0/891204992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4105ac0 con 0x7fbfa4076c10 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.748+0000 7fbfa8e21700 1 --2- 172.21.15.160:0/891204992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4105440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbf980098e0 tx=0x7fbf98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290bb0e558e6fe98 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.748+0000 7fbfa37fe700 1 -- 172.21.15.160:0/891204992 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf9800b860 con 0x7fbfa4076c10 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.748+0000 7fbfa37fe700 1 -- 172.21.15.160:0/891204992 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9800b9c0 con 0x7fbfa4076c10 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.748+0000 7fbfa37fe700 1 -- 172.21.15.160:0/891204992 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf9800bc90 con 0x7fbfa4076c10 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 -- 172.21.15.160:0/891204992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 msgr2=0x7fbfa4105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 --2- 172.21.15.160:0/891204992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4105440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbf980098e0 tx=0x7fbf98009250 comp rx=0 tx=0).stop 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 -- 172.21.15.160:0/891204992 shutdown_connections 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 --2- 172.21.15.160:0/891204992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4105440 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 -- 172.21.15.160:0/891204992 >> 172.21.15.160:0/891204992 conn(0x7fbfa40fcef0 msgr2=0x7fbfa40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 -- 172.21.15.160:0/891204992 shutdown_connections 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 -- 172.21.15.160:0/891204992 wait complete. 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 Processor -- start 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.749+0000 7fbfab085700 1 -- start start 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfab085700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4110e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfab085700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4111390 con 0x7fbfa4076c10 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa8e21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa8e21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4110e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51164/0 (socket says 172.21.15.160:51164) 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa8e21700 1 -- 172.21.15.160:0/3781437888 learned_addr learned my addr 172.21.15.160:0/3781437888 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa8e21700 1 -- 172.21.15.160:0/3781437888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980094f0 con 0x7fbfa4076c10 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa8e21700 1 --2- 172.21.15.160:0/3781437888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4110e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009290 tx=0x7fbf98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf9801c5c0 con 0x7fbfa4076c10 2024-05-25T06:01:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9801c740 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa41115f0 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf9801ca50 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.750+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa4111fc0 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.751+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fbf9801cbb0 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.751+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbf9801abf0 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.751+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa404bed0 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.753+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fbf9801ae50 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.822+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fbfa4112b50 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.822+0000 7fbfa17fa700 1 -- 172.21.15.160:0/3781437888 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fbf98025440 con 0x7fbfa4076c10 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 msgr2=0x7fbfa4110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 --2- 172.21.15.160:0/3781437888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4110e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009290 tx=0x7fbf98016000 comp rx=0 tx=0).stop 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 shutdown_connections 2024-05-25T06:01:29.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 --2- 172.21.15.160:0/3781437888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfa4076c10 0x7fbfa4110e50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:29.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 >> 172.21.15.160:0/3781437888 conn(0x7fbfa40fcef0 msgr2=0x7fbfa40fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:29.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 shutdown_connections 2024-05-25T06:01:29.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:28.823+0000 7fbfab085700 1 -- 172.21.15.160:0/3781437888 wait complete. 2024-05-25T06:01:29.328 INFO:teuthology.orchestra.run.smithi160.stdout:mon is available 2024-05-25T06:01:29.328 INFO:teuthology.orchestra.run.smithi160.stdout:Assimilating anything we can from ceph.conf... 2024-05-25T06:01:29.439 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:29 smithi160 ceph-mon[28671]: mon.smithi160 is new leader, mons smithi160 in quorum (ranks 0) 2024-05-25T06:01:29.440 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:29 smithi160 ceph-mon[28671]: monmap e1: 1 mons at {smithi160=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-25T06:01:29.440 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:29 smithi160 ceph-mon[28671]: fsmap 2024-05-25T06:01:29.440 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:29 smithi160 ceph-mon[28671]: osdmap e1: 0 total, 0 up, 0 in 2024-05-25T06:01:29.440 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:29 smithi160 ceph-mon[28671]: mgrmap e1: no daemons active 2024-05-25T06:01:29.440 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:29 smithi160 ceph-mon[28671]: from='client.? 172.21.15.160:0/3781437888' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-25T06:01:30.445 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:30 smithi160 ceph-mon[28671]: from='client.? 172.21.15.160:0/1616011722' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-25T06:01:30.445 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:30 smithi160 ceph-mon[28671]: from='client.? 172.21.15.160:0/1616011722' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [global] 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout fsid = 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.160:3300,v1:172.21.15.160:6789] 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-25T06:01:30.453 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [osd] 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58daea6700 1 Processor -- start 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58daea6700 1 -- start start 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58daea6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d4101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58daea6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4102080 con 0x7f58d4101730 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58d8c42700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d4101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:30.454 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58d8c42700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d4101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51172/0 (socket says 172.21.15.160:51172) 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.754+0000 7f58d8c42700 1 -- 172.21.15.160:0/59213648 learned_addr learned my addr 172.21.15.160:0/59213648 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58d8c42700 1 -- 172.21.15.160:0/59213648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d41021c0 con 0x7f58d4101730 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58d8c42700 1 --2- 172.21.15.160:0/59213648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d4101b40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f58c80098e0 tx=0x7f58c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903bc65b47ca8871 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58d37fe700 1 -- 172.21.15.160:0/59213648 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58c800b860 con 0x7f58d4101730 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58d37fe700 1 -- 172.21.15.160:0/59213648 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c800b9c0 con 0x7f58d4101730 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58d37fe700 1 -- 172.21.15.160:0/59213648 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58c800bc90 con 0x7f58d4101730 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58daea6700 1 -- 172.21.15.160:0/59213648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 msgr2=0x7f58d4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58daea6700 1 --2- 172.21.15.160:0/59213648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d4101b40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f58c80098e0 tx=0x7f58c8009250 comp rx=0 tx=0).stop 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58daea6700 1 -- 172.21.15.160:0/59213648 shutdown_connections 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.755+0000 7f58daea6700 1 --2- 172.21.15.160:0/59213648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d4101b40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 -- 172.21.15.160:0/59213648 >> 172.21.15.160:0/59213648 conn(0x7f58d40fcef0 msgr2=0x7f58d40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 -- 172.21.15.160:0/59213648 shutdown_connections 2024-05-25T06:01:30.455 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 -- 172.21.15.160:0/59213648 wait complete. 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 Processor -- start 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 -- start start 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d410fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.756+0000 7f58daea6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41101e0 con 0x7f58d4101730 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d8c42700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d410fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d8c42700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d410fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51176/0 (socket says 172.21.15.160:51176) 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d8c42700 1 -- 172.21.15.160:0/1616011722 learned_addr learned my addr 172.21.15.160:0/1616011722 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d8c42700 1 -- 172.21.15.160:0/1616011722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80094f0 con 0x7f58d4101730 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d8c42700 1 --2- 172.21.15.160:0/1616011722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d410fca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f58c8003080 tx=0x7f58c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58c801c5c0 con 0x7f58d4101730 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c801c720 con 0x7f58d4101730 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58c801ca30 con 0x7f58d4101730 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d41103e0 con 0x7f58d4101730 2024-05-25T06:01:30.456 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d4110f50 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f58c801ccb0 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.757+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f58c801ac90 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.758+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d405f5c0 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.760+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f58c801a450 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.826+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f58d4102be0 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.830+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f58c802e8d0 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.830+0000 7f58d17fa700 1 -- 172.21.15.160:0/1616011722 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f58d4102be0 con 0x7f58d4101730 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 msgr2=0x7f58d410fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 --2- 172.21.15.160:0/1616011722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d410fca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f58c8003080 tx=0x7f58c8016000 comp rx=0 tx=0).stop 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 shutdown_connections 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 --2- 172.21.15.160:0/1616011722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58d4101730 0x7f58d410fca0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 >> 172.21.15.160:0/1616011722 conn(0x7f58d40fcef0 msgr2=0x7f58d4104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:30.457 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 shutdown_connections 2024-05-25T06:01:30.458 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:29.831+0000 7f58daea6700 1 -- 172.21.15.160:0/1616011722 wait complete. 2024-05-25T06:01:30.458 INFO:teuthology.orchestra.run.smithi160.stdout:Generating new minimal ceph.conf... 2024-05-25T06:01:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:31 smithi160 ceph-mon[28671]: from='client.? 172.21.15.160:0/1239479802' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:01:31.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.845+0000 7fcb153e4700 1 Processor -- start 2024-05-25T06:01:31.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.845+0000 7fcb153e4700 1 -- start start 2024-05-25T06:01:31.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.845+0000 7fcb153e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.845+0000 7fcb153e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10070d90 con 0x7fcb10103ed0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51190/0 (socket says 172.21.15.160:51190) 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0effd700 1 -- 172.21.15.160:0/1186402075 learned_addr learned my addr 172.21.15.160:0/1186402075 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0effd700 1 -- 172.21.15.160:0/1186402075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10104820 con 0x7fcb10103ed0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0effd700 1 --2- 172.21.15.160:0/1186402075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101042e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb080098e0 tx=0x7fcb08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa11596809fa157 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0dffb700 1 -- 172.21.15.160:0/1186402075 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb0800b860 con 0x7fcb10103ed0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0dffb700 1 -- 172.21.15.160:0/1186402075 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb0800be50 con 0x7fcb10103ed0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.846+0000 7fcb0dffb700 1 -- 172.21.15.160:0/1186402075 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb08010380 con 0x7fcb10103ed0 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 -- 172.21.15.160:0/1186402075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 msgr2=0x7fcb101042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 --2- 172.21.15.160:0/1186402075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101042e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb080098e0 tx=0x7fcb08009250 comp rx=0 tx=0).stop 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 -- 172.21.15.160:0/1186402075 shutdown_connections 2024-05-25T06:01:31.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 --2- 172.21.15.160:0/1186402075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101042e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 -- 172.21.15.160:0/1186402075 >> 172.21.15.160:0/1186402075 conn(0x7fcb100fc6f0 msgr2=0x7fcb100feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 -- 172.21.15.160:0/1186402075 shutdown_connections 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.847+0000 7fcb153e4700 1 -- 172.21.15.160:0/1186402075 wait complete. 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb153e4700 1 Processor -- start 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb153e4700 1 -- start start 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb153e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb153e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10111b00 con 0x7fcb10103ed0 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb0effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb0effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51194/0 (socket says 172.21.15.160:51194) 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb0effd700 1 -- 172.21.15.160:0/1239479802 learned_addr learned my addr 172.21.15.160:0/1239479802 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb0effd700 1 -- 172.21.15.160:0/1239479802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080094f0 con 0x7fcb10103ed0 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb0effd700 1 --2- 172.21.15.160:0/1239479802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101115c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcb08000c00 tx=0x7fcb08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb080107c0 con 0x7fcb10103ed0 2024-05-25T06:01:31.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb08010db0 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb0801ad00 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.848+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb10111d00 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.849+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb10112870 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.849+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcb08024440 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.849+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcb08019710 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.850+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1005f5c0 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.852+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fcb08042790 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.917+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fcb10101340 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.918+0000 7fcb077fe700 1 -- 172.21.15.160:0/1239479802 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fcb08031020 con 0x7fcb10103ed0 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.918+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 msgr2=0x7fcb101115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.918+0000 7fcb153e4700 1 --2- 172.21.15.160:0/1239479802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101115c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcb08000c00 tx=0x7fcb08009250 comp rx=0 tx=0).stop 2024-05-25T06:01:31.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.918+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 shutdown_connections 2024-05-25T06:01:31.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.918+0000 7fcb153e4700 1 --2- 172.21.15.160:0/1239479802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb10103ed0 0x7fcb101115c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:31.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.918+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 >> 172.21.15.160:0/1239479802 conn(0x7fcb100fc6f0 msgr2=0x7fcb100fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:31.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.919+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 shutdown_connections 2024-05-25T06:01:31.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:30.919+0000 7fcb153e4700 1 -- 172.21.15.160:0/1239479802 wait complete. 2024-05-25T06:01:31.487 INFO:teuthology.orchestra.run.smithi160.stdout:Restarting the monitor... 2024-05-25T06:01:31.738 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:31 smithi160 systemd[1]: Stopping Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:01:32.134 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:31 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[28667]: 2024-05-25T06:01:31.737+0000 7f3990706700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi160 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:01:32.134 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:31 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[28667]: 2024-05-25T06:01:31.737+0000 7f3990706700 -1 mon.smithi160@0(leader) e1 *** Got Signal Terminated *** 2024-05-25T06:01:32.134 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:31 smithi160 podman[28918]: 2024-05-25 06:01:31.983538111 +0000 UTC m=+0.340791207 container died 96242d2ddd621642ef4df2ca0ca79a30b31626c439feecf8d5f1d133acad24fd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-7b01852, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-25T06:01:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 podman[28918]: 2024-05-25 06:01:32.254605304 +0000 UTC m=+0.611858390 container cleanup 96242d2ddd621642ef4df2ca0ca79a30b31626c439feecf8d5f1d133acad24fd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=quincy-7b01852) 2024-05-25T06:01:32.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 bash[28918]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160 2024-05-25T06:01:32.839 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 podman[28932]: 2024-05-25 06:01:32.491967655 +0000 UTC m=+0.508083270 container remove 96242d2ddd621642ef4df2ca0ca79a30b31626c439feecf8d5f1d133acad24fd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-7b01852, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-25T06:01:32.839 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service: Deactivated successfully. 2024-05-25T06:01:32.839 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 systemd[1]: Stopped Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:01:32.839 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 systemd[1]: Starting Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 podman[29022]: 2024-05-25 06:01:32.839608021 +0000 UTC m=+0.111867180 container create 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=quincy-7b01852, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:32 smithi160 podman[29022]: 2024-05-25 06:01:32.745000548 +0000 UTC m=+0.017259720 image pull baf1547092e7c72c2f7d4e5f13d7eaa7f754aad671d1b31b838412ad770f1454 quay.ceph.io/ceph-ci/ceph:quincy 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 podman[29022]: 2024-05-25 06:01:33.025050038 +0000 UTC m=+0.297309214 container init 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=quincy-7b01852, org.label-schema.vendor=CentOS) 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 podman[29022]: 2024-05-25 06:01:33.028070167 +0000 UTC m=+0.300329340 container start 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, ceph=True, RELEASE=quincy-7b01852, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: set uid:gid to 167:167 (ceph:ceph) 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable), process ceph-mon, pid 2 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: pidfile_write: ignore empty --pid-file 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: load: jerasure load: lrc 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: RocksDB version: 6.15.5 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-05-25T06:01:33.100 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Compile date May 23 2024 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: DB SUMMARY 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: DB Session ID: ZXSVQ6N188OWFT7R7KPW 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: CURRENT file: CURRENT 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: IDENTITY file: IDENTITY 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi160/store.db dir, Total Num: 1, files: 000008.sst 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi160/store.db: 000010.log size: 98192 ; 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.error_if_exists: 0 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.create_if_missing: 0 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.paranoid_checks: 1 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.env: 0x560a44788b00 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.fs: Posix File System 2024-05-25T06:01:33.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.info_log: 0x560a45ac5620 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_file_opening_threads: 16 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.statistics: (nil) 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.use_fsync: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_log_file_size: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.keep_log_file_num: 1000 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.recycle_log_file_num: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.allow_fallocate: 1 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.allow_mmap_reads: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.allow_mmap_writes: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.use_direct_reads: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-25T06:01:33.102 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.create_missing_column_families: 0 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.db_log_dir: 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi160/store.db 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.advise_random_on_open: 1 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.db_write_buffer_size: 0 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.write_buffer_manager: 0x560a467b89c0 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-05-25T06:01:33.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-25T06:01:33.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-25T06:01:33.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.rate_limiter: (nil) 2024-05-25T06:01:33.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-25T06:01:33.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.wal_recovery_mode: 2 2024-05-25T06:01:33.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.enable_thread_tracking: 0 2024-05-25T06:01:33.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.enable_pipelined_write: 0 2024-05-25T06:01:33.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.unordered_write: 0 2024-05-25T06:01:33.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-25T06:01:33.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-25T06:01:33.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-25T06:01:33.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-25T06:01:33.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.row_cache: None 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.wal_filter: None 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.allow_ingest_behind: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.preserve_deletes: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.two_write_queues: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.manual_wal_flush: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.atomic_flush: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.log_readahead_size: 0 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-25T06:01:33.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.best_efforts_recovery: 0 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.allow_data_in_errors: 0 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.db_host_id: __hostname__ 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_background_jobs: 2 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_background_compactions: -1 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_subcompactions: 1 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_total_wal_size: 0 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-25T06:01:33.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_open_files: -1 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bytes_per_sync: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_readahead_size: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_background_flushes: -1 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Compression algorithms supported: 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kZSTD supported: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kXpressCompression supported: 0 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kLZ4HCCompression supported: 1 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kZlibCompression supported: 1 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kSnappyCompression supported: 1 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kLZ4Compression supported: 1 2024-05-25T06:01:33.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: kBZip2Compression supported: 0 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi160/store.db/MANIFEST-000009 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.merge_operator: 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_filter: None 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_filter_factory: None 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.sst_partitioner_factory: None 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560a45a98f58) 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: cache_index_and_filter_blocks: 1 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-25T06:01:33.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: pin_top_level_index_and_filter: 1 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: index_type: 0 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: data_block_index_type: 0 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: index_shortening: 1 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: hash_index_allow_collision: 1 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: checksum: 1 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: no_block_cache: 0 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache: 0x560a45acb090 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache_name: BinnedLRUCache 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache_options: 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: capacity : 536870912 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: num_shard_bits : 4 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: strict_capacity_limit : 0 2024-05-25T06:01:33.110 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: high_pri_pool_ratio: 0.000 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache_compressed: (nil) 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: persistent_cache: (nil) 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_size: 4096 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_size_deviation: 10 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_restart_interval: 16 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: index_block_restart_interval: 1 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: metadata_block_size: 4096 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: partition_filters: 0 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: use_delta_encoding: 1 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: whole_key_filtering: 1 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: verify_compression: 0 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: read_amp_bytes_per_bit: 0 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: format_version: 4 2024-05-25T06:01:33.111 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: enable_index_compression: 1 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_align: 0 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.write_buffer_size: 33554432 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_write_buffer_number: 2 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression: NoCompression 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression: Disabled 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.prefix_extractor: nullptr 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.num_levels: 7 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-25T06:01:33.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.level: 32767 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.strategy: 0 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compression_opts.enabled: false 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-25T06:01:33.113 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.target_file_size_base: 67108864 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-25T06:01:33.114 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.arena_block_size: 4194304 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.disable_auto_compactions: 0 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-25T06:01:33.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.table_properties_collectors: 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.inplace_update_support: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.bloom_locality: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.max_successive_merges: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.paranoid_file_checks: 0 2024-05-25T06:01:33.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.force_consistency_checks: 1 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.report_bg_io_stats: 0 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.ttl: 2592000 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.enable_blob_files: false 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.min_blob_size: 0 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.blob_file_size: 268435456 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi160/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-25T06:01:33.117 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716616893055042, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716616893055920, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93398, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91510, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84074, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1716616893, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "55c71099-df43-4919-9c1a-658159af5fb0", "db_session_id": "ZXSVQ6N188OWFT7R7KPW"}} 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716616893056237, "job": 1, "event": "recovery_finished"} 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi160/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x560a45a5f180 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: DB pointer 0x560a45b34000 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-05-25T06:01:33.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** DB Stats ** 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:01:33.119 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: L0 2/0 93.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Sum 2/0 93.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-25T06:01:33.121 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative compaction: 0.00 GB write, 27.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval compaction: 0.00 GB write, 27.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-25T06:01:33.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: L0 2/0 93.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Sum 2/0 93.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.1 0.00 0.00 1 0.001 0 0 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-25T06:01:33.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative compaction: 0.00 GB write, 27.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: starting mon.smithi160 rank 0 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi160 fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???) e1 preinit fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:33.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).mds e1 new map 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).mds e1 print_map 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: e1 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: legacy client fscid: -1 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: No filesystems configured 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 bash[29022]: 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 2024-05-25T06:01:33.125 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 systemd[1]: Started Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:01:33.126 INFO:teuthology.orchestra.run.smithi160.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-25T06:01:33.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mon.smithi160 is new leader, mons smithi160 in quorum (ranks 0) 2024-05-25T06:01:33.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: monmap e1: 1 mons at {smithi160=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-25T06:01:33.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: fsmap 2024-05-25T06:01:33.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: osdmap e1: 0 total, 0 up, 0 in 2024-05-25T06:01:33.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:33 smithi160 ceph-mon[29041]: mgrmap e1: no daemons active 2024-05-25T06:01:34.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.577+0000 7f405e52d700 1 Processor -- start 2024-05-25T06:01:34.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.577+0000 7f405e52d700 1 -- start start 2024-05-25T06:01:34.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f405e52d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f4058107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:34.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f405e52d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4058103230 con 0x7f4058076c20 2024-05-25T06:01:34.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f4057fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f4058107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f4057fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f4058107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51198/0 (socket says 172.21.15.160:51198) 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f4057fff700 1 -- 172.21.15.160:0/3272901546 learned_addr learned my addr 172.21.15.160:0/3272901546 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f4057fff700 1 -- 172.21.15.160:0/3272901546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4058107d80 con 0x7f4058076c20 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.578+0000 7f4057fff700 1 --2- 172.21.15.160:0/3272901546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f4058107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40480098e0 tx=0x7f4048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8545c73d3e60bf server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f40577fe700 1 -- 172.21.15.160:0/3272901546 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f404800b860 con 0x7f4058076c20 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f40577fe700 1 -- 172.21.15.160:0/3272901546 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f404800be50 con 0x7f4058076c20 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f40577fe700 1 -- 172.21.15.160:0/3272901546 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4048010380 con 0x7f4058076c20 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 -- 172.21.15.160:0/3272901546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 msgr2=0x7f4058107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 --2- 172.21.15.160:0/3272901546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f4058107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40480098e0 tx=0x7f4048009250 comp rx=0 tx=0).stop 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 -- 172.21.15.160:0/3272901546 shutdown_connections 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 --2- 172.21.15.160:0/3272901546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f4058107840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 -- 172.21.15.160:0/3272901546 >> 172.21.15.160:0/3272901546 conn(0x7f40580fced0 msgr2=0x7f40580ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 -- 172.21.15.160:0/3272901546 shutdown_connections 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.579+0000 7f405e52d700 1 -- 172.21.15.160:0/3272901546 wait complete. 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f405e52d700 1 Processor -- start 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f405e52d700 1 -- start start 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f405e52d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f40581137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f405e52d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4058113d10 con 0x7f4058076c20 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f4057fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f40581137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f4057fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f40581137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51206/0 (socket says 172.21.15.160:51206) 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.580+0000 7f4057fff700 1 -- 172.21.15.160:0/4104134162 learned_addr learned my addr 172.21.15.160:0/4104134162 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f4057fff700 1 -- 172.21.15.160:0/4104134162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480094f0 con 0x7f4058076c20 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f4057fff700 1 --2- 172.21.15.160:0/4104134162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f40581137d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4048006b00 tx=0x7f4048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40480107c0 con 0x7f4058076c20 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4048010db0 con 0x7f4058076c20 2024-05-25T06:01:34.140 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4058113f10 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f404801acf0 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4058114a80 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f4048024440 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.581+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4048019710 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.582+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405805f5c0 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.584+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f4048042790 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.649+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f4058108a70 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.652+0000 7f40557fa700 1 -- 172.21.15.160:0/4104134162 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4048031020 con 0x7f4058076c20 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 msgr2=0x7f40581137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 --2- 172.21.15.160:0/4104134162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f40581137d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4048006b00 tx=0x7f4048009250 comp rx=0 tx=0).stop 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 shutdown_connections 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 --2- 172.21.15.160:0/4104134162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4058076c20 0x7f40581137d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:34.141 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 >> 172.21.15.160:0/4104134162 conn(0x7f40580fced0 msgr2=0x7f40580fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:34.142 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 shutdown_connections 2024-05-25T06:01:34.142 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:33.653+0000 7f405e52d700 1 -- 172.21.15.160:0/4104134162 wait complete. 2024-05-25T06:01:34.142 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-25T06:01:34.142 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:01:34.142 INFO:teuthology.orchestra.run.smithi160.stdout:Creating mgr... 2024-05-25T06:01:34.142 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying port 9283 ... 2024-05-25T06:01:34.305 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mgr.smithi160.nalzlg 2024-05-25T06:01:34.306 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mgr.smithi160.nalzlg.service: Unit ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mgr.smithi160.nalzlg.service not loaded. 2024-05-25T06:01:34.456 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968.target.wants/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mgr.smithi160.nalzlg.service → /etc/systemd/system/ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@.service. 2024-05-25T06:01:34.802 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:34 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/4104134162' entity='client.admin' 2024-05-25T06:01:34.957 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-25T06:01:34.958 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-05-25T06:01:34.965 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-25T06:01:34.965 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-05-25T06:01:34.966 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-05-25T06:01:34.966 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to enable service . firewalld.service is not available 2024-05-25T06:01:34.974 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-25T06:01:34.974 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-05-25T06:01:34.982 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-25T06:01:34.982 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-05-25T06:01:34.982 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-05-25T06:01:34.982 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-05-25T06:01:34.983 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr to start... 2024-05-25T06:01:34.983 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr... 2024-05-25T06:01:35.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:35 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3747526860' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-25T06:01:36.110 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:36.110 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:01:36.110 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "0ed6924e-1a5c-11ef-bc9b-c7b262605968", 2024-05-25T06:01:36.110 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-05-25T06:01:36.110 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "smithi160" 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-25T06:01:36.111 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-25T06:01:36.112 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.113 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-05-25T06:01:28.264738+0000", 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb5802de700 1 Processor -- start 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb5802de700 1 -- start start 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb5802de700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb578105450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb5802de700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578105990 con 0x7fb5780712d0 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb57e07a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb578105450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb57e07a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb578105450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51252/0 (socket says 172.21.15.160:51252) 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.403+0000 7fb57e07a700 1 -- 172.21.15.160:0/1071810161 learned_addr learned my addr 172.21.15.160:0/1071810161 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.404+0000 7fb57e07a700 1 -- 172.21.15.160:0/1071810161 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578105ad0 con 0x7fb5780712d0 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.404+0000 7fb57e07a700 1 --2- 172.21.15.160:0/1071810161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb578105450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb5740098e0 tx=0x7fb574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d1928a7a9242f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.404+0000 7fb57d078700 1 -- 172.21.15.160:0/1071810161 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb57400b860 con 0x7fb5780712d0 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.404+0000 7fb57d078700 1 -- 172.21.15.160:0/1071810161 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb57400be50 con 0x7fb5780712d0 2024-05-25T06:01:36.115 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.404+0000 7fb57d078700 1 -- 172.21.15.160:0/1071810161 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb574010380 con 0x7fb5780712d0 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 -- 172.21.15.160:0/1071810161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 msgr2=0x7fb578105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 --2- 172.21.15.160:0/1071810161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb578105450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb5740098e0 tx=0x7fb574009250 comp rx=0 tx=0).stop 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 -- 172.21.15.160:0/1071810161 shutdown_connections 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 --2- 172.21.15.160:0/1071810161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb578105450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 -- 172.21.15.160:0/1071810161 >> 172.21.15.160:0/1071810161 conn(0x7fb5780fcec0 msgr2=0x7fb5780ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 -- 172.21.15.160:0/1071810161 shutdown_connections 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 -- 172.21.15.160:0/1071810161 wait complete. 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 Processor -- start 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.405+0000 7fb5802de700 1 -- start start 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb5802de700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb5781115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb5802de700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578111b00 con 0x7fb5780712d0 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb57e07a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb5781115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb57e07a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb5781115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51260/0 (socket says 172.21.15.160:51260) 2024-05-25T06:01:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb57e07a700 1 -- 172.21.15.160:0/3747526860 learned_addr learned my addr 172.21.15.160:0/3747526860 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb57e07a700 1 -- 172.21.15.160:0/3747526860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740094f0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb57e07a700 1 --2- 172.21.15.160:0/3747526860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb5781115c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb574006b00 tx=0x7fb574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5740107c0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb574010db0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb57401acf0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb578111d00 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.406+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb578112870 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.407+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb574024440 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.407+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb574019710 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.407+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57805f5c0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.409+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fb57401a530 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.482+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb57804bed0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.483+0000 7fb56effd700 1 -- 172.21.15.160:0/3747526860 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fb57804bed0 con 0x7fb5780712d0 2024-05-25T06:01:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.483+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 msgr2=0x7fb5781115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.483+0000 7fb5802de700 1 --2- 172.21.15.160:0/3747526860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb5781115c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb574006b00 tx=0x7fb574009250 comp rx=0 tx=0).stop 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.483+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 shutdown_connections 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.483+0000 7fb5802de700 1 --2- 172.21.15.160:0/3747526860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5780712d0 0x7fb5781115c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.483+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 >> 172.21.15.160:0/3747526860 conn(0x7fb5780fcec0 msgr2=0x7fb5780fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.484+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 shutdown_connections 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:35.484+0000 7fb5802de700 1 -- 172.21.15.160:0/3747526860 wait complete. 2024-05-25T06:01:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (1/15)... 2024-05-25T06:01:38.956 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:38 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/472288697' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "0ed6924e-1a5c-11ef-bc9b-c7b262605968", 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.202 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "smithi160" 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-25T06:01:39.203 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:39.204 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-25T06:01:39.204 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-25T06:01:39.204 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-25T06:01:39.211 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-25T06:01:39.212 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-05-25T06:01:28.264738+0000", 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.491+0000 7fdd21fce700 1 Processor -- start 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.491+0000 7fdd21fce700 1 -- start start 2024-05-25T06:01:39.213 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.491+0000 7fdd21fce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.491+0000 7fdd21fce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1031c0 con 0x7fdd1c102870 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33806/0 (socket says 172.21.15.160:33806) 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1b7fe700 1 -- 172.21.15.160:0/3716559644 learned_addr learned my addr 172.21.15.160:0/3716559644 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1b7fe700 1 -- 172.21.15.160:0/3716559644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c103300 con 0x7fdd1c102870 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1b7fe700 1 --2- 172.21.15.160:0/3716559644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c102c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000fa00 tx=0x7fdd10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f97ad2aff7f0f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1a7fc700 1 -- 172.21.15.160:0/3716559644 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd10009cb0 con 0x7fdd1c102870 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1a7fc700 1 -- 172.21.15.160:0/3716559644 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdd10009e10 con 0x7fdd1c102870 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.492+0000 7fdd1a7fc700 1 -- 172.21.15.160:0/3716559644 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd10009b50 con 0x7fdd1c102870 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 -- 172.21.15.160:0/3716559644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 msgr2=0x7fdd1c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 --2- 172.21.15.160:0/3716559644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c102c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000fa00 tx=0x7fdd10009250 comp rx=0 tx=0).stop 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 -- 172.21.15.160:0/3716559644 shutdown_connections 2024-05-25T06:01:39.214 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 --2- 172.21.15.160:0/3716559644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c102c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 -- 172.21.15.160:0/3716559644 >> 172.21.15.160:0/3716559644 conn(0x7fdd1c0fdf90 msgr2=0x7fdd1c100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 -- 172.21.15.160:0/3716559644 shutdown_connections 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.493+0000 7fdd21fce700 1 -- 172.21.15.160:0/3716559644 wait complete. 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd21fce700 1 Processor -- start 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd21fce700 1 -- start start 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd21fce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c10f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd21fce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10f8b0 con 0x7fdd1c102870 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c10f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c10f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33814/0 (socket says 172.21.15.160:33814) 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.494+0000 7fdd1b7fe700 1 -- 172.21.15.160:0/472288697 learned_addr learned my addr 172.21.15.160:0/472288697 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd1b7fe700 1 -- 172.21.15.160:0/472288697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1000f610 con 0x7fdd1c102870 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd1b7fe700 1 --2- 172.21.15.160:0/472288697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c10f370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006fd0 tx=0x7fdd10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd100177c0 con 0x7fdd1c102870 2024-05-25T06:01:39.215 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdd10020440 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c10fab0 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c10fec0 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd1001f540 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fdd1001e070 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.495+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdd10029450 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.496+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd080052e0 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.500+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fdd1002e060 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.572+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdd08005860 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.573+0000 7fdd03fff700 1 -- 172.21.15.160:0/472288697 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fdd10024070 con 0x7fdd1c102870 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 msgr2=0x7fdd1c10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 --2- 172.21.15.160:0/472288697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c10f370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006fd0 tx=0x7fdd10008040 comp rx=0 tx=0).stop 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 shutdown_connections 2024-05-25T06:01:39.216 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 --2- 172.21.15.160:0/472288697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd1c102870 0x7fdd1c10f370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:39.217 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 >> 172.21.15.160:0/472288697 conn(0x7fdd1c0fdf90 msgr2=0x7fdd1c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:39.217 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 shutdown_connections 2024-05-25T06:01:39.217 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:38.574+0000 7fdd21fce700 1 -- 172.21.15.160:0/472288697 wait complete. 2024-05-25T06:01:39.217 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (2/15)... 2024-05-25T06:01:41.541 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:01:41.541 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: mgrmap e2: smithi160.nalzlg(active, starting, since 0.00275008s) 2024-05-25T06:01:41.541 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' 2024-05-25T06:01:41.542 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:41 smithi160 ceph-mon[29041]: from='mgr.14100 172.21.15.160:0/960787546' entity='mgr.smithi160.nalzlg' 2024-05-25T06:01:42.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "0ed6924e-1a5c-11ef-bc9b-c7b262605968", 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-25T06:01:42.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "smithi160" 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-25T06:01:42.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-25T06:01:42.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-25T06:01:42.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-25T06:01:42.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-25T06:01:42.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-25T06:01:42.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-05-25T06:01:28.264738+0000", 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.635+0000 7f4af72bd700 1 Processor -- start 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.635+0000 7f4af72bd700 1 -- start start 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.635+0000 7f4af72bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af0104170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.635+0000 7f4af72bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0104780 con 0x7f4af0101d90 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4af5059700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af0104170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4af5059700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af0104170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33922/0 (socket says 172.21.15.160:33922) 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4af5059700 1 -- 172.21.15.160:0/3806737063 learned_addr learned my addr 172.21.15.160:0/3806737063 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4af5059700 1 -- 172.21.15.160:0/3806737063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af01048c0 con 0x7f4af0101d90 2024-05-25T06:01:42.303 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4af5059700 1 --2- 172.21.15.160:0/3806737063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af0104170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00bb80 tx=0x7f4aec00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61531397250b9ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4ae7fff700 1 -- 172.21.15.160:0/3806737063 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4aec00dd00 con 0x7f4af0101d90 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4ae7fff700 1 -- 172.21.15.160:0/3806737063 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4aec00de60 con 0x7f4af0101d90 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.636+0000 7f4ae7fff700 1 -- 172.21.15.160:0/3806737063 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4aec0188a0 con 0x7f4af0101d90 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- 172.21.15.160:0/3806737063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 msgr2=0x7f4af0104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 --2- 172.21.15.160:0/3806737063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af0104170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00bb80 tx=0x7f4aec00b4f0 comp rx=0 tx=0).stop 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- 172.21.15.160:0/3806737063 shutdown_connections 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 --2- 172.21.15.160:0/3806737063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af0104170 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- 172.21.15.160:0/3806737063 >> 172.21.15.160:0/3806737063 conn(0x7f4af00fc6f0 msgr2=0x7f4af00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- 172.21.15.160:0/3806737063 shutdown_connections 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- 172.21.15.160:0/3806737063 wait complete. 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 Processor -- start 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- start start 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af010f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.637+0000 7f4af72bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af010f860 con 0x7f4af0101d90 2024-05-25T06:01:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af5059700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af010f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af5059700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af010f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33938/0 (socket says 172.21.15.160:33938) 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af5059700 1 -- 172.21.15.160:0/2184337149 learned_addr learned my addr 172.21.15.160:0/2184337149 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af5059700 1 -- 172.21.15.160:0/2184337149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec00b790 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af5059700 1 --2- 172.21.15.160:0/2184337149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af010f320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4aec018600 tx=0x7f4aec00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4aec010bd0 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4aec01d050 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af010fa60 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4aec022590 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.638+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01105d0 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.639+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 2) v1 ==== 43132+0+0 (secure 0 0 0) 0x7f4aec022790 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.639+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af005f5c0 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.639+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4aec014070 con 0x7f4af0101d90 2024-05-25T06:01:42.305 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.641+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f4aec00e360 con 0x7f4af0101d90 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.709+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4af004bed0 con 0x7f4af0101d90 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.710+0000 7f4ae5ffb700 1 -- 172.21.15.160:0/2184337149 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f4aec0202d0 con 0x7f4af0101d90 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 msgr2=0x7f4af010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 --2- 172.21.15.160:0/2184337149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af010f320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4aec018600 tx=0x7f4aec00b4f0 comp rx=0 tx=0).stop 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 shutdown_connections 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 --2- 172.21.15.160:0/2184337149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4af0101d90 0x7f4af010f320 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 >> 172.21.15.160:0/2184337149 conn(0x7f4af00fc6f0 msgr2=0x7f4af00fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 shutdown_connections 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:41.711+0000 7f4af72bd700 1 -- 172.21.15.160:0/2184337149 wait complete. 2024-05-25T06:01:42.306 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (3/15)... 2024-05-25T06:01:42.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:42 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2184337149' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-25T06:01:42.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:42 smithi160 ceph-mon[29041]: mgrmap e3: smithi160.nalzlg(active, since 1.00575s) 2024-05-25T06:01:44.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:44 smithi160 ceph-mon[29041]: mgrmap e4: smithi160.nalzlg(active, since 2s) 2024-05-25T06:01:47.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:47 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/4159539850' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-25T06:01:53.512 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:53.514 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:01:53.514 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "0ed6924e-1a5c-11ef-bc9b-c7b262605968", 2024-05-25T06:01:53.514 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "smithi160" 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:53.515 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-25T06:01:53.516 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-25T06:01:53.524 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-25T06:01:53.525 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:53.526 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-05-25T06:01:28.264738+0000", 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.134+0000 7fcdbbe2d700 1 Processor -- start 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.134+0000 7fcdbbe2d700 1 -- start start 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.134+0000 7fcdbbe2d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.134+0000 7fcdbbe2d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4079070 con 0x7fcdb4078c60 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb9bc9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb9bc9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57694/0 (socket says 172.21.15.160:57694) 2024-05-25T06:01:53.527 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb9bc9700 1 -- 172.21.15.160:0/2690471293 learned_addr learned my addr 172.21.15.160:0/2690471293 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb9bc9700 1 -- 172.21.15.160:0/2690471293 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4077750 con 0x7fcdb4078c60 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb9bc9700 1 --2- 172.21.15.160:0/2690471293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4077210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00098e0 tx=0x7fcdb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c14462d1dc60085 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb8bc7700 1 -- 172.21.15.160:0/2690471293 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb000b860 con 0x7fcdb4078c60 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb8bc7700 1 -- 172.21.15.160:0/2690471293 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcdb000be50 con 0x7fcdb4078c60 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.135+0000 7fcdb8bc7700 1 -- 172.21.15.160:0/2690471293 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb0010380 con 0x7fcdb4078c60 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/2690471293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 msgr2=0x7fcdb4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 --2- 172.21.15.160:0/2690471293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4077210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00098e0 tx=0x7fcdb0009250 comp rx=0 tx=0).stop 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/2690471293 shutdown_connections 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 --2- 172.21.15.160:0/2690471293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4077210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/2690471293 >> 172.21.15.160:0/2690471293 conn(0x7fcdb40fe060 msgr2=0x7fcdb4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/2690471293 shutdown_connections 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.136+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/2690471293 wait complete. 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdbbe2d700 1 Processor -- start 2024-05-25T06:01:53.528 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdbbe2d700 1 -- start start 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdbbe2d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdbbe2d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4113d80 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdb9bc9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdb9bc9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57698/0 (socket says 172.21.15.160:57698) 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdb9bc9700 1 -- 172.21.15.160:0/4159539850 learned_addr learned my addr 172.21.15.160:0/4159539850 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdb9bc9700 1 -- 172.21.15.160:0/4159539850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb00094f0 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.137+0000 7fcdb9bc9700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4113840 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0006fd0 tx=0x7fcdb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.138+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb0010760 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.138+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcdb0010d50 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.138+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb4113f80 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.138+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb001ae10 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.138+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb4114af0 con 0x7fcdb4078c60 2024-05-25T06:01:53.529 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.138+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fcdb0019410 con 0x7fcdb4078c60 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.139+0000 7fcdaa7fc700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fcda00367a0 0x7fcda0038c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.139+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcdb0049fd0 con 0x7fcdb4078c60 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.139+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb405f5c0 con 0x7fcdb4078c60 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.139+0000 7fcdb93c8700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fcda00367a0 0x7fcda0038c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.139+0000 7fcdb93c8700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fcda00367a0 0x7fcda0038c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009990 tx=0x7fcda4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.144+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcdb0022c90 con 0x7fcdb4078c60 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.404+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcdb404bed0 con 0x7fcdb4078c60 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.405+0000 7fcdaa7fc700 1 -- 172.21.15.160:0/4159539850 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fcdb0022ab0 con 0x7fcdb4078c60 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fcda00367a0 msgr2=0x7fcda0038c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fcda00367a0 0x7fcda0038c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009990 tx=0x7fcda4006e40 comp rx=0 tx=0).stop 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 msgr2=0x7fcdb4113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4113840 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0006fd0 tx=0x7fcdb0009250 comp rx=0 tx=0).stop 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 shutdown_connections 2024-05-25T06:01:53.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fcda00367a0 0x7fcda0038c50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:53.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 --2- 172.21.15.160:0/4159539850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4078c60 0x7fcdb4113840 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:53.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 >> 172.21.15.160:0/4159539850 conn(0x7fcdb40fe060 msgr2=0x7fcdb40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:53.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 shutdown_connections 2024-05-25T06:01:53.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:47.407+0000 7fcdbbe2d700 1 -- 172.21.15.160:0/4159539850 wait complete. 2024-05-25T06:01:53.531 INFO:teuthology.orchestra.run.smithi160.stdout:mgr is available 2024-05-25T06:01:55.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:55 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3571996865' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-25T06:01:56.046 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [global] 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout fsid = 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [osd] 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-25T06:01:56.047 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a2aa7700 1 Processor -- start 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a2aa7700 1 -- start start 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a2aa7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a2aa7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c070d90 con 0x7f309c103ed0 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a0843700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a0843700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57706/0 (socket says 172.21.15.160:57706) 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.292+0000 7f30a0843700 1 -- 172.21.15.160:0/838110240 learned_addr learned my addr 172.21.15.160:0/838110240 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f30a0843700 1 -- 172.21.15.160:0/838110240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c104820 con 0x7f309c103ed0 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f30a0843700 1 --2- 172.21.15.160:0/838110240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1042e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30900098e0 tx=0x7f3090009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef553aa8cbab377c server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f309b7fe700 1 -- 172.21.15.160:0/838110240 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f309000b860 con 0x7f309c103ed0 2024-05-25T06:01:56.048 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f309b7fe700 1 -- 172.21.15.160:0/838110240 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f309000be50 con 0x7f309c103ed0 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f309b7fe700 1 -- 172.21.15.160:0/838110240 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3090010380 con 0x7f309c103ed0 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f30a2aa7700 1 -- 172.21.15.160:0/838110240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 msgr2=0x7f309c1042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.293+0000 7f30a2aa7700 1 --2- 172.21.15.160:0/838110240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1042e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30900098e0 tx=0x7f3090009250 comp rx=0 tx=0).stop 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 -- 172.21.15.160:0/838110240 shutdown_connections 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 --2- 172.21.15.160:0/838110240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1042e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 -- 172.21.15.160:0/838110240 >> 172.21.15.160:0/838110240 conn(0x7f309c0fc6f0 msgr2=0x7f309c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 -- 172.21.15.160:0/838110240 shutdown_connections 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 -- 172.21.15.160:0/838110240 wait complete. 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 Processor -- start 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.294+0000 7f30a2aa7700 1 -- start start 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a2aa7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a2aa7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c103b20 con 0x7f309c103ed0 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a0843700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:56.049 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a0843700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57720/0 (socket says 172.21.15.160:57720) 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a0843700 1 -- 172.21.15.160:0/3571996865 learned_addr learned my addr 172.21.15.160:0/3571996865 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a0843700 1 -- 172.21.15.160:0/3571996865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30900094f0 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30a0843700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1035e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f309000b440 tx=0x7f3090009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3090010760 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3090010d50 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.295+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f309001ae20 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.296+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c101b90 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.296+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c102080 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.297+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f30900108c0 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.297+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c04bed0 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.297+0000 7f30997fa700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f308c036850 0x7f308c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.297+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f309004a510 con 0x7f309c103ed0 2024-05-25T06:01:56.050 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.297+0000 7f309bfff700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f308c036850 0x7f308c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.297+0000 7f309bfff700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f308c036850 0x7f308c038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3088009990 tx=0x7f3088006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.302+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3090018920 con 0x7f309c103ed0 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.488+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f309c102500 con 0x7f309c103ed0 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.489+0000 7f30997fa700 1 -- 172.21.15.160:0/3571996865 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f3090018350 con 0x7f309c103ed0 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f308c036850 msgr2=0x7f308c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f308c036850 0x7f308c038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3088009990 tx=0x7f3088006e40 comp rx=0 tx=0).stop 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 msgr2=0x7f309c1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1035e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f309000b440 tx=0x7f3090009250 comp rx=0 tx=0).stop 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 shutdown_connections 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f308c036850 0x7f308c038d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 --2- 172.21.15.160:0/3571996865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c103ed0 0x7f309c1035e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 >> 172.21.15.160:0/3571996865 conn(0x7f309c0fc6f0 msgr2=0x7f309c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:56.051 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 shutdown_connections 2024-05-25T06:01:56.052 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:55.491+0000 7f30a2aa7700 1 -- 172.21.15.160:0/3571996865 wait complete. 2024-05-25T06:01:56.052 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling cephadm module... 2024-05-25T06:01:57.044 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:56 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2546621567' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f791d5fd700 1 Processor -- start 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f791d5fd700 1 -- start start 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f791d5fd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f791d5fd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7918102080 con 0x7f7918101730 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42454/0 (socket says 172.21.15.160:42454) 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.438+0000 7f7916ffd700 1 -- 172.21.15.160:0/4228624056 learned_addr learned my addr 172.21.15.160:0/4228624056 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.439+0000 7f7916ffd700 1 -- 172.21.15.160:0/4228624056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79181021c0 con 0x7f7918101730 2024-05-25T06:01:57.336 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.439+0000 7f7916ffd700 1 --2- 172.21.15.160:0/4228624056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918101b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79080098e0 tx=0x7f7908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e4f929d5b2f8d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.439+0000 7f79167fc700 1 -- 172.21.15.160:0/4228624056 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f790800b860 con 0x7f7918101730 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.439+0000 7f79167fc700 1 -- 172.21.15.160:0/4228624056 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f790800be50 con 0x7f7918101730 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.439+0000 7f79167fc700 1 -- 172.21.15.160:0/4228624056 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7908010380 con 0x7f7918101730 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 -- 172.21.15.160:0/4228624056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 msgr2=0x7f7918101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 --2- 172.21.15.160:0/4228624056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918101b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79080098e0 tx=0x7f7908009250 comp rx=0 tx=0).stop 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 -- 172.21.15.160:0/4228624056 shutdown_connections 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 --2- 172.21.15.160:0/4228624056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918101b40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 -- 172.21.15.160:0/4228624056 >> 172.21.15.160:0/4228624056 conn(0x7f79180fcef0 msgr2=0x7f79180ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 -- 172.21.15.160:0/4228624056 shutdown_connections 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 -- 172.21.15.160:0/4228624056 wait complete. 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 Processor -- start 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.440+0000 7f791d5fd700 1 -- start start 2024-05-25T06:01:57.337 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f791d5fd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918076b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f791d5fd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79181142d0 con 0x7f7918101730 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918076b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918076b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42458/0 (socket says 172.21.15.160:42458) 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f7916ffd700 1 -- 172.21.15.160:0/2546621567 learned_addr learned my addr 172.21.15.160:0/2546621567 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f7916ffd700 1 -- 172.21.15.160:0/2546621567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080094f0 con 0x7f7918101730 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f7916ffd700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918076b20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f790800b440 tx=0x7f7908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7908010760 con 0x7f7918101730 2024-05-25T06:01:57.338 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7908010d50 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7918114410 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f790801ae20 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.441+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79181148a0 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.442+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f790802e440 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.442+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791804bed0 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.442+0000 7f790f7fe700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f79040367e0 0x7f7904038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.442+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f790804a7d0 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.443+0000 7f790ffff700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f79040367e0 0x7f7904038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.443+0000 7f790ffff700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f79040367e0 0x7f7904038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7900009990 tx=0x7f7900006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.447+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7908018940 con 0x7f7918101730 2024-05-25T06:01:57.339 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.681+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f7918103550 con 0x7f7918101730 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.734+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f79080243f0 con 0x7f7918101730 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.735+0000 7f790f7fe700 1 -- 172.21.15.160:0/2546621567 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f7908018530 con 0x7f7918101730 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f79040367e0 msgr2=0x7f7904038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f79040367e0 0x7f7904038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7900009990 tx=0x7f7900006e40 comp rx=0 tx=0).stop 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 msgr2=0x7f7918076b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918076b20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f790800b440 tx=0x7f7908009250 comp rx=0 tx=0).stop 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 shutdown_connections 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f79040367e0 0x7f7904038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 --2- 172.21.15.160:0/2546621567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7918101730 0x7f7918076b20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 >> 172.21.15.160:0/2546621567 conn(0x7f79180fcef0 msgr2=0x7f79180fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:57.340 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 shutdown_connections 2024-05-25T06:01:57.341 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:56.738+0000 7f791d5fd700 1 -- 172.21.15.160:0/2546621567 wait complete. 2024-05-25T06:01:57.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:57 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2546621567' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-25T06:01:57.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:57 smithi160 ceph-mon[29041]: mgrmap e5: smithi160.nalzlg(active, since 15s) 2024-05-25T06:01:58.543 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:01:58.543 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-25T06:01:58.544 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-25T06:01:58.544 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "active_name": "smithi160.nalzlg", 2024-05-25T06:01:58.544 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-25T06:01:58.544 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:01:58.544 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f49a2067700 1 Processor -- start 2024-05-25T06:01:58.544 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f49a2067700 1 -- start start 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f49a2067700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f49a2067700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0790b0 con 0x7f499c078ca0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42480/0 (socket says 172.21.15.160:42480) 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.788+0000 7f499b7fe700 1 -- 172.21.15.160:0/3349806592 learned_addr learned my addr 172.21.15.160:0/3349806592 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.789+0000 7f499b7fe700 1 -- 172.21.15.160:0/3349806592 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c077790 con 0x7f499c078ca0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.789+0000 7f499b7fe700 1 --2- 172.21.15.160:0/3349806592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c077250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f498c0098e0 tx=0x7f498c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbcd8119b29c67d server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.789+0000 7f499a7fc700 1 -- 172.21.15.160:0/3349806592 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f498c00b860 con 0x7f499c078ca0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.789+0000 7f499a7fc700 1 -- 172.21.15.160:0/3349806592 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f498c00be50 con 0x7f499c078ca0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.789+0000 7f499a7fc700 1 -- 172.21.15.160:0/3349806592 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f498c010380 con 0x7f499c078ca0 2024-05-25T06:01:58.545 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 -- 172.21.15.160:0/3349806592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 msgr2=0x7f499c077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 --2- 172.21.15.160:0/3349806592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c077250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f498c0098e0 tx=0x7f498c009250 comp rx=0 tx=0).stop 2024-05-25T06:01:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 -- 172.21.15.160:0/3349806592 shutdown_connections 2024-05-25T06:01:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 --2- 172.21.15.160:0/3349806592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c077250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 -- 172.21.15.160:0/3349806592 >> 172.21.15.160:0/3349806592 conn(0x7f499c0fe060 msgr2=0x7f499c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 -- 172.21.15.160:0/3349806592 shutdown_connections 2024-05-25T06:01:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 -- 172.21.15.160:0/3349806592 wait complete. 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 Processor -- start 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.790+0000 7f49a2067700 1 -- start start 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f49a2067700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f49a2067700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c113da0 con 0x7f499c078ca0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42488/0 (socket says 172.21.15.160:42488) 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f499b7fe700 1 -- 172.21.15.160:0/2088192166 learned_addr learned my addr 172.21.15.160:0/2088192166 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f499b7fe700 1 -- 172.21.15.160:0/2088192166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0094f0 con 0x7f499c078ca0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f499b7fe700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c113860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f498c00b440 tx=0x7f498c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f498c010760 con 0x7f499c078ca0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c113fa0 con 0x7f499c078ca0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f498c010d50 con 0x7f499c078ca0 2024-05-25T06:01:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f498c01adc0 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.791+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c114b10 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.792+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f498c019410 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.792+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c05f5c0 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.792+0000 7f4983fff700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 0x7f4984038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.793+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f498c01a660 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.793+0000 7f499affd700 1 -- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 msgr2=0x7f4984038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.793+0000 7f499affd700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 0x7f4984038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.797+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f498c023940 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.994+0000 7f499affd700 1 -- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 msgr2=0x7f4984038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:57.994+0000 7f499affd700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 0x7f4984038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.076+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f499c114eb0 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.077+0000 7f4983fff700 1 -- 172.21.15.160:0/2088192166 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f498c023350 con 0x7f499c078ca0 2024-05-25T06:01:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 msgr2=0x7f4984038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 0x7f4984038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 msgr2=0x7f499c113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c113860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f498c00b440 tx=0x7f498c009250 comp rx=0 tx=0).stop 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 shutdown_connections 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7f4984036830 0x7f4984038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 --2- 172.21.15.160:0/2088192166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f499c078ca0 0x7f499c113860 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 >> 172.21.15.160:0/2088192166 conn(0x7f499c0fe060 msgr2=0x7f499c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 shutdown_connections 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.079+0000 7f49a2067700 1 -- 172.21.15.160:0/2088192166 wait complete. 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for the mgr to restart... 2024-05-25T06:01:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr epoch 5... 2024-05-25T06:01:58.924 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:01:58 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2088192166' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-25T06:02:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: Active manager daemon smithi160.nalzlg restarted 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: osdmap e2: 0 total, 0 up, 0 in 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: mgrmap e6: smithi160.nalzlg(active, starting, since 0.00330421s) 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:03.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:03.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:03.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:02:03.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:03 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:02:04.648 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:04 smithi160 ceph-mon[29041]: Found migration_current of "None". Setting to last migration. 2024-05-25T06:02:04.648 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:04 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:04.649 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:04 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:04.649 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:04 smithi160 ceph-mon[29041]: mgrmap e7: smithi160.nalzlg(active, since 1.00571s) 2024-05-25T06:02:04.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:02:04.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-25T06:02:04.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-25T06:02:04.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:02:04.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52ead5700 1 Processor -- start 2024-05-25T06:02:04.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52ead5700 1 -- start start 2024-05-25T06:02:04.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52ead5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:04.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52ead5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528105a20 con 0x7fa528076b70 2024-05-25T06:02:04.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52c871700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:04.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52c871700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42502/0 (socket says 172.21.15.160:42502) 2024-05-25T06:02:04.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.984+0000 7fa52c871700 1 -- 172.21.15.160:0/1073462980 learned_addr learned my addr 172.21.15.160:0/1073462980 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:04.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.985+0000 7fa52c871700 1 -- 172.21.15.160:0/1073462980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528105b60 con 0x7fa528076b70 2024-05-25T06:02:04.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.985+0000 7fa52c871700 1 --2- 172.21.15.160:0/1073462980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281054e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0098e0 tx=0x7fa51c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe59ef350ba6168 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:04.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.985+0000 7fa5277fe700 1 -- 172.21.15.160:0/1073462980 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa51c00b860 con 0x7fa528076b70 2024-05-25T06:02:04.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.985+0000 7fa5277fe700 1 -- 172.21.15.160:0/1073462980 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa51c00be50 con 0x7fa528076b70 2024-05-25T06:02:04.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.985+0000 7fa5277fe700 1 -- 172.21.15.160:0/1073462980 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa51c010380 con 0x7fa528076b70 2024-05-25T06:02:04.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 -- 172.21.15.160:0/1073462980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 msgr2=0x7fa5281054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:04.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 --2- 172.21.15.160:0/1073462980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281054e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0098e0 tx=0x7fa51c009250 comp rx=0 tx=0).stop 2024-05-25T06:02:04.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 -- 172.21.15.160:0/1073462980 shutdown_connections 2024-05-25T06:02:04.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 --2- 172.21.15.160:0/1073462980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281054e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:04.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 -- 172.21.15.160:0/1073462980 >> 172.21.15.160:0/1073462980 conn(0x7fa5280fced0 msgr2=0x7fa5280ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:04.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 -- 172.21.15.160:0/1073462980 shutdown_connections 2024-05-25T06:02:04.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 -- 172.21.15.160:0/1073462980 wait complete. 2024-05-25T06:02:04.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 Processor -- start 2024-05-25T06:02:04.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.986+0000 7fa52ead5700 1 -- start start 2024-05-25T06:02:04.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52ead5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:04.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52ead5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528111b10 con 0x7fa528076b70 2024-05-25T06:02:04.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52c871700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:04.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52c871700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42516/0 (socket says 172.21.15.160:42516) 2024-05-25T06:02:04.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52c871700 1 -- 172.21.15.160:0/3413347958 learned_addr learned my addr 172.21.15.160:0/3413347958 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:04.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52c871700 1 -- 172.21.15.160:0/3413347958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0094f0 con 0x7fa528076b70 2024-05-25T06:02:04.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa52c871700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281115d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa51c006b00 tx=0x7fa51c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:04.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.987+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa51c0107c0 con 0x7fa528076b70 2024-05-25T06:02:04.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.988+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa528111d10 con 0x7fa528076b70 2024-05-25T06:02:04.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.988+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa51c010db0 con 0x7fa528076b70 2024-05-25T06:02:04.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.988+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa51c01acf0 con 0x7fa528076b70 2024-05-25T06:02:04.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.988+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa528112880 con 0x7fa528076b70 2024-05-25T06:02:04.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.989+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa51c0193c0 con 0x7fa528076b70 2024-05-25T06:02:04.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.989+0000 7fa5257fa700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:04.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.989+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 --> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa518039410 con 0x7fa518036850 2024-05-25T06:02:04.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.989+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa51c04a610 con 0x7fa528076b70 2024-05-25T06:02:04.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.989+0000 7fa527fff700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 msgr2=0x7fa518038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:02:04.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:58.989+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:02:04.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:59.190+0000 7fa527fff700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 msgr2=0x7fa518038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:02:04.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:59.190+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-25T06:02:04.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:59.591+0000 7fa527fff700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 msgr2=0x7fa518038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:02:04.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:01:59.591+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-25T06:02:04.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:00.392+0000 7fa527fff700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 msgr2=0x7fa518038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:02:04.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:00.392+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-25T06:02:04.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:01.993+0000 7fa527fff700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 msgr2=0x7fa518038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/4006145193 2024-05-25T06:02:04.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:01.993+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-25T06:02:04.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:03.340+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7fa51c0198a0 con 0x7fa528076b70 2024-05-25T06:02:04.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:03.340+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 msgr2=0x7fa518038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:02:04.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:03.340+0000 7fa5257fa700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:04.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.342+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7fa51c019c20 con 0x7fa528076b70 2024-05-25T06:02:04.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.342+0000 7fa5257fa700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa518039ff0 0x7fa51803c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:04.884 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.342+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa518039410 con 0x7fa518039ff0 2024-05-25T06:02:04.884 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.342+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa518039ff0 0x7fa51803c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:04.884 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.343+0000 7fa527fff700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa518039ff0 0x7fa51803c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa5140039d0 tx=0x7fa514009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:04.884 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.344+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fa518039410 con 0x7fa518039ff0 2024-05-25T06:02:04.884 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa52805f5c0 con 0x7fa518039ff0 2024-05-25T06:02:04.884 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa5257fa700 1 -- 172.21.15.160:0/3413347958 <== mgr.14120 v2:172.21.15.160:6800/3392624553 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa52805f5c0 con 0x7fa518039ff0 2024-05-25T06:02:04.885 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa518039ff0 msgr2=0x7fa51803c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:04.885 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa518039ff0 0x7fa51803c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa5140039d0 tx=0x7fa514009350 comp rx=0 tx=0).stop 2024-05-25T06:02:04.885 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 msgr2=0x7fa5281115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:04.885 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281115d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa51c006b00 tx=0x7fa51c009250 comp rx=0 tx=0).stop 2024-05-25T06:02:04.885 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 shutdown_connections 2024-05-25T06:02:04.886 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa518039ff0 0x7fa51803c3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:04.886 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528076b70 0x7fa5281115d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:04.886 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 --2- 172.21.15.160:0/3413347958 >> [v2:172.21.15.160:6800/4006145193,v1:172.21.15.160:6801/4006145193] conn(0x7fa518036850 0x7fa518038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:04.886 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 >> 172.21.15.160:0/3413347958 conn(0x7fa5280fced0 msgr2=0x7fa5280fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:04.886 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 shutdown_connections 2024-05-25T06:02:04.886 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:04.350+0000 7fa52ead5700 1 -- 172.21.15.160:0/3413347958 wait complete. 2024-05-25T06:02:04.887 INFO:teuthology.orchestra.run.smithi160.stdout:mgr epoch 5 is available 2024-05-25T06:02:04.887 INFO:teuthology.orchestra.run.smithi160.stdout:Setting orchestrator backend to cephadm... 2024-05-25T06:02:05.492 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:05 smithi160 ceph-mon[29041]: [25/May/2024:06:02:04] ENGINE Bus STARTING 2024-05-25T06:02:05.493 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:05 smithi160 ceph-mon[29041]: [25/May/2024:06:02:04] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:02:05.493 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:05 smithi160 ceph-mon[29041]: [25/May/2024:06:02:04] ENGINE Bus STARTED 2024-05-25T06:02:05.493 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:05 smithi160 ceph-mon[29041]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-25T06:02:05.493 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:05 smithi160 ceph-mon[29041]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.257+0000 7f89145bc700 1 Processor -- start 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.257+0000 7f89145bc700 1 -- start start 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f89145bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f89145bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c1031d0 con 0x7f890c102880 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f8912358700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f8912358700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42576/0 (socket says 172.21.15.160:42576) 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f8912358700 1 -- 172.21.15.160:0/3498748307 learned_addr learned my addr 172.21.15.160:0/3498748307 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f8912358700 1 -- 172.21.15.160:0/3498748307 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c103310 con 0x7f890c102880 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.258+0000 7f8912358700 1 --2- 172.21.15.160:0/3498748307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c102c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89080098e0 tx=0x7f8908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b35bd685d8fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f8911356700 1 -- 172.21.15.160:0/3498748307 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f890800b860 con 0x7f890c102880 2024-05-25T06:02:06.079 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f8911356700 1 -- 172.21.15.160:0/3498748307 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f890800be50 con 0x7f890c102880 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f8911356700 1 -- 172.21.15.160:0/3498748307 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8908010380 con 0x7f890c102880 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f89145bc700 1 -- 172.21.15.160:0/3498748307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 msgr2=0x7f890c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f89145bc700 1 --2- 172.21.15.160:0/3498748307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c102c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89080098e0 tx=0x7f8908009250 comp rx=0 tx=0).stop 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f89145bc700 1 -- 172.21.15.160:0/3498748307 shutdown_connections 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f89145bc700 1 --2- 172.21.15.160:0/3498748307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c102c90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f89145bc700 1 -- 172.21.15.160:0/3498748307 >> 172.21.15.160:0/3498748307 conn(0x7f890c0fe060 msgr2=0x7f890c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.259+0000 7f89145bc700 1 -- 172.21.15.160:0/3498748307 shutdown_connections 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f89145bc700 1 -- 172.21.15.160:0/3498748307 wait complete. 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f89145bc700 1 Processor -- start 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f89145bc700 1 -- start start 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f89145bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:06.080 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f89145bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c111b30 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f8912358700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f8912358700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42578/0 (socket says 172.21.15.160:42578) 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f8912358700 1 -- 172.21.15.160:0/2746312447 learned_addr learned my addr 172.21.15.160:0/2746312447 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.260+0000 7f8912358700 1 -- 172.21.15.160:0/2746312447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080094f0 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.261+0000 7f8912358700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c1115f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8908000c00 tx=0x7f8908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.261+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89080107c0 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.261+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8908010db0 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.261+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c111d30 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.261+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f890801ad00 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.261+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c1128a0 con 0x7f890c102880 2024-05-25T06:02:06.081 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.262+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f89080193c0 con 0x7f890c102880 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.262+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c05f5c0 con 0x7f890c102880 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.262+0000 7f88feffd700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f88f8036720 0x7f88f8038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.262+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f890804a600 con 0x7f890c102880 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.262+0000 7f8911b57700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f88f8036720 0x7f88f8038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.262+0000 7f8911b57700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f88f8036720 0x7f88f8038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8900009990 tx=0x7f8900006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.267+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8908018ca0 con 0x7f890c102880 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.393+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f890801a530 con 0x7f890c102880 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.470+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f890c0ff200 con 0x7f88f8036720 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.479+0000 7f88feffd700 1 -- 172.21.15.160:0/2746312447 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f890c0ff200 con 0x7f88f8036720 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f88f8036720 msgr2=0x7f88f8038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f88f8036720 0x7f88f8038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8900009990 tx=0x7f8900006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 msgr2=0x7f890c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:06.082 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c1115f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8908000c00 tx=0x7f8908009250 comp rx=0 tx=0).stop 2024-05-25T06:02:06.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 shutdown_connections 2024-05-25T06:02:06.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f88f8036720 0x7f88f8038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:06.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 --2- 172.21.15.160:0/2746312447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f890c102880 0x7f890c1115f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:06.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 >> 172.21.15.160:0/2746312447 conn(0x7f890c0fe060 msgr2=0x7f890c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:06.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 shutdown_connections 2024-05-25T06:02:06.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:05.481+0000 7f89145bc700 1 -- 172.21.15.160:0/2746312447 wait complete. 2024-05-25T06:02:06.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:06 smithi160 ceph-mon[29041]: mgrmap e8: smithi160.nalzlg(active, since 2s) 2024-05-25T06:02:06.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:06 smithi160 ceph-mon[29041]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:06.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:06 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:06.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:06 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:07.246 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-25T06:02:07.246 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.475+0000 7fcb91da0700 1 Processor -- start 2024-05-25T06:02:07.247 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.475+0000 7fcb91da0700 1 -- start start 2024-05-25T06:02:07.247 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.475+0000 7fcb91da0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:07.247 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.475+0000 7fcb91da0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c103090 con 0x7fcb8c102740 2024-05-25T06:02:07.247 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb90d9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:07.247 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb90d9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54804/0 (socket says 172.21.15.160:54804) 2024-05-25T06:02:07.247 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb90d9e700 1 -- 172.21.15.160:0/2811752018 learned_addr learned my addr 172.21.15.160:0/2811752018 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:07.248 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb90d9e700 1 -- 172.21.15.160:0/2811752018 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c1031d0 con 0x7fcb8c102740 2024-05-25T06:02:07.248 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb90d9e700 1 --2- 172.21.15.160:0/2811752018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c102b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009e50 tx=0x7fcb88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3cb73f1b559732 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:07.248 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb837fe700 1 -- 172.21.15.160:0/2811752018 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb880092e0 con 0x7fcb8c102740 2024-05-25T06:02:07.248 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb837fe700 1 -- 172.21.15.160:0/2811752018 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb88012430 con 0x7fcb8c102740 2024-05-25T06:02:07.248 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.476+0000 7fcb837fe700 1 -- 172.21.15.160:0/2811752018 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb880075c0 con 0x7fcb8c102740 2024-05-25T06:02:07.248 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 -- 172.21.15.160:0/2811752018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 msgr2=0x7fcb8c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:07.249 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 --2- 172.21.15.160:0/2811752018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c102b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009e50 tx=0x7fcb88009840 comp rx=0 tx=0).stop 2024-05-25T06:02:07.249 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 -- 172.21.15.160:0/2811752018 shutdown_connections 2024-05-25T06:02:07.249 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 --2- 172.21.15.160:0/2811752018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c102b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:07.249 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 -- 172.21.15.160:0/2811752018 >> 172.21.15.160:0/2811752018 conn(0x7fcb8c0fdea0 msgr2=0x7fcb8c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:07.249 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 -- 172.21.15.160:0/2811752018 shutdown_connections 2024-05-25T06:02:07.249 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 -- 172.21.15.160:0/2811752018 wait complete. 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 Processor -- start 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.477+0000 7fcb91da0700 1 -- start start 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb91da0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb91da0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c079160 con 0x7fcb8c102740 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb90d9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb90d9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54810/0 (socket says 172.21.15.160:54810) 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb90d9e700 1 -- 172.21.15.160:0/4160473939 learned_addr learned my addr 172.21.15.160:0/4160473939 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:07.250 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb90d9e700 1 -- 172.21.15.160:0/4160473939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb88009ae0 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb90d9e700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c078c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009ab0 tx=0x7fcb88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb880092e0 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.478+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb88007d10 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.479+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c077230 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.479+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb8801c990 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.479+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c077720 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.480+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fcb88019070 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.480+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c04bed0 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.480+0000 7fcb817fa700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fcb740368a0 0x7fcb74038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.480+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fcb8804a040 con 0x7fcb8c102740 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.480+0000 7fcb83fff700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fcb740368a0 0x7fcb74038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:07.251 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.480+0000 7fcb83fff700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fcb740368a0 0x7fcb74038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009990 tx=0x7fcb7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:07.252 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.485+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcb8807c250 con 0x7fcb8c102740 2024-05-25T06:02:07.252 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.673+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb8c077e30 con 0x7fcb740368a0 2024-05-25T06:02:07.252 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.674+0000 7fcb817fa700 1 -- 172.21.15.160:0/4160473939 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fcb8c077e30 con 0x7fcb740368a0 2024-05-25T06:02:07.252 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fcb740368a0 msgr2=0x7fcb74038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:07.252 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fcb740368a0 0x7fcb74038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009990 tx=0x7fcb7c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 msgr2=0x7fcb8c078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c078c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009ab0 tx=0x7fcb88009840 comp rx=0 tx=0).stop 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 shutdown_connections 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fcb740368a0 0x7fcb74038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 --2- 172.21.15.160:0/4160473939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb8c102740 0x7fcb8c078c20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 >> 172.21.15.160:0/4160473939 conn(0x7fcb8c0fdea0 msgr2=0x7fcb8c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:07.253 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 shutdown_connections 2024-05-25T06:02:07.254 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:06.676+0000 7fcb91da0700 1 -- 172.21.15.160:0/4160473939 wait complete. 2024-05-25T06:02:07.254 INFO:teuthology.orchestra.run.smithi160.stdout:Generating ssh key... 2024-05-25T06:02:07.547 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:07 smithi160 ceph-mon[29041]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdd04ec0700 1 Processor -- start 2024-05-25T06:02:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdd04ec0700 1 -- start start 2024-05-25T06:02:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdd04ec0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdd04ec0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd001059a0 con 0x7fdd00076bf0 2024-05-25T06:02:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54812/0 (socket says 172.21.15.160:54812) 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.643+0000 7fdcfe59c700 1 -- 172.21.15.160:0/3531052591 learned_addr learned my addr 172.21.15.160:0/3531052591 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdcfe59c700 1 -- 172.21.15.160:0/3531052591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00105ae0 con 0x7fdd00076bf0 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdcfe59c700 1 --2- 172.21.15.160:0/3531052591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00105460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00098e0 tx=0x7fdcf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1539270e621c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdcfdd9b700 1 -- 172.21.15.160:0/3531052591 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcf000b860 con 0x7fdd00076bf0 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdcfdd9b700 1 -- 172.21.15.160:0/3531052591 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdcf000be50 con 0x7fdd00076bf0 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdcfdd9b700 1 -- 172.21.15.160:0/3531052591 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcf0010380 con 0x7fdd00076bf0 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3531052591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 msgr2=0x7fdd00105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.644+0000 7fdd04ec0700 1 --2- 172.21.15.160:0/3531052591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00105460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00098e0 tx=0x7fdcf0009250 comp rx=0 tx=0).stop 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3531052591 shutdown_connections 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 --2- 172.21.15.160:0/3531052591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00105460 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3531052591 >> 172.21.15.160:0/3531052591 conn(0x7fdd000fcef0 msgr2=0x7fdd000ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3531052591 shutdown_connections 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3531052591 wait complete. 2024-05-25T06:02:08.483 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 Processor -- start 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 -- start start 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdd04ec0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00111b40 con 0x7fdd00076bf0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54822/0 (socket says 172.21.15.160:54822) 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.645+0000 7fdcfe59c700 1 -- 172.21.15.160:0/3051412418 learned_addr learned my addr 172.21.15.160:0/3051412418 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdcfe59c700 1 -- 172.21.15.160:0/3051412418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00094f0 con 0x7fdd00076bf0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdcfe59c700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00111600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0003050 tx=0x7fdcf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcf0010760 con 0x7fdd00076bf0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd00111d40 con 0x7fdd00076bf0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdcf0010d50 con 0x7fdd00076bf0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcf001ae20 con 0x7fdd00076bf0 2024-05-25T06:02:08.484 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.646+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd001128b0 con 0x7fdd00076bf0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.647+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdcf002e440 con 0x7fdd00076bf0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.647+0000 7fdcf6ffd700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdcec036830 0x7fdcec038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.647+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdcf004a500 con 0x7fdd00076bf0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.647+0000 7fdcf7fff700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdcec036830 0x7fdcec038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.647+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0005f5c0 con 0x7fdd00076bf0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.647+0000 7fdcf7fff700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdcec036830 0x7fdcec038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009990 tx=0x7fdce8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.652+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdcf00243f0 con 0x7fdd00076bf0 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.837+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd000fe100 con 0x7fdcec036830 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.935+0000 7fdcf6ffd700 1 -- 172.21.15.160:0/3051412418 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdd000fe100 con 0x7fdcec036830 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.937+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdcec036830 msgr2=0x7fdcec038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.937+0000 7fdd04ec0700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdcec036830 0x7fdcec038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009990 tx=0x7fdce8006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.937+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 msgr2=0x7fdd00111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:08.485 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.937+0000 7fdd04ec0700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00111600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0003050 tx=0x7fdcf0009250 comp rx=0 tx=0).stop 2024-05-25T06:02:08.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.938+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 shutdown_connections 2024-05-25T06:02:08.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.938+0000 7fdd04ec0700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdcec036830 0x7fdcec038ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:08.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.938+0000 7fdd04ec0700 1 --2- 172.21.15.160:0/3051412418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd00076bf0 0x7fdd00111600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:08.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.938+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 >> 172.21.15.160:0/3051412418 conn(0x7fdd000fcef0 msgr2=0x7fdd000fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:08.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.938+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 shutdown_connections 2024-05-25T06:02:08.486 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:07.938+0000 7fdd04ec0700 1 -- 172.21.15.160:0/3051412418 wait complete. 2024-05-25T06:02:09.084 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:08 smithi160 ceph-mon[29041]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:09.084 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:08 smithi160 ceph-mon[29041]: Generating ssh key... 2024-05-25T06:02:09.084 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:08 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:09.084 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:08 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:09.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDafYLNn3O30qJnlzWG/t7lfP+9J94R33Xr2IwXwK0QjU2wOsjYI9Yiya0EqJNiUXiPt4wlZCmc3J31YIav234XMpCalQ3eg2Bk7mkQjmWlFq1BxsGzJ9WYwtBW8PyRt2sg7aNBw+7uzI43NPMDqTbAraVq99+0s6X1FaeDEXr1MMBkQmEtBAQupaJU+NusDCO+/V7+iCJY6JCqeu93KvNRZLmoc4Cu5N6KdurpAbQ89ooUzhOJwIMg29fKpzut+QrAOLMs+wIUzSS8HP3G48GyTzn+qBajdU35raE/0P3vDyIXca564mxJ+kOExE0zZdw/nziSz3N3Zjtk3SJZ1Txl9OP88i9sHyWpSiJBAT+2Taf5NJ6st4U/bKwjRy8jqceH/6SGy3ys+hdCJ0pOPVFb7k1scs2mw/rZOCU97vnYCjXpMKRHIZ/YBBsO7mWzVPrEgFmsOwSiN5Y7ARfXMMhHYIPbTTEd7nAMvWp+6bjfeNuKrDkrodCykP8/LXDApyU= ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:02:09.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.889+0000 7fa7072c5700 1 Processor -- start 2024-05-25T06:02:09.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.889+0000 7fa7072c5700 1 -- start start 2024-05-25T06:02:09.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa7072c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700102150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:09.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa7072c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700102690 con 0x7fa700101d40 2024-05-25T06:02:09.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa705061700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:09.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa705061700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54824/0 (socket says 172.21.15.160:54824) 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa705061700 1 -- 172.21.15.160:0/1623910265 learned_addr learned my addr 172.21.15.160:0/1623910265 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa705061700 1 -- 172.21.15.160:0/1623910265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7001027d0 con 0x7fa700101d40 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa705061700 1 --2- 172.21.15.160:0/1623910265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700102150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00098e0 tx=0x7fa6f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699a49db7a666084 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa6f7fff700 1 -- 172.21.15.160:0/1623910265 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6f000b860 con 0x7fa700101d40 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa6f7fff700 1 -- 172.21.15.160:0/1623910265 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa6f000be50 con 0x7fa700101d40 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.890+0000 7fa6f7fff700 1 -- 172.21.15.160:0/1623910265 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6f0010380 con 0x7fa700101d40 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 -- 172.21.15.160:0/1623910265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 msgr2=0x7fa700102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 --2- 172.21.15.160:0/1623910265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700102150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00098e0 tx=0x7fa6f0009250 comp rx=0 tx=0).stop 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 -- 172.21.15.160:0/1623910265 shutdown_connections 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 --2- 172.21.15.160:0/1623910265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700102150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 -- 172.21.15.160:0/1623910265 >> 172.21.15.160:0/1623910265 conn(0x7fa7000fc6f0 msgr2=0x7fa7000feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 -- 172.21.15.160:0/1623910265 shutdown_connections 2024-05-25T06:02:09.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.891+0000 7fa7072c5700 1 -- 172.21.15.160:0/1623910265 wait complete. 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa7072c5700 1 Processor -- start 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa7072c5700 1 -- start start 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa7072c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa7072c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001142d0 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa705061700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa705061700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54826/0 (socket says 172.21.15.160:54826) 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa705061700 1 -- 172.21.15.160:0/3628148383 learned_addr learned my addr 172.21.15.160:0/3628148383 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa705061700 1 -- 172.21.15.160:0/3628148383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00094f0 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa705061700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700101540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000b440 tx=0x7fa6f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.892+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6f0010760 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.893+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa6f0010d50 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.893+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa700114410 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.893+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6f001ae20 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.893+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7001148a0 con 0x7fa700101d40 2024-05-25T06:02:09.599 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.894+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa6f002e440 con 0x7fa700101d40 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.894+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70004bed0 con 0x7fa700101d40 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.894+0000 7fa6f5ffb700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa6ec0368a0 0x7fa6ec038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.894+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa6f004a5f0 con 0x7fa700101d40 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.894+0000 7fa704860700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa6ec0368a0 0x7fa6ec038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.894+0000 7fa704860700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa6ec0368a0 0x7fa6ec038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009990 tx=0x7fa6fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:08.899+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa6f0018940 con 0x7fa700101d40 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.084+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7000fd8e0 con 0x7fa6ec0368a0 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.084+0000 7fa6f5ffb700 1 -- 172.21.15.160:0/3628148383 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa7000fd8e0 con 0x7fa6ec0368a0 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa6ec0368a0 msgr2=0x7fa6ec038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa6ec0368a0 0x7fa6ec038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009990 tx=0x7fa6fc006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 msgr2=0x7fa700101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:09.600 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700101540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000b440 tx=0x7fa6f0009250 comp rx=0 tx=0).stop 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 shutdown_connections 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fa6ec0368a0 0x7fa6ec038d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 --2- 172.21.15.160:0/3628148383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa700101d40 0x7fa700101540 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 >> 172.21.15.160:0/3628148383 conn(0x7fa7000fc6f0 msgr2=0x7fa7000fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 shutdown_connections 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:09.087+0000 7fa7072c5700 1 -- 172.21.15.160:0/3628148383 wait complete. 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:Adding key to root@localhost authorized_keys... 2024-05-25T06:02:09.601 INFO:teuthology.orchestra.run.smithi160.stdout:Adding host smithi160... 2024-05-25T06:02:10.199 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:09 smithi160 ceph-mon[29041]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:11.053 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:10 smithi160 ceph-mon[29041]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "addr": "172.21.15.160", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:11.053 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:10 smithi160 ceph-mon[29041]: Deploying cephadm binary to smithi160 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Added host 'smithi160' with addr '172.21.15.160' 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.000+0000 7f7513d74700 1 Processor -- start 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.000+0000 7f7513d74700 1 -- start start 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7513d74700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7513d74700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c102080 con 0x7f750c101730 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7511b10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7511b10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54832/0 (socket says 172.21.15.160:54832) 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7511b10700 1 -- 172.21.15.160:0/4006219982 learned_addr learned my addr 172.21.15.160:0/4006219982 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7511b10700 1 -- 172.21.15.160:0/4006219982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c1021c0 con 0x7f750c101730 2024-05-25T06:02:12.588 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7511b10700 1 --2- 172.21.15.160:0/4006219982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c101b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75080098e0 tx=0x7f7508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6f66022d68e039 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7510b0e700 1 -- 172.21.15.160:0/4006219982 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f750800b860 con 0x7f750c101730 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7510b0e700 1 -- 172.21.15.160:0/4006219982 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f750800be50 con 0x7f750c101730 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.001+0000 7f7510b0e700 1 -- 172.21.15.160:0/4006219982 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7508010380 con 0x7f750c101730 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 -- 172.21.15.160:0/4006219982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 msgr2=0x7f750c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 --2- 172.21.15.160:0/4006219982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c101b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75080098e0 tx=0x7f7508009250 comp rx=0 tx=0).stop 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 -- 172.21.15.160:0/4006219982 shutdown_connections 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 --2- 172.21.15.160:0/4006219982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c101b40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 -- 172.21.15.160:0/4006219982 >> 172.21.15.160:0/4006219982 conn(0x7f750c0fcef0 msgr2=0x7f750c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 -- 172.21.15.160:0/4006219982 shutdown_connections 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 -- 172.21.15.160:0/4006219982 wait complete. 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.002+0000 7f7513d74700 1 Processor -- start 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7513d74700 1 -- start start 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7513d74700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c076b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:12.589 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7513d74700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1142d0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7511b10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c076b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7511b10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c076b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54836/0 (socket says 172.21.15.160:54836) 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7511b10700 1 -- 172.21.15.160:0/2707695416 learned_addr learned my addr 172.21.15.160:0/2707695416 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7511b10700 1 -- 172.21.15.160:0/2707695416 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75080094f0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7511b10700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c076b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7508000c00 tx=0x7f7508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f75080107c0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7508010db0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c114410 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f750801ad00 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.003+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c1148a0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.004+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f75080193c0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.004+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c04bed0 con 0x7f750c101730 2024-05-25T06:02:12.590 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.004+0000 7f75027fc700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f74f8036850 0x7f74f8038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.004+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f750804a7d0 con 0x7f750c101730 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.004+0000 7f751130f700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f74f8036850 0x7f74f8038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.005+0000 7f751130f700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f74f8036850 0x7f74f8038d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009990 tx=0x7f74fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.009+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7508018b00 con 0x7f750c101730 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:10.198+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "addr": "172.21.15.160", "target": ["mon-mgr", ""]}) v1 -- 0x7f750c114ae0 con 0x7f74f8036850 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.030+0000 7f75027fc700 1 -- 172.21.15.160:0/2707695416 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f750c114ae0 con 0x7f74f8036850 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f74f8036850 msgr2=0x7f74f8038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f74f8036850 0x7f74f8038d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009990 tx=0x7f74fc006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 msgr2=0x7f750c076b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c076b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7508000c00 tx=0x7f7508009250 comp rx=0 tx=0).stop 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 shutdown_connections 2024-05-25T06:02:12.591 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f74f8036850 0x7f74f8038d00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:12.592 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 --2- 172.21.15.160:0/2707695416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c101730 0x7f750c076b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:12.592 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 >> 172.21.15.160:0/2707695416 conn(0x7f750c0fcef0 msgr2=0x7f750c0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:12.592 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 shutdown_connections 2024-05-25T06:02:12.592 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.033+0000 7f7513d74700 1 -- 172.21.15.160:0/2707695416 wait complete. 2024-05-25T06:02:12.592 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying mon service with default placement... 2024-05-25T06:02:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:13 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:13 smithi160 ceph-mon[29041]: Added host smithi160 2024-05-25T06:02:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:13 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:13.971 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-25T06:02:13.971 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.973+0000 7f360ffaf700 1 Processor -- start 2024-05-25T06:02:13.971 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.973+0000 7f360ffaf700 1 -- start start 2024-05-25T06:02:13.971 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.973+0000 7f360ffaf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608102c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.973+0000 7f360ffaf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36081031a0 con 0x7f3608102850 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360dd4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360dd4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54852/0 (socket says 172.21.15.160:54852) 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360dd4b700 1 -- 172.21.15.160:0/1977908971 learned_addr learned my addr 172.21.15.160:0/1977908971 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360dd4b700 1 -- 172.21.15.160:0/1977908971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36081032e0 con 0x7f3608102850 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360dd4b700 1 --2- 172.21.15.160:0/1977908971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608102c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35f80098e0 tx=0x7f35f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac687d154e4dac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360cd49700 1 -- 172.21.15.160:0/1977908971 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35f800b860 con 0x7f3608102850 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360cd49700 1 -- 172.21.15.160:0/1977908971 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f35f800be50 con 0x7f3608102850 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.974+0000 7f360cd49700 1 -- 172.21.15.160:0/1977908971 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35f8010380 con 0x7f3608102850 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 -- 172.21.15.160:0/1977908971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 msgr2=0x7f3608102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 --2- 172.21.15.160:0/1977908971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608102c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35f80098e0 tx=0x7f35f8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 -- 172.21.15.160:0/1977908971 shutdown_connections 2024-05-25T06:02:13.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 --2- 172.21.15.160:0/1977908971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608102c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 -- 172.21.15.160:0/1977908971 >> 172.21.15.160:0/1977908971 conn(0x7f36080fe010 msgr2=0x7f3608100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 -- 172.21.15.160:0/1977908971 shutdown_connections 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.975+0000 7f360ffaf700 1 -- 172.21.15.160:0/1977908971 wait complete. 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360ffaf700 1 Processor -- start 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360ffaf700 1 -- start start 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360ffaf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360ffaf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3608111ab0 con 0x7f3608102850 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360dd4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360dd4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54862/0 (socket says 172.21.15.160:54862) 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360dd4b700 1 -- 172.21.15.160:0/2229272001 learned_addr learned my addr 172.21.15.160:0/2229272001 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360dd4b700 1 -- 172.21.15.160:0/2229272001 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80094f0 con 0x7f3608102850 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360dd4b700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608111570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b440 tx=0x7f35f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35f8010760 con 0x7f3608102850 2024-05-25T06:02:13.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f35f8010d50 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3608111cb0 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.976+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35f801ae20 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.977+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3608112820 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.977+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f35f802e440 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.978+0000 7f35fe7fc700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f35f4036830 0x7f35f4038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.978+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f35f804a590 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.978+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360805f5c0 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.978+0000 7f360d54a700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f35f4036830 0x7f35f4038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.978+0000 7f360d54a700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f35f4036830 0x7f35f4038ce0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3604009990 tx=0x7f3604006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:12.983+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35f8018940 con 0x7f3608102850 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.201+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f36080ff160 con 0x7f35f4036830 2024-05-25T06:02:13.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.206+0000 7f35fe7fc700 1 -- 172.21.15.160:0/2229272001 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f36080ff160 con 0x7f35f4036830 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f35f4036830 msgr2=0x7f35f4038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f35f4036830 0x7f35f4038ce0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3604009990 tx=0x7f3604006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 msgr2=0x7f3608111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608111570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b440 tx=0x7f35f8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 shutdown_connections 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f35f4036830 0x7f35f4038ce0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 --2- 172.21.15.160:0/2229272001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3608102850 0x7f3608111570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 >> 172.21.15.160:0/2229272001 conn(0x7f36080fe010 msgr2=0x7f36080fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 shutdown_connections 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:13.209+0000 7f360ffaf700 1 -- 172.21.15.160:0/2229272001 wait complete. 2024-05-25T06:02:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying mgr service with default placement... 2024-05-25T06:02:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:14 smithi160 ceph-mon[29041]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:14 smithi160 ceph-mon[29041]: Saving service mon spec with placement count:5 2024-05-25T06:02:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:14 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:15.834 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:15 smithi160 ceph-mon[29041]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:15.834 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:15 smithi160 ceph-mon[29041]: Saving service mgr spec with placement count:2 2024-05-25T06:02:15.834 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:15 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:15.834 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:15 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:15.841 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-25T06:02:15.841 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffbaaa7b700 1 Processor -- start 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffbaaa7b700 1 -- start start 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffbaaa7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba4101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffbaaa7b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41020a0 con 0x7ffba4101750 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffba8817700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba4101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffba8817700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba4101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54874/0 (socket says 172.21.15.160:54874) 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.350+0000 7ffba8817700 1 -- 172.21.15.160:0/2609564330 learned_addr learned my addr 172.21.15.160:0/2609564330 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffba8817700 1 -- 172.21.15.160:0/2609564330 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba41021e0 con 0x7ffba4101750 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffba8817700 1 --2- 172.21.15.160:0/2609564330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba4101b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098e0 tx=0x7ffb98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860b244ff2cfc42a server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffba37fe700 1 -- 172.21.15.160:0/2609564330 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb9800b860 con 0x7ffba4101750 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffba37fe700 1 -- 172.21.15.160:0/2609564330 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb9800be50 con 0x7ffba4101750 2024-05-25T06:02:15.842 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffba37fe700 1 -- 172.21.15.160:0/2609564330 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb98010380 con 0x7ffba4101750 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/2609564330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 msgr2=0x7ffba4101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.351+0000 7ffbaaa7b700 1 --2- 172.21.15.160:0/2609564330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba4101b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098e0 tx=0x7ffb98009250 comp rx=0 tx=0).stop 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/2609564330 shutdown_connections 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 --2- 172.21.15.160:0/2609564330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba4101b60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/2609564330 >> 172.21.15.160:0/2609564330 conn(0x7ffba40fcef0 msgr2=0x7ffba40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/2609564330 shutdown_connections 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/2609564330 wait complete. 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 Processor -- start 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 -- start start 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.352+0000 7ffbaaa7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba41104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffbaaa7b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4110a10 con 0x7ffba4101750 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba8817700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba41104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:15.843 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba8817700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba41104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54888/0 (socket says 172.21.15.160:54888) 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba8817700 1 -- 172.21.15.160:0/459706766 learned_addr learned my addr 172.21.15.160:0/459706766 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba8817700 1 -- 172.21.15.160:0/459706766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb980094f0 con 0x7ffba4101750 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba8817700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba41104d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800b440 tx=0x7ffb98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb98010760 con 0x7ffba4101750 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4110c10 con 0x7ffba4101750 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb98010d50 con 0x7ffba4101750 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb9801ae20 con 0x7ffba4101750 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.353+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba4111780 con 0x7ffba4101750 2024-05-25T06:02:15.844 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.354+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ffb9802e440 con 0x7ffba4101750 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.354+0000 7ffba17fa700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ffb940368a0 0x7ffb94038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.354+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba405f5c0 con 0x7ffba4101750 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.355+0000 7ffba3fff700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ffb940368a0 0x7ffb94038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.355+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ffb9804a690 con 0x7ffba4101750 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.355+0000 7ffba3fff700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ffb940368a0 0x7ffb94038d50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009990 tx=0x7ffb90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.359+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffb98018710 con 0x7ffba4101750 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.563+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7ffba40fe090 con 0x7ffb940368a0 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.570+0000 7ffba17fa700 1 -- 172.21.15.160:0/459706766 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ffba40fe090 con 0x7ffb940368a0 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ffb940368a0 msgr2=0x7ffb94038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:15.845 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ffb940368a0 0x7ffb94038d50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009990 tx=0x7ffb90006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 msgr2=0x7ffba41104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba41104d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800b440 tx=0x7ffb98009250 comp rx=0 tx=0).stop 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 shutdown_connections 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ffb940368a0 0x7ffb94038d50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 --2- 172.21.15.160:0/459706766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba4101750 0x7ffba41104d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 >> 172.21.15.160:0/459706766 conn(0x7ffba40fcef0 msgr2=0x7ffba40fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 shutdown_connections 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:14.572+0000 7ffbaaa7b700 1 -- 172.21.15.160:0/459706766 wait complete. 2024-05-25T06:02:15.846 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying crash service with default placement... 2024-05-25T06:02:16.972 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe3435ad700 1 Processor -- start 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe3435ad700 1 -- start start 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe3435ad700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c103230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe3435ad700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c103770 con 0x7fe33c102e20 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe341349700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe341349700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35454/0 (socket says 172.21.15.160:35454) 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.237+0000 7fe341349700 1 -- 172.21.15.160:0/3634338570 learned_addr learned my addr 172.21.15.160:0/3634338570 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.238+0000 7fe341349700 1 -- 172.21.15.160:0/3634338570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c1038b0 con 0x7fe33c102e20 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.238+0000 7fe341349700 1 --2- 172.21.15.160:0/3634338570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c103230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098e0 tx=0x7fe338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af48ab6a5abbcd46 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.238+0000 7fe333fff700 1 -- 172.21.15.160:0/3634338570 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe33800b860 con 0x7fe33c102e20 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.238+0000 7fe333fff700 1 -- 172.21.15.160:0/3634338570 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe33800be90 con 0x7fe33c102e20 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.238+0000 7fe333fff700 1 -- 172.21.15.160:0/3634338570 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3380103c0 con 0x7fe33c102e20 2024-05-25T06:02:16.973 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 -- 172.21.15.160:0/3634338570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 msgr2=0x7fe33c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 --2- 172.21.15.160:0/3634338570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c103230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098e0 tx=0x7fe338009250 comp rx=0 tx=0).stop 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 -- 172.21.15.160:0/3634338570 shutdown_connections 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 --2- 172.21.15.160:0/3634338570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c103230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 -- 172.21.15.160:0/3634338570 >> 172.21.15.160:0/3634338570 conn(0x7fe33c0fc690 msgr2=0x7fe33c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 -- 172.21.15.160:0/3634338570 shutdown_connections 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.239+0000 7fe3435ad700 1 -- 172.21.15.160:0/3634338570 wait complete. 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe3435ad700 1 Processor -- start 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe3435ad700 1 -- start start 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe3435ad700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe3435ad700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c113d70 con 0x7fe33c102e20 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe341349700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe341349700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35456/0 (socket says 172.21.15.160:35456) 2024-05-25T06:02:16.974 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe341349700 1 -- 172.21.15.160:0/1898097136 learned_addr learned my addr 172.21.15.160:0/1898097136 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe341349700 1 -- 172.21.15.160:0/1898097136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380094f0 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.240+0000 7fe341349700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c113830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3380090b0 tx=0x7fe338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.241+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe338010840 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.241+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe338010e70 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.241+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe338024e40 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.241+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c113f70 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.241+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c114ae0 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.242+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe338024430 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.242+0000 7fe331ffb700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fe328036800 0x7fe328038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.242+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c05f5c0 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.242+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe33804a750 con 0x7fe33c102e20 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.242+0000 7fe340b48700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fe328036800 0x7fe328038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.243+0000 7fe340b48700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fe328036800 0x7fe328038cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009990 tx=0x7fe32c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.247+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe33802c520 con 0x7fe33c102e20 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.458+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fe33c0fd8a0 con 0x7fe328036800 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.465+0000 7fe331ffb700 1 -- 172.21.15.160:0/1898097136 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fe33c0fd8a0 con 0x7fe328036800 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fe328036800 msgr2=0x7fe328038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fe328036800 0x7fe328038cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009990 tx=0x7fe32c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 msgr2=0x7fe33c113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c113830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3380090b0 tx=0x7fe338009250 comp rx=0 tx=0).stop 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 shutdown_connections 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fe328036800 0x7fe328038cb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 --2- 172.21.15.160:0/1898097136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe33c102e20 0x7fe33c113830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 >> 172.21.15.160:0/1898097136 conn(0x7fe33c0fc690 msgr2=0x7fe33c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 shutdown_connections 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:16.467+0000 7fe3435ad700 1 -- 172.21.15.160:0/1898097136 wait complete. 2024-05-25T06:02:16.976 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying prometheus service with default placement... 2024-05-25T06:02:17.357 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:17 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:17.358 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:17 smithi160 ceph-mon[29041]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:17.358 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:17 smithi160 ceph-mon[29041]: Saving service crash spec with placement * 2024-05-25T06:02:17.358 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:17 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.356+0000 7f4e6d22d700 1 Processor -- start 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.356+0000 7f4e6d22d700 1 -- start start 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.356+0000 7f4e6d22d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.356+0000 7f4e6d22d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681059b0 con 0x7f4e680712d0 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.357+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.357+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35460/0 (socket says 172.21.15.160:35460) 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.357+0000 7f4e66d9d700 1 -- 172.21.15.160:0/673815308 learned_addr learned my addr 172.21.15.160:0/673815308 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.357+0000 7f4e66d9d700 1 -- 172.21.15.160:0/673815308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68105af0 con 0x7f4e680712d0 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.357+0000 7f4e66d9d700 1 --2- 172.21.15.160:0/673815308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68105470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e580098e0 tx=0x7f4e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e885fabbc6223b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.357+0000 7f4e65d9b700 1 -- 172.21.15.160:0/673815308 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e5800b860 con 0x7f4e680712d0 2024-05-25T06:02:18.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e65d9b700 1 -- 172.21.15.160:0/673815308 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e5800be90 con 0x7f4e680712d0 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e65d9b700 1 -- 172.21.15.160:0/673815308 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e580103c0 con 0x7f4e680712d0 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 -- 172.21.15.160:0/673815308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 msgr2=0x7f4e68105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 --2- 172.21.15.160:0/673815308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68105470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e580098e0 tx=0x7f4e58009250 comp rx=0 tx=0).stop 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 -- 172.21.15.160:0/673815308 shutdown_connections 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 --2- 172.21.15.160:0/673815308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68105470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 -- 172.21.15.160:0/673815308 >> 172.21.15.160:0/673815308 conn(0x7f4e680fcea0 msgr2=0x7f4e680ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 -- 172.21.15.160:0/673815308 shutdown_connections 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.358+0000 7f4e6d22d700 1 -- 172.21.15.160:0/673815308 wait complete. 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.359+0000 7f4e6d22d700 1 Processor -- start 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.359+0000 7f4e6d22d700 1 -- start start 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.359+0000 7f4e6d22d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.359+0000 7f4e6d22d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e68103300 con 0x7f4e680712d0 2024-05-25T06:02:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.359+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.359+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35462/0 (socket says 172.21.15.160:35462) 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e66d9d700 1 -- 172.21.15.160:0/2985163135 learned_addr learned my addr 172.21.15.160:0/2985163135 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e66d9d700 1 -- 172.21.15.160:0/2985163135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580094f0 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e66d9d700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68102dc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e580094c0 tx=0x7f4e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e58011920 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e68101370 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e58010d60 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e5801ae70 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.360+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e68101860 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.361+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f4e5802e440 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.361+0000 7f4e4f7fe700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f4e50036800 0x7f4e50038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.361+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4e5804b700 con 0x7f4e680712d0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.361+0000 7f4e6659c700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f4e50036800 0x7f4e50038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:18.107 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.362+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6804bed0 con 0x7f4e680712d0 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.362+0000 7f4e6659c700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f4e50036800 0x7f4e50038cb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009990 tx=0x7f4e5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.367+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4e5802c930 con 0x7f4e680712d0 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.576+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e68101d10 con 0x7f4e50036800 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.581+0000 7f4e4f7fe700 1 -- 172.21.15.160:0/2985163135 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f4e68101d10 con 0x7f4e50036800 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f4e50036800 msgr2=0x7f4e50038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f4e50036800 0x7f4e50038cb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009990 tx=0x7f4e5c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 msgr2=0x7f4e68102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68102dc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e580094c0 tx=0x7f4e58009250 comp rx=0 tx=0).stop 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 shutdown_connections 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f4e50036800 0x7f4e50038cb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 --2- 172.21.15.160:0/2985163135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e680712d0 0x7f4e68102dc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 >> 172.21.15.160:0/2985163135 conn(0x7f4e680fcea0 msgr2=0x7f4e680fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:18.108 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 shutdown_connections 2024-05-25T06:02:18.109 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:17.583+0000 7f4e6d22d700 1 -- 172.21.15.160:0/2985163135 wait complete. 2024-05-25T06:02:18.109 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying grafana service with default placement... 2024-05-25T06:02:18.886 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:18 smithi160 ceph-mon[29041]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:18.886 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:18 smithi160 ceph-mon[29041]: Saving service prometheus spec with placement count:1 2024-05-25T06:02:18.886 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:18 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ecb0a2700 1 Processor -- start 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ecb0a2700 1 -- start start 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ecb0a2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec4077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ecb0a2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4079150 con 0x7f2ec4078d40 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ec8e3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec4077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ec8e3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec4077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35472/0 (socket says 172.21.15.160:35472) 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ec8e3e700 1 -- 172.21.15.160:0/1659243512 learned_addr learned my addr 172.21.15.160:0/1659243512 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ec8e3e700 1 -- 172.21.15.160:0/1659243512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec4077790 con 0x7f2ec4078d40 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.483+0000 7f2ec8e3e700 1 --2- 172.21.15.160:0/1659243512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec4077250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80098e0 tx=0x7f2eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7a7db289bd6b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:19.463 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ec37fe700 1 -- 172.21.15.160:0/1659243512 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eb800b860 con 0x7f2ec4078d40 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ec37fe700 1 -- 172.21.15.160:0/1659243512 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb800be90 con 0x7f2ec4078d40 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ec37fe700 1 -- 172.21.15.160:0/1659243512 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eb80103c0 con 0x7f2ec4078d40 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/1659243512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 msgr2=0x7f2ec4077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 --2- 172.21.15.160:0/1659243512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec4077250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80098e0 tx=0x7f2eb8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/1659243512 shutdown_connections 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 --2- 172.21.15.160:0/1659243512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec4077250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/1659243512 >> 172.21.15.160:0/1659243512 conn(0x7f2ec40fdfd0 msgr2=0x7f2ec41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/1659243512 shutdown_connections 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.484+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/1659243512 wait complete. 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ecb0a2700 1 Processor -- start 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ecb0a2700 1 -- start start 2024-05-25T06:02:19.464 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ecb0a2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec41138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ecb0a2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4113df0 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ec8e3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec41138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ec8e3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec41138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35476/0 (socket says 172.21.15.160:35476) 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ec8e3e700 1 -- 172.21.15.160:0/4049514264 learned_addr learned my addr 172.21.15.160:0/4049514264 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.485+0000 7f2ec8e3e700 1 -- 172.21.15.160:0/4049514264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80094f0 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.486+0000 7f2ec8e3e700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec41138b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80105b0 tx=0x7f2eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.486+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eb8010840 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.486+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb8010e70 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.486+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eb801ae30 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.486+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec4113ff0 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.486+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec4114b60 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.487+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f2eb802d3f0 con 0x7f2ec4078d40 2024-05-25T06:02:19.465 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.487+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec405f5c0 con 0x7f2ec4078d40 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.487+0000 7f2ec17fa700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2eb40368a0 0x7f2eb4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.487+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2eb804a140 con 0x7f2ec4078d40 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.487+0000 7f2ec3fff700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2eb40368a0 0x7f2eb4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.487+0000 7f2ec3fff700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2eb40368a0 0x7f2eb4038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009990 tx=0x7f2eb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.491+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2eb802b4d0 con 0x7f2ec4078d40 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.689+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec40ff230 con 0x7f2eb40368a0 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.694+0000 7f2ec17fa700 1 -- 172.21.15.160:0/4049514264 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2ec40ff230 con 0x7f2eb40368a0 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2eb40368a0 msgr2=0x7f2eb4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2eb40368a0 0x7f2eb4038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009990 tx=0x7f2eb0006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 msgr2=0x7f2ec41138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec41138b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80105b0 tx=0x7f2eb8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 shutdown_connections 2024-05-25T06:02:19.466 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2eb40368a0 0x7f2eb4038d50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:19.467 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 --2- 172.21.15.160:0/4049514264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ec4078d40 0x7f2ec41138b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:19.467 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 >> 172.21.15.160:0/4049514264 conn(0x7f2ec40fdfd0 msgr2=0x7f2ec40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:19.467 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 shutdown_connections 2024-05-25T06:02:19.467 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:18.696+0000 7f2ecb0a2700 1 -- 172.21.15.160:0/4049514264 wait complete. 2024-05-25T06:02:19.467 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying node-exporter service with default placement... 2024-05-25T06:02:19.825 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:19 smithi160 ceph-mon[29041]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:19.825 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:19 smithi160 ceph-mon[29041]: Saving service grafana spec with placement count:1 2024-05-25T06:02:19.825 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:19 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:19.825 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:19 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:20.635 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.849+0000 7ff48e609700 1 Processor -- start 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.849+0000 7ff48e609700 1 -- start start 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff48e609700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff48e609700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488105970 con 0x7ff488071310 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff487fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff487fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35482/0 (socket says 172.21.15.160:35482) 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff487fff700 1 -- 172.21.15.160:0/341132669 learned_addr learned my addr 172.21.15.160:0/341132669 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff487fff700 1 -- 172.21.15.160:0/341132669 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488105ab0 con 0x7ff488071310 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff487fff700 1 --2- 172.21.15.160:0/341132669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488105430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4780098e0 tx=0x7ff478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b8fcebfee63306 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff486ffd700 1 -- 172.21.15.160:0/341132669 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff47800b860 con 0x7ff488071310 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff486ffd700 1 -- 172.21.15.160:0/341132669 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff47800be90 con 0x7ff488071310 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.850+0000 7ff486ffd700 1 -- 172.21.15.160:0/341132669 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4780103c0 con 0x7ff488071310 2024-05-25T06:02:20.636 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 -- 172.21.15.160:0/341132669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 msgr2=0x7ff488105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 --2- 172.21.15.160:0/341132669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488105430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4780098e0 tx=0x7ff478009250 comp rx=0 tx=0).stop 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 -- 172.21.15.160:0/341132669 shutdown_connections 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 --2- 172.21.15.160:0/341132669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488105430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 -- 172.21.15.160:0/341132669 >> 172.21.15.160:0/341132669 conn(0x7ff4880fcea0 msgr2=0x7ff4880ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 -- 172.21.15.160:0/341132669 shutdown_connections 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.851+0000 7ff48e609700 1 -- 172.21.15.160:0/341132669 wait complete. 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff48e609700 1 Processor -- start 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff48e609700 1 -- start start 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff48e609700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff48e609700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881032c0 con 0x7ff488071310 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff487fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff487fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35492/0 (socket says 172.21.15.160:35492) 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff487fff700 1 -- 172.21.15.160:0/3206128699 learned_addr learned my addr 172.21.15.160:0/3206128699 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:20.637 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff487fff700 1 -- 172.21.15.160:0/3206128699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780094f0 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.852+0000 7ff487fff700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488102d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff478006d40 tx=0x7ff478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.853+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff478010840 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.853+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff478010e70 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.853+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff478024ea0 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.853+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488101330 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.853+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488101820 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.854+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff47801a510 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.854+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48804bed0 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.854+0000 7ff484ff9700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ff470036800 0x7ff470038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.854+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff47804a710 con 0x7ff488071310 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.854+0000 7ff4877fe700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ff470036800 0x7ff470038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.855+0000 7ff4877fe700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ff470036800 0x7ff470038cb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009990 tx=0x7ff47c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:20.638 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:19.859+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff478018940 con 0x7ff488071310 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.067+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7ff488101e00 con 0x7ff470036800 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.079+0000 7ff484ff9700 1 -- 172.21.15.160:0/3206128699 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7ff488101e00 con 0x7ff470036800 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ff470036800 msgr2=0x7ff470038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ff470036800 0x7ff470038cb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009990 tx=0x7ff47c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 msgr2=0x7ff488102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488102d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff478006d40 tx=0x7ff478009250 comp rx=0 tx=0).stop 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 shutdown_connections 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7ff470036800 0x7ff470038cb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 --2- 172.21.15.160:0/3206128699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff488071310 0x7ff488102d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.081+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 >> 172.21.15.160:0/3206128699 conn(0x7ff4880fcea0 msgr2=0x7ff4880fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.082+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 shutdown_connections 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:20.082+0000 7ff48e609700 1 -- 172.21.15.160:0/3206128699 wait complete. 2024-05-25T06:02:20.639 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying alertmanager service with default placement... 2024-05-25T06:02:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:21 smithi160 ceph-mon[29041]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:21 smithi160 ceph-mon[29041]: Saving service node-exporter spec with placement * 2024-05-25T06:02:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:21 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:21.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:21 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:22.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:22 smithi160 ceph-mon[29041]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:22.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:22 smithi160 ceph-mon[29041]: Saving service alertmanager spec with placement count:1 2024-05-25T06:02:22.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:22 smithi160 ceph-mon[29041]: from='mgr.14120 172.21.15.160:0/3415429581' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.108+0000 7f3044062700 1 Processor -- start 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.108+0000 7f3044062700 1 -- start start 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.108+0000 7f3044062700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3044062700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c1020a0 con 0x7f303c101750 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3041dfe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3041dfe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35498/0 (socket says 172.21.15.160:35498) 2024-05-25T06:02:22.753 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3041dfe700 1 -- 172.21.15.160:0/3269689230 learned_addr learned my addr 172.21.15.160:0/3269689230 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3041dfe700 1 -- 172.21.15.160:0/3269689230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c1021e0 con 0x7f303c101750 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3041dfe700 1 --2- 172.21.15.160:0/3269689230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c101b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30380098e0 tx=0x7f3038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627cf291683ffe2e server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3040dfc700 1 -- 172.21.15.160:0/3269689230 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f303800b860 con 0x7f303c101750 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3040dfc700 1 -- 172.21.15.160:0/3269689230 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f303800be90 con 0x7f303c101750 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.109+0000 7f3040dfc700 1 -- 172.21.15.160:0/3269689230 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30380103c0 con 0x7f303c101750 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 -- 172.21.15.160:0/3269689230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 msgr2=0x7f303c101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 --2- 172.21.15.160:0/3269689230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c101b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30380098e0 tx=0x7f3038009250 comp rx=0 tx=0).stop 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 -- 172.21.15.160:0/3269689230 shutdown_connections 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 --2- 172.21.15.160:0/3269689230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c101b60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 -- 172.21.15.160:0/3269689230 >> 172.21.15.160:0/3269689230 conn(0x7f303c0fcef0 msgr2=0x7f303c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 -- 172.21.15.160:0/3269689230 shutdown_connections 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.110+0000 7f3044062700 1 -- 172.21.15.160:0/3269689230 wait complete. 2024-05-25T06:02:22.754 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3044062700 1 Processor -- start 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3044062700 1 -- start start 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3044062700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3044062700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c110a50 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3041dfe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3041dfe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c110510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35510/0 (socket says 172.21.15.160:35510) 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3041dfe700 1 -- 172.21.15.160:0/143747242 learned_addr learned my addr 172.21.15.160:0/143747242 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3041dfe700 1 -- 172.21.15.160:0/143747242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380094f0 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3041dfe700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c110510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30380090b0 tx=0x7f3038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3038010840 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3038010e70 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.111+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c110c50 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.112+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3038024e40 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.112+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c1117c0 con 0x7f303c101750 2024-05-25T06:02:22.755 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.112+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f303801a530 con 0x7f303c101750 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.112+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c05f5c0 con 0x7f303c101750 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.112+0000 7f30327fc700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f3028036800 0x7f3028038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.113+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f303804a980 con 0x7f303c101750 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.113+0000 7f30415fd700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f3028036800 0x7f3028038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.113+0000 7f30415fd700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f3028036800 0x7f3028038cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f302c009990 tx=0x7f302c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.117+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3038018530 con 0x7f303c101750 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.336+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f303c0fe0e0 con 0x7f3028036800 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.343+0000 7f30327fc700 1 -- 172.21.15.160:0/143747242 <== mgr.14120 v2:172.21.15.160:6800/3392624553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f303c0fe0e0 con 0x7f3028036800 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f3028036800 msgr2=0x7f3028038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f3028036800 0x7f3028038cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f302c009990 tx=0x7f302c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 msgr2=0x7f303c110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c110510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30380090b0 tx=0x7f3038009250 comp rx=0 tx=0).stop 2024-05-25T06:02:22.756 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 shutdown_connections 2024-05-25T06:02:22.757 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f3028036800 0x7f3028038cb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:22.757 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 --2- 172.21.15.160:0/143747242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f303c101750 0x7f303c110510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:22.757 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 >> 172.21.15.160:0/143747242 conn(0x7f303c0fcef0 msgr2=0x7f303c0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:22.757 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 shutdown_connections 2024-05-25T06:02:22.757 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:21.345+0000 7f3044062700 1 -- 172.21.15.160:0/143747242 wait complete. 2024-05-25T06:02:23.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c35edd700 1 Processor -- start 2024-05-25T06:02:23.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c35edd700 1 -- start start 2024-05-25T06:02:23.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c35edd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c301021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c35edd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301026f0 con 0x7f2c30101da0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c301021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c301021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35520/0 (socket says 172.21.15.160:35520) 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.148+0000 7f2c2f7fe700 1 -- 172.21.15.160:0/3862367656 learned_addr learned my addr 172.21.15.160:0/3862367656 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.149+0000 7f2c2f7fe700 1 -- 172.21.15.160:0/3862367656 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c30102830 con 0x7f2c30101da0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.149+0000 7f2c2f7fe700 1 --2- 172.21.15.160:0/3862367656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c301021b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000bb80 tx=0x7f2c2000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3173514ccdf09d48 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.149+0000 7f2c2e7fc700 1 -- 172.21.15.160:0/3862367656 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c2000dd00 con 0x7f2c30101da0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.149+0000 7f2c2e7fc700 1 -- 172.21.15.160:0/3862367656 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2000de60 con 0x7f2c30101da0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.149+0000 7f2c2e7fc700 1 -- 172.21.15.160:0/3862367656 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c20018920 con 0x7f2c30101da0 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.149+0000 7f2c35edd700 1 -- 172.21.15.160:0/3862367656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 msgr2=0x7f2c301021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 --2- 172.21.15.160:0/3862367656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c301021b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000bb80 tx=0x7f2c2000b4f0 comp rx=0 tx=0).stop 2024-05-25T06:02:23.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 -- 172.21.15.160:0/3862367656 shutdown_connections 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 --2- 172.21.15.160:0/3862367656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c301021b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 -- 172.21.15.160:0/3862367656 >> 172.21.15.160:0/3862367656 conn(0x7f2c300fc6f0 msgr2=0x7f2c300feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 -- 172.21.15.160:0/3862367656 shutdown_connections 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 -- 172.21.15.160:0/3862367656 wait complete. 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 Processor -- start 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 -- start start 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c3010f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.150+0000 7f2c35edd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3010f8b0 con 0x7f2c30101da0 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c3010f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c3010f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35536/0 (socket says 172.21.15.160:35536) 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c2f7fe700 1 -- 172.21.15.160:0/489336109 learned_addr learned my addr 172.21.15.160:0/489336109 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c2f7fe700 1 -- 172.21.15.160:0/489336109 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2000b790 con 0x7f2c30101da0 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c2f7fe700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c3010f370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c20006b00 tx=0x7f2c2000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:23.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c20010bd0 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3010fab0 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2001d050 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c20022590 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.151+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c30110620 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.152+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f2c20022810 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.152+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3005f5c0 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.152+0000 7f2c17fff700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2c18036850 0x7f2c18038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.152+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2c20014070 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.153+0000 7f2c2effd700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2c18036850 0x7f2c18038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.153+0000 7f2c2effd700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2c18036850 0x7f2c18038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009990 tx=0x7f2c24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.157+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2c200237f0 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.338+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f2c3004bed0 con 0x7f2c30101da0 2024-05-25T06:02:23.882 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.342+0000 7f2c17fff700 1 -- 172.21.15.160:0/489336109 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2c2000fbc0 con 0x7f2c30101da0 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2c18036850 msgr2=0x7f2c18038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2c18036850 0x7f2c18038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009990 tx=0x7f2c24006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 msgr2=0x7f2c3010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c3010f370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c20006b00 tx=0x7f2c2000b4f0 comp rx=0 tx=0).stop 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 shutdown_connections 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f2c18036850 0x7f2c18038d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 --2- 172.21.15.160:0/489336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c30101da0 0x7f2c3010f370 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c20006b00 tx=0x7f2c2000b4f0 comp rx=0 tx=0).stop 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 >> 172.21.15.160:0/489336109 conn(0x7f2c300fc6f0 msgr2=0x7f2c30106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 shutdown_connections 2024-05-25T06:02:23.883 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:23.345+0000 7f2c35edd700 1 -- 172.21.15.160:0/489336109 wait complete. 2024-05-25T06:02:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:24 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/489336109' entity='client.admin' 2024-05-25T06:02:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:24 smithi160 ceph-mon[29041]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:02:24.985 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.248+0000 7f70d9a95700 1 Processor -- start 2024-05-25T06:02:24.985 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.248+0000 7f70d9a95700 1 -- start start 2024-05-25T06:02:24.985 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.248+0000 7f70d9a95700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:24.985 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.248+0000 7f70d9a95700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d41059a0 con 0x7f70d4076bf0 2024-05-25T06:02:24.985 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:24.985 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35542/0 (socket says 172.21.15.160:35542) 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d37fe700 1 -- 172.21.15.160:0/81060807 learned_addr learned my addr 172.21.15.160:0/81060807 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d37fe700 1 -- 172.21.15.160:0/81060807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4105ae0 con 0x7f70d4076bf0 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d37fe700 1 --2- 172.21.15.160:0/81060807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4105460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009e50 tx=0x7f70c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb5d880b05e1a8f server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d27fc700 1 -- 172.21.15.160:0/81060807 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40092e0 con 0x7f70d4076bf0 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d27fc700 1 -- 172.21.15.160:0/81060807 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70c4012430 con 0x7f70d4076bf0 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.249+0000 7f70d27fc700 1 -- 172.21.15.160:0/81060807 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40075a0 con 0x7f70d4076bf0 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 -- 172.21.15.160:0/81060807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 msgr2=0x7f70d4105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 --2- 172.21.15.160:0/81060807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4105460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009e50 tx=0x7f70c4009840 comp rx=0 tx=0).stop 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 -- 172.21.15.160:0/81060807 shutdown_connections 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 --2- 172.21.15.160:0/81060807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4105460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 -- 172.21.15.160:0/81060807 >> 172.21.15.160:0/81060807 conn(0x7f70d40fcef0 msgr2=0x7f70d40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 -- 172.21.15.160:0/81060807 shutdown_connections 2024-05-25T06:02:24.986 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 -- 172.21.15.160:0/81060807 wait complete. 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.250+0000 7f70d9a95700 1 Processor -- start 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d9a95700 1 -- start start 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d9a95700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d9a95700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4111b90 con 0x7f70d4076bf0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4111650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35552/0 (socket says 172.21.15.160:35552) 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d37fe700 1 -- 172.21.15.160:0/2985381313 learned_addr learned my addr 172.21.15.160:0/2985381313 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d37fe700 1 -- 172.21.15.160:0/2985381313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c4009ae0 con 0x7f70d4076bf0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d37fe700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4111650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009fd0 tx=0x7f70c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40092e0 con 0x7f70d4076bf0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d4111d90 con 0x7f70d4076bf0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70c4007ce0 con 0x7f70d4076bf0 2024-05-25T06:02:24.987 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.251+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40259c0 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.252+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d4112900 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.252+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f70c4019050 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.253+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d405f5c0 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.253+0000 7f70bbfff700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f70bc036850 0x7f70bc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.253+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f70c4049fc0 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.253+0000 7f70d2ffd700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f70bc036850 0x7f70bc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.253+0000 7f70d2ffd700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f70bc036850 0x7f70bc038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.257+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f70c402d9a0 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.443+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f70d404bed0 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.449+0000 7f70bbfff700 1 -- 172.21.15.160:0/2985381313 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f70c4020030 con 0x7f70d4076bf0 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f70bc036850 msgr2=0x7f70bc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f70bc036850 0x7f70bc038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:24.988 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 msgr2=0x7f70d4111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4111650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009fd0 tx=0x7f70c4009840 comp rx=0 tx=0).stop 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 shutdown_connections 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f70bc036850 0x7f70bc038d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 --2- 172.21.15.160:0/2985381313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70d4076bf0 0x7f70d4111650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.451+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 >> 172.21.15.160:0/2985381313 conn(0x7f70d40fcef0 msgr2=0x7f70d40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.452+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 shutdown_connections 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:24.452+0000 7f70d9a95700 1 -- 172.21.15.160:0/2985381313 wait complete. 2024-05-25T06:02:24.989 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling the dashboard module... 2024-05-25T06:02:25.643 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:25 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2985381313' entity='client.admin' 2024-05-25T06:02:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:26 smithi160 ceph-mon[29041]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:02:26.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:26 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3986113162' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-25T06:02:27.083 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdec0b09700 1 Processor -- start 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdec0b09700 1 -- start start 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdec0b09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdec0b09700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc1059a0 con 0x7fdebc076bf0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdeba59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdeba59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35558/0 (socket says 172.21.15.160:35558) 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.395+0000 7fdeba59c700 1 -- 172.21.15.160:0/2839381378 learned_addr learned my addr 172.21.15.160:0/2839381378 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.396+0000 7fdeba59c700 1 -- 172.21.15.160:0/2839381378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc105ae0 con 0x7fdebc076bf0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.396+0000 7fdeba59c700 1 --2- 172.21.15.160:0/2839381378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc105460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0098e0 tx=0x7fdeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a3c9415d56c25c server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.396+0000 7fdeb959a700 1 -- 172.21.15.160:0/2839381378 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdeac00b860 con 0x7fdebc076bf0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.396+0000 7fdeb959a700 1 -- 172.21.15.160:0/2839381378 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdeac00be90 con 0x7fdebc076bf0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.396+0000 7fdeb959a700 1 -- 172.21.15.160:0/2839381378 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdeac0103c0 con 0x7fdebc076bf0 2024-05-25T06:02:27.084 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- 172.21.15.160:0/2839381378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 msgr2=0x7fdebc105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 --2- 172.21.15.160:0/2839381378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc105460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0098e0 tx=0x7fdeac009250 comp rx=0 tx=0).stop 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- 172.21.15.160:0/2839381378 shutdown_connections 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 --2- 172.21.15.160:0/2839381378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc105460 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- 172.21.15.160:0/2839381378 >> 172.21.15.160:0/2839381378 conn(0x7fdebc0fcef0 msgr2=0x7fdebc0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- 172.21.15.160:0/2839381378 shutdown_connections 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- 172.21.15.160:0/2839381378 wait complete. 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 Processor -- start 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- start start 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.397+0000 7fdec0b09700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc111b80 con 0x7fdebc076bf0 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeba59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeba59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35560/0 (socket says 172.21.15.160:35560) 2024-05-25T06:02:27.085 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeba59c700 1 -- 172.21.15.160:0/3986113162 learned_addr learned my addr 172.21.15.160:0/3986113162 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeba59c700 1 -- 172.21.15.160:0/3986113162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0094f0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeba59c700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc111640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0094c0 tx=0x7fdeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdeac010890 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdeac010ec0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc111d80 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdeac0243f0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.398+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc1128f0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.399+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdeac0109f0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.399+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc05f5c0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.399+0000 7fdeaaffd700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdea40368a0 0x7fdea4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.399+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdeac04a1e0 con 0x7fdebc076bf0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.399+0000 7fdeb9d9b700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdea40368a0 0x7fdea4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:27.086 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.400+0000 7fdeb9d9b700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdea40368a0 0x7fdea4038d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009990 tx=0x7fdeb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.404+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdeac02b4d0 con 0x7fdebc076bf0 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:25.643+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fdebc04bed0 con 0x7fdebc076bf0 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.460+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fdeac01a7e0 con 0x7fdebc076bf0 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.461+0000 7fdeaaffd700 1 -- 172.21.15.160:0/3986113162 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fdeac02c530 con 0x7fdebc076bf0 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdea40368a0 msgr2=0x7fdea4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdea40368a0 0x7fdea4038d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009990 tx=0x7fdeb0006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 msgr2=0x7fdebc111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc111640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0094c0 tx=0x7fdeac009250 comp rx=0 tx=0).stop 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 shutdown_connections 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fdea40368a0 0x7fdea4038d50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 --2- 172.21.15.160:0/3986113162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdebc076bf0 0x7fdebc111640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.464+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 >> 172.21.15.160:0/3986113162 conn(0x7fdebc0fcef0 msgr2=0x7fdebc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.465+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 shutdown_connections 2024-05-25T06:02:27.087 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:26.465+0000 7fdec0b09700 1 -- 172.21.15.160:0/3986113162 wait complete. 2024-05-25T06:02:27.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:27 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3986113162' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-25T06:02:27.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:27 smithi160 ceph-mon[29041]: mgrmap e9: smithi160.nalzlg(active, since 23s) 2024-05-25T06:02:28.474 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:02:28.474 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-25T06:02:28.474 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-25T06:02:28.474 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "active_name": "smithi160.nalzlg", 2024-05-25T06:02:28.474 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36e1fdf700 1 Processor -- start 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36e1fdf700 1 -- start start 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36e1fdf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc1054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36e1fdf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc105a00 con 0x7f36dc076b70 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc1054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc1054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40198/0 (socket says 172.21.15.160:40198) 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.554+0000 7f36db7fe700 1 -- 172.21.15.160:0/3601280588 learned_addr learned my addr 172.21.15.160:0/3601280588 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.555+0000 7f36db7fe700 1 -- 172.21.15.160:0/3601280588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc105b40 con 0x7f36dc076b70 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.555+0000 7f36db7fe700 1 --2- 172.21.15.160:0/3601280588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc1054c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098e0 tx=0x7f36cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741b5e6f913e1a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.555+0000 7f36da7fc700 1 -- 172.21.15.160:0/3601280588 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36cc00b860 con 0x7f36dc076b70 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.555+0000 7f36da7fc700 1 -- 172.21.15.160:0/3601280588 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36cc00be90 con 0x7f36dc076b70 2024-05-25T06:02:28.475 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.555+0000 7f36da7fc700 1 -- 172.21.15.160:0/3601280588 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36cc0103c0 con 0x7f36dc076b70 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 -- 172.21.15.160:0/3601280588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 msgr2=0x7f36dc1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 --2- 172.21.15.160:0/3601280588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc1054c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098e0 tx=0x7f36cc009250 comp rx=0 tx=0).stop 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 -- 172.21.15.160:0/3601280588 shutdown_connections 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 --2- 172.21.15.160:0/3601280588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc1054c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 -- 172.21.15.160:0/3601280588 >> 172.21.15.160:0/3601280588 conn(0x7f36dc0fced0 msgr2=0x7f36dc0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 -- 172.21.15.160:0/3601280588 shutdown_connections 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 -- 172.21.15.160:0/3601280588 wait complete. 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.556+0000 7f36e1fdf700 1 Processor -- start 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36e1fdf700 1 -- start start 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36e1fdf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36e1fdf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc103350 con 0x7f36dc076b70 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:28.476 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40204/0 (socket says 172.21.15.160:40204) 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36db7fe700 1 -- 172.21.15.160:0/4221128511 learned_addr learned my addr 172.21.15.160:0/4221128511 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36db7fe700 1 -- 172.21.15.160:0/4221128511 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc0094f0 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36db7fe700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc102e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36cc006d40 tx=0x7f36cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.557+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36cc010840 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.558+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36cc010e70 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.558+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc1013c0 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.558+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36cc024ea0 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.558+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc1018b0 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.558+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc04bed0 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.558+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f36cc01a4c0 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.559+0000 7f36c3fff700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 0x7f36c4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.559+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f36cc04a770 con 0x7f36dc076b70 2024-05-25T06:02:28.477 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.559+0000 7f36daffd700 1 -- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 msgr2=0x7f36c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.559+0000 7f36daffd700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 0x7f36c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.563+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f36cc018940 con 0x7f36dc076b70 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.760+0000 7f36daffd700 1 -- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 msgr2=0x7f36c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.760+0000 7f36daffd700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 0x7f36c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.841+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f36dc102020 con 0x7f36dc076b70 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.842+0000 7f36c3fff700 1 -- 172.21.15.160:0/4221128511 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f36cc02d3a0 con 0x7f36dc076b70 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 msgr2=0x7f36c4038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:02:28.478 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 0x7f36c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 msgr2=0x7f36dc102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc102e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36cc006d40 tx=0x7f36cc009250 comp rx=0 tx=0).stop 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 shutdown_connections 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7f36c40368a0 0x7f36c4038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 --2- 172.21.15.160:0/4221128511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36dc076b70 0x7f36dc102e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.844+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 >> 172.21.15.160:0/4221128511 conn(0x7f36dc0fced0 msgr2=0x7f36dc0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.845+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 shutdown_connections 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:27.845+0000 7f36e1fdf700 1 -- 172.21.15.160:0/4221128511 wait complete. 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for the mgr to restart... 2024-05-25T06:02:28.479 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr epoch 9... 2024-05-25T06:02:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:28 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/4221128511' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-25T06:02:33.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: Active manager daemon smithi160.nalzlg restarted 2024-05-25T06:02:33.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:02:33.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: osdmap e3: 0 total, 0 up, 0 in 2024-05-25T06:02:33.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: mgrmap e10: smithi160.nalzlg(active, starting, since 0.00347502s) 2024-05-25T06:02:33.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:02:33.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:02:33.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:33 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:02:34.716 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:34 smithi160 ceph-mon[29041]: [25/May/2024:06:02:33] ENGINE Bus STARTING 2024-05-25T06:02:34.716 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:34 smithi160 ceph-mon[29041]: [25/May/2024:06:02:33] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:02:34.716 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:34 smithi160 ceph-mon[29041]: [25/May/2024:06:02:33] ENGINE Bus STARTED 2024-05-25T06:02:34.716 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:34 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:34.716 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:34 smithi160 ceph-mon[29041]: mgrmap e11: smithi160.nalzlg(active, since 1.00577s) 2024-05-25T06:02:34.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-05-25T06:02:34.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-25T06:02:34.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-25T06:02:34.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-05-25T06:02:34.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5dca8a700 1 Processor -- start 2024-05-25T06:02:34.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5dca8a700 1 -- start start 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5dca8a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5dca8a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8105970 con 0x7fb5d8071310 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40214/0 (socket says 172.21.15.160:40214) 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.917+0000 7fb5d659c700 1 -- 172.21.15.160:0/3522110178 learned_addr learned my addr 172.21.15.160:0/3522110178 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.918+0000 7fb5d659c700 1 -- 172.21.15.160:0/3522110178 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8105ab0 con 0x7fb5d8071310 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.918+0000 7fb5d659c700 1 --2- 172.21.15.160:0/3522110178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8105430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80098e0 tx=0x7fb5c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f701a7a1bf462a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.918+0000 7fb5d559a700 1 -- 172.21.15.160:0/3522110178 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c800b860 con 0x7fb5d8071310 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.918+0000 7fb5d559a700 1 -- 172.21.15.160:0/3522110178 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5c800be90 con 0x7fb5d8071310 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.918+0000 7fb5d559a700 1 -- 172.21.15.160:0/3522110178 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c80103c0 con 0x7fb5d8071310 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 -- 172.21.15.160:0/3522110178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 msgr2=0x7fb5d8105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/3522110178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8105430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80098e0 tx=0x7fb5c8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:34.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 -- 172.21.15.160:0/3522110178 shutdown_connections 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/3522110178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8105430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 -- 172.21.15.160:0/3522110178 >> 172.21.15.160:0/3522110178 conn(0x7fb5d80fcea0 msgr2=0x7fb5d80ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 -- 172.21.15.160:0/3522110178 shutdown_connections 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 -- 172.21.15.160:0/3522110178 wait complete. 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 Processor -- start 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.919+0000 7fb5dca8a700 1 -- start start 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5dca8a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5dca8a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8111b80 con 0x7fb5d8071310 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40230/0 (socket says 172.21.15.160:40230) 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5d659c700 1 -- 172.21.15.160:0/1689708787 learned_addr learned my addr 172.21.15.160:0/1689708787 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5d659c700 1 -- 172.21.15.160:0/1689708787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80094f0 con 0x7fb5d8071310 2024-05-25T06:02:34.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5d659c700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8111640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80094c0 tx=0x7fb5c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c8010890 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5c8010ec0 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d8111d80 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c80243f0 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.920+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d81128f0 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.921+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fb5c802d3f0 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.921+0000 7fb5c6ffd700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.921+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 --> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5c00394b0 con 0x7fb5c00368f0 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.921+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb5c804a290 con 0x7fb5d8071310 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.921+0000 7fb5d5d9b700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 msgr2=0x7fb5c0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:28.921+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:29.122+0000 7fb5d5d9b700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 msgr2=0x7fb5c0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:34.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:29.122+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:29.523+0000 7fb5d5d9b700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 msgr2=0x7fb5c0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:29.523+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:30.324+0000 7fb5d5d9b700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 msgr2=0x7fb5c0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:30.324+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:31.925+0000 7fb5d5d9b700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 msgr2=0x7fb5c0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3392624553 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:31.925+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:33.126+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7fb5c801a7e0 con 0x7fb5d8071310 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:33.126+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 msgr2=0x7fb5c0038da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:33.126+0000 7fb5c6ffd700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.128+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7fb5c802dac0 con 0x7fb5d8071310 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.128+0000 7fb5c6ffd700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb5c003a3a0 0x7fb5c003c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:34.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.128+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5c00394b0 con 0x7fb5c003a3a0 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.128+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb5c003a3a0 0x7fb5c003c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.129+0000 7fb5d5d9b700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb5c003a3a0 0x7fb5c003c780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0039d0 tx=0x7fb5cc009350 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.129+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fb5c00394b0 con 0x7fb5c003a3a0 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb5d8112210 con 0x7fb5c003a3a0 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5c6ffd700 1 -- 172.21.15.160:0/1689708787 <== mgr.14162 v2:172.21.15.160:6800/3983131949 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fb5d8112210 con 0x7fb5c003a3a0 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb5c003a3a0 msgr2=0x7fb5c003c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb5c003a3a0 0x7fb5c003c780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0039d0 tx=0x7fb5cc009350 comp rx=0 tx=0).stop 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 msgr2=0x7fb5d8111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8111640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80094c0 tx=0x7fb5c8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 shutdown_connections 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb5c003a3a0 0x7fb5c003c780 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d8071310 0x7fb5d8111640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:34.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 --2- 172.21.15.160:0/1689708787 >> [v2:172.21.15.160:6800/3392624553,v1:172.21.15.160:6801/3392624553] conn(0x7fb5c00368f0 0x7fb5c0038da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:34.731 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.136+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 >> 172.21.15.160:0/1689708787 conn(0x7fb5d80fcea0 msgr2=0x7fb5d80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:34.731 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.137+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 shutdown_connections 2024-05-25T06:02:34.731 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:34.137+0000 7fb5dca8a700 1 -- 172.21.15.160:0/1689708787 wait complete. 2024-05-25T06:02:34.731 INFO:teuthology.orchestra.run.smithi160.stdout:mgr epoch 9 is available 2024-05-25T06:02:34.731 INFO:teuthology.orchestra.run.smithi160.stdout:Generating a dashboard self-signed certificate... 2024-05-25T06:02:35.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:35 smithi160 ceph-mon[29041]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-25T06:02:35.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:35 smithi160 ceph-mon[29041]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-25T06:02:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:35 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:36.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea6dde700 1 Processor -- start 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea6dde700 1 -- start start 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea6dde700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea6dde700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01031d0 con 0x7f6ea0102880 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea4b7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea4b7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40306/0 (socket says 172.21.15.160:40306) 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.173+0000 7f6ea4b7a700 1 -- 172.21.15.160:0/1989321510 learned_addr learned my addr 172.21.15.160:0/1989321510 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea4b7a700 1 -- 172.21.15.160:0/1989321510 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0103310 con 0x7f6ea0102880 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea4b7a700 1 --2- 172.21.15.160:0/1989321510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0102c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e940098e0 tx=0x7f6e94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d266e4d97c06f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6e9f7fe700 1 -- 172.21.15.160:0/1989321510 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e9400b860 con 0x7f6ea0102880 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6e9f7fe700 1 -- 172.21.15.160:0/1989321510 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e9400be90 con 0x7f6ea0102880 2024-05-25T06:02:36.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6e9f7fe700 1 -- 172.21.15.160:0/1989321510 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e940103c0 con 0x7f6ea0102880 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea6dde700 1 -- 172.21.15.160:0/1989321510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 msgr2=0x7f6ea0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea6dde700 1 --2- 172.21.15.160:0/1989321510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0102c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e940098e0 tx=0x7f6e94009250 comp rx=0 tx=0).stop 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea6dde700 1 -- 172.21.15.160:0/1989321510 shutdown_connections 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea6dde700 1 --2- 172.21.15.160:0/1989321510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0102c90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.174+0000 7f6ea6dde700 1 -- 172.21.15.160:0/1989321510 >> 172.21.15.160:0/1989321510 conn(0x7f6ea00fe060 msgr2=0x7f6ea0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea6dde700 1 -- 172.21.15.160:0/1989321510 shutdown_connections 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea6dde700 1 -- 172.21.15.160:0/1989321510 wait complete. 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea6dde700 1 Processor -- start 2024-05-25T06:02:36.131 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea6dde700 1 -- start start 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea6dde700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea6dde700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea0111b60 con 0x7f6ea0102880 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea4b7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea4b7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40318/0 (socket says 172.21.15.160:40318) 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.175+0000 7f6ea4b7a700 1 -- 172.21.15.160:0/3971496 learned_addr learned my addr 172.21.15.160:0/3971496 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6ea4b7a700 1 -- 172.21.15.160:0/3971496 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940094f0 con 0x7f6ea0102880 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6ea4b7a700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e94011680 tx=0x7f6e94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e94010890 con 0x7f6ea0102880 2024-05-25T06:02:36.132 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea0111d60 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e94010ec0 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e940243f0 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.176+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea01128d0 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.177+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f6e940109f0 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.177+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea005f5c0 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.177+0000 7f6e9d7fa700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6e90036770 0x7f6e90038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.177+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6e9404a250 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.177+0000 7f6e9ffff700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6e90036770 0x7f6e90038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.178+0000 7f6e9ffff700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6e90036770 0x7f6e90038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009990 tx=0x7f6e8c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.182+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6e9402b940 con 0x7f6ea0102880 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.386+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ea00ff250 con 0x7f6e90036770 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.541+0000 7f6e9d7fa700 1 -- 172.21.15.160:0/3971496 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f6ea00ff250 con 0x7f6e90036770 2024-05-25T06:02:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6e90036770 msgr2=0x7f6e90038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6e90036770 0x7f6e90038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009990 tx=0x7f6e8c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 msgr2=0x7f6ea0111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e94011680 tx=0x7f6e94009250 comp rx=0 tx=0).stop 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 shutdown_connections 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6e90036770 0x7f6e90038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 --2- 172.21.15.160:0/3971496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea0102880 0x7f6ea0111620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 >> 172.21.15.160:0/3971496 conn(0x7f6ea00fe060 msgr2=0x7f6ea00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 shutdown_connections 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:35.544+0000 7f6ea6dde700 1 -- 172.21.15.160:0/3971496 wait complete. 2024-05-25T06:02:36.134 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial admin user... 2024-05-25T06:02:36.516 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:36 smithi160 ceph-mon[29041]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:36.517 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:36 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:36.517 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:36 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:36.517 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:36 smithi160 ceph-mon[29041]: mgrmap e12: smithi160.nalzlg(active, since 2s) 2024-05-25T06:02:36.517 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:36 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:37.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$VfkhHpsu1Ey41iJUy5DNEe.nELzxOlHD6hZaM/n/A5h5iCZIbq9ce", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716616956, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-25T06:02:37.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f646db85700 1 Processor -- start 2024-05-25T06:02:37.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f646db85700 1 -- start start 2024-05-25T06:02:37.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f646db85700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:37.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f646db85700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468102080 con 0x7f6468101730 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f64677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f64677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33940/0 (socket says 172.21.15.160:33940) 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f64677fe700 1 -- 172.21.15.160:0/3634278149 learned_addr learned my addr 172.21.15.160:0/3634278149 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f64677fe700 1 -- 172.21.15.160:0/3634278149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64681021c0 con 0x7f6468101730 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.525+0000 7f64677fe700 1 --2- 172.21.15.160:0/3634278149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468101b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64580098e0 tx=0x7f6458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3454afd276744353 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f64667fc700 1 -- 172.21.15.160:0/3634278149 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f645800b860 con 0x7f6468101730 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f64667fc700 1 -- 172.21.15.160:0/3634278149 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f645800be90 con 0x7f6468101730 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f64667fc700 1 -- 172.21.15.160:0/3634278149 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64580103c0 con 0x7f6468101730 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 -- 172.21.15.160:0/3634278149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 msgr2=0x7f6468101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 --2- 172.21.15.160:0/3634278149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468101b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64580098e0 tx=0x7f6458009250 comp rx=0 tx=0).stop 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 -- 172.21.15.160:0/3634278149 shutdown_connections 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 --2- 172.21.15.160:0/3634278149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468101b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:37.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 -- 172.21.15.160:0/3634278149 >> 172.21.15.160:0/3634278149 conn(0x7f64680fcef0 msgr2=0x7f64680ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 -- 172.21.15.160:0/3634278149 shutdown_connections 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.526+0000 7f646db85700 1 -- 172.21.15.160:0/3634278149 wait complete. 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f646db85700 1 Processor -- start 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f646db85700 1 -- start start 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f646db85700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468076b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f646db85700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64681142f0 con 0x7f6468101730 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f64677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468076b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f64677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468076b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33942/0 (socket says 172.21.15.160:33942) 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.527+0000 7f64677fe700 1 -- 172.21.15.160:0/1786363221 learned_addr learned my addr 172.21.15.160:0/1786363221 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f64677fe700 1 -- 172.21.15.160:0/1786363221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580094f0 con 0x7f6468101730 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f64677fe700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468076b20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6458011680 tx=0x7f6458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6458010890 con 0x7f6468101730 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6468114430 con 0x7f6468101730 2024-05-25T06:02:37.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6458010ec0 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64681148c0 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.528+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64580243f0 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.529+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f6458024550 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.529+0000 7f644ffff700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f64500368a0 0x7f6450038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.529+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646804bed0 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.529+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f645804a870 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.529+0000 7f6466ffd700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f64500368a0 0x7f6450038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.529+0000 7f6466ffd700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f64500368a0 0x7f6450038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f645c009990 tx=0x7f645c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.534+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6458023ca0 con 0x7f6468101730 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.744+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f64681035e0 con 0x7f64500368a0 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.968+0000 7f644ffff700 1 -- 172.21.15.160:0/1786363221 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f64681035e0 con 0x7f64500368a0 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f64500368a0 msgr2=0x7f6450038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f64500368a0 0x7f6450038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f645c009990 tx=0x7f645c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:37.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 msgr2=0x7f6468076b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468076b20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6458011680 tx=0x7f6458009250 comp rx=0 tx=0).stop 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 shutdown_connections 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f64500368a0 0x7f6450038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 --2- 172.21.15.160:0/1786363221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6468101730 0x7f6468076b20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 >> 172.21.15.160:0/1786363221 conn(0x7f64680fcef0 msgr2=0x7f64680fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 shutdown_connections 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:36.970+0000 7f646db85700 1 -- 172.21.15.160:0/1786363221 wait complete. 2024-05-25T06:02:37.785 INFO:teuthology.orchestra.run.smithi160.stdout:Fetching dashboard port number... 2024-05-25T06:02:38.084 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:37 smithi160 ceph-mon[29041]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:38.084 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:37 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:39.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:38 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/421729805' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-25T06:02:39.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 8443 2024-05-25T06:02:39.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.187+0000 7fd585774700 1 Processor -- start 2024-05-25T06:02:39.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd585774700 1 -- start start 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd585774700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd585774700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801031c0 con 0x7fd580102870 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd57effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd57effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33946/0 (socket says 172.21.15.160:33946) 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd57effd700 1 -- 172.21.15.160:0/80697012 learned_addr learned my addr 172.21.15.160:0/80697012 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd57effd700 1 -- 172.21.15.160:0/80697012 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580103300 con 0x7fd580102870 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.188+0000 7fd57effd700 1 --2- 172.21.15.160:0/80697012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580102c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098e0 tx=0x7fd570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188708b6307e4311 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd57dffb700 1 -- 172.21.15.160:0/80697012 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd57000b860 con 0x7fd580102870 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd57dffb700 1 -- 172.21.15.160:0/80697012 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd57000be90 con 0x7fd580102870 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd57dffb700 1 -- 172.21.15.160:0/80697012 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5700103c0 con 0x7fd580102870 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 -- 172.21.15.160:0/80697012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 msgr2=0x7fd580102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 --2- 172.21.15.160:0/80697012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580102c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098e0 tx=0x7fd570009250 comp rx=0 tx=0).stop 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 -- 172.21.15.160:0/80697012 shutdown_connections 2024-05-25T06:02:39.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 --2- 172.21.15.160:0/80697012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580102c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 -- 172.21.15.160:0/80697012 >> 172.21.15.160:0/80697012 conn(0x7fd5800fe030 msgr2=0x7fd580100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 -- 172.21.15.160:0/80697012 shutdown_connections 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.189+0000 7fd585774700 1 -- 172.21.15.160:0/80697012 wait complete. 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd585774700 1 Processor -- start 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd585774700 1 -- start start 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd585774700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd585774700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd580079160 con 0x7fd580102870 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd57effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd57effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33962/0 (socket says 172.21.15.160:33962) 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.190+0000 7fd57effd700 1 -- 172.21.15.160:0/421729805 learned_addr learned my addr 172.21.15.160:0/421729805 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd57effd700 1 -- 172.21.15.160:0/421729805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700094f0 con 0x7fd580102870 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd57effd700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580078c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd570011680 tx=0x7fd570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd570010890 con 0x7fd580102870 2024-05-25T06:02:39.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd570010ec0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5800771d0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5700243f0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.191+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5800776c0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.192+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fd57002d3f0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.192+0000 7fd5677fe700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fd5680368a0 0x7fd568038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.192+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd57004b5b0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.192+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58004bed0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.192+0000 7fd57e7fc700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fd5680368a0 0x7fd568038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.193+0000 7fd57e7fc700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fd5680368a0 0x7fd568038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd574009990 tx=0x7fd574006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.197+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd57002b8e0 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.405+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd580104610 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.406+0000 7fd5677fe700 1 -- 172.21.15.160:0/421729805 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd570006d70 con 0x7fd580102870 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fd5680368a0 msgr2=0x7fd568038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:39.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fd5680368a0 0x7fd568038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd574009990 tx=0x7fd574006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 msgr2=0x7fd580078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580078c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd570011680 tx=0x7fd570009250 comp rx=0 tx=0).stop 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 shutdown_connections 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fd5680368a0 0x7fd568038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 --2- 172.21.15.160:0/421729805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd580102870 0x7fd580078c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 >> 172.21.15.160:0/421729805 conn(0x7fd5800fe030 msgr2=0x7fd5800feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 shutdown_connections 2024-05-25T06:02:39.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:38.408+0000 7fd585774700 1 -- 172.21.15.160:0/421729805 wait complete. 2024-05-25T06:02:39.286 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-25T06:02:39.286 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-05-25T06:02:39.295 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-25T06:02:39.295 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-05-25T06:02:39.295 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-05-25T06:02:39.295 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-25T06:02:39.297 INFO:teuthology.orchestra.run.smithi160.stdout:Ceph Dashboard is now available at: 2024-05-25T06:02:39.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:39.297 INFO:teuthology.orchestra.run.smithi160.stdout: URL: https://smithi160.front.sepia.ceph.com:8443/ 2024-05-25T06:02:39.297 INFO:teuthology.orchestra.run.smithi160.stdout: User: admin 2024-05-25T06:02:39.297 INFO:teuthology.orchestra.run.smithi160.stdout: Password: u44gf647c3 2024-05-25T06:02:39.298 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:39.298 INFO:teuthology.orchestra.run.smithi160.stdout:Saving cluster configuration to /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config directory 2024-05-25T06:02:39.298 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling autotune for osd_memory_target 2024-05-25T06:02:40.436 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.700+0000 7fe4b3730700 1 Processor -- start 2024-05-25T06:02:40.436 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.700+0000 7fe4b3730700 1 -- start start 2024-05-25T06:02:40.436 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.700+0000 7fe4b3730700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.700+0000 7fe4b3730700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac105940 con 0x7fe4ac076c50 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.700+0000 7fe4b14cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4b14cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33970/0 (socket says 172.21.15.160:33970) 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4b14cc700 1 -- 172.21.15.160:0/4128960162 learned_addr learned my addr 172.21.15.160:0/4128960162 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4b14cc700 1 -- 172.21.15.160:0/4128960162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac105a80 con 0x7fe4ac076c50 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4b14cc700 1 --2- 172.21.15.160:0/4128960162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac105400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80098e0 tx=0x7fe4a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6c0aab00b1fbd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4a3fff700 1 -- 172.21.15.160:0/4128960162 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4a800b860 con 0x7fe4ac076c50 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4a3fff700 1 -- 172.21.15.160:0/4128960162 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4a800be90 con 0x7fe4ac076c50 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.701+0000 7fe4a3fff700 1 -- 172.21.15.160:0/4128960162 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4a80103c0 con 0x7fe4ac076c50 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 -- 172.21.15.160:0/4128960162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 msgr2=0x7fe4ac105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 --2- 172.21.15.160:0/4128960162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac105400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80098e0 tx=0x7fe4a8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 -- 172.21.15.160:0/4128960162 shutdown_connections 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 --2- 172.21.15.160:0/4128960162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac105400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:40.437 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 -- 172.21.15.160:0/4128960162 >> 172.21.15.160:0/4128960162 conn(0x7fe4ac0fcef0 msgr2=0x7fe4ac0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 -- 172.21.15.160:0/4128960162 shutdown_connections 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 -- 172.21.15.160:0/4128960162 wait complete. 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 Processor -- start 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.702+0000 7fe4b3730700 1 -- start start 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b3730700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b3730700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac111b60 con 0x7fe4ac076c50 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b14cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b14cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33982/0 (socket says 172.21.15.160:33982) 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b14cc700 1 -- 172.21.15.160:0/2682699576 learned_addr learned my addr 172.21.15.160:0/2682699576 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b14cc700 1 -- 172.21.15.160:0/2682699576 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80094f0 con 0x7fe4ac076c50 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b14cc700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80094c0 tx=0x7fe4a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4a8010890 con 0x7fe4ac076c50 2024-05-25T06:02:40.438 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4a8010ec0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac111d60 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4a80243f0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.703+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac1128d0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.704+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fe4a80109f0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.704+0000 7fe4a1ffb700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fe4980368f0 0x7fe498038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.704+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe4a804a370 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.704+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac05f5c0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.705+0000 7fe4b0ccb700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fe4980368f0 0x7fe498038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.705+0000 7fe4b0ccb700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fe4980368f0 0x7fe498038da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009990 tx=0x7fe49c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.709+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe4a802b940 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.893+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fe4ac04bed0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.893+0000 7fe4a1ffb700 1 -- 172.21.15.160:0/2682699576 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fe4a802b2f0 con 0x7fe4ac076c50 2024-05-25T06:02:40.439 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fe4980368f0 msgr2=0x7fe498038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fe4980368f0 0x7fe498038da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009990 tx=0x7fe49c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 msgr2=0x7fe4ac111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80094c0 tx=0x7fe4a8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 shutdown_connections 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fe4980368f0 0x7fe498038da0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 --2- 172.21.15.160:0/2682699576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac076c50 0x7fe4ac111620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 >> 172.21.15.160:0/2682699576 conn(0x7fe4ac0fcef0 msgr2=0x7fe4ac0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 shutdown_connections 2024-05-25T06:02:40.440 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:39.896+0000 7fe4b3730700 1 -- 172.21.15.160:0/2682699576 wait complete. 2024-05-25T06:02:41.165 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:40 smithi160 ceph-mon[29041]: mgrmap e13: smithi160.nalzlg(active, since 6s) 2024-05-25T06:02:41.835 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.864+0000 7fb9ef617700 1 Processor -- start 2024-05-25T06:02:41.835 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ef617700 1 -- start start 2024-05-25T06:02:41.835 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ef617700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:41.835 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ef617700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81059a0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ed3b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ed3b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33990/0 (socket says 172.21.15.160:33990) 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ed3b3700 1 -- 172.21.15.160:0/650676758 learned_addr learned my addr 172.21.15.160:0/650676758 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.865+0000 7fb9ed3b3700 1 -- 172.21.15.160:0/650676758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8105ae0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ed3b3700 1 --2- 172.21.15.160:0/650676758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8105460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098e0 tx=0x7fb9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1f92773513d97e server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9dffff700 1 -- 172.21.15.160:0/650676758 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d800b860 con 0x7fb9e8076bf0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9dffff700 1 -- 172.21.15.160:0/650676758 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9d800be90 con 0x7fb9e8076bf0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9dffff700 1 -- 172.21.15.160:0/650676758 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d80103c0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 -- 172.21.15.160:0/650676758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 msgr2=0x7fb9e8105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 --2- 172.21.15.160:0/650676758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8105460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098e0 tx=0x7fb9d8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 -- 172.21.15.160:0/650676758 shutdown_connections 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 --2- 172.21.15.160:0/650676758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8105460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:41.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 -- 172.21.15.160:0/650676758 >> 172.21.15.160:0/650676758 conn(0x7fb9e80fcef0 msgr2=0x7fb9e80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 -- 172.21.15.160:0/650676758 shutdown_connections 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.866+0000 7fb9ef617700 1 -- 172.21.15.160:0/650676758 wait complete. 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ef617700 1 Processor -- start 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ef617700 1 -- start start 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ef617700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ef617700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81032f0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ed3b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ed3b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34006/0 (socket says 172.21.15.160:34006) 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ed3b3700 1 -- 172.21.15.160:0/1312892381 learned_addr learned my addr 172.21.15.160:0/1312892381 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.867+0000 7fb9ed3b3700 1 -- 172.21.15.160:0/1312892381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80094f0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.868+0000 7fb9ed3b3700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8102db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8006d40 tx=0x7fb9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.868+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d8010840 con 0x7fb9e8076bf0 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.868+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9d8010e70 con 0x7fb9e8076bf0 2024-05-25T06:02:41.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.868+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e8101360 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.868+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d8024ea0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.868+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e8101850 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.869+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb9d801a4c0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.869+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e804bed0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.869+0000 7fb9ddffb700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb9d4036940 0x7fb9d4038df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.869+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb9d804a8b0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.869+0000 7fb9ecbb2700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb9d4036940 0x7fb9d4038df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.869+0000 7fb9ecbb2700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb9d4036940 0x7fb9d4038df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009990 tx=0x7fb9e4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:40.874+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb9d8018940 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.165+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fb9e8101cd0 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.167+0000 7fb9ddffb700 1 -- 172.21.15.160:0/1312892381 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fb9d8018350 con 0x7fb9e8076bf0 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb9d4036940 msgr2=0x7fb9d4038df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:41.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb9d4036940 0x7fb9d4038df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009990 tx=0x7fb9e4006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 msgr2=0x7fb9e8102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8102db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8006d40 tx=0x7fb9d8009250 comp rx=0 tx=0).stop 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 shutdown_connections 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb9d4036940 0x7fb9d4038df0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 --2- 172.21.15.160:0/1312892381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8076bf0 0x7fb9e8102db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 >> 172.21.15.160:0/1312892381 conn(0x7fb9e80fcef0 msgr2=0x7fb9e80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 shutdown_connections 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-05-25T06:02:41.170+0000 7fb9ef617700 1 -- 172.21.15.160:0/1312892381 wait complete. 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:02:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout:Or, if you are only running a single cluster on this host: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: ceph telemetry on 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout:For more information see: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:41.840 INFO:teuthology.orchestra.run.smithi160.stdout:Bootstrap complete. 2024-05-25T06:02:41.868 INFO:tasks.cephadm:Fetching config... 2024-05-25T06:02:41.869 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:02:41.869 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-25T06:02:41.926 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-25T06:02:41.926 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:02:41.926 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-25T06:02:41.982 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-25T06:02:41.982 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:02:41.982 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/keyring of=/dev/stdout 2024-05-25T06:02:42.045 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-25T06:02:42.045 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:02:42.046 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-25T06:02:42.102 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-25T06:02:42.102 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDafYLNn3O30qJnlzWG/t7lfP+9J94R33Xr2IwXwK0QjU2wOsjYI9Yiya0EqJNiUXiPt4wlZCmc3J31YIav234XMpCalQ3eg2Bk7mkQjmWlFq1BxsGzJ9WYwtBW8PyRt2sg7aNBw+7uzI43NPMDqTbAraVq99+0s6X1FaeDEXr1MMBkQmEtBAQupaJU+NusDCO+/V7+iCJY6JCqeu93KvNRZLmoc4Cu5N6KdurpAbQ89ooUzhOJwIMg29fKpzut+QrAOLMs+wIUzSS8HP3G48GyTzn+qBajdU35raE/0P3vDyIXca564mxJ+kOExE0zZdw/nziSz3N3Zjtk3SJZ1Txl9OP88i9sHyWpSiJBAT+2Taf5NJ6st4U/bKwjRy8jqceH/6SGy3ys+hdCJ0pOPVFb7k1scs2mw/rZOCU97vnYCjXpMKRHIZ/YBBsO7mWzVPrEgFmsOwSiN5Y7ARfXMMhHYIPbTTEd7nAMvWp+6bjfeNuKrDkrodCykP8/LXDApyU= ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-25T06:02:42.179 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDafYLNn3O30qJnlzWG/t7lfP+9J94R33Xr2IwXwK0QjU2wOsjYI9Yiya0EqJNiUXiPt4wlZCmc3J31YIav234XMpCalQ3eg2Bk7mkQjmWlFq1BxsGzJ9WYwtBW8PyRt2sg7aNBw+7uzI43NPMDqTbAraVq99+0s6X1FaeDEXr1MMBkQmEtBAQupaJU+NusDCO+/V7+iCJY6JCqeu93KvNRZLmoc4Cu5N6KdurpAbQ89ooUzhOJwIMg29fKpzut+QrAOLMs+wIUzSS8HP3G48GyTzn+qBajdU35raE/0P3vDyIXca564mxJ+kOExE0zZdw/nziSz3N3Zjtk3SJZ1Txl9OP88i9sHyWpSiJBAT+2Taf5NJ6st4U/bKwjRy8jqceH/6SGy3ys+hdCJ0pOPVFb7k1scs2mw/rZOCU97vnYCjXpMKRHIZ/YBBsO7mWzVPrEgFmsOwSiN5Y7ARfXMMhHYIPbTTEd7nAMvWp+6bjfeNuKrDkrodCykP8/LXDApyU= ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:02:42.190 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDafYLNn3O30qJnlzWG/t7lfP+9J94R33Xr2IwXwK0QjU2wOsjYI9Yiya0EqJNiUXiPt4wlZCmc3J31YIav234XMpCalQ3eg2Bk7mkQjmWlFq1BxsGzJ9WYwtBW8PyRt2sg7aNBw+7uzI43NPMDqTbAraVq99+0s6X1FaeDEXr1MMBkQmEtBAQupaJU+NusDCO+/V7+iCJY6JCqeu93KvNRZLmoc4Cu5N6KdurpAbQ89ooUzhOJwIMg29fKpzut+QrAOLMs+wIUzSS8HP3G48GyTzn+qBajdU35raE/0P3vDyIXca564mxJ+kOExE0zZdw/nziSz3N3Zjtk3SJZ1Txl9OP88i9sHyWpSiJBAT+2Taf5NJ6st4U/bKwjRy8jqceH/6SGy3ys+hdCJ0pOPVFb7k1scs2mw/rZOCU97vnYCjXpMKRHIZ/YBBsO7mWzVPrEgFmsOwSiN5Y7ARfXMMhHYIPbTTEd7nAMvWp+6bjfeNuKrDkrodCykP8/LXDApyU= ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-25T06:02:42.231 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDafYLNn3O30qJnlzWG/t7lfP+9J94R33Xr2IwXwK0QjU2wOsjYI9Yiya0EqJNiUXiPt4wlZCmc3J31YIav234XMpCalQ3eg2Bk7mkQjmWlFq1BxsGzJ9WYwtBW8PyRt2sg7aNBw+7uzI43NPMDqTbAraVq99+0s6X1FaeDEXr1MMBkQmEtBAQupaJU+NusDCO+/V7+iCJY6JCqeu93KvNRZLmoc4Cu5N6KdurpAbQ89ooUzhOJwIMg29fKpzut+QrAOLMs+wIUzSS8HP3G48GyTzn+qBajdU35raE/0P3vDyIXca564mxJ+kOExE0zZdw/nziSz3N3Zjtk3SJZ1Txl9OP88i9sHyWpSiJBAT+2Taf5NJ6st4U/bKwjRy8jqceH/6SGy3ys+hdCJ0pOPVFb7k1scs2mw/rZOCU97vnYCjXpMKRHIZ/YBBsO7mWzVPrEgFmsOwSiN5Y7ARfXMMhHYIPbTTEd7nAMvWp+6bjfeNuKrDkrodCykP8/LXDApyU= ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:02:42.244 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-25T06:02:42.374 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:42 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1312892381' entity='client.admin' 2024-05-25T06:02:42.497 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 -- 172.21.15.160:0/1641661004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 msgr2=0x7f55a80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 --2- 172.21.15.160:0/1641661004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a80fce40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55980048c0 tx=0x7f5598009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 -- 172.21.15.160:0/1641661004 shutdown_connections 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 --2- 172.21.15.160:0/1641661004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a80fce40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 -- 172.21.15.160:0/1641661004 >> 172.21.15.160:0/1641661004 conn(0x7f55a80f8190 msgr2=0x7f55a80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 -- 172.21.15.160:0/1641661004 shutdown_connections 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.313+0000 7f55ad2d0700 1 -- 172.21.15.160:0/1641661004 wait complete. 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55ad2d0700 1 Processor -- start 2024-05-25T06:02:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55ad2d0700 1 -- start start 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55ad2d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55ad2d0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8072240 con 0x7f55a80fca30 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55a6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55a6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a8071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34020/0 (socket says 172.21.15.160:34020) 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.314+0000 7f55a6ffd700 1 -- 172.21.15.160:0/2901746670 learned_addr learned my addr 172.21.15.160:0/2901746670 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f55a6ffd700 1 -- 172.21.15.160:0/2901746670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598005040 con 0x7f55a80fca30 2024-05-25T06:02:44.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f55a6ffd700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a8071d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55980096b0 tx=0x7f5598009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f559800bed0 con 0x7f55a80fca30 2024-05-25T06:02:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a8070310 con 0x7f55a80fca30 2024-05-25T06:02:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f559800b5c0 con 0x7f55a80fca30 2024-05-25T06:02:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f559801c5b0 con 0x7f55a80fca30 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.315+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8070800 con 0x7f55a80fca30 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.316+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f559801c820 con 0x7f55a80fca30 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.316+0000 7f558f7fe700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f5590036880 0x7f5590038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.316+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5598049bf0 con 0x7f55a80fca30 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.316+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a805f5c0 con 0x7f55a80fca30 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.316+0000 7f55a67fc700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f5590036880 0x7f5590038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.317+0000 7f55a67fc700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f5590036880 0x7f5590038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f559c009990 tx=0x7f559c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:44.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.321+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5598023300 con 0x7f55a80fca30 2024-05-25T06:02:44.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.514+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f55a804bed0 con 0x7f55a80fca30 2024-05-25T06:02:44.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.517+0000 7f558f7fe700 1 -- 172.21.15.160:0/2901746670 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5598011b60 con 0x7f55a80fca30 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.519+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f5590036880 msgr2=0x7f5590038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.519+0000 7f55ad2d0700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f5590036880 0x7f5590038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f559c009990 tx=0x7f559c006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.519+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 msgr2=0x7f55a8071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.519+0000 7f55ad2d0700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a8071d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55980096b0 tx=0x7f5598009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.519+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 shutdown_connections 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.520+0000 7f55ad2d0700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f5590036880 0x7f5590038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.520+0000 7f55ad2d0700 1 --2- 172.21.15.160:0/2901746670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80fca30 0x7f55a8071d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.520+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 >> 172.21.15.160:0/2901746670 conn(0x7f55a80f8190 msgr2=0x7f55a8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.520+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 shutdown_connections 2024-05-25T06:02:44.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:44.520+0000 7f55ad2d0700 1 -- 172.21.15.160:0/2901746670 wait complete. 2024-05-25T06:02:45.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2901746670' entity='client.admin' 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-25T06:02:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:45 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:02:45.796 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-25T06:02:45.796 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-25T06:02:46.717 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:02:46.842 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:46 smithi160 ceph-mon[29041]: Deploying daemon crash.smithi160 on smithi160 2024-05-25T06:02:49.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.277+0000 7f2de896a700 1 -- 172.21.15.160:0/1710987806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 msgr2=0x7f2de40fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.277+0000 7f2de896a700 1 --2- 172.21.15.160:0/1710987806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de40fbd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40048c0 tx=0x7f2dd4009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 -- 172.21.15.160:0/1710987806 shutdown_connections 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 --2- 172.21.15.160:0/1710987806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de40fbd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 -- 172.21.15.160:0/1710987806 >> 172.21.15.160:0/1710987806 conn(0x7f2de40f7070 msgr2=0x7f2de40f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 -- 172.21.15.160:0/1710987806 shutdown_connections 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 -- 172.21.15.160:0/1710987806 wait complete. 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 Processor -- start 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 -- start start 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de410bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.278+0000 7f2de896a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de410c4f0 con 0x7f2de40fb930 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de410bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de410bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44750/0 (socket says 172.21.15.160:44750) 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de259c700 1 -- 172.21.15.160:0/4060399965 learned_addr learned my addr 172.21.15.160:0/4060399965 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de259c700 1 -- 172.21.15.160:0/4060399965 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd4005040 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de259c700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de410bfb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40096b0 tx=0x7f2dd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dd400bed0 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de410c6f0 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2dd400b5c0 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dd401c5b0 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.279+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de410d260 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.280+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f2dd401c820 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.280+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de4101e40 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.280+0000 7f2ddaffd700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2dd0036880 0x7f2dd0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.280+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2dd4049c80 con 0x7f2de40fb930 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.280+0000 7f2ddbfff700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2dd0036880 0x7f2dd0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:49.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.281+0000 7f2ddbfff700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2dd0036880 0x7f2dd0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009990 tx=0x7f2dcc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:49.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.285+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2dd4050080 con 0x7f2de40fb930 2024-05-25T06:02:49.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.489+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f2de410d4a0 con 0x7f2dd0036880 2024-05-25T06:02:49.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.492+0000 7f2ddaffd700 1 -- 172.21.15.160:0/4060399965 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2de410d4a0 con 0x7f2dd0036880 2024-05-25T06:02:49.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2dd0036880 msgr2=0x7f2dd0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:49.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2dd0036880 0x7f2dd0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009990 tx=0x7f2dcc006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 msgr2=0x7f2de410bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de410bfb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40096b0 tx=0x7f2dd4009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 shutdown_connections 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2dd0036880 0x7f2dd0038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 --2- 172.21.15.160:0/4060399965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de40fb930 0x7f2de410bfb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 >> 172.21.15.160:0/4060399965 conn(0x7f2de40f7070 msgr2=0x7f2de4102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 shutdown_connections 2024-05-25T06:02:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:49.494+0000 7f2de896a700 1 -- 172.21.15.160:0/4060399965 wait complete. 2024-05-25T06:02:50.096 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-05-25T06:02:50.096 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:02:50.096 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-05-25T06:02:50.117 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:02:50.117 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:02:50.175 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-05-25T06:02:50.175 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch host add smithi203 2024-05-25T06:02:50.447 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:02:50.597 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:50 smithi160 ceph-mon[29041]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:50.598 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:50 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:50.598 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:50 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:50.598 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:50 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:50.598 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:50 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:50.598 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:50 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:51.645 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:51 smithi160 ceph-mon[29041]: Deploying daemon node-exporter.smithi160 on smithi160 2024-05-25T06:02:51.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 -- 172.21.15.160:0/1833285091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 msgr2=0x7fb4e80fe4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:51.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 --2- 172.21.15.160:0/1833285091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e80fe4a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80048c0 tx=0x7fb4d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:51.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 -- 172.21.15.160:0/1833285091 shutdown_connections 2024-05-25T06:02:51.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 --2- 172.21.15.160:0/1833285091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e80fe4a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:51.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 -- 172.21.15.160:0/1833285091 >> 172.21.15.160:0/1833285091 conn(0x7fb4e80f6850 msgr2=0x7fb4e80f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:51.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 -- 172.21.15.160:0/1833285091 shutdown_connections 2024-05-25T06:02:51.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.947+0000 7fb4ee233700 1 -- 172.21.15.160:0/1833285091 wait complete. 2024-05-25T06:02:51.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4ee233700 1 Processor -- start 2024-05-25T06:02:51.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4ee233700 1 -- start start 2024-05-25T06:02:51.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4ee233700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:51.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4ee233700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e810d5f0 con 0x7fb4e80fe090 2024-05-25T06:02:51.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44770/0 (socket says 172.21.15.160:44770) 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.948+0000 7fb4e77fe700 1 -- 172.21.15.160:0/1692689598 learned_addr learned my addr 172.21.15.160:0/1692689598 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4e77fe700 1 -- 172.21.15.160:0/1692689598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d8005040 con 0x7fb4e80fe090 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4e77fe700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e810d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80096b0 tx=0x7fb4d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4d800bed0 con 0x7fb4e80fe090 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4d800b5c0 con 0x7fb4e80fe090 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e810d7f0 con 0x7fb4e80fe090 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4d801c5b0 con 0x7fb4e80fe090 2024-05-25T06:02:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.949+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e810e360 con 0x7fb4e80fe090 2024-05-25T06:02:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.950+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb4d801c820 con 0x7fb4e80fe090 2024-05-25T06:02:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.950+0000 7fb4e4ff9700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb4d4036880 0x7fb4d4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.950+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb4d804aaa0 con 0x7fb4e80fe090 2024-05-25T06:02:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.950+0000 7fb4deffd700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb4d4036880 0x7fb4d4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.950+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e8102f60 con 0x7fb4e80fe090 2024-05-25T06:02:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.950+0000 7fb4deffd700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb4d4036880 0x7fb4d4038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009990 tx=0x7fb4d0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:51.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:51.955+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb4d8023300 con 0x7fb4e80fe090 2024-05-25T06:02:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:52.165+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4e8102940 con 0x7fb4d4036880 2024-05-25T06:02:52.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:52 smithi160 ceph-mon[29041]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:02:54.206 INFO:teuthology.orchestra.run.smithi160.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-05-25T06:02:54.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.205+0000 7fb4e4ff9700 1 -- 172.21.15.160:0/1692689598 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb4d0009590 con 0x7fb4d4036880 2024-05-25T06:02:54.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb4d4036880 msgr2=0x7fb4d4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:54.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb4d4036880 0x7fb4d4038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009990 tx=0x7fb4d0006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:54.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 msgr2=0x7fb4e810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:54.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e810d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80096b0 tx=0x7fb4d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:54.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 shutdown_connections 2024-05-25T06:02:54.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb4d4036880 0x7fb4d4038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:54.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 --2- 172.21.15.160:0/1692689598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4e80fe090 0x7fb4e810d0b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:54.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 >> 172.21.15.160:0/1692689598 conn(0x7fb4e80f6850 msgr2=0x7fb4e8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:54.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 shutdown_connections 2024-05-25T06:02:54.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:54.208+0000 7fb4ee233700 1 -- 172.21.15.160:0/1692689598 wait complete. 2024-05-25T06:02:54.442 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:54 smithi160 ceph-mon[29041]: Deploying cephadm binary to smithi203 2024-05-25T06:02:54.442 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:54 smithi160 ceph-mon[29041]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:02:54.442 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:54 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:55.262 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-25T06:02:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:55 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:55.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:55 smithi160 ceph-mon[29041]: Added host smithi203 2024-05-25T06:02:55.775 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:02:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:56 smithi160 ceph-mon[29041]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:02:56.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:56 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:56.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:56 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:56.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:56 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:56.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:56 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:57.257 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:57 smithi160 ceph-mon[29041]: Deploying daemon alertmanager.smithi160 on smithi160 2024-05-25T06:02:57.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.352+0000 7f961c8af700 1 -- 172.21.15.160:0/2341166065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 msgr2=0x7f96180ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:57.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.352+0000 7f961c8af700 1 --2- 172.21.15.160:0/2341166065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180ff630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f960c0048c0 tx=0x7f960c009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:57.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.353+0000 7f961c8af700 1 -- 172.21.15.160:0/2341166065 shutdown_connections 2024-05-25T06:02:57.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.353+0000 7f961c8af700 1 --2- 172.21.15.160:0/2341166065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180ff630 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:57.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.353+0000 7f961c8af700 1 -- 172.21.15.160:0/2341166065 >> 172.21.15.160:0/2341166065 conn(0x7f96180f70a0 msgr2=0x7f96180f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:57.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.353+0000 7f961c8af700 1 -- 172.21.15.160:0/2341166065 shutdown_connections 2024-05-25T06:02:57.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.353+0000 7f961c8af700 1 -- 172.21.15.160:0/2341166065 wait complete. 2024-05-25T06:02:57.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.353+0000 7f961c8af700 1 Processor -- start 2024-05-25T06:02:57.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961c8af700 1 -- start start 2024-05-25T06:02:57.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961c8af700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:57.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961c8af700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180fd4c0 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40394/0 (socket says 172.21.15.160:40394) 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961659c700 1 -- 172.21.15.160:0/693145815 learned_addr learned my addr 172.21.15.160:0/693145815 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961659c700 1 -- 172.21.15.160:0/693145815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c005040 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.354+0000 7f961659c700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180fcf80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f960c009680 tx=0x7f960c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.355+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f960c00bed0 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.355+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f960c00b5c0 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.355+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f960c01c5b0 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.355+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96180fb590 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.355+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96180fba80 con 0x7f96180669c0 2024-05-25T06:02:57.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.355+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f960c01c710 con 0x7f96180669c0 2024-05-25T06:02:57.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.356+0000 7f9606ffd700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f96000368d0 0x7f9600038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:02:57.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.356+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961805f5c0 con 0x7f96180669c0 2024-05-25T06:02:57.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.356+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f960c049960 con 0x7f96180669c0 2024-05-25T06:02:57.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.356+0000 7f9615d9b700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f96000368d0 0x7f9600038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:02:57.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.356+0000 7f9615d9b700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f96000368d0 0x7f9600038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9608009990 tx=0x7f9608006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:02:57.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.361+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f960c023300 con 0x7f96180669c0 2024-05-25T06:02:57.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.562+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9618106480 con 0x7f96000368d0 2024-05-25T06:02:57.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.563+0000 7f9606ffd700 1 -- 172.21.15.160:0/693145815 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f9618106480 con 0x7f96000368d0 2024-05-25T06:02:57.563 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:02:57.563 INFO:teuthology.orchestra.run.smithi160.stdout:[{"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-05-25T06:02:57.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f96000368d0 msgr2=0x7f9600038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f96000368d0 0x7f9600038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9608009990 tx=0x7f9608006e40 comp rx=0 tx=0).stop 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 msgr2=0x7f96180fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180fcf80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f960c009680 tx=0x7f960c009e30 comp rx=0 tx=0).stop 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 shutdown_connections 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f96000368d0 0x7f9600038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 --2- 172.21.15.160:0/693145815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96180669c0 0x7f96180fcf80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 >> 172.21.15.160:0/693145815 conn(0x7f96180f70a0 msgr2=0x7f9618103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 shutdown_connections 2024-05-25T06:02:57.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:02:57.565+0000 7f961c8af700 1 -- 172.21.15.160:0/693145815 wait complete. 2024-05-25T06:02:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:58 smithi160 ceph-mon[29041]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:02:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:58 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:02:58.751 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-25T06:02:58.752 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-25T06:02:59.436 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:02:59.560 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:02:59 smithi160 ceph-mon[29041]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-25T06:03:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:00 smithi160 ceph-mon[29041]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:02 smithi160 ceph-mon[29041]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:02.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 -- 172.21.15.160:0/638678919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 msgr2=0x7f16200fd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 --2- 172.21.15.160:0/638678919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fd360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16140048c0 tx=0x7f1614009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 -- 172.21.15.160:0/638678919 shutdown_connections 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 --2- 172.21.15.160:0/638678919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fd360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 -- 172.21.15.160:0/638678919 >> 172.21.15.160:0/638678919 conn(0x7f16200f6850 msgr2=0x7f16200f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 -- 172.21.15.160:0/638678919 shutdown_connections 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 -- 172.21.15.160:0/638678919 wait complete. 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 Processor -- start 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.917+0000 7f1626ddb700 1 -- start start 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1626ddb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1626ddb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162010fd30 con 0x7f16200fcf50 2024-05-25T06:03:02.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1624b77700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1624b77700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40416/0 (socket says 172.21.15.160:40416) 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1624b77700 1 -- 172.21.15.160:0/2743436208 learned_addr learned my addr 172.21.15.160:0/2743436208 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1624b77700 1 -- 172.21.15.160:0/2743436208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614005040 con 0x7f16200fcf50 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1624b77700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fb760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1614009680 tx=0x7f1614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f161400b5c0 con 0x7f16200fcf50 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f161400be70 con 0x7f16200fcf50 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162010fed0 con 0x7f16200fcf50 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16140135a0 con 0x7f16200fcf50 2024-05-25T06:03:02.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.918+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1620110360 con 0x7f16200fcf50 2024-05-25T06:03:02.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.919+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f1614013860 con 0x7f16200fcf50 2024-05-25T06:03:02.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.919+0000 7f161d7fa700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f1610036880 0x7f1610038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:02.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.920+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1614049350 con 0x7f16200fcf50 2024-05-25T06:03:02.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.920+0000 7f161ffff700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f1610036880 0x7f1610038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:02.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.920+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1620105900 con 0x7f16200fcf50 2024-05-25T06:03:02.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.920+0000 7f161ffff700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f1610036880 0x7f1610038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f160c009990 tx=0x7f160c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:02.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:02.925+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1614021030 con 0x7f16200fcf50 2024-05-25T06:03:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.137+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1620101800 con 0x7f16200fcf50 2024-05-25T06:03:03.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.231+0000 7f161d7fa700 1 -- 172.21.15.160:0/2743436208 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1614013bb0 con 0x7f16200fcf50 2024-05-25T06:03:03.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f1610036880 msgr2=0x7f1610038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:03.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f1610036880 0x7f1610038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f160c009990 tx=0x7f160c006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 msgr2=0x7f16200fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fb760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1614009680 tx=0x7f1614009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 shutdown_connections 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f1610036880 0x7f1610038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 --2- 172.21.15.160:0/2743436208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16200fcf50 0x7f16200fb760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 >> 172.21.15.160:0/2743436208 conn(0x7f16200f6850 msgr2=0x7f1620102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 shutdown_connections 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:03:03.233+0000 7f1626ddb700 1 -- 172.21.15.160:0/2743436208 wait complete. 2024-05-25T06:03:03.234 INFO:teuthology.orchestra.run.smithi160.stderr:adjusted tunables profile to default 2024-05-25T06:03:03.398 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:03 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2743436208' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-25T06:03:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:04 smithi160 ceph-mon[29041]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:04 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2743436208' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-25T06:03:04.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:04 smithi160 ceph-mon[29041]: osdmap e4: 0 total, 0 up, 0 in 2024-05-25T06:03:05.455 INFO:tasks.cephadm:Adding mon.smithi160 on smithi160 2024-05-25T06:03:05.455 INFO:tasks.cephadm:Adding mon.smithi203 on smithi203 2024-05-25T06:03:05.455 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch apply mon '2;smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203' 2024-05-25T06:03:05.706 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-25T06:03:06.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:06 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 -- 172.21.15.203:0/1633589514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 msgr2=0x7f940c0ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 --2- 172.21.15.203:0/1633589514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c0ff630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0048c0 tx=0x7f93fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 -- 172.21.15.203:0/1633589514 shutdown_connections 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 --2- 172.21.15.203:0/1633589514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c0ff630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 -- 172.21.15.203:0/1633589514 >> 172.21.15.203:0/1633589514 conn(0x7f940c0f70a0 msgr2=0x7f940c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 -- 172.21.15.203:0/1633589514 shutdown_connections 2024-05-25T06:03:07.356 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 -- 172.21.15.203:0/1633589514 wait complete. 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.356+0000 7f9412194700 1 Processor -- start 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f9412194700 1 -- start start 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f9412194700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f9412194700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c10d600 con 0x7f940c0669c0 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f940b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f940b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:41658/0 (socket says 172.21.15.203:41658) 2024-05-25T06:03:07.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f940b7fe700 1 -- 172.21.15.203:0/2105665298 learned_addr learned my addr 172.21.15.203:0/2105665298 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f940b7fe700 1 -- 172.21.15.203:0/2105665298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc005040 con 0x7f940c0669c0 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.357+0000 7f940b7fe700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c10d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009680 tx=0x7f93fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.358+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93fc00bed0 con 0x7f940c0669c0 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.358+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93fc00b5c0 con 0x7f940c0669c0 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.358+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c10d800 con 0x7f940c0669c0 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.358+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93fc01c5b0 con 0x7f940c0669c0 2024-05-25T06:03:07.359 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.358+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c10e370 con 0x7f940c0669c0 2024-05-25T06:03:07.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.358+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f93fc01c710 con 0x7f940c0669c0 2024-05-25T06:03:07.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.359+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c102f40 con 0x7f940c0669c0 2024-05-25T06:03:07.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.359+0000 7f93f3fff700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f93f4036880 0x7f93f4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:07.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.359+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f93fc049b00 con 0x7f940c0669c0 2024-05-25T06:03:07.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.359+0000 7f940affd700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f93f4036880 0x7f93f4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:07.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.359+0000 7f940affd700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f93f4036880 0x7f93f4038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9400009990 tx=0x7f9400006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:07.364 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.364+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f93fc025400 con 0x7f940c0669c0 2024-05-25T06:03:07.558 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.557+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 --> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203", "target": ["mon-mgr", ""]}) v1 -- 0x7f940c10e670 con 0x7f93f4036880 2024-05-25T06:03:07.565 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.565+0000 7f93f3fff700 1 -- 172.21.15.203:0/2105665298 <== mgr.14162 v2:172.21.15.160:6800/3983131949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f940c10e670 con 0x7f93f4036880 2024-05-25T06:03:07.565 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f93f4036880 msgr2=0x7f93f4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f93f4036880 0x7f93f4038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9400009990 tx=0x7f9400006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 msgr2=0x7f940c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c10d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009680 tx=0x7f93fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 shutdown_connections 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f93f4036880 0x7f93f4038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 --2- 172.21.15.203:0/2105665298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940c0669c0 0x7f940c10d0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:07.568 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 >> 172.21.15.203:0/2105665298 conn(0x7f940c0f70a0 msgr2=0x7f940c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:07.568 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 shutdown_connections 2024-05-25T06:03:07.568 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:07.567+0000 7f9412194700 1 -- 172.21.15.203:0/2105665298 wait complete. 2024-05-25T06:03:07.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:07 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-25T06:03:07.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:07 smithi160 ceph-mon[29041]: Deploying daemon grafana.smithi160 on smithi160 2024-05-25T06:03:08.185 DEBUG:teuthology.orchestra.run.smithi203:mon.smithi203> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203.service 2024-05-25T06:03:08.188 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:08.188 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:08.467 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:08.515 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:08 smithi160 ceph-mon[29041]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:08 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:08.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:08 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:09.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:09 smithi160 ceph-mon[29041]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:03:09.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:09 smithi160 ceph-mon[29041]: Saving service mon spec with placement smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203;count:2 2024-05-25T06:03:10.053 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 -- 172.21.15.203:0/2518237496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 msgr2=0x7fb3500fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:10.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 --2- 172.21.15.203:0/2518237496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb3500fce40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3400048c0 tx=0x7fb340009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:10.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 -- 172.21.15.203:0/2518237496 shutdown_connections 2024-05-25T06:03:10.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 --2- 172.21.15.203:0/2518237496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb3500fce40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:10.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 -- 172.21.15.203:0/2518237496 >> 172.21.15.203:0/2518237496 conn(0x7fb3500f8170 msgr2=0x7fb3500fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:10.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 -- 172.21.15.203:0/2518237496 shutdown_connections 2024-05-25T06:03:10.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.053+0000 7fb3552d2700 1 -- 172.21.15.203:0/2518237496 wait complete. 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb3552d2700 1 Processor -- start 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb3552d2700 1 -- start start 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb3552d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb35010f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb3552d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35010f800 con 0x7fb3500fca30 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb34effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb35010f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb34effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb35010f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:41666/0 (socket says 172.21.15.203:41666) 2024-05-25T06:03:10.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb34effd700 1 -- 172.21.15.203:0/2613563092 learned_addr learned my addr 172.21.15.203:0/2613563092 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.054+0000 7fb34effd700 1 -- 172.21.15.203:0/2613563092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340005040 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.055+0000 7fb34effd700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb35010f2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3400096b0 tx=0x7fb340009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.055+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb34000bed0 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.055+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35010fa00 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.055+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb34000b5c0 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.055+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb34001c5b0 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.055+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb350110570 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.056+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb340013420 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.056+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb350102f40 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.056+0000 7fb3377fe700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb338036880 0x7fb338038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.056+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb340049b10 con 0x7fb3500fca30 2024-05-25T06:03:10.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.056+0000 7fb34e7fc700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb338036880 0x7fb338038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:10.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.056+0000 7fb34e7fc700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb338036880 0x7fb338038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb344009990 tx=0x7fb344006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:10.065 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.065+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb340025400 con 0x7fb3500fca30 2024-05-25T06:03:10.324 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.323+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb3501107b0 con 0x7fb3500fca30 2024-05-25T06:03:10.324 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.324+0000 7fb3377fe700 1 -- 172.21.15.203:0/2613563092 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fb340004d30 con 0x7fb3500fca30 2024-05-25T06:03:10.324 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:10.324 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb338036880 msgr2=0x7fb338038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb338036880 0x7fb338038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb344009990 tx=0x7fb344006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 msgr2=0x7fb35010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb35010f2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3400096b0 tx=0x7fb340009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 shutdown_connections 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fb338036880 0x7fb338038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 --2- 172.21.15.203:0/2613563092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3500fca30 0x7fb35010f2c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:10.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 >> 172.21.15.203:0/2613563092 conn(0x7fb3500f8170 msgr2=0x7fb3500fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:10.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 shutdown_connections 2024-05-25T06:03:10.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:10.326+0000 7fb3552d2700 1 -- 172.21.15.203:0/2613563092 wait complete. 2024-05-25T06:03:10.327 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:10 smithi160 ceph-mon[29041]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:11.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:11 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2613563092' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:11.980 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:11.980 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:12.234 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:12.283 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:12 smithi160 ceph-mon[29041]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:13.669 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 -- 172.21.15.203:0/959729247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 msgr2=0x7f60480fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:13.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 --2- 172.21.15.203:0/959729247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f60480fe490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60440048c0 tx=0x7f6044009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:13.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 -- 172.21.15.203:0/959729247 shutdown_connections 2024-05-25T06:03:13.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 --2- 172.21.15.203:0/959729247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f60480fe490 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:13.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 -- 172.21.15.203:0/959729247 >> 172.21.15.203:0/959729247 conn(0x7f60480f6800 msgr2=0x7f60480f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:13.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 -- 172.21.15.203:0/959729247 shutdown_connections 2024-05-25T06:03:13.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.668+0000 7f604fbf1700 1 -- 172.21.15.203:0/959729247 wait complete. 2024-05-25T06:03:13.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604fbf1700 1 Processor -- start 2024-05-25T06:03:13.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604fbf1700 1 -- start start 2024-05-25T06:03:13.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604fbf1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f604810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:13.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604fbf1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604810d5e0 con 0x7f60480fe080 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604d98d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f604810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604d98d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f604810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:54432/0 (socket says 172.21.15.203:54432) 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.669+0000 7f604d98d700 1 -- 172.21.15.203:0/737399514 learned_addr learned my addr 172.21.15.203:0/737399514 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f604d98d700 1 -- 172.21.15.203:0/737399514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044005040 con 0x7f60480fe080 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f604d98d700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f604810d0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f60440096b0 tx=0x7f6044009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f604400bed0 con 0x7f60480fe080 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604810d7e0 con 0x7f60480fe080 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f604400b5c0 con 0x7f60480fe080 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f604401c5b0 con 0x7f60480fe080 2024-05-25T06:03:13.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.670+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604810e350 con 0x7f60480fe080 2024-05-25T06:03:13.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.671+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6048102f50 con 0x7f60480fe080 2024-05-25T06:03:13.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.671+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f6044004d30 con 0x7f60480fe080 2024-05-25T06:03:13.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.671+0000 7f603e7fc700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6034036880 0x7f6034038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:13.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.671+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6044049a50 con 0x7f60480fe080 2024-05-25T06:03:13.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.671+0000 7f604d18c700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6034036880 0x7f6034038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:13.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.672+0000 7f604d18c700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6034036880 0x7f6034038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:13.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.677+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6044011940 con 0x7f60480fe080 2024-05-25T06:03:13.935 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.934+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f604810ed60 con 0x7f60480fe080 2024-05-25T06:03:13.936 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.935+0000 7f603e7fc700 1 -- 172.21.15.203:0/737399514 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f6044011370 con 0x7f60480fe080 2024-05-25T06:03:13.936 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:13.936 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6034036880 msgr2=0x7f6034038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6034036880 0x7f6034038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 msgr2=0x7f604810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f604810d0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f60440096b0 tx=0x7f6044009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 shutdown_connections 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f6034036880 0x7f6034038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:13.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 --2- 172.21.15.203:0/737399514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60480fe080 0x7f604810d0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:13.939 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 >> 172.21.15.203:0/737399514 conn(0x7f60480f6800 msgr2=0x7f6048103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:13.939 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 shutdown_connections 2024-05-25T06:03:13.939 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:13.938+0000 7f604fbf1700 1 -- 172.21.15.203:0/737399514 wait complete. 2024-05-25T06:03:13.939 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:14.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:14 smithi160 ceph-mon[29041]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:14 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/737399514' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:15.525 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:15.526 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:15.777 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:15.825 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:16.538 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:16 smithi160 ceph-mon[29041]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:17.314 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 -- 172.21.15.203:0/2158841056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 msgr2=0x7f2e8c0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 --2- 172.21.15.203:0/2158841056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c0fce40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e800048c0 tx=0x7f2e80009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 -- 172.21.15.203:0/2158841056 shutdown_connections 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 --2- 172.21.15.203:0/2158841056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c0fce40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 -- 172.21.15.203:0/2158841056 >> 172.21.15.203:0/2158841056 conn(0x7f2e8c0f8170 msgr2=0x7f2e8c0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 -- 172.21.15.203:0/2158841056 shutdown_connections 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 -- 172.21.15.203:0/2158841056 wait complete. 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 Processor -- start 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.314+0000 7f2e92fba700 1 -- start start 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e92fba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:17.315 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e92fba700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c10d5f0 con 0x7f2e8c0fca30 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e90d56700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e90d56700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:54444/0 (socket says 172.21.15.203:54444) 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e90d56700 1 -- 172.21.15.203:0/376958826 learned_addr learned my addr 172.21.15.203:0/376958826 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e90d56700 1 -- 172.21.15.203:0/376958826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80005040 con 0x7f2e8c0fca30 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e90d56700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c10d0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e800096b0 tx=0x7f2e80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.315+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e8000bed0 con 0x7f2e8c0fca30 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.316+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c10d7f0 con 0x7f2e8c0fca30 2024-05-25T06:03:17.317 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.316+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e8000b5c0 con 0x7f2e8c0fca30 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.316+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e8001c5b0 con 0x7f2e8c0fca30 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.316+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c10e360 con 0x7f2e8c0fca30 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.316+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f2e80013420 con 0x7f2e8c0fca30 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.316+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c102f40 con 0x7f2e8c0fca30 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.317+0000 7f2e897fa700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2e7c0368d0 0x7f2e7c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.317+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2e80049ac0 con 0x7f2e8c0fca30 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.317+0000 7f2e8bfff700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2e7c0368d0 0x7f2e7c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:17.318 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.317+0000 7f2e8bfff700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2e7c0368d0 0x7f2e7c038d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009990 tx=0x7f2e78006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:17.322 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.322+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2e80025400 con 0x7f2e8c0fca30 2024-05-25T06:03:17.582 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.582+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2e8c10ed70 con 0x7f2e8c0fca30 2024-05-25T06:03:17.583 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.583+0000 7f2e897fa700 1 -- 172.21.15.203:0/376958826 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2e80004d30 con 0x7f2e8c0fca30 2024-05-25T06:03:17.583 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:17.584 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:17.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.585+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2e7c0368d0 msgr2=0x7f2e7c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:17.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.585+0000 7f2e92fba700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2e7c0368d0 0x7f2e7c038d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009990 tx=0x7f2e78006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:17.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.585+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 msgr2=0x7f2e8c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:17.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.585+0000 7f2e92fba700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c10d0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e800096b0 tx=0x7f2e80009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:17.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.586+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 shutdown_connections 2024-05-25T06:03:17.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.586+0000 7f2e92fba700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2e7c0368d0 0x7f2e7c038d80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:17.587 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.586+0000 7f2e92fba700 1 --2- 172.21.15.203:0/376958826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e8c0fca30 0x7f2e8c10d0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:17.587 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.586+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 >> 172.21.15.203:0/376958826 conn(0x7f2e8c0f8170 msgr2=0x7f2e8c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:17.587 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.586+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 shutdown_connections 2024-05-25T06:03:17.587 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:17.586+0000 7f2e92fba700 1 -- 172.21.15.203:0/376958826 wait complete. 2024-05-25T06:03:17.587 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:18 smithi160 ceph-mon[29041]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:18 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/376958826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:19.170 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:19.170 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:19.420 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:19.468 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:20.191 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:20.191 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:20.191 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:20.191 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:20.191 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:20.192 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:20.192 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: Deploying daemon prometheus.smithi160 on smithi160 2024-05-25T06:03:20.192 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:20 smithi160 ceph-mon[29041]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:20.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 -- 172.21.15.203:0/2239676283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 msgr2=0x7fccf00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:20.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 --2- 172.21.15.203:0/2239676283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf00fce40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fccec0048c0 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:20.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 -- 172.21.15.203:0/2239676283 shutdown_connections 2024-05-25T06:03:20.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 --2- 172.21.15.203:0/2239676283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf00fce40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:20.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 -- 172.21.15.203:0/2239676283 >> 172.21.15.203:0/2239676283 conn(0x7fccf00f8170 msgr2=0x7fccf00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:20.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 -- 172.21.15.203:0/2239676283 shutdown_connections 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.926+0000 7fccf7b8c700 1 -- 172.21.15.203:0/2239676283 wait complete. 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf7b8c700 1 Processor -- start 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf7b8c700 1 -- start start 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf7b8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf7b8c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf010d5f0 con 0x7fccf00fca30 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf5928700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf5928700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:54458/0 (socket says 172.21.15.203:54458) 2024-05-25T06:03:20.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf5928700 1 -- 172.21.15.203:0/1823448785 learned_addr learned my addr 172.21.15.203:0/1823448785 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf5928700 1 -- 172.21.15.203:0/1823448785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec005040 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.927+0000 7fccf5928700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf010d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fccec0096b0 tx=0x7fccec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.928+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccec00bed0 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.928+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccec00b5c0 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.928+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf010d7f0 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.928+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccec01c5b0 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.928+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf010e360 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.928+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fccec01c820 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.929+0000 7fcce67fc700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fccdc036880 0x7fccdc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.929+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fccec049a80 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.929+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf0102f40 con 0x7fccf00fca30 2024-05-25T06:03:20.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.929+0000 7fccf5127700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fccdc036880 0x7fccdc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:20.931 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.929+0000 7fccf5127700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fccdc036880 0x7fccdc038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009990 tx=0x7fcce0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:20.934 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:20.934+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fccec023300 con 0x7fccf00fca30 2024-05-25T06:03:21.197 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.197+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fccf005f5c0 con 0x7fccf00fca30 2024-05-25T06:03:21.197 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.197+0000 7fcce67fc700 1 -- 172.21.15.203:0/1823448785 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fccf005f5c0 con 0x7fccf00fca30 2024-05-25T06:03:21.198 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:21.198 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fccdc036880 msgr2=0x7fccdc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fccdc036880 0x7fccdc038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009990 tx=0x7fcce0006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 msgr2=0x7fccf010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf010d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fccec0096b0 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 shutdown_connections 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fccdc036880 0x7fccdc038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 --2- 172.21.15.203:0/1823448785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf00fca30 0x7fccf010d0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 >> 172.21.15.203:0/1823448785 conn(0x7fccf00f8170 msgr2=0x7fccf0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.199+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 shutdown_connections 2024-05-25T06:03:21.200 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:21.200+0000 7fccf7b8c700 1 -- 172.21.15.203:0/1823448785 wait complete. 2024-05-25T06:03:21.201 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:22 smithi160 ceph-mon[29041]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:22 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1823448785' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:22.785 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:22.786 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:23.046 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:23.095 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:24 smithi160 ceph-mon[29041]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:24 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.578+0000 7f8817305700 1 -- 172.21.15.203:0/614063142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 msgr2=0x7f88100fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.578+0000 7f8817305700 1 --2- 172.21.15.203:0/614063142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f88100fb4d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f880c0049d0 tx=0x7f880c00af40 comp rx=0 tx=0).stop 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 -- 172.21.15.203:0/614063142 shutdown_connections 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 --2- 172.21.15.203:0/614063142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f88100fb4d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 -- 172.21.15.203:0/614063142 >> 172.21.15.203:0/614063142 conn(0x7f88100f6800 msgr2=0x7f88100f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 -- 172.21.15.203:0/614063142 shutdown_connections 2024-05-25T06:03:24.579 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 -- 172.21.15.203:0/614063142 wait complete. 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 Processor -- start 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 -- start start 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f881010d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.579+0000 7f8817305700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881010d6b0 con 0x7f88100fb0c0 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f88150a1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f881010d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f88150a1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f881010d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:57094/0 (socket says 172.21.15.203:57094) 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f88150a1700 1 -- 172.21.15.203:0/2635412007 learned_addr learned my addr 172.21.15.203:0/2635412007 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f88150a1700 1 -- 172.21.15.203:0/2635412007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c005040 con 0x7f88100fb0c0 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f88150a1700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f881010d170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f880c004710 tx=0x7f880c00af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880c00c8c0 con 0x7f88100fb0c0 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881010d8b0 con 0x7f88100fb0c0 2024-05-25T06:03:24.581 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f880c00a8b0 con 0x7f88100fb0c0 2024-05-25T06:03:24.582 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880c014750 con 0x7f88100fb0c0 2024-05-25T06:03:24.582 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.580+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881010e420 con 0x7f88100fb0c0 2024-05-25T06:03:24.582 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.581+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f880c0149d0 con 0x7f88100fb0c0 2024-05-25T06:03:24.583 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.581+0000 7f8805ffb700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f87fc036880 0x7f87fc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:24.583 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.581+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f880c04a300 con 0x7f88100fb0c0 2024-05-25T06:03:24.583 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.581+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8810100d50 con 0x7f88100fb0c0 2024-05-25T06:03:24.583 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.582+0000 7f88148a0700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f87fc036880 0x7f87fc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:24.583 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.582+0000 7f88148a0700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f87fc036880 0x7f87fc038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8800009990 tx=0x7f8800006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:24.586 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.586+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f880c01c3e0 con 0x7f88100fb0c0 2024-05-25T06:03:24.844 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.843+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8810100620 con 0x7f88100fb0c0 2024-05-25T06:03:24.844 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.844+0000 7f8805ffb700 1 -- 172.21.15.203:0/2635412007 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f880c012370 con 0x7f88100fb0c0 2024-05-25T06:03:24.844 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:24.844 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f87fc036880 msgr2=0x7f87fc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f87fc036880 0x7f87fc038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8800009990 tx=0x7f8800006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 msgr2=0x7f881010d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f881010d170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f880c004710 tx=0x7f880c00af40 comp rx=0 tx=0).stop 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 shutdown_connections 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f87fc036880 0x7f87fc038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:24.846 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 --2- 172.21.15.203:0/2635412007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88100fb0c0 0x7f881010d170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:24.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 >> 172.21.15.203:0/2635412007 conn(0x7f88100f6800 msgr2=0x7f8810101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:24.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 shutdown_connections 2024-05-25T06:03:24.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:24.846+0000 7f8817305700 1 -- 172.21.15.203:0/2635412007 wait complete. 2024-05-25T06:03:24.847 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:25 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2635412007' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:26.412 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:26.412 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:26 smithi160 ceph-mon[29041]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:26.663 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:26.711 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 -- 172.21.15.203:0/1112853782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 msgr2=0x7f23280fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 --2- 172.21.15.203:0/1112853782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f23280fbd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23240048c0 tx=0x7f2324009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 -- 172.21.15.203:0/1112853782 shutdown_connections 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 --2- 172.21.15.203:0/1112853782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f23280fbd50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 -- 172.21.15.203:0/1112853782 >> 172.21.15.203:0/1112853782 conn(0x7f23280f70a0 msgr2=0x7f23280f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 -- 172.21.15.203:0/1112853782 shutdown_connections 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 -- 172.21.15.203:0/1112853782 wait complete. 2024-05-25T06:03:28.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 Processor -- start 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.018+0000 7f23303ac700 1 -- start start 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f23303ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f2328065bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f23303ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232810fdb0 con 0x7f23280fb940 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f232e148700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f2328065bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f232e148700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f2328065bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:57104/0 (socket says 172.21.15.203:57104) 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f232e148700 1 -- 172.21.15.203:0/3322279985 learned_addr learned my addr 172.21.15.203:0/3322279985 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f232e148700 1 -- 172.21.15.203:0/3322279985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2324005040 con 0x7f23280fb940 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f232e148700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f2328065bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f23240096b0 tx=0x7f2324009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.019+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f232400bed0 con 0x7f23280fb940 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.020+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232810fef0 con 0x7f23280fb940 2024-05-25T06:03:28.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.020+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f232400b5c0 con 0x7f23280fb940 2024-05-25T06:03:28.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.020+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f232401c5b0 con 0x7f23280fb940 2024-05-25T06:03:28.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.020+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23281103e0 con 0x7f23280fb940 2024-05-25T06:03:28.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.020+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2328105900 con 0x7f23280fb940 2024-05-25T06:03:28.023 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.020+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f2324013a80 con 0x7f23280fb940 2024-05-25T06:03:28.023 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.021+0000 7f231affd700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2314036880 0x7f2314038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:28.023 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.021+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2324049a20 con 0x7f23280fb940 2024-05-25T06:03:28.023 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.021+0000 7f232d947700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2314036880 0x7f2314038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:28.023 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.021+0000 7f232d947700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2314036880 0x7f2314038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f231c009990 tx=0x7f231c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:28.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.025+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2324011940 con 0x7f23280fb940 2024-05-25T06:03:28.288 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.287+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f232805f5c0 con 0x7f23280fb940 2024-05-25T06:03:28.288 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.288+0000 7f231affd700 1 -- 172.21.15.203:0/3322279985 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2324011370 con 0x7f23280fb940 2024-05-25T06:03:28.289 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:28.289 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2314036880 msgr2=0x7f2314038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2314036880 0x7f2314038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f231c009990 tx=0x7f231c006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 msgr2=0x7f2328065bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f2328065bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f23240096b0 tx=0x7f2324009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 shutdown_connections 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f2314036880 0x7f2314038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 --2- 172.21.15.203:0/3322279985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23280fb940 0x7f2328065bb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 >> 172.21.15.203:0/3322279985 conn(0x7f23280f70a0 msgr2=0x7f2328102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 shutdown_connections 2024-05-25T06:03:28.291 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:28.290+0000 7f23303ac700 1 -- 172.21.15.203:0/3322279985 wait complete. 2024-05-25T06:03:28.292 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:28 smithi160 ceph-mon[29041]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:29 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/3322279985' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:29.824 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:29.824 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:30.075 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:30.123 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:30.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:30 smithi160 ceph-mon[29041]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:31.553 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/1516527360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 msgr2=0x7f0d340fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 --2- 172.21.15.203:0/1516527360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d340fd410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d300048c0 tx=0x7f0d30009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/1516527360 shutdown_connections 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 --2- 172.21.15.203:0/1516527360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d340fd410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/1516527360 >> 172.21.15.203:0/1516527360 conn(0x7f0d340f7070 msgr2=0x7f0d340f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/1516527360 shutdown_connections 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/1516527360 wait complete. 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.553+0000 7f0d3b0c9700 1 Processor -- start 2024-05-25T06:03:31.554 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d3b0c9700 1 -- start start 2024-05-25T06:03:31.555 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d3b0c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d34113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:31.555 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d3b0c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34113c80 con 0x7f0d340669a0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d38e65700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d34113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d38e65700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d34113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:57130/0 (socket says 172.21.15.203:57130) 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d38e65700 1 -- 172.21.15.203:0/2440192787 learned_addr learned my addr 172.21.15.203:0/2440192787 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.554+0000 7f0d38e65700 1 -- 172.21.15.203:0/2440192787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30005040 con 0x7f0d340669a0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.555+0000 7f0d38e65700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d34113740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009680 tx=0x7f0d30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.555+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d3000bed0 con 0x7f0d340669a0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.555+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d34113e80 con 0x7f0d340669a0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.555+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d3000b5c0 con 0x7f0d340669a0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.555+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d3001c5b0 con 0x7f0d340669a0 2024-05-25T06:03:31.556 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.555+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341149a0 con 0x7f0d340669a0 2024-05-25T06:03:31.557 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.556+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f0d30013420 con 0x7f0d340669a0 2024-05-25T06:03:31.557 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.556+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d34100d20 con 0x7f0d340669a0 2024-05-25T06:03:31.557 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.556+0000 7f0d297fa700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f0d1c036880 0x7f0d1c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:31.557 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.556+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0d30049a00 con 0x7f0d340669a0 2024-05-25T06:03:31.558 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.556+0000 7f0d2bfff700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f0d1c036880 0x7f0d1c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:31.558 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.557+0000 7f0d2bfff700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f0d1c036880 0x7f0d1c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009990 tx=0x7f0d24006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:31.561 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.561+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0d30013a80 con 0x7f0d340669a0 2024-05-25T06:03:31.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.820+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0d341153b0 con 0x7f0d340669a0 2024-05-25T06:03:31.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.820+0000 7f0d297fa700 1 -- 172.21.15.203:0/2440192787 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f0d30011370 con 0x7f0d340669a0 2024-05-25T06:03:31.821 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:31.821 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f0d1c036880 msgr2=0x7f0d1c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f0d1c036880 0x7f0d1c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009990 tx=0x7f0d24006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 msgr2=0x7f0d34113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d34113740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009680 tx=0x7f0d30009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 shutdown_connections 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7f0d1c036880 0x7f0d1c038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 --2- 172.21.15.203:0/2440192787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d340669a0 0x7f0d34113740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:31.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 >> 172.21.15.203:0/2440192787 conn(0x7f0d340f7070 msgr2=0x7f0d3410a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:31.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 shutdown_connections 2024-05-25T06:03:31.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:31.823+0000 7f0d3b0c9700 1 -- 172.21.15.203:0/2440192787 wait complete. 2024-05-25T06:03:31.824 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:32.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:31 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:31 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:31 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:31 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-25T06:03:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:31 smithi160 ceph-mon[29041]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:31 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2440192787' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:33.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:32 smithi160 ceph-mon[29041]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-25T06:03:33.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:32 smithi160 ceph-mon[29041]: mgrmap e14: smithi160.nalzlg(active, since 58s) 2024-05-25T06:03:33.411 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:33.411 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:33.662 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:33.710 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:35.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.224+0000 7fecb1f9f700 1 -- 172.21.15.203:0/2441338165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 msgr2=0x7fecac0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.224+0000 7fecb1f9f700 1 --2- 172.21.15.203:0/2441338165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac0fbd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0048c0 tx=0x7fec9c009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.224+0000 7fecb1f9f700 1 -- 172.21.15.203:0/2441338165 shutdown_connections 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.224+0000 7fecb1f9f700 1 --2- 172.21.15.203:0/2441338165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac0fbd50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.224+0000 7fecb1f9f700 1 -- 172.21.15.203:0/2441338165 >> 172.21.15.203:0/2441338165 conn(0x7fecac0f70a0 msgr2=0x7fecac0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecb1f9f700 1 -- 172.21.15.203:0/2441338165 shutdown_connections 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecb1f9f700 1 -- 172.21.15.203:0/2441338165 wait complete. 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecb1f9f700 1 Processor -- start 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecb1f9f700 1 -- start start 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecb1f9f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac065bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecb1f9f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac10fdb0 con 0x7fecac0fb940 2024-05-25T06:03:35.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac065bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac065bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:57528/0 (socket says 172.21.15.203:57528) 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.225+0000 7fecab7fe700 1 -- 172.21.15.203:0/1041615956 learned_addr learned my addr 172.21.15.203:0/1041615956 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7fecab7fe700 1 -- 172.21.15.203:0/1041615956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c005040 con 0x7fecac0fb940 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7fecab7fe700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac065bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0096b0 tx=0x7fec9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec9c00bed0 con 0x7fecac0fb940 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec9c00b5c0 con 0x7fecac0fb940 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac10fef0 con 0x7fecac0fb940 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec9c01c5b0 con 0x7fecac0fb940 2024-05-25T06:03:35.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.226+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac1103e0 con 0x7fecac0fb940 2024-05-25T06:03:35.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.227+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7fec9c013420 con 0x7fecac0fb940 2024-05-25T06:03:35.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.227+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac105900 con 0x7fecac0fb940 2024-05-25T06:03:35.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.227+0000 7feca8ff9700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 0x7fec98038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:35.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.227+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fec9c049b80 con 0x7fecac0fb940 2024-05-25T06:03:35.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.227+0000 7feca3fff700 1 -- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 msgr2=0x7fec98038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3983131949 2024-05-25T06:03:35.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.227+0000 7feca3fff700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 0x7fec98038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:03:35.232 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.232+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fec9c04be60 con 0x7fecac0fb940 2024-05-25T06:03:35.428 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.428+0000 7feca3fff700 1 -- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 msgr2=0x7fec98038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3983131949 2024-05-25T06:03:35.428 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.428+0000 7feca3fff700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 0x7fec98038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-25T06:03:35.493 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.492+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fecac101800 con 0x7fecac0fb940 2024-05-25T06:03:35.493 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.493+0000 7feca8ff9700 1 -- 172.21.15.203:0/1041615956 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fec9c025400 con 0x7fecac0fb940 2024-05-25T06:03:35.493 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:35.493 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:35.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 msgr2=0x7fec98038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:03:35.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 0x7fec98038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 msgr2=0x7fecac065bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac065bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0096b0 tx=0x7fec9c009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 shutdown_connections 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:6800/3983131949,v1:172.21.15.160:6801/3983131949] conn(0x7fec980368d0 0x7fec98038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 --2- 172.21.15.203:0/1041615956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecac0fb940 0x7fecac065bb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 >> 172.21.15.203:0/1041615956 conn(0x7fecac0f70a0 msgr2=0x7fecac102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 shutdown_connections 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:35.495+0000 7fecb1f9f700 1 -- 172.21.15.203:0/1041615956 wait complete. 2024-05-25T06:03:35.496 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:35.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:35 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1041615956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:37.075 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:37.075 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:37.326 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:37.374 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 -- 172.21.15.203:0/2624051278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 msgr2=0x7fa2540fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 --2- 172.21.15.203:0/2624051278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa2540fbd40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009c60 tx=0x7fa24c004d10 comp rx=0 tx=0).stop 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 -- 172.21.15.203:0/2624051278 shutdown_connections 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 --2- 172.21.15.203:0/2624051278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa2540fbd40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 -- 172.21.15.203:0/2624051278 >> 172.21.15.203:0/2624051278 conn(0x7fa2540f7070 msgr2=0x7fa2540f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 -- 172.21.15.203:0/2624051278 shutdown_connections 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.846+0000 7fa259a2d700 1 -- 172.21.15.203:0/2624051278 wait complete. 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa259a2d700 1 Processor -- start 2024-05-25T06:03:38.847 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa259a2d700 1 -- start start 2024-05-25T06:03:38.848 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa259a2d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa254066820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa259a2d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25410fdd0 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa252ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa254066820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa252ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa254066820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:57542/0 (socket says 172.21.15.203:57542) 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa252ffd700 1 -- 172.21.15.203:0/2271651508 learned_addr learned my addr 172.21.15.203:0/2271651508 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa252ffd700 1 -- 172.21.15.203:0/2271651508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c009940 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.847+0000 7fa252ffd700 1 --2- 172.21.15.203:0/2271651508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa254066820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009500 tx=0x7fa24c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.848+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa24c007c80 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.848+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa24c01baf0 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.848+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa24c012bb0 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.848+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25410bbb0 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.848+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25410c0a0 con 0x7fa2540fb930 2024-05-25T06:03:38.849 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.849+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) v1 ==== 43170+0+0 (secure 0 0 0) 0x7fa24c02d440 con 0x7fa2540fb930 2024-05-25T06:03:38.850 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.849+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2541058f0 con 0x7fa2540fb930 2024-05-25T06:03:38.850 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.849+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa24c04d8e0 con 0x7fa2540fb930 2024-05-25T06:03:38.854 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:38.854+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa24c02db10 con 0x7fa2540fb930 2024-05-25T06:03:38.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: Active manager daemon smithi160.nalzlg restarted 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: osdmap e5: 0 total, 0 up, 0 in 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: mgrmap e15: smithi160.nalzlg(active, starting, since 0.00315578s) 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:03:38.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:39.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.128+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa25405f5c0 con 0x7fa2540fb930 2024-05-25T06:03:39.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.128+0000 7fa24b7fe700 1 -- 172.21.15.203:0/2271651508 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fa24c028070 con 0x7fa2540fb930 2024-05-25T06:03:39.129 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:39.129 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 msgr2=0x7fa254066820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 --2- 172.21.15.203:0/2271651508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa254066820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009500 tx=0x7fa24c004d10 comp rx=0 tx=0).stop 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 shutdown_connections 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 --2- 172.21.15.203:0/2271651508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2540fb930 0x7fa254066820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 >> 172.21.15.203:0/2271651508 conn(0x7fa2540f7070 msgr2=0x7fa254102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 shutdown_connections 2024-05-25T06:03:39.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:39.131+0000 7fa259a2d700 1 -- 172.21.15.203:0/2271651508 wait complete. 2024-05-25T06:03:39.132 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2271651508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:39 smithi160 ceph-mon[29041]: mgrmap e16: smithi160.nalzlg(active, since 1.00761s) 2024-05-25T06:03:40.727 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:40.727 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:41.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:40 smithi160 ceph-mon[29041]: [25/May/2024:06:03:39] ENGINE Bus STARTING 2024-05-25T06:03:41.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:40 smithi160 ceph-mon[29041]: [25/May/2024:06:03:39] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:03:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:40 smithi160 ceph-mon[29041]: [25/May/2024:06:03:39] ENGINE Bus STARTED 2024-05-25T06:03:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:41.816 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:41.870 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-25T06:03:42.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:42 smithi160 ceph-mon[29041]: mgrmap e17: smithi160.nalzlg(active, since 2s) 2024-05-25T06:03:43.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:03:43.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:03:44.541 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.540+0000 7ff36be66700 1 -- 172.21.15.203:0/1390600010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 msgr2=0x7ff364102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:44.541 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.540+0000 7ff36be66700 1 --2- 172.21.15.203:0/1390600010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364102e20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff3600048c0 tx=0x7ff360009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:44.541 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.540+0000 7ff36be66700 1 -- 172.21.15.203:0/1390600010 shutdown_connections 2024-05-25T06:03:44.541 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.540+0000 7ff36be66700 1 --2- 172.21.15.203:0/1390600010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364102e20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:44.541 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.540+0000 7ff36be66700 1 -- 172.21.15.203:0/1390600010 >> 172.21.15.203:0/1390600010 conn(0x7ff364073090 msgr2=0x7ff364073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:44.541 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.540+0000 7ff36be66700 1 -- 172.21.15.203:0/1390600010 shutdown_connections 2024-05-25T06:03:44.542 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.541+0000 7ff36be66700 1 -- 172.21.15.203:0/1390600010 wait complete. 2024-05-25T06:03:44.542 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.541+0000 7ff36be66700 1 Processor -- start 2024-05-25T06:03:44.542 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.541+0000 7ff36be66700 1 -- start start 2024-05-25T06:03:44.542 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff36be66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364113110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:44.543 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff36be66700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364113650 con 0x7ff364102a50 2024-05-25T06:03:44.543 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff369c02700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364113110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:44.543 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff369c02700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364113110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:43784/0 (socket says 172.21.15.203:43784) 2024-05-25T06:03:44.543 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff369c02700 1 -- 172.21.15.203:0/358051160 learned_addr learned my addr 172.21.15.203:0/358051160 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:44.543 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff369c02700 1 -- 172.21.15.203:0/358051160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360005040 con 0x7ff364102a50 2024-05-25T06:03:44.543 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.542+0000 7ff369c02700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364113110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b7f0 tx=0x7ff360009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:44.544 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.544+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff360009370 con 0x7ff364102a50 2024-05-25T06:03:44.544 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.544+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff364113850 con 0x7ff364102a50 2024-05-25T06:03:44.544 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.544+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff360013420 con 0x7ff364102a50 2024-05-25T06:03:44.544 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.544+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff36001c590 con 0x7ff364102a50 2024-05-25T06:03:44.544 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.544+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff364113d40 con 0x7ff364102a50 2024-05-25T06:03:44.545 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.544+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7ff360026440 con 0x7ff364102a50 2024-05-25T06:03:44.545 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.545+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3640fe350 con 0x7ff364102a50 2024-05-25T06:03:44.545 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.545+0000 7ff35a7fc700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff350036970 0x7ff350038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:44.545 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.545+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff3600494d0 con 0x7ff364102a50 2024-05-25T06:03:44.545 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.545+0000 7ff369401700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff350036970 0x7ff350038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:44.545 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.545+0000 7ff369401700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff350036970 0x7ff350038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff354009990 tx=0x7ff354006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:44.549 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.550+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff360011780 con 0x7ff364102a50 2024-05-25T06:03:44.822 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.821+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff36405f5c0 con 0x7ff364102a50 2024-05-25T06:03:44.822 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.821+0000 7ff35a7fc700 1 -- 172.21.15.203:0/358051160 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7ff360021030 con 0x7ff364102a50 2024-05-25T06:03:44.822 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:44.822 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.823+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff350036970 msgr2=0x7ff350038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff350036970 0x7ff350038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff354009990 tx=0x7ff354006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 msgr2=0x7ff364113110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364113110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b7f0 tx=0x7ff360009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 shutdown_connections 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff350036970 0x7ff350038e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 --2- 172.21.15.203:0/358051160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff364102a50 0x7ff364113110 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 >> 172.21.15.203:0/358051160 conn(0x7ff364073090 msgr2=0x7ff3640f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:44.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 shutdown_connections 2024-05-25T06:03:44.825 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:44.824+0000 7ff36be66700 1 -- 172.21.15.203:0/358051160 wait complete. 2024-05-25T06:03:44.825 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:44.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:44 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:03:44.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:44 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:03:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:44 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:03:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:44 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:03:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:44 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:44 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:45.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: Deploying daemon crash.smithi203 on smithi203 2024-05-25T06:03:45.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:45 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/358051160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:46.371 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:46.371 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:46.779 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1907217957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 msgr2=0x7f40b40fbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 --2- 172.21.15.203:0/1907217957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b40fbc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40a40048c0 tx=0x7f40a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1907217957 shutdown_connections 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 --2- 172.21.15.203:0/1907217957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b40fbc00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1907217957 >> 172.21.15.203:0/1907217957 conn(0x7f40b40f74c0 msgr2=0x7f40b40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1907217957 shutdown_connections 2024-05-25T06:03:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1907217957 wait complete. 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.736+0000 7f40ba9f4700 1 Processor -- start 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.737+0000 7f40ba9f4700 1 -- start start 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.737+0000 7f40ba9f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b4115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.737+0000 7f40ba9f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4115840 con 0x7f40b40fb830 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.737+0000 7f40b3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b4115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.737+0000 7f40b3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b4115300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:43818/0 (socket says 172.21.15.203:43818) 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.737+0000 7f40b3fff700 1 -- 172.21.15.203:0/1645185032 learned_addr learned my addr 172.21.15.203:0/1645185032 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40b3fff700 1 -- 172.21.15.203:0/1645185032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4005040 con 0x7f40b40fb830 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40b3fff700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b4115300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b820 tx=0x7f40a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40a40093c0 con 0x7f40b40fb830 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40a4013420 con 0x7f40b40fb830 2024-05-25T06:03:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b4115a40 con 0x7f40b40fb830 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40a401c5b0 con 0x7f40b40fb830 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.738+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b4115f30 con 0x7f40b40fb830 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.739+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f40a401c7f0 con 0x7f40b40fb830 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.739+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b4066140 con 0x7f40b40fb830 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.739+0000 7f40b0ff9700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f409c036970 0x7f409c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.739+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f40a4049ad0 con 0x7f40b40fb830 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.739+0000 7f40b37fe700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f409c036970 0x7f409c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.740+0000 7f40b37fe700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f409c036970 0x7f409c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009990 tx=0x7f40a8006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:48.744+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f40a4009520 con 0x7f40b40fb830 2024-05-25T06:03:49.029 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.027+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f40b4066140 con 0x7f40b40fb830 2024-05-25T06:03:49.029 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.028+0000 7f40b0ff9700 1 -- 172.21.15.203:0/1645185032 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f40a4011b60 con 0x7f40b40fb830 2024-05-25T06:03:49.029 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:49.029 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f409c036970 msgr2=0x7f409c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f409c036970 0x7f409c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009990 tx=0x7f40a8006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 msgr2=0x7f40b4115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b4115300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b820 tx=0x7f40a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 shutdown_connections 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f409c036970 0x7f409c038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 --2- 172.21.15.203:0/1645185032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b40fb830 0x7f40b4115300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.030+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 >> 172.21.15.203:0/1645185032 conn(0x7f40b40f74c0 msgr2=0x7f40b40f8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.031+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 shutdown_connections 2024-05-25T06:03:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:49.031+0000 7f40ba9f4700 1 -- 172.21.15.203:0/1645185032 wait complete. 2024-05-25T06:03:49.032 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:49.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: Deploying daemon node-exporter.smithi203 on smithi203 2024-05-25T06:03:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:49 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1645185032' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:50.637 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:50.637 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:50.931 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:03:53.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 -- 172.21.15.203:0/3207152490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 msgr2=0x7fbf84101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:53.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 --2- 172.21.15.203:0/3207152490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf84101da0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0048c0 tx=0x7fbf7c009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:53.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 -- 172.21.15.203:0/3207152490 shutdown_connections 2024-05-25T06:03:53.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 --2- 172.21.15.203:0/3207152490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf84101da0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:53.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 -- 172.21.15.203:0/3207152490 >> 172.21.15.203:0/3207152490 conn(0x7fbf84071800 msgr2=0x7fbf84071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:53.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 -- 172.21.15.203:0/3207152490 shutdown_connections 2024-05-25T06:03:53.822 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 -- 172.21.15.203:0/3207152490 wait complete. 2024-05-25T06:03:53.822 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 Processor -- start 2024-05-25T06:03:53.822 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.818+0000 7fbf8957f700 1 -- start start 2024-05-25T06:03:53.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.819+0000 7fbf8957f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf8410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:53.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.819+0000 7fbf8957f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf84065fc0 con 0x7fbf841019d0 2024-05-25T06:03:53.823 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.819+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf8410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.819+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf8410e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:38240/0 (socket says 172.21.15.203:38240) 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.819+0000 7fbf82ffd700 1 -- 172.21.15.203:0/2150535543 learned_addr learned my addr 172.21.15.203:0/2150535543 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.819+0000 7fbf82ffd700 1 -- 172.21.15.203:0/2150535543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c005040 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.820+0000 7fbf82ffd700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf8410e270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00b820 tx=0x7fbf7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.820+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf7c0093c0 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.820+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf7c013420 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.820+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf7c01c5b0 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.820+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8410e7b0 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.820+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8410eca0 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.821+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fbf7c009520 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.821+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf84102b20 con 0x7fbf841019d0 2024-05-25T06:03:53.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.821+0000 7fbf7b7fe700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbf70036970 0x7fbf70038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:53.825 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.821+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbf7c049a60 con 0x7fbf841019d0 2024-05-25T06:03:53.825 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.821+0000 7fbf827fc700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbf70036970 0x7fbf70038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:53.825 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.822+0000 7fbf827fc700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbf70036970 0x7fbf70038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009990 tx=0x7fbf6c006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:53.826 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:53.826+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbf7c013a90 con 0x7fbf841019d0 2024-05-25T06:03:54.088 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.088+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbf840fd0d0 con 0x7fbf841019d0 2024-05-25T06:03:54.089 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.089+0000 7fbf7b7fe700 1 -- 172.21.15.203:0/2150535543 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fbf7c011370 con 0x7fbf841019d0 2024-05-25T06:03:54.089 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:54.090 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbf70036970 msgr2=0x7fbf70038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbf70036970 0x7fbf70038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009990 tx=0x7fbf6c006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 msgr2=0x7fbf8410e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf8410e270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00b820 tx=0x7fbf7c009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 shutdown_connections 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbf70036970 0x7fbf70038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 --2- 172.21.15.203:0/2150535543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf841019d0 0x7fbf8410e270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 >> 172.21.15.203:0/2150535543 conn(0x7fbf84071800 msgr2=0x7fbf84103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 shutdown_connections 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:54.091+0000 7fbf8957f700 1 -- 172.21.15.203:0/2150535543 wait complete. 2024-05-25T06:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: Deploying daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:03:55.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:54 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2150535543' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:55.833 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:03:55.833 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:03:56.327 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:03:57.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:03:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:03:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:03:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:57 smithi160 ceph-mon[29041]: Deploying daemon mon.smithi203 on smithi203 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 -- 172.21.15.203:0/1055436854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 msgr2=0x7f09740fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 --2- 172.21.15.203:0/1055436854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f09740fde20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f09700048c0 tx=0x7f0970009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 -- 172.21.15.203:0/1055436854 shutdown_connections 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 --2- 172.21.15.203:0/1055436854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f09740fde20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 -- 172.21.15.203:0/1055436854 >> 172.21.15.203:0/1055436854 conn(0x7f09740f74c0 msgr2=0x7f09740f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 -- 172.21.15.203:0/1055436854 shutdown_connections 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.799+0000 7f097b2c9700 1 -- 172.21.15.203:0/1055436854 wait complete. 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.800+0000 7f097b2c9700 1 Processor -- start 2024-05-25T06:03:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.800+0000 7f097b2c9700 1 -- start start 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.800+0000 7f097b2c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f0974113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.800+0000 7f097b2c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741136c0 con 0x7f09740fda50 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0979065700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f0974113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0979065700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f0974113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:38272/0 (socket says 172.21.15.203:38272) 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0979065700 1 -- 172.21.15.203:0/1169750956 learned_addr learned my addr 172.21.15.203:0/1169750956 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0979065700 1 -- 172.21.15.203:0/1169750956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970005040 con 0x7f09740fda50 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0979065700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f0974113180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f097000b820 tx=0x7f0970009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09700093c0 con 0x7f09740fda50 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09741138c0 con 0x7f09740fda50 2024-05-25T06:03:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0970013420 con 0x7f09740fda50 2024-05-25T06:03:57.804 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.801+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f097001c5b0 con 0x7f09740fda50 2024-05-25T06:03:57.804 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.802+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0974113d30 con 0x7f09740fda50 2024-05-25T06:03:57.804 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.802+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09740661c0 con 0x7f09740fda50 2024-05-25T06:03:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.802+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f0970013a90 con 0x7f09740fda50 2024-05-25T06:03:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.803+0000 7f0969ffb700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0960036970 0x7f0960038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:03:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.803+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0970049b60 con 0x7f09740fda50 2024-05-25T06:03:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.803+0000 7f0978864700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0960036970 0x7f0960038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:03:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.803+0000 7f0978864700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0960036970 0x7f0960038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0964009990 tx=0x7f0964006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:03:57.808 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:57.808+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f09700098a0 con 0x7f09740fda50 2024-05-25T06:03:58.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.094+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f097405f5c0 con 0x7f09740fda50 2024-05-25T06:03:58.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.094+0000 7f0969ffb700 1 -- 172.21.15.203:0/1169750956 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f0970011370 con 0x7f09740fda50 2024-05-25T06:03:58.096 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:03:58.096 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0960036970 msgr2=0x7f0960038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0960036970 0x7f0960038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0964009990 tx=0x7f0964006e40 comp rx=0 tx=0).stop 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 msgr2=0x7f0974113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f0974113180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f097000b820 tx=0x7f0970009e30 comp rx=0 tx=0).stop 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 shutdown_connections 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0960036970 0x7f0960038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 --2- 172.21.15.203:0/1169750956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09740fda50 0x7f0974113180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.096+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 >> 172.21.15.203:0/1169750956 conn(0x7f09740f74c0 msgr2=0x7f0974103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.097+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 shutdown_connections 2024-05-25T06:03:58.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:03:58.097+0000 7f097b2c9700 1 -- 172.21.15.203:0/1169750956 wait complete. 2024-05-25T06:03:58.099 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:03:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:58 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1169750956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:03:59.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:59 smithi160 ceph-mon[29041]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:03:59.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:03:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:00.111 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:04:00.112 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:04:00.618 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:04:01.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:01 smithi160 ceph-mon[29041]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.161 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 -- 172.21.15.203:0/1716079460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 msgr2=0x7fe910102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:02.161 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 --2- 172.21.15.203:0/1716079460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910102f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9000048c0 tx=0x7fe900009e30 comp rx=0 tx=0).stop 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 -- 172.21.15.203:0/1716079460 shutdown_connections 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 --2- 172.21.15.203:0/1716079460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910102f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 -- 172.21.15.203:0/1716079460 >> 172.21.15.203:0/1716079460 conn(0x7fe9100f86d0 msgr2=0x7fe9100faae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 -- 172.21.15.203:0/1716079460 shutdown_connections 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.159+0000 7fe9169c9700 1 -- 172.21.15.203:0/1716079460 wait complete. 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe9169c9700 1 Processor -- start 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe9169c9700 1 -- start start 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe9169c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe9169c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910113750 con 0x7fe910102b80 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:38296/0 (socket says 172.21.15.203:38296) 2024-05-25T06:04:02.162 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.160+0000 7fe90ffff700 1 -- 172.21.15.203:0/2043328125 learned_addr learned my addr 172.21.15.203:0/2043328125 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe90ffff700 1 -- 172.21.15.203:0/2043328125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe900005040 con 0x7fe910102b80 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe90ffff700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910113210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe90000b7f0 tx=0x7fe900009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe900009370 con 0x7fe910102b80 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe900013420 con 0x7fe910102b80 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe910113950 con 0x7fe910102b80 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe90001c590 con 0x7fe910102b80 2024-05-25T06:04:02.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.161+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe910113e40 con 0x7fe910102b80 2024-05-25T06:04:02.164 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.162+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fe900026440 con 0x7fe910102b80 2024-05-25T06:04:02.164 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.162+0000 7fe90cff9700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8f8036a00 0x7fe8f8038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:02.164 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.162+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe90004a590 con 0x7fe910102b80 2024-05-25T06:04:02.164 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.162+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9100fe2d0 con 0x7fe910102b80 2024-05-25T06:04:02.164 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.163+0000 7fe90f7fe700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8f8036a00 0x7fe8f8038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:02.164 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.163+0000 7fe90f7fe700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8f8036a00 0x7fe8f8038eb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe904009990 tx=0x7fe904006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:02.168 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.168+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe900021030 con 0x7fe910102b80 2024-05-25T06:04:02.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 podman[35980]: 2024-05-25 06:04:02.292374328 +0000 UTC m=+0.130900096 container create 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-7b01852, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-25T06:04:02.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.442+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe91004bed0 con 0x7fe910102b80 2024-05-25T06:04:02.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.443+0000 7fe90cff9700 1 -- 172.21.15.203:0/2043328125 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fe9000123c0 con 0x7fe910102b80 2024-05-25T06:04:02.444 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:04:02.444 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:01:25.425099Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-25T06:04:02.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8f8036a00 msgr2=0x7fe8f8038eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:02.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8f8036a00 0x7fe8f8038eb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe904009990 tx=0x7fe904006e40 comp rx=0 tx=0).stop 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 msgr2=0x7fe910113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910113210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe90000b7f0 tx=0x7fe900009e30 comp rx=0 tx=0).stop 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 shutdown_connections 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8f8036a00 0x7fe8f8038eb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 --2- 172.21.15.203:0/2043328125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe910102b80 0x7fe910113210 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 >> 172.21.15.203:0/2043328125 conn(0x7fe9100f86d0 msgr2=0x7fe9100f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 shutdown_connections 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:02.445+0000 7fe9169c9700 1 -- 172.21.15.203:0/2043328125 wait complete. 2024-05-25T06:04:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-05-25T06:04:02.756 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 podman[35980]: 2024-05-25 06:04:02.496305376 +0000 UTC m=+0.334831156 container init 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=quincy-7b01852, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-25T06:04:02.756 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 podman[35980]: 2024-05-25 06:04:02.500087262 +0000 UTC m=+0.338613025 container start 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-7b01852, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: set uid:gid to 167:167 (ceph:ceph) 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable), process ceph-mon, pid 2 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pidfile_write: ignore empty --pid-file 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: load: jerasure load: lrc 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: RocksDB version: 6.15.5 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Compile date May 23 2024 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: DB SUMMARY 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: DB Session ID: MOTHM75Z2PSSRRNECGUW 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: CURRENT file: CURRENT 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: IDENTITY file: IDENTITY 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-05-25T06:04:02.757 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi203/store.db dir, Total Num: 0, files: 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi203/store.db: 000004.log size: 511 ; 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.error_if_exists: 0 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.create_if_missing: 0 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.paranoid_checks: 1 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.env: 0x557357a10b00 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.fs: Posix File System 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.info_log: 0x557358f45620 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_file_opening_threads: 16 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.statistics: (nil) 2024-05-25T06:04:02.758 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.use_fsync: 0 2024-05-25T06:04:02.760 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_log_file_size: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.keep_log_file_num: 1000 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.recycle_log_file_num: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.allow_fallocate: 1 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.allow_mmap_reads: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.allow_mmap_writes: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.use_direct_reads: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.create_missing_column_families: 0 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.db_log_dir: 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi203/store.db 2024-05-25T06:04:02.761 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.advise_random_on_open: 1 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.db_write_buffer_size: 0 2024-05-25T06:04:02.762 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.write_buffer_manager: 0x557359c389c0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.rate_limiter: (nil) 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.wal_recovery_mode: 2 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.enable_thread_tracking: 0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.enable_pipelined_write: 0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.unordered_write: 0 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-25T06:04:02.763 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.row_cache: None 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.wal_filter: None 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.allow_ingest_behind: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.preserve_deletes: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.two_write_queues: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.manual_wal_flush: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.atomic_flush: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.log_readahead_size: 0 2024-05-25T06:04:02.764 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-25T06:04:02.765 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.best_efforts_recovery: 0 2024-05-25T06:04:02.765 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-25T06:04:02.765 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-25T06:04:02.765 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.allow_data_in_errors: 0 2024-05-25T06:04:02.765 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.db_host_id: __hostname__ 2024-05-25T06:04:02.765 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_background_jobs: 2 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_background_compactions: -1 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_subcompactions: 1 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_total_wal_size: 0 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_open_files: -1 2024-05-25T06:04:02.766 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bytes_per_sync: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_readahead_size: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_background_flushes: -1 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Compression algorithms supported: 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kZSTD supported: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kXpressCompression supported: 0 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kLZ4HCCompression supported: 1 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kZlibCompression supported: 1 2024-05-25T06:04:02.767 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kSnappyCompression supported: 1 2024-05-25T06:04:02.768 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kLZ4Compression supported: 1 2024-05-25T06:04:02.768 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: kBZip2Compression supported: 0 2024-05-25T06:04:02.768 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-25T06:04:02.768 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi203/store.db/MANIFEST-000003 2024-05-25T06:04:02.768 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-05-25T06:04:02.769 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-25T06:04:02.769 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.merge_operator: 2024-05-25T06:04:02.769 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_filter: None 2024-05-25T06:04:02.769 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_filter_factory: None 2024-05-25T06:04:02.769 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.sst_partitioner_factory: None 2024-05-25T06:04:02.769 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557358f18f58) 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: cache_index_and_filter_blocks: 1 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: pin_top_level_index_and_filter: 1 2024-05-25T06:04:02.770 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: index_type: 0 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: data_block_index_type: 0 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: index_shortening: 1 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: hash_index_allow_collision: 1 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: checksum: 1 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: no_block_cache: 0 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache: 0x557358f4b090 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache_name: BinnedLRUCache 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache_options: 2024-05-25T06:04:02.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: capacity : 536870912 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: num_shard_bits : 4 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: strict_capacity_limit : 0 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: high_pri_pool_ratio: 0.000 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache_compressed: (nil) 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: persistent_cache: (nil) 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_size: 4096 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_size_deviation: 10 2024-05-25T06:04:02.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_restart_interval: 16 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: index_block_restart_interval: 1 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: metadata_block_size: 4096 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: partition_filters: 0 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: use_delta_encoding: 1 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: whole_key_filtering: 1 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: verify_compression: 0 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: read_amp_bytes_per_bit: 0 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: format_version: 4 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: enable_index_compression: 1 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_align: 0 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.write_buffer_size: 33554432 2024-05-25T06:04:02.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_write_buffer_number: 2 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression: NoCompression 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression: Disabled 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.prefix_extractor: nullptr 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.num_levels: 7 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-25T06:04:02.774 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.level: 32767 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.strategy: 0 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compression_opts.enabled: false 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.target_file_size_base: 67108864 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-25T06:04:02.775 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.arena_block_size: 4194304 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-25T06:04:02.776 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.disable_auto_compactions: 0 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.table_properties_collectors: 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.inplace_update_support: 0 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-25T06:04:02.777 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.bloom_locality: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.max_successive_merges: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.paranoid_file_checks: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.force_consistency_checks: 1 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.report_bg_io_stats: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.ttl: 2592000 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.enable_blob_files: false 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.min_blob_size: 0 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.blob_file_size: 268435456 2024-05-25T06:04:02.778 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi203/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617042527081, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617042527390, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1716617042, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "1e92b2d3-c240-4f00-b8d7-320dab0d5694", "db_session_id": "MOTHM75Z2PSSRRNECGUW"}} 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617042527705, "job": 1, "event": "recovery_finished"} 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi203/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x557358edf180 2024-05-25T06:04:02.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: DB pointer 0x557358fb4000 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** DB Stats ** 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:04:02.780 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-25T06:04:02.781 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-05-25T06:04:02.782 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-25T06:04:02.783 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203 does not exist in monmap, will attempt to join an existing cluster 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: starting mon.smithi203 rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi203 fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(???) e0 preinit fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).mds e1 new map 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).mds e1 print_map 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: e1 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: legacy client fscid: -1 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: No filesystems configured 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-25T06:04:02.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying cephadm binary to smithi203 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.785 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Added host smithi203 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon alertmanager.smithi160 on smithi160 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2743436208' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-25T06:04:02.786 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2743436208' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: osdmap e4: 0 total, 0 up, 0 in 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon grafana.smithi160 on smithi160 2024-05-25T06:04:02.787 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Saving service mon spec with placement smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203;count:2 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2613563092' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/737399514' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.788 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/376958826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon prometheus.smithi160 on smithi160 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.789 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1823448785' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2635412007' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/3322279985' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.790 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2440192787' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14162 172.21.15.160:0/912000960' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mgrmap e14: smithi160.nalzlg(active, since 58s) 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1041615956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Active manager daemon smithi160.nalzlg restarted 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: osdmap e5: 0 total, 0 up, 0 in 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mgrmap e15: smithi160.nalzlg(active, starting, since 0.00315578s) 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:04:02.791 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2271651508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mgrmap e16: smithi160.nalzlg(active, since 1.00761s) 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: [25/May/2024:06:03:39] ENGINE Bus STARTING 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: [25/May/2024:06:03:39] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:04:02.792 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: [25/May/2024:06:03:39] ENGINE Bus STARTED 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mgrmap e17: smithi160.nalzlg(active, since 2s) 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:04:02.794 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon crash.smithi203 on smithi203 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/358051160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.795 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon node-exporter.smithi203 on smithi203 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1645185032' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:04:02.796 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2150535543' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: Deploying daemon mon.smithi203 on smithi203 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1169750956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:02.797 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 ceph-mon[36030]: mon.smithi203@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-05-25T06:04:03.033 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 bash[35980]: 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a 2024-05-25T06:04:03.033 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:02 smithi203 systemd[1]: Started Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:04:04.265 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-25T06:04:04.265 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-25T06:04:04.527 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: mon.smithi160 calling monitor election 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: mon.smithi203 calling monitor election 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:07.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: mon.smithi160 is new leader, mons smithi160,smithi203 in quorum (ranks 0,1) 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: monmap e2: 2 mons at {smithi160=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],smithi203=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: fsmap 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: osdmap e5: 0 total, 0 up, 0 in 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: mgrmap e17: smithi160.nalzlg(active, since 28s) 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: overall HEALTH_OK 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.902 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.902 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:07 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:04:07.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: mon.smithi160 calling monitor election 2024-05-25T06:04:07.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:07.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: mon.smithi203 calling monitor election 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: mon.smithi160 is new leader, mons smithi160,smithi203 in quorum (ranks 0,1) 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: monmap e2: 2 mons at {smithi160=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],smithi203=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: fsmap 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: osdmap e5: 0 total, 0 up, 0 in 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: mgrmap e17: smithi160.nalzlg(active, since 28s) 2024-05-25T06:04:07.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: overall HEALTH_OK 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/938093605' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:07.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:07 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:08.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:08 smithi203 ceph-mon[36030]: mgrmap e18: smithi160.nalzlg(active, since 29s), standbys: smithi203.rotqex 2024-05-25T06:04:08.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:04:08.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:08.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:08.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:08 smithi160 ceph-mon[29041]: mgrmap e18: smithi160.nalzlg(active, since 29s), standbys: smithi203.rotqex 2024-05-25T06:04:08.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:04:08.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:08.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.982+0000 7f7c0defc700 1 -- 172.21.15.203:0/3304979107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 msgr2=0x7f7bf0005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.982+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3304979107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7bf0005a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c080655f0 tx=0x7f7bf800b040 comp rx=0 tx=0).stop 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.982+0000 7f7c0defc700 1 -- 172.21.15.203:0/3304979107 shutdown_connections 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.982+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3304979107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7bf0005a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.982+0000 7f7c0defc700 1 -- 172.21.15.203:0/3304979107 >> 172.21.15.203:0/3304979107 conn(0x7f7c08072080 msgr2=0x7f7c08072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.982+0000 7f7c0defc700 1 -- 172.21.15.203:0/3304979107 shutdown_connections 2024-05-25T06:04:08.983 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.983+0000 7f7c0defc700 1 -- 172.21.15.203:0/3304979107 wait complete. 2024-05-25T06:04:08.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.983+0000 7f7c0defc700 1 Processor -- start 2024-05-25T06:04:08.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.983+0000 7f7c0defc700 1 -- start start 2024-05-25T06:04:08.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c0defc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7c080fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:08.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c0defc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:08.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c0defc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08119550 con 0x7f7bf0003580 2024-05-25T06:04:08.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c0defc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08119690 con 0x7f7c080fc830 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7c080fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:38112/0 (socket says 172.21.15.203:38112) 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7c080fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.203:42308/0 (socket says 172.21.15.203:42308) 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 -- 172.21.15.203:0/3561619601 learned_addr learned my addr 172.21.15.203:0/3561619601 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 -- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 msgr2=0x7f7c080fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 -- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 msgr2=0x7f7c080fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c06ffd700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c077fe700 1 -- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 msgr2=0x7f7c080fcca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c077fe700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:08.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c077fe700 1 -- 172.21.15.203:0/3561619601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8005040 con 0x7f7bf0003580 2024-05-25T06:04:08.987 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c077fe700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7c080fc2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009ce0 tx=0x7f7bf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:08.987 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf801b070 con 0x7f7bf0003580 2024-05-25T06:04:08.987 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c08119830 con 0x7f7bf0003580 2024-05-25T06:04:08.987 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf8016d10 con 0x7f7bf0003580 2024-05-25T06:04:08.987 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.984+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf801f620 con 0x7f7bf0003580 2024-05-25T06:04:08.987 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.985+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c08119d70 con 0x7f7bf0003580 2024-05-25T06:04:08.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.986+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c08107380 con 0x7f7bf0003580 2024-05-25T06:04:08.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.986+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f7bf8020440 con 0x7f7bf0003580 2024-05-25T06:04:08.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.986+0000 7f7beffff700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f7bf4069320 0x7f7bf406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:08.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.987+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7bf8087f70 con 0x7f7bf0003580 2024-05-25T06:04:08.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.987+0000 7f7c06ffd700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f7bf4069320 0x7f7bf406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:08.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.987+0000 7f7c06ffd700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f7bf4069320 0x7f7bf406b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:08.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:08.992+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7bf8055260 con 0x7f7bf0003580 2024-05-25T06:04:09.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.256+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7c080fe400 con 0x7f7bf0003580 2024-05-25T06:04:09.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.257+0000 7f7beffff700 1 -- 172.21.15.203:0/3561619601 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7f7bf8024410 con 0x7f7bf0003580 2024-05-25T06:04:09.258 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-25T06:04:09.258 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","modified":"2024-05-25T06:04:02.551194Z","created":"2024-05-25T06:01:25.425099Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi160","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi203","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-25T06:04:09.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f7bf4069320 msgr2=0x7f7bf406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:09.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f7bf4069320 0x7f7bf406b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc008040 comp rx=0 tx=0).stop 2024-05-25T06:04:09.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 msgr2=0x7f7c080fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:09.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7c080fc2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009ce0 tx=0x7f7bf800b040 comp rx=0 tx=0).stop 2024-05-25T06:04:09.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 shutdown_connections 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f7bf4069320 0x7f7bf406b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0003580 0x7f7c080fc2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 --2- 172.21.15.203:0/3561619601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c080fc830 0x7f7c080fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 >> 172.21.15.203:0/3561619601 conn(0x7f7c08072080 msgr2=0x7f7c080fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 shutdown_connections 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:04:09.260+0000 7f7c0defc700 1 -- 172.21.15.203:0/3561619601 wait complete. 2024-05-25T06:04:09.261 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-05-25T06:04:09.846 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:09 smithi203 ceph-mon[36030]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:09.846 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:09.846 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:09.846 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:09 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/3561619601' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:09.869 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-25T06:04:09.870 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-25T06:04:09.897 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:09 smithi160 ceph-mon[29041]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:09.897 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:09.897 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:09.898 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:09 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/3561619601' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-25T06:04:10.163 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.297 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.297 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.297 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:04:11.297 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:04:11.297 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:04:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/2180481355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4102b20 msgr2=0x7f8ee4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/2180481355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4102b20 0x7f8ee4102ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009c60 tx=0x7f8ed4004d10 comp rx=0 tx=0).stop 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/2180481355 shutdown_connections 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/2180481355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4102b20 0x7f8ee4102ef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/2180481355 >> 172.21.15.160:0/2180481355 conn(0x7f8ee4073060 msgr2=0x7f8ee4073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/2180481355 shutdown_connections 2024-05-25T06:04:11.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/2180481355 wait complete. 2024-05-25T06:04:11.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 Processor -- start 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.748+0000 7f8ee9b4b700 1 -- start start 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee9b4b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee4102b20 0x7f8ee4113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee9b4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 0x7f8ee410d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee9b4b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee4113ea0 con 0x7f8ee4113890 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee9b4b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee410d7f0 con 0x7f8ee4102b20 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 0x7f8ee410d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:11.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 0x7f8ee410d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49770/0 (socket says 172.21.15.160:49770) 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 -- 172.21.15.160:0/443590924 learned_addr learned my addr 172.21.15.160:0/443590924 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee37fe700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee4102b20 0x7f8ee4113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 -- 172.21.15.160:0/443590924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee4102b20 msgr2=0x7f8ee4113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee4102b20 0x7f8ee4113350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 -- 172.21.15.160:0/443590924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4009940 con 0x7f8ee4113890 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee2ffd700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 0x7f8ee410d2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b790 tx=0x7f8ecc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ecc010870 con 0x7f8ee4113890 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ecc00f3e0 con 0x7f8ee4113890 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee410dad0 con 0x7f8ee4113890 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ecc010ea0 con 0x7f8ee4113890 2024-05-25T06:04:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.749+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee410e010 con 0x7f8ee4113890 2024-05-25T06:04:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.750+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee40fe350 con 0x7f8ee4113890 2024-05-25T06:04:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.751+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f8ecc00f550 con 0x7f8ee4113890 2024-05-25T06:04:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.751+0000 7f8edbfff700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ed0069320 0x7f8ed006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.751+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8ecc086690 con 0x7f8ee4113890 2024-05-25T06:04:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.751+0000 7f8ee37fe700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ed0069320 0x7f8ed006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.752+0000 7f8ee37fe700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ed0069320 0x7f8ed006b7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4015040 tx=0x7f8ed4004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:11.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.755+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8ecc0574b0 con 0x7f8ee4113890 2024-05-25T06:04:11.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.947+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8ee4106e40 con 0x7f8ee4113890 2024-05-25T06:04:11.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.948+0000 7f8edbfff700 1 -- 172.21.15.160:0/443590924 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f8ecc014100 con 0x7f8ee4113890 2024-05-25T06:04:11.948 INFO:teuthology.orchestra.run.smithi160.stdout:# minimal ceph.conf for 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:04:11.948 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2024-05-25T06:04:11.948 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:04:11.948 INFO:teuthology.orchestra.run.smithi160.stdout: mon_host = [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ed0069320 msgr2=0x7f8ed006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ed0069320 0x7f8ed006b7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4015040 tx=0x7f8ed4004d10 comp rx=0 tx=0).stop 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 msgr2=0x7f8ee410d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 0x7f8ee410d2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b790 tx=0x7f8ecc009250 comp rx=0 tx=0).stop 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 shutdown_connections 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ed0069320 0x7f8ed006b7d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee4102b20 0x7f8ee4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:11.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 --2- 172.21.15.160:0/443590924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ee4113890 0x7f8ee410d2b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:11.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 >> 172.21.15.160:0/443590924 conn(0x7f8ee4073060 msgr2=0x7f8ee40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:11.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 shutdown_connections 2024-05-25T06:04:11.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:11.950+0000 7f8ee9b4b700 1 -- 172.21.15.160:0/443590924 wait complete. 2024-05-25T06:04:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:12 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:04:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:12 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:04:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:12 smithi160 ceph-mon[29041]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:12 smithi160 ceph-mon[29041]: Reconfiguring mon.smithi160 (unknown last config time)... 2024-05-25T06:04:12.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:12 smithi160 ceph-mon[29041]: Reconfiguring daemon mon.smithi160 on smithi160 2024-05-25T06:04:12.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:12 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/443590924' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:12.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:12 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:04:12.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:12 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:04:12.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:12 smithi203 ceph-mon[36030]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:12.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:12 smithi203 ceph-mon[36030]: Reconfiguring mon.smithi160 (unknown last config time)... 2024-05-25T06:04:12.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:12 smithi203 ceph-mon[36030]: Reconfiguring daemon mon.smithi160 on smithi160 2024-05-25T06:04:12.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:12 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/443590924' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:12.528 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-25T06:04:12.528 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:04:12.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-05-25T06:04:12.596 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:04:12.596 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:04:12.663 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:04:12.663 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-05-25T06:04:12.692 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:04:12.692 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:04:12.758 INFO:tasks.cephadm:Deploying OSDs... 2024-05-25T06:04:12.758 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:04:12.758 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-05-25T06:04:12.775 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-25T06:04:12.775 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-25 06:03:30.291836328 +0000 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-25 05:57:23.504492728 +0000 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-25 05:57:23.504492728 +0000 2024-05-25T06:04:12.830 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-25 05:57:23.504492728 +0000 2024-05-25T06:04:12.831 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-25T06:04:12.893 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-25T06:04:12.893 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-25T06:04:12.893 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000268211 s, 1.9 MB/s 2024-05-25T06:04:12.894 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-25T06:04:12.950 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-05-25T06:04:13.006 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-25T06:04:13.006 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:13.006 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-25T06:04:13.006 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:13.006 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:13.006 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-25 06:03:30.292836314 +0000 2024-05-25T06:04:13.007 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-25 05:57:23.845487706 +0000 2024-05-25T06:04:13.007 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-25 05:57:23.845487706 +0000 2024-05-25T06:04:13.007 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-25 05:57:23.845487706 +0000 2024-05-25T06:04:13.007 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-25T06:04:13.069 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-25T06:04:13.069 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-25T06:04:13.069 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000210174 s, 2.4 MB/s 2024-05-25T06:04:13.070 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-25T06:04:13.127 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-25 06:03:30.292836314 +0000 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-25 05:57:24.173482875 +0000 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-25 05:57:24.173482875 +0000 2024-05-25T06:04:13.183 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-25 05:57:24.173482875 +0000 2024-05-25T06:04:13.184 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-25T06:04:13.246 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-25T06:04:13.246 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-25T06:04:13.246 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000175535 s, 2.9 MB/s 2024-05-25T06:04:13.247 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-25T06:04:13.303 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-25 06:03:30.293836299 +0000 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-25 05:57:24.494478148 +0000 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-25 05:57:24.494478148 +0000 2024-05-25T06:04:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-25 05:57:24.494478148 +0000 2024-05-25T06:04:13.359 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-25T06:04:13.423 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-25T06:04:13.423 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-25T06:04:13.423 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000223288 s, 2.3 MB/s 2024-05-25T06:04:13.424 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-25T06:04:13.481 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:04:13.482 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-05-25T06:04:13.498 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-25T06:04:13.498 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-05-25T06:04:13.554 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-25 06:04:00.850899282 +0000 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-25 05:57:23.303053590 +0000 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-25 05:57:23.303053590 +0000 2024-05-25T06:04:13.555 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-25 05:57:23.303053590 +0000 2024-05-25T06:04:13.555 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-25T06:04:13.618 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-25T06:04:13.619 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-25T06:04:13.619 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000196958 s, 2.6 MB/s 2024-05-25T06:04:13.620 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-25T06:04:13.676 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-05-25T06:04:13.731 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-25T06:04:13.731 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:13.731 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-25T06:04:13.731 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:13.731 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:13.732 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-25 06:04:00.851899260 +0000 2024-05-25T06:04:13.732 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-25 05:57:23.643045873 +0000 2024-05-25T06:04:13.732 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-25 05:57:23.643045873 +0000 2024-05-25T06:04:13.732 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-25 05:57:23.643045873 +0000 2024-05-25T06:04:13.732 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-25T06:04:13.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:13 smithi203 ceph-mon[36030]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:13.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:13.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:13.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:04:13.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:04:13.793 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:13.795 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-25T06:04:13.795 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-25T06:04:13.795 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000218962 s, 2.3 MB/s 2024-05-25T06:04:13.796 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-25T06:04:13.852 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-05-25T06:04:13.860 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:13 smithi160 ceph-mon[29041]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:13.860 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:13.860 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:13.861 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:04:13.861 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:04:13.861 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:13.907 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-25T06:04:13.907 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:13.907 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-25T06:04:13.908 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:13.908 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:13.908 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-25 06:04:00.851899260 +0000 2024-05-25T06:04:13.908 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-25 05:57:23.974038360 +0000 2024-05-25T06:04:13.908 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-25 05:57:23.974038360 +0000 2024-05-25T06:04:13.908 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-25 05:57:23.974038360 +0000 2024-05-25T06:04:13.908 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-25T06:04:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-25T06:04:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-25T06:04:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000217592 s, 2.4 MB/s 2024-05-25T06:04:13.972 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-25T06:04:14.029 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-25 06:04:00.851899260 +0000 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-25 05:57:24.300030961 +0000 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-25 05:57:24.300030961 +0000 2024-05-25T06:04:14.084 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-25 05:57:24.300030961 +0000 2024-05-25T06:04:14.085 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-25T06:04:14.147 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-25T06:04:14.147 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-25T06:04:14.147 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000216637 s, 2.4 MB/s 2024-05-25T06:04:14.148 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-25T06:04:14.205 INFO:tasks.cephadm:Deploying osd.0 on smithi160 with /dev/vg_nvme/lv_4... 2024-05-25T06:04:14.205 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-25T06:04:14.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:14 smithi160 ceph-mon[29041]: Reconfiguring mgr.smithi160.nalzlg (unknown last config time)... 2024-05-25T06:04:14.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:14 smithi160 ceph-mon[29041]: Reconfiguring daemon mgr.smithi160.nalzlg on smithi160 2024-05-25T06:04:14.886 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:04:14.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:14 smithi203 ceph-mon[36030]: Reconfiguring mgr.smithi160.nalzlg (unknown last config time)... 2024-05-25T06:04:14.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:14 smithi203 ceph-mon[36030]: Reconfiguring daemon mgr.smithi160.nalzlg on smithi160 2024-05-25T06:04:15.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: Reconfiguring crash.smithi160 (monmap changed)... 2024-05-25T06:04:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:04:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:15 smithi160 ceph-mon[29041]: Reconfiguring daemon crash.smithi160 on smithi160 2024-05-25T06:04:16.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:16.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:16.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:16.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: Reconfiguring crash.smithi160 (monmap changed)... 2024-05-25T06:04:16.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:04:16.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:16.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:15 smithi203 ceph-mon[36030]: Reconfiguring daemon crash.smithi160 on smithi160 2024-05-25T06:04:17.345 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-25T06:04:17.345 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-25T06:04:17.346 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-25T06:04:17.346 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-25T06:04:17.346 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114629 s, 915 MB/s 2024-05-25T06:04:17.346 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-25T06:04:18.082 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-05-25T06:04:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:17 smithi203 ceph-mon[36030]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:17 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:17 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:17 smithi160 ceph-mon[29041]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:18.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:17 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:18.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:17 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:18 smithi203 ceph-mon[36030]: Reconfiguring alertmanager.smithi160 (dependencies changed)... 2024-05-25T06:04:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:18 smithi203 ceph-mon[36030]: Reconfiguring daemon alertmanager.smithi160 on smithi160 2024-05-25T06:04:19.209 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:18 smithi160 ceph-mon[29041]: Reconfiguring alertmanager.smithi160 (dependencies changed)... 2024-05-25T06:04:19.209 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:18 smithi160 ceph-mon[29041]: Reconfiguring daemon alertmanager.smithi160 on smithi160 2024-05-25T06:04:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:19 smithi203 ceph-mon[36030]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:20.151 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:19 smithi160 ceph-mon[29041]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:20.590 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:04:22.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 -- 172.21.15.160:0/4188534306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 msgr2=0x7f1f900fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:22.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 --2- 172.21.15.160:0/4188534306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f900fd3e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f84004940 tx=0x7f1f84009e30 comp rx=0 tx=0).stop 2024-05-25T06:04:22.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 -- 172.21.15.160:0/4188534306 shutdown_connections 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 --2- 172.21.15.160:0/4188534306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f900fd3e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 --2- 172.21.15.160:0/4188534306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f900fbcc0 0x7f1f900fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 -- 172.21.15.160:0/4188534306 >> 172.21.15.160:0/4188534306 conn(0x7f1f900f7420 msgr2=0x7f1f900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 -- 172.21.15.160:0/4188534306 shutdown_connections 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.083+0000 7f1f955fa700 1 -- 172.21.15.160:0/4188534306 wait complete. 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f955fa700 1 Processor -- start 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f955fa700 1 -- start start 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f955fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f900fbcc0 0x7f1f9010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f955fa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f9010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f955fa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9010e5d0 con 0x7f1f900fcf70 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f955fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9010edd0 con 0x7f1f900fbcc0 2024-05-25T06:04:22.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f9010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f9010dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35794/0 (socket says 172.21.15.160:35794) 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 -- 172.21.15.160:0/610630063 learned_addr learned my addr 172.21.15.160:0/610630063 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8effd700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f900fbcc0 0x7f1f9010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 -- 172.21.15.160:0/610630063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f900fbcc0 msgr2=0x7f1f9010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f900fbcc0 0x7f1f9010da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 -- 172.21.15.160:0/610630063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84005040 con 0x7f1f900fcf70 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.084+0000 7f1f8e7fc700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f9010dfd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400b3b0 tx=0x7f1f84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.085+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8400e040 con 0x7f1f900fcf70 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.085+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f8400bbf0 con 0x7f1f900fcf70 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.085+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9010f050 con 0x7f1f900fcf70 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.085+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8401c630 con 0x7f1f900fcf70 2024-05-25T06:04:22.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.085+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9010f590 con 0x7f1f900fcf70 2024-05-25T06:04:22.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.086+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90101b00 con 0x7f1f900fcf70 2024-05-25T06:04:22.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.086+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f1f8400bd60 con 0x7f1f900fcf70 2024-05-25T06:04:22.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.087+0000 7f1f777fe700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f1f78069210 0x7f1f7806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:22.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.087+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1f84086c80 con 0x7f1f900fcf70 2024-05-25T06:04:22.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.087+0000 7f1f8effd700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f1f78069210 0x7f1f7806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:22.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.087+0000 7f1f8effd700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f1f78069210 0x7f1f7806b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f80004f40 tx=0x7f1f8000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:22.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.091+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1f84057bc0 con 0x7f1f900fcf70 2024-05-25T06:04:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:21 smithi203 ceph-mon[36030]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:22.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:21 smithi160 ceph-mon[29041]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:22.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:22.289+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f901048c0 con 0x7f1f78069210 2024-05-25T06:04:23.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:04:23.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:04:23.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:23.218 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:04:23.218 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:04:23.218 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:23 smithi203 ceph-mon[36030]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:04:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:23 smithi203 ceph-mon[36030]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:24.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:23 smithi160 ceph-mon[29041]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:04:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:23 smithi160 ceph-mon[29041]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:26.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:25 smithi203 ceph-mon[36030]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:26.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:26.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:26.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:25 smithi203 ceph-mon[36030]: Reconfiguring grafana.smithi160 (dependencies changed)... 2024-05-25T06:04:26.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:25 smithi203 ceph-mon[36030]: Reconfiguring daemon grafana.smithi160 on smithi160 2024-05-25T06:04:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:25 smithi160 ceph-mon[29041]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:25 smithi160 ceph-mon[29041]: Reconfiguring grafana.smithi160 (dependencies changed)... 2024-05-25T06:04:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:25 smithi160 ceph-mon[29041]: Reconfiguring daemon grafana.smithi160 on smithi160 2024-05-25T06:04:26.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:26 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3334681202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0838a204-c967-4d16-bf9c-3574b6649cc9"}]: dispatch 2024-05-25T06:04:26.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:26 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3334681202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0838a204-c967-4d16-bf9c-3574b6649cc9"}]': finished 2024-05-25T06:04:26.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:26 smithi160 ceph-mon[29041]: osdmap e6: 1 total, 0 up, 1 in 2024-05-25T06:04:26.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:26.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:26 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3412284624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:04:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:26 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3334681202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0838a204-c967-4d16-bf9c-3574b6649cc9"}]: dispatch 2024-05-25T06:04:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:26 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3334681202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0838a204-c967-4d16-bf9c-3574b6649cc9"}]': finished 2024-05-25T06:04:27.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:26 smithi203 ceph-mon[36030]: osdmap e6: 1 total, 0 up, 1 in 2024-05-25T06:04:27.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:27.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:26 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3412284624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:04:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:27 smithi160 ceph-mon[29041]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:27 smithi203 ceph-mon[36030]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:30.116 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:29 smithi160 ceph-mon[29041]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:29 smithi203 ceph-mon[36030]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:32.269 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:31 smithi160 ceph-mon[29041]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:31 smithi203 ceph-mon[36030]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:34.198 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:33 smithi160 ceph-mon[29041]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:33 smithi203 ceph-mon[36030]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:36.078 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:35 smithi160 ceph-mon[29041]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:36.078 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:36.079 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:35 smithi203 ceph-mon[36030]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:36.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:36.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:37.276 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:36 smithi160 ceph-mon[29041]: Reconfiguring prometheus.smithi160 (dependencies changed)... 2024-05-25T06:04:37.276 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:36 smithi160 ceph-mon[29041]: Reconfiguring daemon prometheus.smithi160 on smithi160 2024-05-25T06:04:37.276 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:36 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-25T06:04:37.276 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:36 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:36 smithi203 ceph-mon[36030]: Reconfiguring prometheus.smithi160 (dependencies changed)... 2024-05-25T06:04:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:36 smithi203 ceph-mon[36030]: Reconfiguring daemon prometheus.smithi160 on smithi160 2024-05-25T06:04:37.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:36 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-25T06:04:37.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:36 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:38.268 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:37 smithi160 ceph-mon[29041]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:38.268 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:37 smithi160 ceph-mon[29041]: Deploying daemon osd.0 on smithi160 2024-05-25T06:04:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:37 smithi203 ceph-mon[36030]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:37 smithi203 ceph-mon[36030]: Deploying daemon osd.0 on smithi160 2024-05-25T06:04:39.266 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:38 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:40.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:39 smithi160 ceph-mon[29041]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:39 smithi203 ceph-mon[36030]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:42.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:41 smithi203 ceph-mon[36030]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:42.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:42.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:42.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:04:42.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:42.214 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:41 smithi160 ceph-mon[29041]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:42.214 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:42.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:42.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:04:42.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: Reconfiguring crash.smithi203 (monmap changed)... 2024-05-25T06:04:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: Reconfiguring daemon crash.smithi203 on smithi203 2024-05-25T06:04:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:04:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:04:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:43.196 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: Reconfiguring crash.smithi203 (monmap changed)... 2024-05-25T06:04:43.196 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: Reconfiguring daemon crash.smithi203 on smithi203 2024-05-25T06:04:43.196 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:43.196 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:43.196 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:04:43.196 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:04:43.197 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:44.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: Reconfiguring mgr.smithi203.rotqex (monmap changed)... 2024-05-25T06:04:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: Reconfiguring daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:04:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:04:44.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:04:44.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:43 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: Reconfiguring mgr.smithi203.rotqex (monmap changed)... 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: Reconfiguring daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:04:44.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:43 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:45.120 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:45 smithi160 ceph-mon[29041]: Reconfiguring mon.smithi203 (monmap changed)... 2024-05-25T06:04:45.120 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:45 smithi160 ceph-mon[29041]: Reconfiguring daemon mon.smithi203 on smithi203 2024-05-25T06:04:45.282 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:45 smithi203 ceph-mon[36030]: Reconfiguring mon.smithi203 (monmap changed)... 2024-05-25T06:04:45.282 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:45 smithi203 ceph-mon[36030]: Reconfiguring daemon mon.smithi203 on smithi203 2024-05-25T06:04:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi160.front.sepia.ceph.com:3000"}]: dispatch 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi160.front.sepia.ceph.com:9095"}]: dispatch 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:46 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi160.front.sepia.ceph.com:3000"}]: dispatch 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi160.front.sepia.ceph.com:9095"}]: dispatch 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:46.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:46 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:47.155 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi160.front.sepia.ceph.com:3000"}]: dispatch 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi160.front.sepia.ceph.com:9095"}]: dispatch 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:47.156 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:47 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:04:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-05-25T06:04:47.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:04:47.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi160.front.sepia.ceph.com:3000"}]: dispatch 2024-05-25T06:04:47.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:04:47.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi160.front.sepia.ceph.com:9095"}]: dispatch 2024-05-25T06:04:47.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:47.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:47 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:48.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:48 smithi160 ceph-mon[29041]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:48.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:48 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:48.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:48 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:48 smithi203 ceph-mon[36030]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:48 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:48 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:49.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.310+0000 7f1f777fe700 1 -- 172.21.15.160:0/610630063 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f901048c0 con 0x7f1f78069210 2024-05-25T06:04:49.310 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 0 on host 'smithi160' 2024-05-25T06:04:49.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f1f78069210 msgr2=0x7f1f7806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:49.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f1f78069210 0x7f1f7806b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f80004f40 tx=0x7f1f8000b040 comp rx=0 tx=0).stop 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 msgr2=0x7f1f9010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f9010dfd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400b3b0 tx=0x7f1f84009e30 comp rx=0 tx=0).stop 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 shutdown_connections 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f1f78069210 0x7f1f7806b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f900fbcc0 0x7f1f9010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 --2- 172.21.15.160:0/610630063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900fcf70 0x7f1f9010dfd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 >> 172.21.15.160:0/610630063 conn(0x7f1f900f7420 msgr2=0x7f1f90066460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 shutdown_connections 2024-05-25T06:04:49.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:49.312+0000 7f1f955fa700 1 -- 172.21.15.160:0/610630063 wait complete. 2024-05-25T06:04:49.880 DEBUG:teuthology.orchestra.run.smithi160:osd.0> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.0.service 2024-05-25T06:04:49.883 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_3... 2024-05-25T06:04:49.883 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-25T06:04:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:50 smithi160 ceph-mon[29041]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:50.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:50.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:50.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:50 smithi160 ceph-mon[29041]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-25T06:04:50.313 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:04:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:50 smithi203 ceph-mon[36030]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:50.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:50 smithi203 ceph-mon[36030]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-25T06:04:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:51 smithi203 ceph-mon[36030]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-25T06:04:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:51 smithi203 ceph-mon[36030]: osdmap e7: 1 total, 0 up, 1 in 2024-05-25T06:04:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:51.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:51 smithi203 ceph-mon[36030]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:04:51.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:51 smithi203 ceph-mon[36030]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:51.707 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:51 smithi160 ceph-mon[29041]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-25T06:04:51.708 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:51 smithi160 ceph-mon[29041]: osdmap e7: 1 total, 0 up, 1 in 2024-05-25T06:04:51.708 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:51.708 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:51 smithi160 ceph-mon[29041]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:04:51.708 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:51 smithi160 ceph-mon[29041]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:51.708 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:04:51 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[46870]: 2024-05-25T06:04:51.307+0000 7fcb5c0a8700 -1 osd.0 0 waiting for initial osdmap 2024-05-25T06:04:51.708 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:04:51 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[46870]: 2024-05-25T06:04:51.311+0000 7fcb58ea7700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:04:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:52 smithi203 ceph-mon[36030]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-25T06:04:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:52 smithi203 ceph-mon[36030]: osdmap e8: 1 total, 0 up, 1 in 2024-05-25T06:04:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:52 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:52.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:52 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:52.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:52 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:52.656 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:52 smithi160 ceph-mon[29041]: from='osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-25T06:04:52.656 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:52 smithi160 ceph-mon[29041]: osdmap e8: 1 total, 0 up, 1 in 2024-05-25T06:04:52.656 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:52 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:52.656 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:52 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:52.656 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:52 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:52.793 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-25T06:04:52.793 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-25T06:04:52.793 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-25T06:04:52.793 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-25T06:04:52.793 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013267 s, 790 MB/s 2024-05-25T06:04:52.793 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-25T06:04:53.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:53 smithi160 ceph-mon[29041]: purged_snaps scrub starts 2024-05-25T06:04:53.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:53 smithi160 ceph-mon[29041]: purged_snaps scrub ok 2024-05-25T06:04:53.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:53 smithi160 ceph-mon[29041]: osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] boot 2024-05-25T06:04:53.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:53 smithi160 ceph-mon[29041]: osdmap e9: 1 total, 1 up, 1 in 2024-05-25T06:04:53.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:53.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:53 smithi160 ceph-mon[29041]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:53.446 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-05-25T06:04:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:53 smithi203 ceph-mon[36030]: purged_snaps scrub starts 2024-05-25T06:04:53.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:53 smithi203 ceph-mon[36030]: purged_snaps scrub ok 2024-05-25T06:04:53.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:53 smithi203 ceph-mon[36030]: osd.0 [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] boot 2024-05-25T06:04:53.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:53 smithi203 ceph-mon[36030]: osdmap e9: 1 total, 1 up, 1 in 2024-05-25T06:04:53.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:04:53.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:53 smithi203 ceph-mon[36030]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-25T06:04:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:04:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:54 smithi203 ceph-mon[36030]: osdmap e10: 1 total, 1 up, 1 in 2024-05-25T06:04:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:54 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:54 smithi160 ceph-mon[29041]: osdmap e10: 1 total, 1 up, 1 in 2024-05-25T06:04:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:54 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:04:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:55 smithi160 ceph-mon[29041]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:04:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:55 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:55 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:55 smithi203 ceph-mon[36030]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:04:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:55 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:55 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 -- 172.21.15.160:0/1918420487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 msgr2=0x7f3e900fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 --2- 172.21.15.160:0/1918420487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e900fd460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e84004940 tx=0x7f3e84009e30 comp rx=0 tx=0).stop 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 -- 172.21.15.160:0/1918420487 shutdown_connections 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 --2- 172.21.15.160:0/1918420487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e900fd460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 --2- 172.21.15.160:0/1918420487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 0x7f3e900fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 -- 172.21.15.160:0/1918420487 >> 172.21.15.160:0/1918420487 conn(0x7f3e900f74c0 msgr2=0x7f3e900f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 -- 172.21.15.160:0/1918420487 shutdown_connections 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.054+0000 7f3e96372700 1 -- 172.21.15.160:0/1918420487 wait complete. 2024-05-25T06:04:56.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e96372700 1 Processor -- start 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e96372700 1 -- start start 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e96372700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 0x7f3e9010daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e96372700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e9010dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e96372700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9010e5e0 con 0x7f3e900fcff0 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e96372700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9010ed90 con 0x7f3e900fbd40 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e9010dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e9010dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46072/0 (socket says 172.21.15.160:46072) 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8f7fe700 1 -- 172.21.15.160:0/4023372919 learned_addr learned my addr 172.21.15.160:0/4023372919 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8ffff700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 0x7f3e9010daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8f7fe700 1 -- 172.21.15.160:0/4023372919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 msgr2=0x7f3e9010daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8f7fe700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 0x7f3e9010daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:04:56.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.055+0000 7f3e8f7fe700 1 -- 172.21.15.160:0/4023372919 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e84005040 con 0x7f3e900fcff0 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.056+0000 7f3e8ffff700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 0x7f3e9010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.056+0000 7f3e8f7fe700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e9010dfe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009150 tx=0x7f3e84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.056+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e8400e040 con 0x7f3e900fcff0 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.056+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e8400bcc0 con 0x7f3e900fcff0 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.056+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9010f010 con 0x7f3e900fcff0 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.056+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9010f550 con 0x7f3e900fcff0 2024-05-25T06:04:56.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.057+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e8401c630 con 0x7f3e900fcff0 2024-05-25T06:04:56.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.058+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90101b80 con 0x7f3e900fcff0 2024-05-25T06:04:56.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.058+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f3e84026440 con 0x7f3e900fcff0 2024-05-25T06:04:56.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.058+0000 7f3e8cff9700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3e78069210 0x7f3e7806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:04:56.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.059+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f3e84086fd0 con 0x7f3e900fcff0 2024-05-25T06:04:56.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.059+0000 7f3e8ffff700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3e78069210 0x7f3e7806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:04:56.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.059+0000 7f3e8ffff700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3e78069210 0x7f3e7806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009fd0 tx=0x7f3e8000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:04:56.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.063+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3e84057cb0 con 0x7f3e900fcff0 2024-05-25T06:04:56.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:04:56.268+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e901048d0 con 0x7f3e78069210 2024-05-25T06:04:56.467 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:56 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:04:56.467 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:56 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:04:56.467 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:56 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:56 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:04:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:56 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:04:56.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:56 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:57.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:57 smithi160 ceph-mon[29041]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:04:57.364 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:57 smithi160 ceph-mon[29041]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:04:57.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:57 smithi203 ceph-mon[36030]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:04:57.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:57 smithi203 ceph-mon[36030]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:04:59.780 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.781 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:04:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:04:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:04:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:04:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:04:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:00.686 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:00 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi160 2024-05-25T06:05:00.686 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:00 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi160 to 12864M 2024-05-25T06:05:00.686 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:00 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/32515883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9"}]: dispatch 2024-05-25T06:05:00.686 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:00 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/32515883' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9"}]': finished 2024-05-25T06:05:00.687 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:00 smithi203 ceph-mon[36030]: osdmap e11: 2 total, 1 up, 2 in 2024-05-25T06:05:00.687 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:00 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:00.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:00 smithi160 ceph-mon[29041]: Detected new or changed devices on smithi160 2024-05-25T06:05:00.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:00 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi160 to 12864M 2024-05-25T06:05:00.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:00 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/32515883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9"}]: dispatch 2024-05-25T06:05:00.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:00 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/32515883' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9"}]': finished 2024-05-25T06:05:00.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:00 smithi160 ceph-mon[29041]: osdmap e11: 2 total, 1 up, 2 in 2024-05-25T06:05:00.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:00 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:02.170 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:02 smithi160 ceph-mon[29041]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:02.170 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:02 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:02.170 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:02 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:02.170 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:02 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3374144593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:05:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:02 smithi203 ceph-mon[36030]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:02.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3374144593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:05:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:04 smithi203 ceph-mon[36030]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:04 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:04 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:04 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:04 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:04 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:04 smithi160 ceph-mon[29041]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:04.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:04 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:04.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:04 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:04.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:04 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:04.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:04 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:04.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:04 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:06 smithi160 ceph-mon[29041]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:06 smithi203 ceph-mon[36030]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:08 smithi203 ceph-mon[36030]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-25T06:05:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:08.465 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:08 smithi160 ceph-mon[29041]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:08.465 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-25T06:05:08.465 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:09.302 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:09 smithi160 ceph-mon[29041]: Deploying daemon osd.1 on smithi160 2024-05-25T06:05:09.302 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:09 smithi203 ceph-mon[36030]: Deploying daemon osd.1 on smithi160 2024-05-25T06:05:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:10.283 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:10 smithi160 ceph-mon[29041]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:10 smithi203 ceph-mon[36030]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:11.210 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:11 smithi160 ceph-mon[29041]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:11 smithi203 ceph-mon[36030]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:13.789 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:13 smithi160 ceph-mon[29041]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:13.789 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:13.789 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:13.789 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:13 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:14.102 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:13 smithi203 ceph-mon[36030]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:14.102 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:14.102 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:14.102 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:13 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:15.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:15 smithi160 ceph-mon[29041]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:15.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:16.044 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:15 smithi203 ceph-mon[36030]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:16.044 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:16.044 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:17.045 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-05-25T06:05:17.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.044+0000 7f3e8cff9700 1 -- 172.21.15.160:0/4023372919 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3e901048d0 con 0x7f3e78069210 2024-05-25T06:05:17.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3e78069210 msgr2=0x7f3e7806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3e78069210 0x7f3e7806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009fd0 tx=0x7f3e8000b040 comp rx=0 tx=0).stop 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 msgr2=0x7f3e9010dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e9010dfe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009150 tx=0x7f3e84009e30 comp rx=0 tx=0).stop 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 shutdown_connections 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3e78069210 0x7f3e7806b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e900fbd40 0x7f3e9010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 --2- 172.21.15.160:0/4023372919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e900fcff0 0x7f3e9010dfe0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.046+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 >> 172.21.15.160:0/4023372919 conn(0x7f3e900f74c0 msgr2=0x7f3e90066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.047+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 shutdown_connections 2024-05-25T06:05:17.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:17.047+0000 7f3e96372700 1 -- 172.21.15.160:0/4023372919 wait complete. 2024-05-25T06:05:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:17 smithi160 ceph-mon[29041]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:17 smithi160 ceph-mon[29041]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-25T06:05:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:17 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:17 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:17 smithi203 ceph-mon[36030]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:17 smithi203 ceph-mon[36030]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-25T06:05:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:17 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:17 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:18.601 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.1.service 2024-05-25T06:05:18.603 INFO:tasks.cephadm:Deploying osd.2 on smithi160 with /dev/vg_nvme/lv_2... 2024-05-25T06:05:18.603 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-25T06:05:19.033 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:05:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:18 smithi203 ceph-mon[36030]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-25T06:05:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:18 smithi203 ceph-mon[36030]: osdmap e12: 2 total, 1 up, 2 in 2024-05-25T06:05:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:18 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:19.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:18 smithi203 ceph-mon[36030]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:05:19.158 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:18 smithi160 ceph-mon[29041]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-25T06:05:19.158 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:18 smithi160 ceph-mon[29041]: osdmap e12: 2 total, 1 up, 2 in 2024-05-25T06:05:19.158 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:18 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:19.158 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:18 smithi160 ceph-mon[29041]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:05:19.158 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:18 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[54545]: 2024-05-25T06:05:18.892+0000 7feab72e6700 -1 osd.1 0 waiting for initial osdmap 2024-05-25T06:05:19.158 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:18 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[54545]: 2024-05-25T06:05:18.896+0000 7feab00dd700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:05:20.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:19 smithi160 ceph-mon[29041]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:20.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:19 smithi160 ceph-mon[29041]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-25T06:05:20.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:19 smithi160 ceph-mon[29041]: osdmap e13: 2 total, 1 up, 2 in 2024-05-25T06:05:20.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:19 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:20.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:19 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:20.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:19 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:19 smithi203 ceph-mon[36030]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:19 smithi203 ceph-mon[36030]: from='osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-25T06:05:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:19 smithi203 ceph-mon[36030]: osdmap e13: 2 total, 1 up, 2 in 2024-05-25T06:05:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:19 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:19 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:19 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:21.012 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: purged_snaps scrub starts 2024-05-25T06:05:21.012 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: purged_snaps scrub ok 2024-05-25T06:05:21.012 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] boot 2024-05-25T06:05:21.012 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: osdmap e14: 2 total, 2 up, 2 in 2024-05-25T06:05:21.012 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:21.012 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:21.013 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:20 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: purged_snaps scrub starts 2024-05-25T06:05:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: purged_snaps scrub ok 2024-05-25T06:05:21.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: osd.1 [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] boot 2024-05-25T06:05:21.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: osdmap e14: 2 total, 2 up, 2 in 2024-05-25T06:05:21.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:05:21.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:21.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:20 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:22.208 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:21 smithi160 ceph-mon[29041]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:22.208 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:21 smithi160 ceph-mon[29041]: osdmap e15: 2 total, 2 up, 2 in 2024-05-25T06:05:22.208 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-25T06:05:22.208 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-25T06:05:22.208 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-25T06:05:22.209 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-25T06:05:22.209 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127689 s, 821 MB/s 2024-05-25T06:05:22.209 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-25T06:05:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:21 smithi203 ceph-mon[36030]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-25T06:05:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:21 smithi203 ceph-mon[36030]: osdmap e15: 2 total, 2 up, 2 in 2024-05-25T06:05:22.882 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-05-25T06:05:23.402 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:05:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:23 smithi160 ceph-mon[29041]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:23 smithi203 ceph-mon[36030]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:25.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 -- 172.21.15.160:0/3753703531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fbcb0 msgr2=0x7f89140fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:25.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 --2- 172.21.15.160:0/3753703531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fbcb0 0x7f89140fc0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8904004940 tx=0x7f8904009e30 comp rx=0 tx=0).stop 2024-05-25T06:05:25.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 -- 172.21.15.160:0/3753703531 shutdown_connections 2024-05-25T06:05:25.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 --2- 172.21.15.160:0/3753703531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fcf60 0x7f89140fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 --2- 172.21.15.160:0/3753703531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fbcb0 0x7f89140fc0c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 -- 172.21.15.160:0/3753703531 >> 172.21.15.160:0/3753703531 conn(0x7f89140f7430 msgr2=0x7f89140f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 -- 172.21.15.160:0/3753703531 shutdown_connections 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.647+0000 7f891b41f700 1 -- 172.21.15.160:0/3753703531 wait complete. 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f891b41f700 1 Processor -- start 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f891b41f700 1 -- start start 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f891b41f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fbcb0 0x7f891410c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f891b41f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 0x7f891410cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f891b41f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410d3c0 con 0x7f89140fcf60 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f891b41f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410dc10 con 0x7f89140fbcb0 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f89189ba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 0x7f891410cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:05:25.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f89189ba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 0x7f891410cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54618/0 (socket says 172.21.15.160:54618) 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f89189ba700 1 -- 172.21.15.160:0/3648731333 learned_addr learned my addr 172.21.15.160:0/3648731333 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.648+0000 7f89191bb700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fbcb0 0x7f891410c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f89189ba700 1 -- 172.21.15.160:0/3648731333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fbcb0 msgr2=0x7f891410c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f89189ba700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fbcb0 0x7f891410c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f89189ba700 1 -- 172.21.15.160:0/3648731333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904005040 con 0x7f89140fcf60 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f89191bb700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fbcb0 0x7f891410c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f89189ba700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 0x7f891410cdc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8910010d50 tx=0x7f891000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f891000ed60 con 0x7f89140fcf60 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89100090d0 con 0x7f89140fcf60 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891410de90 con 0x7f89140fcf60 2024-05-25T06:05:25.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8910012670 con 0x7f89140fcf60 2024-05-25T06:05:25.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.649+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891410e3d0 con 0x7f89140fcf60 2024-05-25T06:05:25.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.650+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891405f5c0 con 0x7f89140fcf60 2024-05-25T06:05:25.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.651+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f8910018030 con 0x7f89140fcf60 2024-05-25T06:05:25.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.651+0000 7f8909ffb700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8900069210 0x7f890006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:05:25.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.651+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f8910016070 con 0x7f89140fcf60 2024-05-25T06:05:25.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.651+0000 7f89191bb700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8900069210 0x7f890006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:05:25.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.652+0000 7f89191bb700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8900069210 0x7f890006b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8904009b30 tx=0x7f8904009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:05:25.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.656+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8910059910 con 0x7f89140fcf60 2024-05-25T06:05:25.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:25.857+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f891410e750 con 0x7f8900069210 2024-05-25T06:05:26.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: Detected new or changed devices on smithi160 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi160 to 6432M 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:26.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:26.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.109 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:26.113 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi160 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi160 to 6432M 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:26.114 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:26.115 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:26.881 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:26 smithi160 ceph-mon[29041]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:05:26.881 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:05:26.881 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:05:26.881 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:26 smithi203 ceph-mon[36030]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:05:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:05:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:05:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:27.895 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:27 smithi160 ceph-mon[29041]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:28.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:27 smithi203 ceph-mon[36030]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:30.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:29 smithi203 ceph-mon[36030]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:30.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:29 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1391281419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63ed9e8f-565f-48c2-bcc6-4be1c0262eef"}]: dispatch 2024-05-25T06:05:30.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:29 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1391281419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63ed9e8f-565f-48c2-bcc6-4be1c0262eef"}]': finished 2024-05-25T06:05:30.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:29 smithi203 ceph-mon[36030]: osdmap e16: 3 total, 2 up, 3 in 2024-05-25T06:05:30.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:30.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:29 smithi160 ceph-mon[29041]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:30.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:29 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1391281419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63ed9e8f-565f-48c2-bcc6-4be1c0262eef"}]: dispatch 2024-05-25T06:05:30.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:29 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1391281419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63ed9e8f-565f-48c2-bcc6-4be1c0262eef"}]': finished 2024-05-25T06:05:30.202 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:29 smithi160 ceph-mon[29041]: osdmap e16: 3 total, 2 up, 3 in 2024-05-25T06:05:30.202 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:31.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:30 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1233320540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:05:31.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:30 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1233320540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:05:32.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:31 smithi160 ceph-mon[29041]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:31 smithi203 ceph-mon[36030]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:32.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:32.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:32.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:32.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:32.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:34.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:33 smithi160 ceph-mon[29041]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:33 smithi203 ceph-mon[36030]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:36.033 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:35 smithi160 ceph-mon[29041]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:35 smithi203 ceph-mon[36030]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:37.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:36 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-25T06:05:37.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:36 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:36 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-25T06:05:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:36 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:37 smithi160 ceph-mon[29041]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:37 smithi160 ceph-mon[29041]: Deploying daemon osd.2 on smithi160 2024-05-25T06:05:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:37 smithi203 ceph-mon[36030]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:37 smithi203 ceph-mon[36030]: Deploying daemon osd.2 on smithi160 2024-05-25T06:05:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:38 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:39.414 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:40.118 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:39 smithi160 ceph-mon[29041]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:39 smithi203 ceph-mon[36030]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:42.214 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:41 smithi160 ceph-mon[29041]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:42.214 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:42.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:42.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:41 smithi203 ceph-mon[36030]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:44.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:43 smithi160 ceph-mon[29041]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:43 smithi203 ceph-mon[36030]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:44.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.660+0000 7f8909ffb700 1 -- 172.21.15.160:0/3648731333 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f891410e750 con 0x7f8900069210 2024-05-25T06:05:44.662 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 2 on host 'smithi160' 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8900069210 msgr2=0x7f890006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8900069210 0x7f890006b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8904009b30 tx=0x7f8904009e30 comp rx=0 tx=0).stop 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 msgr2=0x7f891410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 0x7f891410cdc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8910010d50 tx=0x7f891000b040 comp rx=0 tx=0).stop 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 shutdown_connections 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8900069210 0x7f890006b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89140fbcb0 0x7f891410c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.662+0000 7f891b41f700 1 --2- 172.21.15.160:0/3648731333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89140fcf60 0x7f891410cdc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.663+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 >> 172.21.15.160:0/3648731333 conn(0x7f89140f7430 msgr2=0x7f89140f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:05:44.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.663+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 shutdown_connections 2024-05-25T06:05:44.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:05:44.663+0000 7f891b41f700 1 -- 172.21.15.160:0/3648731333 wait complete. 2024-05-25T06:05:45.612 DEBUG:teuthology.orchestra.run.smithi160:osd.2> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.2.service 2024-05-25T06:05:45.615 INFO:tasks.cephadm:Deploying osd.3 on smithi203 with /dev/vg_nvme/lv_4... 2024-05-25T06:05:45.615 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-25T06:05:45.849 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:45 smithi160 ceph-mon[29041]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:45.849 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:45.849 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:45.850 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:45 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[61828]: 2024-05-25T06:05:45.732+0000 7fbee4e79540 -1 osd.2 0 log_to_monitors true 2024-05-25T06:05:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:05:45.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:45 smithi203 ceph-mon[36030]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:45.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:45 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:45.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:45 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:46.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:46 smithi203 ceph-mon[36030]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-25T06:05:46.909 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:46 smithi160 ceph-mon[29041]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-25T06:05:47.650 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-25T06:05:47.650 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-25T06:05:47.650 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-25T06:05:47.650 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-25T06:05:47.651 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112752 s, 930 MB/s 2024-05-25T06:05:47.651 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-25T06:05:47.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:47 smithi160 ceph-mon[29041]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:47.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:47 smithi160 ceph-mon[29041]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-25T06:05:47.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:47 smithi160 ceph-mon[29041]: osdmap e17: 3 total, 2 up, 3 in 2024-05-25T06:05:47.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:47 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:47.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:47 smithi160 ceph-mon[29041]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:05:47.970 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:47 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[61828]: 2024-05-25T06:05:47.656+0000 7fbedb64a700 -1 osd.2 0 waiting for initial osdmap 2024-05-25T06:05:47.970 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:47 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[61828]: 2024-05-25T06:05:47.663+0000 7fbed5c44700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:05:48.037 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:47 smithi203 ceph-mon[36030]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:48.037 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:47 smithi203 ceph-mon[36030]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-25T06:05:48.038 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:47 smithi203 ceph-mon[36030]: osdmap e17: 3 total, 2 up, 3 in 2024-05-25T06:05:48.038 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:47 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:48.038 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:47 smithi203 ceph-mon[36030]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:05:48.196 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-05-25T06:05:48.449 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:05:49.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:48 smithi203 ceph-mon[36030]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-25T06:05:49.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:48 smithi203 ceph-mon[36030]: osdmap e18: 3 total, 2 up, 3 in 2024-05-25T06:05:49.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:48 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:49.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:48 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:49.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:48 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:49.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:48 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:49.188 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:48 smithi160 ceph-mon[29041]: from='osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-25T06:05:49.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:48 smithi160 ceph-mon[29041]: osdmap e18: 3 total, 2 up, 3 in 2024-05-25T06:05:49.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:48 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:49.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:48 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:49.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:48 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:49.189 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:48 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:50.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 -- 172.21.15.203:0/4186357796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fe080 msgr2=0x7fa5080fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:50.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 --2- 172.21.15.203:0/4186357796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fe080 0x7fa5080fe4f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc004940 tx=0x7fa4fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:05:50.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 -- 172.21.15.203:0/4186357796 shutdown_connections 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 --2- 172.21.15.203:0/4186357796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fe080 0x7fa5080fe4f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 --2- 172.21.15.203:0/4186357796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5080fce80 0x7fa5080fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 -- 172.21.15.203:0/4186357796 >> 172.21.15.203:0/4186357796 conn(0x7fa5080f8590 msgr2=0x7fa5080fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 -- 172.21.15.203:0/4186357796 shutdown_connections 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.029+0000 7fa50e851700 1 -- 172.21.15.203:0/4186357796 wait complete. 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa50e851700 1 Processor -- start 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa50e851700 1 -- start start 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa50e851700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 0x7fa508071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa50e851700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5080fe080 0x7fa5080702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa50e851700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508070860 con 0x7fa5080fe080 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa50e851700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080709d0 con 0x7fa5080fce80 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa507fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 0x7fa508071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:05:50.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa507fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 0x7fa508071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54406/0 (socket says 172.21.15.203:54406) 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.030+0000 7fa507fff700 1 -- 172.21.15.203:0/2535284908 learned_addr learned my addr 172.21.15.203:0/2535284908 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa5077fe700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5080fe080 0x7fa5080702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa507fff700 1 -- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5080fe080 msgr2=0x7fa5080702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa507fff700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5080fe080 0x7fa5080702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa507fff700 1 -- 172.21.15.203:0/2535284908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc005040 con 0x7fa5080fce80 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa507fff700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 0x7fa508071de0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8002a80 tx=0x7fa4f8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4f8007cc0 con 0x7fa5080fce80 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa508070cb0 con 0x7fa5080fce80 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4f800f040 con 0x7fa5080fce80 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4f800da30 con 0x7fa5080fce80 2024-05-25T06:05:50.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.031+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa508115a30 con 0x7fa5080fce80 2024-05-25T06:05:50.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.032+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50805f5c0 con 0x7fa5080fce80 2024-05-25T06:05:50.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.033+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7fa4f800db90 con 0x7fa5080fce80 2024-05-25T06:05:50.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.033+0000 7fa504ff9700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fa4f0069210 0x7fa4f006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:05:50.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.033+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7fa4f8086540 con 0x7fa5080fce80 2024-05-25T06:05:50.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.033+0000 7fa5077fe700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fa4f0069210 0x7fa4f006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:05:50.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.034+0000 7fa5077fe700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fa4f0069210 0x7fa4f006b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00b970 tx=0x7fa4fc009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:05:50.038 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.038+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa4f8053190 con 0x7fa5080fce80 2024-05-25T06:05:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: purged_snaps scrub starts 2024-05-25T06:05:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: purged_snaps scrub ok 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] boot 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: osdmap e19: 3 total, 3 up, 3 in 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: osdmap e20: 3 total, 3 up, 3 in 2024-05-25T06:05:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: purged_snaps scrub starts 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: purged_snaps scrub ok 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: osd.2 [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] boot 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: osdmap e19: 3 total, 3 up, 3 in 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-25T06:05:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: osdmap e20: 3 total, 3 up, 3 in 2024-05-25T06:05:50.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-25T06:05:50.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:50.228+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa508102440 con 0x7fa4f0069210 2024-05-25T06:05:51.219 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-25T06:05:51.220 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:51.220 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66235]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:51.220 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66235]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:51.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:05:51.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:05:51.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:51.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-25T06:05:51.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:50 smithi160 ceph-mon[29041]: osdmap e21: 3 total, 3 up, 3 in 2024-05-25T06:05:51.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:05:51.318 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:05:51.318 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:51.318 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-25T06:05:51.318 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:50 smithi203 ceph-mon[36030]: osdmap e21: 3 total, 3 up, 3 in 2024-05-25T06:05:51.620 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66369]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-25T06:05:51.620 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66369]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:51.620 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66369]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:51.620 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66369]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:51.885 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66375]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-25T06:05:51.885 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66375]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:51.885 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66375]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:51.885 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66375]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:52.138 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66410]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-25T06:05:52.138 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66410]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:52.138 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66410]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:52.138 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:05:51 smithi160 sudo[66410]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:52.138 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:05:52.138 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:05:52.138 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-25T06:05:52.138 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.138 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: osdmap e22: 3 total, 3 up, 3 in 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:52.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:05:52.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:05:52.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-25T06:05:52.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: osdmap e22: 3 total, 3 up, 3 in 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:52.409 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-25T06:05:52.409 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:52.409 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66479]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:52.409 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66479]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:52.659 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66567]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-25T06:05:52.659 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66567]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:52.659 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66567]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:52.659 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66567]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:52.916 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-25T06:05:52.916 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:52.916 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66576]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:52.916 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66576]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:53.182 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:52 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi160 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:52 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi160 to 4288M 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:52 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:52 smithi203 ceph-mon[36030]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:53.183 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39482]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:53.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-25T06:05:53.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:53.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66590]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:53.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 sudo[66590]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:53.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 ceph-mon[29041]: Detected new or changed devices on smithi160 2024-05-25T06:05:53.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi160 to 4288M 2024-05-25T06:05:53.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:05:53.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:52 smithi160 ceph-mon[29041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-25T06:05:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-25T06:05:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-25T06:05:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39485]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-25T06:05:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 sudo[39485]: pam_unix(sudo:session): session closed for user root 2024-05-25T06:05:53.932 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:05:53.932+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa4f8053560 con 0x7fa5080fce80 2024-05-25T06:05:54.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:05:54.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/474526791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6a9eac-ed34-4283-9693-053d313f1524"}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6a9eac-ed34-4283-9693-053d313f1524"}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee6a9eac-ed34-4283-9693-053d313f1524"}]': finished 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: osdmap e23: 4 total, 3 up, 4 in 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-25T06:05:54.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: Cluster is now healthy 2024-05-25T06:05:54.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-25T06:05:54.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:54.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/474526791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6a9eac-ed34-4283-9693-053d313f1524"}]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6a9eac-ed34-4283-9693-053d313f1524"}]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee6a9eac-ed34-4283-9693-053d313f1524"}]': finished 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: osdmap e23: 4 total, 3 up, 4 in 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:05:54.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:05:54.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:05:54.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-25T06:05:54.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: Cluster is now healthy 2024-05-25T06:05:54.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-25T06:05:54.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:54.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:05:55.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:54 smithi160 ceph-mon[29041]: mgrmap e19: smithi160.nalzlg(active, since 2m), standbys: smithi203.rotqex 2024-05-25T06:05:55.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:54 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/3438217689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:05:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:54 smithi203 ceph-mon[36030]: mgrmap e19: smithi160.nalzlg(active, since 2m), standbys: smithi203.rotqex 2024-05-25T06:05:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:54 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/3438217689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:05:55.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:55 smithi160 ceph-mon[29041]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:05:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:55 smithi203 ceph-mon[36030]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:05:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:57.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:57.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:57.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:57.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:57 smithi203 ceph-mon[36030]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.5 KiB/s rd, 264 KiB/s wr, 18 op/s 2024-05-25T06:05:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:57.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:05:57.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:05:57.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:05:57.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:57 smithi160 ceph-mon[29041]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.5 KiB/s rd, 264 KiB/s wr, 18 op/s 2024-05-25T06:05:59.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:05:59 smithi160 ceph-mon[29041]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.9 KiB/s rd, 188 KiB/s wr, 12 op/s 2024-05-25T06:06:00.092 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:05:59 smithi203 ceph-mon[36030]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.9 KiB/s rd, 188 KiB/s wr, 12 op/s 2024-05-25T06:06:00.934 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:00 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-25T06:06:00.934 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:00 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:00.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:00 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-25T06:06:00.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:00 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:01.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:01 smithi203 ceph-mon[36030]: Deploying daemon osd.3 on smithi203 2024-05-25T06:06:01.784 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:01 smithi203 ceph-mon[36030]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 146 KiB/s wr, 10 op/s 2024-05-25T06:06:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:01 smithi160 ceph-mon[29041]: Deploying daemon osd.3 on smithi203 2024-05-25T06:06:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:01 smithi160 ceph-mon[29041]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 146 KiB/s wr, 10 op/s 2024-05-25T06:06:03.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:03 smithi203 ceph-mon[36030]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-25T06:06:03.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:03 smithi160 ceph-mon[29041]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-25T06:06:05.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:05 smithi160 ceph-mon[29041]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 112 KiB/s wr, 7 op/s 2024-05-25T06:06:05.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:06.016 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:05 smithi203 ceph-mon[36030]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 112 KiB/s wr, 7 op/s 2024-05-25T06:06:06.016 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:06.016 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:06.016 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:07.735 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:07 smithi203 ceph-mon[36030]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-25T06:06:07.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:07 smithi160 ceph-mon[29041]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-25T06:06:08.439 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 3 on host 'smithi203' 2024-05-25T06:06:08.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.438+0000 7fa504ff9700 1 -- 172.21.15.203:0/2535284908 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa508102440 con 0x7fa4f0069210 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fa4f0069210 msgr2=0x7fa4f006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fa4f0069210 0x7fa4f006b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00b970 tx=0x7fa4fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 msgr2=0x7fa508071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 0x7fa508071de0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8002a80 tx=0x7fa4f8004600 comp rx=0 tx=0).stop 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 shutdown_connections 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fa4f0069210 0x7fa4f006b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.440+0000 7fa50e851700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5080fce80 0x7fa508071de0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.441+0000 7fa50e851700 1 --2- 172.21.15.203:0/2535284908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5080fe080 0x7fa5080702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.441+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 >> 172.21.15.203:0/2535284908 conn(0x7fa5080f8590 msgr2=0x7fa5080f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:06:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.441+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 shutdown_connections 2024-05-25T06:06:08.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:08.441+0000 7fa50e851700 1 -- 172.21.15.203:0/2535284908 wait complete. 2024-05-25T06:06:09.044 DEBUG:teuthology.orchestra.run.smithi203:osd.3> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.3.service 2024-05-25T06:06:09.047 INFO:tasks.cephadm:Deploying osd.4 on smithi203 with /dev/vg_nvme/lv_3... 2024-05-25T06:06:09.047 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-25T06:06:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:09.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:09 smithi203 ceph-mon[36030]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:06:09.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:09.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:09 smithi203 ceph-mon[36030]: from='osd.3 [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:06:09.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:09 smithi203 ceph-mon[36030]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:06:09.650 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:06:09 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[41551]: 2024-05-25T06:06:09.329+0000 7fe96e346540 -1 osd.3 0 log_to_monitors true 2024-05-25T06:06:09.715 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:06:09.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:09.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:09.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:09 smithi160 ceph-mon[29041]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:06:09.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:09.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:09 smithi160 ceph-mon[29041]: from='osd.3 [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:06:09.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:09 smithi160 ceph-mon[29041]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:06:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:10 smithi160 ceph-mon[29041]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-25T06:06:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:10 smithi160 ceph-mon[29041]: osdmap e24: 4 total, 3 up, 4 in 2024-05-25T06:06:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:10 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:10 smithi160 ceph-mon[29041]: from='osd.3 [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:10 smithi160 ceph-mon[29041]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:10 smithi203 ceph-mon[36030]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-25T06:06:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:10 smithi203 ceph-mon[36030]: osdmap e24: 4 total, 3 up, 4 in 2024-05-25T06:06:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:10 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:10.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:10 smithi203 ceph-mon[36030]: from='osd.3 [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:10.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:10 smithi203 ceph-mon[36030]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:10.650 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:06:10 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[41551]: 2024-05-25T06:06:10.412+0000 7fe964b17700 -1 osd.3 0 waiting for initial osdmap 2024-05-25T06:06:10.650 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:06:10 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[41551]: 2024-05-25T06:06:10.419+0000 7fe961916700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:06:11.567 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-25T06:06:11.568 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-25T06:06:11.568 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-25T06:06:11.568 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-25T06:06:11.568 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012773 s, 821 MB/s 2024-05-25T06:06:11.568 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-25T06:06:11.670 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-25T06:06:11.671 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: osdmap e25: 4 total, 3 up, 4 in 2024-05-25T06:06:11.671 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:11.671 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:11.671 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:06:11.671 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:11.672 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:11.672 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: osd.3 [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] boot 2024-05-25T06:06:11.672 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: osdmap e26: 4 total, 4 up, 4 in 2024-05-25T06:06:11.673 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:11 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:11.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: osdmap e25: 4 total, 3 up, 4 in 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: osd.3 [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] boot 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: osdmap e26: 4 total, 4 up, 4 in 2024-05-25T06:06:11.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:11 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:06:12.589 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:12 smithi203 ceph-mon[36030]: purged_snaps scrub starts 2024-05-25T06:06:12.589 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:12 smithi203 ceph-mon[36030]: purged_snaps scrub ok 2024-05-25T06:06:12.589 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:12 smithi203 ceph-mon[36030]: osdmap e27: 4 total, 4 up, 4 in 2024-05-25T06:06:12.616 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-05-25T06:06:12.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:12 smithi160 ceph-mon[29041]: purged_snaps scrub starts 2024-05-25T06:06:12.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:12 smithi160 ceph-mon[29041]: purged_snaps scrub ok 2024-05-25T06:06:12.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:12 smithi160 ceph-mon[29041]: osdmap e27: 4 total, 4 up, 4 in 2024-05-25T06:06:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:13 smithi203 ceph-mon[36030]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:13 smithi203 ceph-mon[36030]: osdmap e28: 4 total, 4 up, 4 in 2024-05-25T06:06:13.711 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:06:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:13 smithi160 ceph-mon[29041]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:13 smithi160 ceph-mon[29041]: osdmap e28: 4 total, 4 up, 4 in 2024-05-25T06:06:15.644 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: osdmap e29: 4 total, 4 up, 4 in 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:15.645 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:15 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: osdmap e29: 4 total, 4 up, 4 in 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:15 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:15.864 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 -- 172.21.15.203:0/4043794918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fed70 msgr2=0x7fd1180faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 --2- 172.21.15.203:0/4043794918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fed70 0x7fd1180faeb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd10c004940 tx=0x7fd10c009e30 comp rx=0 tx=0).stop 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 -- 172.21.15.203:0/4043794918 shutdown_connections 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 --2- 172.21.15.203:0/4043794918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fed70 0x7fd1180faeb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 --2- 172.21.15.203:0/4043794918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1180fe420 0x7fd1180fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 -- 172.21.15.203:0/4043794918 >> 172.21.15.203:0/4043794918 conn(0x7fd1180f6be0 msgr2=0x7fd1180f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 -- 172.21.15.203:0/4043794918 shutdown_connections 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 -- 172.21.15.203:0/4043794918 wait complete. 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.864+0000 7fd11e97f700 1 Processor -- start 2024-05-25T06:06:15.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd11e97f700 1 -- start start 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd11e97f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 0x7fd1180fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd11e97f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1180fed70 0x7fd1180fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd11e97f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180fe030 con 0x7fd1180fed70 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd11e97f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180fc620 con 0x7fd1180fe420 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 0x7fd1180fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 0x7fd1180fda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:33208/0 (socket says 172.21.15.203:33208) 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 -- 172.21.15.203:0/3375732931 learned_addr learned my addr 172.21.15.203:0/3375732931 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd1177fe700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1180fed70 0x7fd1180fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:06:15.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 -- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1180fed70 msgr2=0x7fd1180fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1180fed70 0x7fd1180fc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 -- 172.21.15.203:0/3375732931 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c005040 con 0x7fd1180fe420 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.865+0000 7fd117fff700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 0x7fd1180fda30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd108002a50 tx=0x7fd1080046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.866+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd108007cc0 con 0x7fd1180fe420 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.866+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd10800f040 con 0x7fd1180fe420 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.866+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1180fc8d0 con 0x7fd1180fe420 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.866+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10800da30 con 0x7fd1180fe420 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.866+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1180fce90 con 0x7fd1180fe420 2024-05-25T06:06:15.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.867+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11804c710 con 0x7fd1180fe420 2024-05-25T06:06:15.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.867+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd10800db90 con 0x7fd1180fe420 2024-05-25T06:06:15.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.868+0000 7fd114ff9700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd1000692a0 0x7fd10006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:06:15.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.868+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fd1080868b0 con 0x7fd1180fe420 2024-05-25T06:06:15.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.868+0000 7fd1177fe700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd1000692a0 0x7fd10006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:06:15.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.868+0000 7fd1177fe700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd1000692a0 0x7fd10006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009fd0 tx=0x7fd10c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:06:15.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:15.872+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd108056c90 con 0x7fd1180fe420 2024-05-25T06:06:16.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:16.067+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1181024e0 con 0x7fd1000692a0 2024-05-25T06:06:16.617 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:16 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi203 2024-05-25T06:06:16.618 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:16 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi203 to 15936M 2024-05-25T06:06:16.618 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:16 smithi203 ceph-mon[36030]: osdmap e30: 4 total, 4 up, 4 in 2024-05-25T06:06:16.618 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:16 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:06:16.618 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:16 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:06:16.618 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:16 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:16 smithi160 ceph-mon[29041]: Detected new or changed devices on smithi203 2024-05-25T06:06:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:16 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi203 to 15936M 2024-05-25T06:06:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:16 smithi160 ceph-mon[29041]: osdmap e30: 4 total, 4 up, 4 in 2024-05-25T06:06:16.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:16 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:06:16.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:16 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:06:16.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:16 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:17.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:17 smithi160 ceph-mon[29041]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:06:17.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:17 smithi160 ceph-mon[29041]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-05-25T06:06:17.755 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:17 smithi203 ceph-mon[36030]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:06:17.755 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:17 smithi203 ceph-mon[36030]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-05-25T06:06:18.883 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:18 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1806736881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ecec6a-830e-456f-8e7a-429c2c16eec1"}]: dispatch 2024-05-25T06:06:18.883 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:18 smithi203 ceph-mon[36030]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ecec6a-830e-456f-8e7a-429c2c16eec1"}]: dispatch 2024-05-25T06:06:18.883 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:18 smithi203 ceph-mon[36030]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4ecec6a-830e-456f-8e7a-429c2c16eec1"}]': finished 2024-05-25T06:06:18.883 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:18 smithi203 ceph-mon[36030]: osdmap e31: 5 total, 4 up, 5 in 2024-05-25T06:06:18.883 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:18 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:18.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:18 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1806736881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ecec6a-830e-456f-8e7a-429c2c16eec1"}]: dispatch 2024-05-25T06:06:18.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:18 smithi160 ceph-mon[29041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ecec6a-830e-456f-8e7a-429c2c16eec1"}]: dispatch 2024-05-25T06:06:18.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:18 smithi160 ceph-mon[29041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4ecec6a-830e-456f-8e7a-429c2c16eec1"}]': finished 2024-05-25T06:06:18.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:18 smithi160 ceph-mon[29041]: osdmap e31: 5 total, 4 up, 5 in 2024-05-25T06:06:18.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:18 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:19 smithi203 ceph-mon[36030]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-25T06:06:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:19 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2415890076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:06:19.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:19 smithi160 ceph-mon[29041]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-25T06:06:19.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:19 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2415890076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:06:21.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:21 smithi160 ceph-mon[29041]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-25T06:06:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:21 smithi203 ceph-mon[36030]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-25T06:06:23.691 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:23 smithi203 ceph-mon[36030]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-25T06:06:23.691 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:23.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:23 smithi160 ceph-mon[29041]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-25T06:06:23.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:25.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:25 smithi203 ceph-mon[36030]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-25T06:06:25.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:25 smithi160 ceph-mon[29041]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-25T06:06:26.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-25T06:06:26.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:26.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:26 smithi160 ceph-mon[29041]: Deploying daemon osd.4 on smithi203 2024-05-25T06:06:27.035 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-25T06:06:27.035 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:27.035 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:26 smithi203 ceph-mon[36030]: Deploying daemon osd.4 on smithi203 2024-05-25T06:06:27.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:27 smithi203 ceph-mon[36030]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:27.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:27 smithi160 ceph-mon[29041]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:29.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:29 smithi203 ceph-mon[36030]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:29 smithi160 ceph-mon[29041]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:31.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:31.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:31.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:31 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:31.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:31 smithi203 ceph-mon[36030]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:31 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:31 smithi160 ceph-mon[29041]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:33 smithi203 ceph-mon[36030]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:33.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:33 smithi160 ceph-mon[29041]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:34.334 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.333+0000 7fd114ff9700 1 -- 172.21.15.203:0/3375732931 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd1181024e0 con 0x7fd1000692a0 2024-05-25T06:06:34.334 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 4 on host 'smithi203' 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.335+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd1000692a0 msgr2=0x7fd10006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.335+0000 7fd11e97f700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd1000692a0 0x7fd10006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009fd0 tx=0x7fd10c009e30 comp rx=0 tx=0).stop 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 msgr2=0x7fd1180fda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 0x7fd1180fda30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd108002a50 tx=0x7fd1080046b0 comp rx=0 tx=0).stop 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 shutdown_connections 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd1000692a0 0x7fd10006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1180fe420 0x7fd1180fda30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 --2- 172.21.15.203:0/3375732931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1180fed70 0x7fd1180fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:34.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 >> 172.21.15.203:0/3375732931 conn(0x7fd1180f6be0 msgr2=0x7fd1180f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:06:34.337 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 shutdown_connections 2024-05-25T06:06:34.337 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:34.336+0000 7fd11e97f700 1 -- 172.21.15.203:0/3375732931 wait complete. 2024-05-25T06:06:34.889 DEBUG:teuthology.orchestra.run.smithi203:osd.4> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.4.service 2024-05-25T06:06:34.891 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_2... 2024-05-25T06:06:34.891 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-25T06:06:35.224 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:06:35.574 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: from='osd.4 [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:06:35.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:06:35.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:35.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: from='osd.4 [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:06:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:06:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:36.547 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:36 smithi203 ceph-mon[36030]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-25T06:06:36.548 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:36 smithi203 ceph-mon[36030]: osdmap e32: 5 total, 4 up, 5 in 2024-05-25T06:06:36.548 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:36 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:36.548 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:36 smithi203 ceph-mon[36030]: from='osd.4 [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:36.548 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:36 smithi203 ceph-mon[36030]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:36.548 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:06:36 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[48383]: 2024-05-25T06:06:36.337+0000 7f473a7f5700 -1 osd.4 0 waiting for initial osdmap 2024-05-25T06:06:36.548 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:06:36 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[48383]: 2024-05-25T06:06:36.348+0000 7f47375f4700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:06:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:36 smithi160 ceph-mon[29041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-25T06:06:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:36 smithi160 ceph-mon[29041]: osdmap e32: 5 total, 4 up, 5 in 2024-05-25T06:06:36.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:36 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:36.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:36 smithi160 ceph-mon[29041]: from='osd.4 [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:36.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:36 smithi160 ceph-mon[29041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: osdmap e33: 5 total, 4 up, 5 in 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: osd.4 [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] boot 2024-05-25T06:06:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:37 smithi203 ceph-mon[36030]: osdmap e34: 5 total, 5 up, 5 in 2024-05-25T06:06:37.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-25T06:06:37.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: osdmap e33: 5 total, 4 up, 5 in 2024-05-25T06:06:37.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:37.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:37.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-25T06:06:37.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: osd.4 [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] boot 2024-05-25T06:06:37.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:37 smithi160 ceph-mon[29041]: osdmap e34: 5 total, 5 up, 5 in 2024-05-25T06:06:37.789 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-25T06:06:37.790 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-25T06:06:37.790 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-25T06:06:37.790 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-25T06:06:37.790 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122735 s, 854 MB/s 2024-05-25T06:06:37.790 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-25T06:06:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:38 smithi203 ceph-mon[36030]: purged_snaps scrub starts 2024-05-25T06:06:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:38 smithi203 ceph-mon[36030]: purged_snaps scrub ok 2024-05-25T06:06:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:38 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:38 smithi160 ceph-mon[29041]: purged_snaps scrub starts 2024-05-25T06:06:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:38 smithi160 ceph-mon[29041]: purged_snaps scrub ok 2024-05-25T06:06:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:06:38.905 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-05-25T06:06:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:39 smithi203 ceph-mon[36030]: osdmap e35: 5 total, 5 up, 5 in 2024-05-25T06:06:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:39 smithi203 ceph-mon[36030]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:39 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:39.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:39 smithi160 ceph-mon[29041]: osdmap e35: 5 total, 5 up, 5 in 2024-05-25T06:06:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:39 smithi160 ceph-mon[29041]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:39.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:39.799 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi203 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi203 to 7968M 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:41.001 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:41.002 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:41.002 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:40 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: Detected new or changed devices on smithi203 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi203 to 7968M 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:41.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:41.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:06:41.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:40 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 -- 172.21.15.203:0/4061644858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fdfd0 msgr2=0x7f8df80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4061644858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fdfd0 0x7f8df80fe440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8dec004940 tx=0x7f8dec009e30 comp rx=0 tx=0).stop 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 -- 172.21.15.203:0/4061644858 shutdown_connections 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4061644858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fdfd0 0x7f8df80fe440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4061644858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8df80fce60 0x7f8df80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 -- 172.21.15.203:0/4061644858 >> 172.21.15.203:0/4061644858 conn(0x7f8df80f8540 msgr2=0x7f8df80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 -- 172.21.15.203:0/4061644858 shutdown_connections 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.400+0000 7f8dfe091700 1 -- 172.21.15.203:0/4061644858 wait complete. 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8dfe091700 1 Processor -- start 2024-05-25T06:06:41.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8dfe091700 1 -- start start 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8dfe091700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 0x7f8df810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8dfe091700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8df80fdfd0 0x7f8df810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8dfe091700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df810e470 con 0x7f8df80fdfd0 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8dfe091700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df810e5e0 con 0x7f8df80fce60 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 0x7f8df810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 0x7f8df810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54718/0 (socket says 172.21.15.203:54718) 2024-05-25T06:06:41.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df77fe700 1 -- 172.21.15.203:0/4251192141 learned_addr learned my addr 172.21.15.203:0/4251192141 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df6ffd700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8df80fdfd0 0x7f8df810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df77fe700 1 -- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8df80fdfd0 msgr2=0x7f8df810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df77fe700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8df80fdfd0 0x7f8df810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.401+0000 7f8df77fe700 1 -- 172.21.15.203:0/4251192141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec005040 con 0x7f8df80fce60 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.402+0000 7f8df77fe700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 0x7f8df810d930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8de8002a50 tx=0x7f8de80046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.402+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de8007cc0 con 0x7f8df80fce60 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.402+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8de800f040 con 0x7f8df80fce60 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.402+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df8112fe0 con 0x7f8df80fce60 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.402+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de800da70 con 0x7f8df80fce60 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.402+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df8113570 con 0x7f8df80fce60 2024-05-25T06:06:41.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.403+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df804c710 con 0x7f8df80fce60 2024-05-25T06:06:41.405 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.403+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8de800dc10 con 0x7f8df80fce60 2024-05-25T06:06:41.405 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.404+0000 7f8ddffff700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8de00692a0 0x7f8de006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:06:41.405 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.404+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f8de8086300 con 0x7f8df80fce60 2024-05-25T06:06:41.405 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.404+0000 7f8df6ffd700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8de00692a0 0x7f8de006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:06:41.406 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.404+0000 7f8df6ffd700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8de00692a0 0x7f8de006b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009fd0 tx=0x7f8dec009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:06:41.408 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.408+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8de8056530 con 0x7f8df80fce60 2024-05-25T06:06:41.601 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:06:41.601+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8df8102390 con 0x7f8de00692a0 2024-05-25T06:06:42.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:41 smithi203 ceph-mon[36030]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:42.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:06:42.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:06:42.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:41 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:41 smithi160 ceph-mon[29041]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-25T06:06:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-25T06:06:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:41 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:42.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:42 smithi160 ceph-mon[29041]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:06:43.223 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:42 smithi203 ceph-mon[36030]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:06:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:43 smithi160 ceph-mon[29041]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:43 smithi203 ceph-mon[36030]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:44 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/2993830665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717a4698-b4f4-457e-840f-23de1aad2734"}]: dispatch 2024-05-25T06:06:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:44 smithi160 ceph-mon[29041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717a4698-b4f4-457e-840f-23de1aad2734"}]: dispatch 2024-05-25T06:06:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:44 smithi160 ceph-mon[29041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717a4698-b4f4-457e-840f-23de1aad2734"}]': finished 2024-05-25T06:06:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:44 smithi160 ceph-mon[29041]: osdmap e36: 6 total, 5 up, 6 in 2024-05-25T06:06:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:44 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:06:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:44 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1365306126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:06:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:44 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/2993830665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717a4698-b4f4-457e-840f-23de1aad2734"}]: dispatch 2024-05-25T06:06:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:44 smithi203 ceph-mon[36030]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717a4698-b4f4-457e-840f-23de1aad2734"}]: dispatch 2024-05-25T06:06:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:44 smithi203 ceph-mon[36030]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717a4698-b4f4-457e-840f-23de1aad2734"}]': finished 2024-05-25T06:06:45.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:44 smithi203 ceph-mon[36030]: osdmap e36: 6 total, 5 up, 6 in 2024-05-25T06:06:45.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:44 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:06:45.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:44 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1365306126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-25T06:06:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:46 smithi203 ceph-mon[36030]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:46.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:46 smithi160 ceph-mon[29041]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:47 smithi203 ceph-mon[36030]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:47 smithi160 ceph-mon[29041]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:49.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:49 smithi160 ceph-mon[29041]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:49.987 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:49 smithi203 ceph-mon[36030]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:51.956 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:51 smithi203 ceph-mon[36030]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:51.956 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-25T06:06:51.956 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:51 smithi160 ceph-mon[29041]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-25T06:06:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:06:52.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:52 smithi160 ceph-mon[29041]: Deploying daemon osd.5 on smithi203 2024-05-25T06:06:52.974 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:52 smithi203 ceph-mon[36030]: Deploying daemon osd.5 on smithi203 2024-05-25T06:06:53.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:53 smithi160 ceph-mon[29041]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:53.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:54.033 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:53 smithi203 ceph-mon[36030]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:54.033 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:06:55.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:55 smithi160 ceph-mon[29041]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:55.974 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:55 smithi203 ceph-mon[36030]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:57.892 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:57.893 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:57.893 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:57.893 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:57 smithi203 ceph-mon[36030]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:57.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:57.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:06:57.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:06:57.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:57 smithi160 ceph-mon[29041]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:59.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:06:59 smithi203 ceph-mon[36030]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:06:59.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:06:59 smithi160 ceph-mon[29041]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:07:00.269 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.268+0000 7f8ddffff700 1 -- 172.21.15.203:0/4251192141 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8df8102390 con 0x7f8de00692a0 2024-05-25T06:07:00.269 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8de00692a0 msgr2=0x7f8de006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8de00692a0 0x7f8de006b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009fd0 tx=0x7f8dec009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 msgr2=0x7f8df810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 0x7f8df810d930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8de8002a50 tx=0x7f8de80046b0 comp rx=0 tx=0).stop 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 shutdown_connections 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8de00692a0 0x7f8de006b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df80fce60 0x7f8df810d930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 --2- 172.21.15.203:0/4251192141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8df80fdfd0 0x7f8df810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 >> 172.21.15.203:0/4251192141 conn(0x7f8df80f8540 msgr2=0x7f8df80f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 shutdown_connections 2024-05-25T06:07:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:00.271+0000 7f8dfe091700 1 -- 172.21.15.203:0/4251192141 wait complete. 2024-05-25T06:07:00.894 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.5.service 2024-05-25T06:07:00.897 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-25T06:07:00.897 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-25T06:07:01.182 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: from='osd.5 [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:07:01.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:07:01.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:07:01.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:01 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: from='osd.5 [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:07:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:07:01.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:07:01.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:01.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:01 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:02.571 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:02 smithi160 ceph-mon[29041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-25T06:07:02.571 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:02 smithi160 ceph-mon[29041]: osdmap e37: 6 total, 5 up, 6 in 2024-05-25T06:07:02.571 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:02 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:02.571 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:02 smithi160 ceph-mon[29041]: from='osd.5 [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:07:02.571 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:02 smithi160 ceph-mon[29041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:07:02.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:02 smithi203 ceph-mon[36030]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-25T06:07:02.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:02 smithi203 ceph-mon[36030]: osdmap e37: 6 total, 5 up, 6 in 2024-05-25T06:07:02.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:02 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:02.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:02 smithi203 ceph-mon[36030]: from='osd.5 [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:07:02.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:02 smithi203 ceph-mon[36030]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:07:02.650 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:07:02 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[55269]: 2024-05-25T06:07:02.232+0000 7f37a8d50700 -1 osd.5 0 waiting for initial osdmap 2024-05-25T06:07:02.650 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:07:02 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[55269]: 2024-05-25T06:07:02.240+0000 7f37a5b4f700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.820+0000 7fb2e4de2700 1 -- 172.21.15.160:0/4192647186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fce60 msgr2=0x7fb2e00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.820+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/4192647186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fce60 0x7fb2e00fd270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8004940 tx=0x7fb2c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.820+0000 7fb2e4de2700 1 -- 172.21.15.160:0/4192647186 shutdown_connections 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.820+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/4192647186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.820+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/4192647186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fce60 0x7fb2e00fd270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.820+0000 7fb2e4de2700 1 -- 172.21.15.160:0/4192647186 >> 172.21.15.160:0/4192647186 conn(0x7fb2e00f8540 msgr2=0x7fb2e00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 -- 172.21.15.160:0/4192647186 shutdown_connections 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 -- 172.21.15.160:0/4192647186 wait complete. 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 Processor -- start 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 -- start start 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e00fce60 0x7fb2e010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e010e500 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.821+0000 7fb2e4de2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e0112ea0 con 0x7fb2e00fce60 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2de59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e00fce60 0x7fb2e010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40662/0 (socket says 172.21.15.160:40662) 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 -- 172.21.15.160:0/1156188347 learned_addr learned my addr 172.21.15.160:0/1156188347 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 -- 172.21.15.160:0/1156188347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e00fce60 msgr2=0x7fb2e010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:02.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e00fce60 0x7fb2e010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:02.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 -- 172.21.15.160:0/1156188347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8005040 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2ddd9b700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e010de70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b790 tx=0x7fb2d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d000f6f0 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e0113100 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2d000fd20 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d00103a0 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.822+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e0113670 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.824+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e004c710 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.824+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb2d0011400 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.825+0000 7fb2d6ffd700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2cc0692a0 0x7fb2cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.825+0000 7fb2de59c700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2cc0692a0 0x7fb2cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:02.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.825+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fb2d0086b30 con 0x7fb2e00fdfd0 2024-05-25T06:07:02.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.825+0000 7fb2de59c700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2cc0692a0 0x7fb2cc06b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800b970 tx=0x7fb2c8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:02.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:02.829+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb2d0056d40 con 0x7fb2e00fdfd0 2024-05-25T06:07:03.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.015+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb2e0105940 con 0x7fb2e00fdfd0 2024-05-25T06:07:03.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.016+0000 7fb2d6ffd700 1 -- 172.21.15.160:0/1156188347 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fb2d00568d0 con 0x7fb2e00fdfd0 2024-05-25T06:07:03.016 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:03.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2cc0692a0 msgr2=0x7fb2cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:03.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2cc0692a0 0x7fb2cc06b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800b970 tx=0x7fb2c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 msgr2=0x7fb2e010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e010de70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b790 tx=0x7fb2d0009250 comp rx=0 tx=0).stop 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 shutdown_connections 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2cc0692a0 0x7fb2cc06b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e00fce60 0x7fb2e010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 --2- 172.21.15.160:0/1156188347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e00fdfd0 0x7fb2e010de70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 >> 172.21.15.160:0/1156188347 conn(0x7fb2e00f8540 msgr2=0x7fb2e00f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 shutdown_connections 2024-05-25T06:07:03.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:03.018+0000 7fb2e4de2700 1 -- 172.21.15.160:0/1156188347 wait complete. 2024-05-25T06:07:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:03 smithi203 ceph-mon[36030]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-25T06:07:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:03 smithi203 ceph-mon[36030]: osdmap e38: 6 total, 5 up, 6 in 2024-05-25T06:07:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:03 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:03 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:03 smithi203 ceph-mon[36030]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:07:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:03 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1156188347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-25T06:07:03.487 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:03 smithi160 ceph-mon[29041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-25T06:07:03.487 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:03 smithi160 ceph-mon[29041]: osdmap e38: 6 total, 5 up, 6 in 2024-05-25T06:07:03.487 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:03 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:03.487 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:03 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:03.487 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:03 smithi160 ceph-mon[29041]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-25T06:07:03.488 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:03 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1156188347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-25T06:07:03.669 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1716617197,"num_in_osds":6,"osd_in_since":1716617204,"num_remapped_pgs":0} 2024-05-25T06:07:04.415 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:04 smithi203 ceph-mon[36030]: purged_snaps scrub starts 2024-05-25T06:07:04.416 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:04 smithi203 ceph-mon[36030]: purged_snaps scrub ok 2024-05-25T06:07:04.416 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:04 smithi203 ceph-mon[36030]: osd.5 [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] boot 2024-05-25T06:07:04.416 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:04 smithi203 ceph-mon[36030]: osdmap e39: 6 total, 6 up, 6 in 2024-05-25T06:07:04.416 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:04 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:04.670 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-25T06:07:04.693 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:04 smithi160 ceph-mon[29041]: purged_snaps scrub starts 2024-05-25T06:07:04.694 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:04 smithi160 ceph-mon[29041]: purged_snaps scrub ok 2024-05-25T06:07:04.694 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:04 smithi160 ceph-mon[29041]: osd.5 [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] boot 2024-05-25T06:07:04.694 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:04 smithi160 ceph-mon[29041]: osdmap e39: 6 total, 6 up, 6 in 2024-05-25T06:07:04.694 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:04 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:07:04.927 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:05.329 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: osdmap e40: 6 total, 6 up, 6 in 2024-05-25T06:07:05.329 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: Detected new or changed devices on smithi203 2024-05-25T06:07:05.329 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi203 to 5312M 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:05.330 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: osdmap e40: 6 total, 6 up, 6 in 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi203 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi203 to 5312M 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:07:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:07:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:07:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 -- 172.21.15.160:0/597947705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 msgr2=0x7f913c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 --2- 172.21.15.160:0/597947705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c102300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9124004940 tx=0x7f9124009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 -- 172.21.15.160:0/597947705 shutdown_connections 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 --2- 172.21.15.160:0/597947705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c102300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 --2- 172.21.15.160:0/597947705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f913c0662c0 0x7f913c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 -- 172.21.15.160:0/597947705 >> 172.21.15.160:0/597947705 conn(0x7f913c0f7430 msgr2=0x7f913c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 -- 172.21.15.160:0/597947705 shutdown_connections 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.543+0000 7f9141ceb700 1 -- 172.21.15.160:0/597947705 wait complete. 2024-05-25T06:07:06.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9141ceb700 1 Processor -- start 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9141ceb700 1 -- start start 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9141ceb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f913c0662c0 0x7f913c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9141ceb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9141ceb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0fbe50 con 0x7f913c0ffee0 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9141ceb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0fbfc0 con 0x7f913c0662c0 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9132ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9132ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46356/0 (socket says 172.21.15.160:46356) 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9132ffd700 1 -- 172.21.15.160:0/1677388827 learned_addr learned my addr 172.21.15.160:0/1677388827 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f913b7fe700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f913c0662c0 0x7f913c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9132ffd700 1 -- 172.21.15.160:0/1677388827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f913c0662c0 msgr2=0x7f913c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9132ffd700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f913c0662c0 0x7f913c0fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.544+0000 7f9132ffd700 1 -- 172.21.15.160:0/1677388827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124005040 con 0x7f913c0ffee0 2024-05-25T06:07:06.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.545+0000 7f9132ffd700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c0fb8a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f912400b3b0 tx=0x7f9124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:06.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.545+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912400e040 con 0x7f913c0ffee0 2024-05-25T06:07:06.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.545+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f912400bcc0 con 0x7f913c0ffee0 2024-05-25T06:07:06.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.545+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c0fc240 con 0x7f913c0ffee0 2024-05-25T06:07:06.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.545+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912401c630 con 0x7f913c0ffee0 2024-05-25T06:07:06.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.545+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c115a30 con 0x7f913c0ffee0 2024-05-25T06:07:06.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.546+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c05f5c0 con 0x7f913c0ffee0 2024-05-25T06:07:06.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.546+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9124013980 con 0x7f913c0ffee0 2024-05-25T06:07:06.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.547+0000 7f9138ff9700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f91280692a0 0x7f912806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:06.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.547+0000 7f913b7fe700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f91280692a0 0x7f912806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:06.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.547+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9124087c90 con 0x7f913c0ffee0 2024-05-25T06:07:06.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.547+0000 7f913b7fe700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f91280692a0 0x7f912806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f913c0fd160 tx=0x7f912c009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:06.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.552+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9124054280 con 0x7f913c0ffee0 2024-05-25T06:07:06.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.739+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f913c115bc0 con 0x7f913c0ffee0 2024-05-25T06:07:06.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.740+0000 7f9138ff9700 1 -- 172.21.15.160:0/1677388827 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f91240578a0 con 0x7f913c0ffee0 2024-05-25T06:07:06.740 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:06.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f91280692a0 msgr2=0x7f912806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:06.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f91280692a0 0x7f912806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f913c0fd160 tx=0x7f912c009ee0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 msgr2=0x7f913c0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:06.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c0fb8a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f912400b3b0 tx=0x7f9124009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:06.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 shutdown_connections 2024-05-25T06:07:06.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f91280692a0 0x7f912806b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f913c0662c0 0x7f913c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 --2- 172.21.15.160:0/1677388827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f913c0ffee0 0x7f913c0fb8a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:06.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 >> 172.21.15.160:0/1677388827 conn(0x7f913c0f7430 msgr2=0x7f913c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:06.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 shutdown_connections 2024-05-25T06:07:06.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:06.742+0000 7f9141ceb700 1 -- 172.21.15.160:0/1677388827 wait complete. 2024-05-25T06:07:07.423 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1716617223,"num_in_osds":6,"osd_in_since":1716617204,"num_remapped_pgs":0} 2024-05-25T06:07:07.423 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-25T06:07:07.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:07 smithi160 ceph-mon[29041]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:07.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:07 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1677388827' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-25T06:07:07.720 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:07.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:07 smithi203 ceph-mon[36030]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:07.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:07 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1677388827' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-25T06:07:09.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/666557114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 msgr2=0x7f3b54070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/666557114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54070cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b44004970 tx=0x7f3b44009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/666557114 shutdown_connections 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/666557114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54070cb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/666557114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b54071f30 0x7f3b54070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/666557114 >> 172.21.15.160:0/666557114 conn(0x7f3b540f8990 msgr2=0x7f3b540fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/666557114 shutdown_connections 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/666557114 wait complete. 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 Processor -- start 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.373+0000 7f3b5bfe1700 1 -- start start 2024-05-25T06:07:09.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5bfe1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5bfe1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b54071f30 0x7f3b5411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5bfe1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5411aba0 con 0x7f3b54070840 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5bfe1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b541140f0 con 0x7f3b54071f30 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46372/0 (socket says 172.21.15.160:46372) 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 -- 172.21.15.160:0/3379501350 learned_addr learned my addr 172.21.15.160:0/3379501350 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5957c700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b54071f30 0x7f3b5411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 -- 172.21.15.160:0/3379501350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b54071f30 msgr2=0x7f3b5411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b54071f30 0x7f3b5411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 -- 172.21.15.160:0/3379501350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44005040 con 0x7f3b54070840 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b59d7d700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54119f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000b790 tx=0x7f3b50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b5000f6f0 con 0x7f3b54070840 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b5000fd20 con 0x7f3b54070840 2024-05-25T06:07:09.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b500103a0 con 0x7f3b54070840 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b54114370 con 0x7f3b54070840 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.374+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b54114880 con 0x7f3b54070840 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.375+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b54071a40 con 0x7f3b54070840 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.376+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3b50011400 con 0x7f3b54070840 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.376+0000 7f3b4a7fc700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3b400692a0 0x7f3b4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.376+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3b50086b60 con 0x7f3b54070840 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.376+0000 7f3b5957c700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3b400692a0 0x7f3b4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.377+0000 7f3b5957c700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3b400692a0 0x7f3b4006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3b44004f50 tx=0x7f3b44009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:09.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.381+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3b50056d00 con 0x7f3b54070840 2024-05-25T06:07:09.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.572+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b541155d0 con 0x7f3b54070840 2024-05-25T06:07:09.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.572+0000 7f3b4a7fc700 1 -- 172.21.15.160:0/3379501350 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11145 (secure 0 0 0) 0x7f3b50056890 con 0x7f3b54070840 2024-05-25T06:07:09.573 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:09.573 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":40,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:07:04.236427+0000","last_up_change":"2024-05-25T06:07:03.230915+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641643302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641643302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6809","nonce":2641643302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641643302}]},"public_addr":"172.21.15.160:6803/2641643302","cluster_addr":"172.21.15.160:6805/2641643302","heartbeat_back_addr":"172.21.15.160:6809/2641643302","heartbeat_front_addr":"172.21.15.160:6807/2641643302","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6811","nonce":3982025942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6813","nonce":3982025942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6817","nonce":3982025942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6815","nonce":3982025942}]},"public_addr":"172.21.15.160:6811/3982025942","cluster_addr":"172.21.15.160:6813/3982025942","heartbeat_back_addr":"172.21.15.160:6817/3982025942","heartbeat_front_addr":"172.21.15.160:6815/3982025942","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6819","nonce":1185286089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6821","nonce":1185286089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6825","nonce":1185286089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6823","nonce":1185286089}]},"public_addr":"172.21.15.160:6819/1185286089","cluster_addr":"172.21.15.160:6821/1185286089","heartbeat_back_addr":"172.21.15.160:6825/1185286089","heartbeat_front_addr":"172.21.15.160:6823/1185286089","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6801","nonce":3678463726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6803","nonce":3678463726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6807","nonce":3678463726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6805","nonce":3678463726}]},"public_addr":"172.21.15.203:6801/3678463726","cluster_addr":"172.21.15.203:6803/3678463726","heartbeat_back_addr":"172.21.15.203:6807/3678463726","heartbeat_front_addr":"172.21.15.203:6805/3678463726","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6809","nonce":2408351414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6811","nonce":2408351414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6815","nonce":2408351414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6813","nonce":2408351414}]},"public_addr":"172.21.15.203:6809/2408351414","cluster_addr":"172.21.15.203:6811/2408351414","heartbeat_back_addr":"172.21.15.203:6815/2408351414","heartbeat_front_addr":"172.21.15.203:6813/2408351414","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6817","nonce":2488256923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6819","nonce":2488256923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6823","nonce":2488256923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6821","nonce":2488256923}]},"public_addr":"172.21.15.203:6817/2488256923","cluster_addr":"172.21.15.203:6819/2488256923","heartbeat_back_addr":"172.21.15.203:6823/2488256923","heartbeat_front_addr":"172.21.15.203:6821/2488256923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3b400692a0 msgr2=0x7f3b4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3b400692a0 0x7f3b4006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3b44004f50 tx=0x7f3b44009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 msgr2=0x7f3b54119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54119f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000b790 tx=0x7f3b50009250 comp rx=0 tx=0).stop 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 shutdown_connections 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3b400692a0 0x7f3b4006b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:09.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54070840 0x7f3b54119f80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:09.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 --2- 172.21.15.160:0/3379501350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b54071f30 0x7f3b5411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:09.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 >> 172.21.15.160:0/3379501350 conn(0x7f3b540f8990 msgr2=0x7f3b540f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:09.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 shutdown_connections 2024-05-25T06:07:09.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:09.575+0000 7f3b5bfe1700 1 -- 172.21.15.160:0/3379501350 wait complete. 2024-05-25T06:07:09.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:09 smithi203 ceph-mon[36030]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:09.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:09 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:09.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:09 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3379501350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:07:09.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:09 smithi160 ceph-mon[29041]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:09.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:09 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:09.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:09 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3379501350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:07:10.287 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-25T06:05:48.910750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-05-25T06:07:10.287 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-25T06:07:10.542 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:11.844 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:11 smithi160 ceph-mon[29041]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:11.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:11 smithi203 ceph-mon[36030]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:12.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 -- 172.21.15.160:0/2259159513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 msgr2=0x7f94780fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:12.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 --2- 172.21.15.160:0/2259159513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94780fe200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9468009ce0 tx=0x7f9468004d10 comp rx=0 tx=0).stop 2024-05-25T06:07:12.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 -- 172.21.15.160:0/2259159513 shutdown_connections 2024-05-25T06:07:12.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 --2- 172.21.15.160:0/2259159513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94780fe7d0 0x7f9478106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 --2- 172.21.15.160:0/2259159513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94780fe200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 -- 172.21.15.160:0/2259159513 >> 172.21.15.160:0/2259159513 conn(0x7f94780f7870 msgr2=0x7f94780f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 -- 172.21.15.160:0/2259159513 shutdown_connections 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.071+0000 7f947db3c700 1 -- 172.21.15.160:0/2259159513 wait complete. 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f947db3c700 1 Processor -- start 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f947db3c700 1 -- start start 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f947db3c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94781139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f947db3c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94780fe7d0 0x7f9478113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f947db3c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781145c0 con 0x7f94780fde30 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f947db3c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947810db50 con 0x7f94780fe7d0 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f94777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94781139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:12.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f94777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94781139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46386/0 (socket says 172.21.15.160:46386) 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f94777fe700 1 -- 172.21.15.160:0/3330148298 learned_addr learned my addr 172.21.15.160:0/3330148298 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f94777fe700 1 -- 172.21.15.160:0/3330148298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94780fe7d0 msgr2=0x7f9478113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f94777fe700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94780fe7d0 0x7f9478113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.072+0000 7f94777fe700 1 -- 172.21.15.160:0/3330148298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468009940 con 0x7f94780fde30 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.073+0000 7f94777fe700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94781139a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f946800c010 tx=0x7f9468004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.073+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946800e040 con 0x7f94780fde30 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.073+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9468004610 con 0x7f94780fde30 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.073+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947810ddd0 con 0x7f94780fde30 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.073+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946801c800 con 0x7f94780fde30 2024-05-25T06:07:12.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.073+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947810e310 con 0x7f94780fde30 2024-05-25T06:07:12.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.074+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94781018c0 con 0x7f94780fde30 2024-05-25T06:07:12.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.074+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9468012410 con 0x7f94780fde30 2024-05-25T06:07:12.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.075+0000 7f945ffff700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f94600692a0 0x7f946006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:12.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.075+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9468088cc0 con 0x7f94780fde30 2024-05-25T06:07:12.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.075+0000 7f9476ffd700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f94600692a0 0x7f946006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:12.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.075+0000 7f9476ffd700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f94600692a0 0x7f946006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f946c006fd0 tx=0x7f946c006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:12.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.079+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f94680552b0 con 0x7f94780fde30 2024-05-25T06:07:12.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.264+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f947810eff0 con 0x7f94780fde30 2024-05-25T06:07:12.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.264+0000 7f945ffff700 1 -- 172.21.15.160:0/3330148298 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94680588d0 con 0x7f94780fde30 2024-05-25T06:07:12.265 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-25T06:07:12.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f94600692a0 msgr2=0x7f946006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:12.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f94600692a0 0x7f946006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f946c006fd0 tx=0x7f946c006db0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 msgr2=0x7f94781139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94781139a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f946800c010 tx=0x7f9468004d10 comp rx=0 tx=0).stop 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 shutdown_connections 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f94600692a0 0x7f946006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94780fde30 0x7f94781139a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 --2- 172.21.15.160:0/3330148298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94780fe7d0 0x7f9478113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 >> 172.21.15.160:0/3330148298 conn(0x7f94780f7870 msgr2=0x7f94780ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.266+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 shutdown_connections 2024-05-25T06:07:12.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:12.267+0000 7f947db3c700 1 -- 172.21.15.160:0/3330148298 wait complete. 2024-05-25T06:07:12.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:12 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3330148298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:07:12.949 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:12 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3330148298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:07:12.977 INFO:tasks.cephadm:Setting up client nodes... 2024-05-25T06:07:12.977 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-25T06:07:13.234 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:13 smithi203 ceph-mon[36030]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:13.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:13 smithi160 ceph-mon[29041]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:14.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.850+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/2964025351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d8066eb0 msgr2=0x7fe7d80ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:14.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.850+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/2964025351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d8066eb0 0x7fe7d80ffa60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004940 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:14.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.850+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/2964025351 shutdown_connections 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.850+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/2964025351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d80fffa0 0x7fe7d8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.850+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/2964025351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d8066eb0 0x7fe7d80ffa60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.850+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/2964025351 >> 172.21.15.160:0/2964025351 conn(0x7fe7d80f7430 msgr2=0x7fe7d80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/2964025351 shutdown_connections 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/2964025351 wait complete. 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 Processor -- start 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 -- start start 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d8066eb0 0x7fe7d810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 0x7fe7d810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d810e520 con 0x7fe7d80fffa0 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.851+0000 7fe7dfcb3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d8112ec0 con 0x7fe7d8066eb0 2024-05-25T06:07:14.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 0x7fe7d810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 0x7fe7d810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46398/0 (socket says 172.21.15.160:46398) 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 -- 172.21.15.160:0/101713847 learned_addr learned my addr 172.21.15.160:0/101713847 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dda4f700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d8066eb0 0x7fe7d810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 -- 172.21.15.160:0/101713847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d8066eb0 msgr2=0x7fe7d810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d8066eb0 0x7fe7d810d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 -- 172.21.15.160:0/101713847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4005040 con 0x7fe7d80fffa0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dd24e700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 0x7fe7d810de90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b790 tx=0x7fe7c8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7c800f6f0 con 0x7fe7d80fffa0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d8113120 con 0x7fe7d80fffa0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d8113690 con 0x7fe7d80fffa0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7c800fd20 con 0x7fe7d80fffa0 2024-05-25T06:07:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.852+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7c80103a0 con 0x7fe7d80fffa0 2024-05-25T06:07:14.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.853+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc0052e0 con 0x7fe7d80fffa0 2024-05-25T06:07:14.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.854+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe7c800f850 con 0x7fe7d80fffa0 2024-05-25T06:07:14.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.854+0000 7fe7ce7fc700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe7c40692b0 0x7fe7c406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:14.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.854+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe7c80869e0 con 0x7fe7d80fffa0 2024-05-25T06:07:14.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.854+0000 7fe7dda4f700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe7c40692b0 0x7fe7c406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:14.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.854+0000 7fe7dda4f700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe7c40692b0 0x7fe7c406b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b3b0 tx=0x7fe7d4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:14.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:14.858+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe7c8056b80 con 0x7fe7d80fffa0 2024-05-25T06:07:15.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.117+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe7bc005590 con 0x7fe7d80fffa0 2024-05-25T06:07:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.121+0000 7fe7ce7fc700 1 -- 172.21.15.160:0/101713847 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe7c8056710 con 0x7fe7d80fffa0 2024-05-25T06:07:15.121 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2024-05-25T06:07:15.121 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQATgFFmcJsTBxAAJNpy0dufo/zu2B7N8mFCYQ== 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.123+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe7c40692b0 msgr2=0x7fe7c406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.123+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe7c40692b0 0x7fe7c406b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b3b0 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.123+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 msgr2=0x7fe7d810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.123+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 0x7fe7d810de90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b790 tx=0x7fe7c8009220 comp rx=0 tx=0).stop 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 shutdown_connections 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe7c40692b0 0x7fe7c406b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d8066eb0 0x7fe7d810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 --2- 172.21.15.160:0/101713847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7d80fffa0 0x7fe7d810de90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 >> 172.21.15.160:0/101713847 conn(0x7fe7d80f7430 msgr2=0x7fe7d80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:15.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 shutdown_connections 2024-05-25T06:07:15.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:15.124+0000 7fe7dfcb3700 1 -- 172.21.15.160:0/101713847 wait complete. 2024-05-25T06:07:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:15 smithi160 ceph-mon[29041]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:15 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/101713847' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-25T06:07:15.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:15 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/101713847' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-25T06:07:15.830 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:07:15.830 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-25T06:07:15.830 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-25T06:07:15.869 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-25T06:07:15.894 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:15 smithi203 ceph-mon[36030]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:15.895 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:15 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/101713847' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-25T06:07:15.895 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:15 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/101713847' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-25T06:07:16.139 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi203/config 2024-05-25T06:07:17.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 -- 172.21.15.203:0/1577188739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 msgr2=0x7f27540faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 --2- 172.21.15.203:0/1577188739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f27540faea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f273c004940 tx=0x7f273c009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 -- 172.21.15.203:0/1577188739 shutdown_connections 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 --2- 172.21.15.203:0/1577188739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f27540faea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 --2- 172.21.15.203:0/1577188739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 0x7f27540fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 -- 172.21.15.203:0/1577188739 >> 172.21.15.203:0/1577188739 conn(0x7f27540f6bb0 msgr2=0x7f27540f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 -- 172.21.15.203:0/1577188739 shutdown_connections 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.697+0000 7f2759532700 1 -- 172.21.15.203:0/1577188739 wait complete. 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2759532700 1 Processor -- start 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2759532700 1 -- start start 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2759532700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 0x7f275410d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:17.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2759532700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f275410de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2759532700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275410e4a0 con 0x7f27540fe410 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2759532700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2754112e40 con 0x7f27540fed60 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f274a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f275410de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f274a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f275410de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:33036/0 (socket says 172.21.15.203:33036) 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f274a7fc700 1 -- 172.21.15.203:0/1109060545 learned_addr learned my addr 172.21.15.203:0/1109060545 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2752ffd700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 0x7f275410d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f274a7fc700 1 -- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 msgr2=0x7f275410d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f274a7fc700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 0x7f275410d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f274a7fc700 1 -- 172.21.15.203:0/1109060545 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c005040 con 0x7f27540fed60 2024-05-25T06:07:17.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.698+0000 7f2752ffd700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 0x7f275410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:07:17.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.699+0000 7f274a7fc700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f275410de10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f273c009150 tx=0x7f273c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:17.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.699+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273c00e040 con 0x7f27540fed60 2024-05-25T06:07:17.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.699+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f273c00bbf0 con 0x7f27540fed60 2024-05-25T06:07:17.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.699+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2754113040 con 0x7f27540fed60 2024-05-25T06:07:17.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.699+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273c01c630 con 0x7f27540fed60 2024-05-25T06:07:17.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.699+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27541135b0 con 0x7f27540fed60 2024-05-25T06:07:17.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.700+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275404c710 con 0x7f27540fed60 2024-05-25T06:07:17.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.700+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f273c00bd60 con 0x7f27540fed60 2024-05-25T06:07:17.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.701+0000 7f274bfff700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f27400692b0 0x7f274006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:17.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.701+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f273c0882b0 con 0x7f27540fed60 2024-05-25T06:07:17.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.701+0000 7f2752ffd700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f27400692b0 0x7f274006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:17.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.701+0000 7f2752ffd700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f27400692b0 0x7f274006b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2744006fd0 tx=0x7f2744008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:17.705 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.705+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f273c058460 con 0x7f27540fed60 2024-05-25T06:07:17.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:17 smithi203 ceph-mon[36030]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:17.962 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.962+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f27541139e0 con 0x7f27540fed60 2024-05-25T06:07:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:17 smithi160 ceph-mon[29041]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:17.973 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.969+0000 7f274bfff700 1 -- 172.21.15.203:0/1109060545 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f273c057ff0 con 0x7f27540fed60 2024-05-25T06:07:17.974 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2024-05-25T06:07:17.974 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAVgFFmMBB8ORAALOyGcksn6Gd9dvO/wzveRg== 2024-05-25T06:07:17.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.971+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f27400692b0 msgr2=0x7f274006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:17.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.971+0000 7f2759532700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f27400692b0 0x7f274006b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2744006fd0 tx=0x7f2744008040 comp rx=0 tx=0).stop 2024-05-25T06:07:17.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.971+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 msgr2=0x7f275410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:17.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.971+0000 7f2759532700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f275410de10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f273c009150 tx=0x7f273c009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 shutdown_connections 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f27400692b0 0x7f274006b760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27540fe410 0x7f275410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 --2- 172.21.15.203:0/1109060545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27540fed60 0x7f275410de10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 >> 172.21.15.203:0/1109060545 conn(0x7f27540f6bb0 msgr2=0x7f27540f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 shutdown_connections 2024-05-25T06:07:17.976 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:07:17.972+0000 7f2759532700 1 -- 172.21.15.203:0/1109060545 wait complete. 2024-05-25T06:07:18.576 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:07:18.577 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-25T06:07:18.577 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-25T06:07:18.657 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-25T06:07:18.658 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-25T06:07:18.658 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-25T06:07:18.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:18 smithi203 ceph-mon[36030]: from='client.? 172.21.15.203:0/1109060545' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-25T06:07:18.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:18 smithi203 ceph-mon[36030]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-25T06:07:18.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:18 smithi203 ceph-mon[36030]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-25T06:07:18.920 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:18.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:18 smithi160 ceph-mon[29041]: from='client.? 172.21.15.203:0/1109060545' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-25T06:07:18.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:18 smithi160 ceph-mon[29041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-25T06:07:18.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:18 smithi160 ceph-mon[29041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-25T06:07:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:19 smithi160 ceph-mon[29041]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:19 smithi203 ceph-mon[36030]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:20.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.563+0000 7f033b834700 1 -- 172.21.15.160:0/1417725674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03341026a0 msgr2=0x7f03340fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:20.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.563+0000 7f033b834700 1 --2- 172.21.15.160:0/1417725674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03341026a0 0x7f03340fbd10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0324004940 tx=0x7f0324009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:20.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 -- 172.21.15.160:0/1417725674 shutdown_connections 2024-05-25T06:07:20.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 --2- 172.21.15.160:0/1417725674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03341026a0 0x7f03340fbd10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:20.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 --2- 172.21.15.160:0/1417725674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0334101d00 0x7f03341020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:20.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 -- 172.21.15.160:0/1417725674 >> 172.21.15.160:0/1417725674 conn(0x7f03340f7870 msgr2=0x7f03340f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:20.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 -- 172.21.15.160:0/1417725674 shutdown_connections 2024-05-25T06:07:20.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 -- 172.21.15.160:0/1417725674 wait complete. 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.564+0000 7f033b834700 1 Processor -- start 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f033b834700 1 -- start start 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f033b834700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 0x7f03341128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f033b834700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03341026a0 0x7f0334112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f033b834700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03341134f0 con 0x7f0334101d00 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f033b834700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033410ca60 con 0x7f03341026a0 2024-05-25T06:07:20.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 0x7f03341128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 0x7f03341128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41916/0 (socket says 172.21.15.160:41916) 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 -- 172.21.15.160:0/377377309 learned_addr learned my addr 172.21.15.160:0/377377309 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 -- 172.21.15.160:0/377377309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03341026a0 msgr2=0x7f0334112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f0338dcf700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03341026a0 0x7f0334112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03341026a0 0x7f0334112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 -- 172.21.15.160:0/377377309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0324005040 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f03395d0700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 0x7f03341128d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f033000b790 tx=0x7f0330009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.565+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f033000f770 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.566+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033410cd40 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.566+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f033000fda0 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.566+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f033000d310 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.566+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033410d300 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.567+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03340fd450 con 0x7f0334101d00 2024-05-25T06:07:20.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.567+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0330010420 con 0x7f0334101d00 2024-05-25T06:07:20.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.568+0000 7f0329ffb700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f03200692a0 0x7f032006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:20.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.568+0000 7f0338dcf700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f03200692a0 0x7f032006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:20.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.568+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0330086e80 con 0x7f0334101d00 2024-05-25T06:07:20.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.568+0000 7f0338dcf700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f03200692a0 0x7f032006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0324009fd0 tx=0x7f0324009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:20.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.572+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0330056f80 con 0x7f0334101d00 2024-05-25T06:07:20.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.813+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0334108e50 con 0x7f0334101d00 2024-05-25T06:07:20.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.816+0000 7f0329ffb700 1 -- 172.21.15.160:0/377377309 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167188 (secure 0 0 0) 0x7f0330056b10 con 0x7f0334101d00 2024-05-25T06:07:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f03200692a0 msgr2=0x7f032006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f03200692a0 0x7f032006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0324009fd0 tx=0x7f0324009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 msgr2=0x7f03341128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 0x7f03341128d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f033000b790 tx=0x7f0330009250 comp rx=0 tx=0).stop 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 shutdown_connections 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f03200692a0 0x7f032006b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:20.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0334101d00 0x7f03341128d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:20.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 --2- 172.21.15.160:0/377377309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03341026a0 0x7f0334112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:20.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 >> 172.21.15.160:0/377377309 conn(0x7f03340f7870 msgr2=0x7f03340f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:20.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 shutdown_connections 2024-05-25T06:07:20.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:20.819+0000 7f033b834700 1 -- 172.21.15.160:0/377377309 wait complete. 2024-05-25T06:07:21.143 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:20 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/377377309' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-25T06:07:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:20 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/377377309' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-25T06:07:21.473 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":19,"flags":0,"active_gid":14210,"active_name":"smithi160.nalzlg","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":4089125744},{"type":"v1","addr":"172.21.15.160:6801","nonce":4089125744}]},"active_addr":"172.21.15.160:6801/4089125744","active_change":"2024-05-25T06:03:38.584094+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi203.rotqex","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.160:8443/","prometheus":"http://172.21.15.160:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2473068109}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3623795045}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2019174179}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1251129035}]}]} 2024-05-25T06:07:21.477 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-25T06:07:21.477 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-25T06:07:21.477 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-25T06:07:21.732 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:22.171 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:21 smithi160 ceph-mon[29041]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:21 smithi203 ceph-mon[36030]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:23.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 -- 172.21.15.160:0/3998028387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 msgr2=0x7f3d100fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:23.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/3998028387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fb990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8004940 tx=0x7f3cf8009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:23.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 -- 172.21.15.160:0/3998028387 shutdown_connections 2024-05-25T06:07:23.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/3998028387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fb990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:23.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/3998028387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 0x7f3d10103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:23.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 -- 172.21.15.160:0/3998028387 >> 172.21.15.160:0/3998028387 conn(0x7f3d100714d0 msgr2=0x7f3d100718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 -- 172.21.15.160:0/3998028387 shutdown_connections 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.402+0000 7f3d15d5e700 1 -- 172.21.15.160:0/3998028387 wait complete. 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d15d5e700 1 Processor -- start 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d15d5e700 1 -- start start 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d15d5e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d15d5e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 0x7f3d100fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d15d5e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100fe650 con 0x7f3d100fb520 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d15d5e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100fd180 con 0x7f3d10102f60 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41948/0 (socket says 172.21.15.160:41948) 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0f7fe700 1 -- 172.21.15.160:0/937935231 learned_addr learned my addr 172.21.15.160:0/937935231 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0effd700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 0x7f3d100fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0f7fe700 1 -- 172.21.15.160:0/937935231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 msgr2=0x7f3d100fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0f7fe700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 0x7f3d100fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:23.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0f7fe700 1 -- 172.21.15.160:0/937935231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf8005040 con 0x7f3d100fb520 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.403+0000 7f3d0effd700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 0x7f3d100fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.404+0000 7f3d0f7fe700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fc700 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c8b0 tx=0x7f3d08004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.404+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0800a760 con 0x7f3d100fb520 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.404+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d0800ad90 con 0x7f3d100fb520 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.404+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d100fd460 con 0x7f3d100fb520 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.404+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d080113a0 con 0x7f3d100fb520 2024-05-25T06:07:23.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.404+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1011a0f0 con 0x7f3d100fb520 2024-05-25T06:07:23.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.405+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d10100250 con 0x7f3d100fb520 2024-05-25T06:07:23.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.405+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3d08012400 con 0x7f3d100fb520 2024-05-25T06:07:23.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.406+0000 7f3d07fff700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3cfc0692a0 0x7f3cfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:23.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.406+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3d08086eb0 con 0x7f3d100fb520 2024-05-25T06:07:23.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.406+0000 7f3d0effd700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3cfc0692a0 0x7f3cfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:23.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.406+0000 7f3d0effd700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3cfc0692a0 0x7f3cfc06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8004f90 tx=0x7f3cf8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:23.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.410+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3d08053520 con 0x7f3d100fb520 2024-05-25T06:07:23.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.595+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d100fe840 con 0x7f3d100fb520 2024-05-25T06:07:23.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.596+0000 7f3d07fff700 1 -- 172.21.15.160:0/937935231 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11145 (secure 0 0 0) 0x7f3d0805c090 con 0x7f3d100fb520 2024-05-25T06:07:23.596 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:23.596 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":40,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:07:04.236427+0000","last_up_change":"2024-05-25T06:07:03.230915+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641643302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641643302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6809","nonce":2641643302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641643302}]},"public_addr":"172.21.15.160:6803/2641643302","cluster_addr":"172.21.15.160:6805/2641643302","heartbeat_back_addr":"172.21.15.160:6809/2641643302","heartbeat_front_addr":"172.21.15.160:6807/2641643302","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6811","nonce":3982025942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6813","nonce":3982025942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6817","nonce":3982025942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6815","nonce":3982025942}]},"public_addr":"172.21.15.160:6811/3982025942","cluster_addr":"172.21.15.160:6813/3982025942","heartbeat_back_addr":"172.21.15.160:6817/3982025942","heartbeat_front_addr":"172.21.15.160:6815/3982025942","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6819","nonce":1185286089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6821","nonce":1185286089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6825","nonce":1185286089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6823","nonce":1185286089}]},"public_addr":"172.21.15.160:6819/1185286089","cluster_addr":"172.21.15.160:6821/1185286089","heartbeat_back_addr":"172.21.15.160:6825/1185286089","heartbeat_front_addr":"172.21.15.160:6823/1185286089","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6801","nonce":3678463726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6803","nonce":3678463726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6807","nonce":3678463726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6805","nonce":3678463726}]},"public_addr":"172.21.15.203:6801/3678463726","cluster_addr":"172.21.15.203:6803/3678463726","heartbeat_back_addr":"172.21.15.203:6807/3678463726","heartbeat_front_addr":"172.21.15.203:6805/3678463726","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6809","nonce":2408351414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6811","nonce":2408351414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6815","nonce":2408351414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6813","nonce":2408351414}]},"public_addr":"172.21.15.203:6809/2408351414","cluster_addr":"172.21.15.203:6811/2408351414","heartbeat_back_addr":"172.21.15.203:6815/2408351414","heartbeat_front_addr":"172.21.15.203:6813/2408351414","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6817","nonce":2488256923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6819","nonce":2488256923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6823","nonce":2488256923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6821","nonce":2488256923}]},"public_addr":"172.21.15.203:6817/2488256923","cluster_addr":"172.21.15.203:6819/2488256923","heartbeat_back_addr":"172.21.15.203:6823/2488256923","heartbeat_front_addr":"172.21.15.203:6821/2488256923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3cfc0692a0 msgr2=0x7f3cfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3cfc0692a0 0x7f3cfc06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8004f90 tx=0x7f3cf8009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 msgr2=0x7f3d100fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fc700 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c8b0 tx=0x7f3d08004470 comp rx=0 tx=0).stop 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 shutdown_connections 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3cfc0692a0 0x7f3cfc06b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d100fb520 0x7f3d100fc700 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 --2- 172.21.15.160:0/937935231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d10102f60 0x7f3d100fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 >> 172.21.15.160:0/937935231 conn(0x7f3d100714d0 msgr2=0x7f3d100fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 shutdown_connections 2024-05-25T06:07:23.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:23.599+0000 7f3d15d5e700 1 -- 172.21.15.160:0/937935231 wait complete. 2024-05-25T06:07:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:23 smithi160 ceph-mon[29041]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:23 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/937935231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:07:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:24.307 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-25T06:07:24.307 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-25T06:07:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:23 smithi203 ceph-mon[36030]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:23 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/937935231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:07:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:24.561 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:26.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 -- 172.21.15.160:0/422742216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 msgr2=0x7f0bd8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:26.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 --2- 172.21.15.160:0/422742216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8103120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8004940 tx=0x7f0bc8009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 -- 172.21.15.160:0/422742216 shutdown_connections 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 --2- 172.21.15.160:0/422742216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd81036f0 0x7f0bd80fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 --2- 172.21.15.160:0/422742216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8103120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 -- 172.21.15.160:0/422742216 >> 172.21.15.160:0/422742216 conn(0x7f0bd80f8900 msgr2=0x7f0bd80fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 -- 172.21.15.160:0/422742216 shutdown_connections 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 -- 172.21.15.160:0/422742216 wait complete. 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 Processor -- start 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.099+0000 7f0bde122700 1 -- start start 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bde122700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bde122700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd81036f0 0x7f0bd81140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bde122700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd81146f0 con 0x7f0bd8102d50 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bde122700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd810dae0 con 0x7f0bd81036f0 2024-05-25T06:07:26.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53938/0 (socket says 172.21.15.160:53938) 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 -- 172.21.15.160:0/1701860176 learned_addr learned my addr 172.21.15.160:0/1701860176 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd6ffd700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd81036f0 0x7f0bd81140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 -- 172.21.15.160:0/1701860176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd81036f0 msgr2=0x7f0bd81140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd81036f0 0x7f0bd81140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 -- 172.21.15.160:0/1701860176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc8005040 con 0x7f0bd8102d50 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bd77fe700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8113b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800b8b0 tx=0x7f0bc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bc800e040 con 0x7f0bd8102d50 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd810dd60 con 0x7f0bd8102d50 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bc8004290 con 0x7f0bd8102d50 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bc801c3f0 con 0x7f0bd8102d50 2024-05-25T06:07:26.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.100+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd810e320 con 0x7f0bd8102d50 2024-05-25T06:07:26.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.101+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd80fe4a0 con 0x7f0bd8102d50 2024-05-25T06:07:26.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.102+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0bc8013420 con 0x7f0bd8102d50 2024-05-25T06:07:26.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.102+0000 7f0bbffff700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0bc00692a0 0x7f0bc006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:26.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.102+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0bc8088100 con 0x7f0bd8102d50 2024-05-25T06:07:26.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.102+0000 7f0bd6ffd700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0bc00692a0 0x7f0bc006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:26.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.103+0000 7f0bd6ffd700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0bc00692a0 0x7f0bc006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bd810eca0 tx=0x7f0bcc00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:26.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.107+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0bc8058200 con 0x7f0bd8102d50 2024-05-25T06:07:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:25 smithi160 ceph-mon[29041]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:26.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.292+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd8109e70 con 0x7f0bd8102d50 2024-05-25T06:07:26.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.293+0000 7f0bbffff700 1 -- 172.21.15.160:0/1701860176 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11145 (secure 0 0 0) 0x7f0bc8057d90 con 0x7f0bd8102d50 2024-05-25T06:07:26.293 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:26.294 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":40,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:07:04.236427+0000","last_up_change":"2024-05-25T06:07:03.230915+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641643302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641643302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6809","nonce":2641643302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641643302}]},"public_addr":"172.21.15.160:6803/2641643302","cluster_addr":"172.21.15.160:6805/2641643302","heartbeat_back_addr":"172.21.15.160:6809/2641643302","heartbeat_front_addr":"172.21.15.160:6807/2641643302","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6811","nonce":3982025942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6813","nonce":3982025942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6817","nonce":3982025942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6815","nonce":3982025942}]},"public_addr":"172.21.15.160:6811/3982025942","cluster_addr":"172.21.15.160:6813/3982025942","heartbeat_back_addr":"172.21.15.160:6817/3982025942","heartbeat_front_addr":"172.21.15.160:6815/3982025942","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6819","nonce":1185286089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6821","nonce":1185286089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6825","nonce":1185286089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6823","nonce":1185286089}]},"public_addr":"172.21.15.160:6819/1185286089","cluster_addr":"172.21.15.160:6821/1185286089","heartbeat_back_addr":"172.21.15.160:6825/1185286089","heartbeat_front_addr":"172.21.15.160:6823/1185286089","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6801","nonce":3678463726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6803","nonce":3678463726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6807","nonce":3678463726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6805","nonce":3678463726}]},"public_addr":"172.21.15.203:6801/3678463726","cluster_addr":"172.21.15.203:6803/3678463726","heartbeat_back_addr":"172.21.15.203:6807/3678463726","heartbeat_front_addr":"172.21.15.203:6805/3678463726","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6809","nonce":2408351414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6811","nonce":2408351414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6815","nonce":2408351414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6813","nonce":2408351414}]},"public_addr":"172.21.15.203:6809/2408351414","cluster_addr":"172.21.15.203:6811/2408351414","heartbeat_back_addr":"172.21.15.203:6815/2408351414","heartbeat_front_addr":"172.21.15.203:6813/2408351414","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6817","nonce":2488256923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6819","nonce":2488256923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6823","nonce":2488256923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6821","nonce":2488256923}]},"public_addr":"172.21.15.203:6817/2488256923","cluster_addr":"172.21.15.203:6819/2488256923","heartbeat_back_addr":"172.21.15.203:6823/2488256923","heartbeat_front_addr":"172.21.15.203:6821/2488256923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0bc00692a0 msgr2=0x7f0bc006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0bc00692a0 0x7f0bc006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bd810eca0 tx=0x7f0bcc00b040 comp rx=0 tx=0).stop 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 msgr2=0x7f0bd8113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8113b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800b8b0 tx=0x7f0bc8009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 shutdown_connections 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0bc00692a0 0x7f0bc006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd8102d50 0x7f0bd8113b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.295+0000 7f0bde122700 1 --2- 172.21.15.160:0/1701860176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd81036f0 0x7f0bd81140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.296+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 >> 172.21.15.160:0/1701860176 conn(0x7f0bd80f8900 msgr2=0x7f0bd80f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:26.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.296+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 shutdown_connections 2024-05-25T06:07:26.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:26.296+0000 7f0bde122700 1 -- 172.21.15.160:0/1701860176 wait complete. 2024-05-25T06:07:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:25 smithi203 ceph-mon[36030]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:26.955 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-25T06:07:26.956 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-25T06:07:26.956 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-25T06:07:26.957 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-25T06:07:26.957 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-25T06:07:26.958 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-25T06:07:27.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:26 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1701860176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:07:27.258 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:27.259 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:26 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1701860176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:07:28.343 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:27 smithi160 ceph-mon[29041]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:27 smithi203 ceph-mon[36030]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:29.575 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:29.577 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:30.053 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:29 smithi160 ceph-mon[29041]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.064+0000 7f67e77b9700 1 -- 172.21.15.160:0/969435004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 msgr2=0x7f67e01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.064+0000 7f67e77b9700 1 --2- 172.21.15.160:0/969435004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01020d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67dc004940 tx=0x7f67dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.064+0000 7f67e77b9700 1 -- 172.21.15.160:0/969435004 shutdown_connections 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.064+0000 7f67e77b9700 1 --2- 172.21.15.160:0/969435004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 0x7f67e00fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.064+0000 7f67e77b9700 1 --2- 172.21.15.160:0/969435004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01020d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.064+0000 7f67e77b9700 1 -- 172.21.15.160:0/969435004 >> 172.21.15.160:0/969435004 conn(0x7f67e00f7870 msgr2=0x7f67e00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 -- 172.21.15.160:0/969435004 shutdown_connections 2024-05-25T06:07:30.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 -- 172.21.15.160:0/969435004 wait complete. 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 Processor -- start 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 -- start start 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 0x7f67e0112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0113430 con 0x7f67e0101d00 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e77b9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e010ca30 con 0x7f67e01026a0 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e5555700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e5555700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53956/0 (socket says 172.21.15.160:53956) 2024-05-25T06:07:30.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.065+0000 7f67e5555700 1 -- 172.21.15.160:0/3697310020 learned_addr learned my addr 172.21.15.160:0/3697310020 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:30.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e4d54700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 0x7f67e0112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e5555700 1 -- 172.21.15.160:0/3697310020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 msgr2=0x7f67e0112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e5555700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 0x7f67e0112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e5555700 1 -- 172.21.15.160:0/3697310020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc005040 con 0x7f67e0101d00 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e4d54700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 0x7f67e0112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e5555700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01128a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b3b0 tx=0x7f67dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67dc00e040 con 0x7f67e0101d00 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67dc00bbc0 con 0x7f67e0101d00 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e010ccb0 con 0x7f67e0101d00 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67dc01c3f0 con 0x7f67e0101d00 2024-05-25T06:07:30.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.066+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e010d270 con 0x7f67e0101d00 2024-05-25T06:07:30.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.067+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67e010d7a0 con 0x7f67e0101d00 2024-05-25T06:07:30.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f67dc00bd30 con 0x7f67e0101d00 2024-05-25T06:07:30.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67d5ffb700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f67cc0692a0 0x7f67cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f67dc088030 con 0x7f67e0101d00 2024-05-25T06:07:30.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67e4d54700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f67cc0692a0 0x7f67cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67d5ffb700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] conn(0x7f67cc06ef70 0x7f67cc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 --> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67cc071a30 con 0x7f67cc06ef70 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67dc08a620 con 0x7f67e0101d00 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.068+0000 7f67e5d56700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] conn(0x7f67cc06ef70 0x7f67cc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.069+0000 7f67e4d54700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f67cc0692a0 0x7f67cc06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67d0007fd0 tx=0x7f67d0009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.069+0000 7f67e5d56700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] conn(0x7f67cc06ef70 0x7f67cc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:30.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.069+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== osd.2 v2:172.21.15.160:6818/1185286089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f67cc071a30 con 0x7f67cc06ef70 2024-05-25T06:07:30.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 --> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67e00f9110 con 0x7f67cc06ef70 2024-05-25T06:07:30.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67d5ffb700 1 -- 172.21.15.160:0/3697310020 <== osd.2 v2:172.21.15.160:6818/1185286089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f67e00f9110 con 0x7f67cc06ef70 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] conn(0x7f67cc06ef70 msgr2=0x7f67cc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] conn(0x7f67cc06ef70 0x7f67cc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f67cc0692a0 msgr2=0x7f67cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f67cc0692a0 0x7f67cc06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67d0007fd0 tx=0x7f67d0009040 comp rx=0 tx=0).stop 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 msgr2=0x7f67e01128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01128a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b3b0 tx=0x7f67dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:30.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 shutdown_connections 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6818/1185286089,v1:172.21.15.160:6819/1185286089] conn(0x7f67cc06ef70 0x7f67cc071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f67cc0692a0 0x7f67cc06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e0101d00 0x7f67e01128a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 --2- 172.21.15.160:0/3697310020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e01026a0 0x7f67e0112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.084+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 >> 172.21.15.160:0/3697310020 conn(0x7f67e00f7870 msgr2=0x7f67e00f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.085+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 shutdown_connections 2024-05-25T06:07:30.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.085+0000 7f67e77b9700 1 -- 172.21.15.160:0/3697310020 wait complete. 2024-05-25T06:07:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:29 smithi203 ceph-mon[36030]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.725+0000 7fbb51213700 1 -- 172.21.15.160:0/1952876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 msgr2=0x7fbb4c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.725+0000 7fbb51213700 1 --2- 172.21.15.160:0/1952876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c101a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbb40004940 tx=0x7fbb40009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 -- 172.21.15.160:0/1952876844 shutdown_connections 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 --2- 172.21.15.160:0/1952876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c101a30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 --2- 172.21.15.160:0/1952876844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 0x7fbb4c0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 -- 172.21.15.160:0/1952876844 >> 172.21.15.160:0/1952876844 conn(0x7fbb4c073350 msgr2=0x7fbb4c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 -- 172.21.15.160:0/1952876844 shutdown_connections 2024-05-25T06:07:30.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 -- 172.21.15.160:0/1952876844 wait complete. 2024-05-25T06:07:30.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 Processor -- start 2024-05-25T06:07:30.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 -- start start 2024-05-25T06:07:30.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 0x7fbb4c113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.726+0000 7fbb51213700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb51213700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1145d0 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb51213700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c10dbb0 con 0x7fbb4c0fb5a0 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53970/0 (socket says 172.21.15.160:53970) 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 -- 172.21.15.160:0/825079783 learned_addr learned my addr 172.21.15.160:0/825079783 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4ad9d700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 0x7fbb4c113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 -- 172.21.15.160:0/825079783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 msgr2=0x7fbb4c113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 0x7fbb4c113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 -- 172.21.15.160:0/825079783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40005040 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4ad9d700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 0x7fbb4c113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb4a59c700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c113f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb40000c00 tx=0x7fbb40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb40021070 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb40013cd0 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c10de30 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb4000e070 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.727+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c10e310 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.728+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb4c10e830 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbb4001caf0 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb337fe700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb340692a0 0x7fbb3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbb40088280 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb4ad9d700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb340692a0 0x7fbb3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb337fe700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] conn(0x7fbb3406ef70 0x7fbb34071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 --> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb34071a30 con 0x7fbb3406ef70 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.729+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4008a870 con 0x7fbb4c0fbf40 2024-05-25T06:07:30.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.730+0000 7fbb4b59e700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] conn(0x7fbb3406ef70 0x7fbb34071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:30.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.730+0000 7fbb4ad9d700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb340692a0 0x7fbb3406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c004600 tx=0x7fbb3c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:30.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.730+0000 7fbb4b59e700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] conn(0x7fbb3406ef70 0x7fbb34071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:30.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.731+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== osd.0 v2:172.21.15.160:6802/2641643302 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbb34071a30 con 0x7fbb3406ef70 2024-05-25T06:07:30.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.746+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 --> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb4c1013e0 con 0x7fbb3406ef70 2024-05-25T06:07:30.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.746+0000 7fbb337fe700 1 -- 172.21.15.160:0/825079783 <== osd.0 v2:172.21.15.160:6802/2641643302 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb4c1013e0 con 0x7fbb3406ef70 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] conn(0x7fbb3406ef70 msgr2=0x7fbb34071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] conn(0x7fbb3406ef70 0x7fbb34071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb340692a0 msgr2=0x7fbb3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb340692a0 0x7fbb3406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c004600 tx=0x7fbb3c00b040 comp rx=0 tx=0).stop 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 msgr2=0x7fbb4c113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c113f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb40000c00 tx=0x7fbb40009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:30.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 shutdown_connections 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb340692a0 0x7fbb3406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb4c0fb5a0 0x7fbb4c113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:6802/2641643302,v1:172.21.15.160:6803/2641643302] conn(0x7fbb3406ef70 0x7fbb34071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 --2- 172.21.15.160:0/825079783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0fbf40 0x7fbb4c113f80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 >> 172.21.15.160:0/825079783 conn(0x7fbb4c073350 msgr2=0x7fbb4c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 shutdown_connections 2024-05-25T06:07:30.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:30.747+0000 7fbb51213700 1 -- 172.21.15.160:0/825079783 wait complete. 2024-05-25T06:07:31.440 INFO:teuthology.orchestra.run.smithi160.stdout:81604378648 2024-05-25T06:07:31.440 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-25T06:07:32.294 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:31 smithi160 ceph-mon[29041]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:31 smithi203 ceph-mon[36030]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:33.591 INFO:teuthology.orchestra.run.smithi160.stdout:38654705699 2024-05-25T06:07:33.591 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-25T06:07:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:33 smithi203 ceph-mon[36030]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:33 smithi160 ceph-mon[29041]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:34.567 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:35.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.388+0000 7f126429f700 1 -- 172.21.15.160:0/1610221169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c103780 msgr2=0x7f125c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.388+0000 7f126429f700 1 --2- 172.21.15.160:0/1610221169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c103780 0x7f125c0fcdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f124c004880 tx=0x7f124c009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.388+0000 7f126429f700 1 -- 172.21.15.160:0/1610221169 shutdown_connections 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.388+0000 7f126429f700 1 --2- 172.21.15.160:0/1610221169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c103780 0x7f125c0fcdf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.388+0000 7f126429f700 1 --2- 172.21.15.160:0/1610221169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c102de0 0x7f125c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 -- 172.21.15.160:0/1610221169 >> 172.21.15.160:0/1610221169 conn(0x7f125c0f8990 msgr2=0x7f125c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 -- 172.21.15.160:0/1610221169 shutdown_connections 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 -- 172.21.15.160:0/1610221169 wait complete. 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 Processor -- start 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 -- start start 2024-05-25T06:07:35.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c102de0 0x7f125c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 0x7f125c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c114500 con 0x7f125c103780 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.389+0000 7f126429f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c10db00 con 0x7f125c102de0 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 0x7f125c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 0x7f125c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53988/0 (socket says 172.21.15.160:53988) 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 -- 172.21.15.160:0/3104402455 learned_addr learned my addr 172.21.15.160:0/3104402455 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126203b700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c102de0 0x7f125c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 -- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c102de0 msgr2=0x7f125c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c102de0 0x7f125c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 -- 172.21.15.160:0/3104402455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258005040 con 0x7f125c103780 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126183a700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 0x7f125c113eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f124c0134d0 tx=0x7f124c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124c00e030 con 0x7f125c103780 2024-05-25T06:07:35.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c005040 con 0x7f125c103780 2024-05-25T06:07:35.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f124c013d60 con 0x7f125c103780 2024-05-25T06:07:35.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124c01cd90 con 0x7f125c103780 2024-05-25T06:07:35.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.390+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c10e1b0 con 0x7f125c103780 2024-05-25T06:07:35.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.391+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f125c10e690 con 0x7f125c103780 2024-05-25T06:07:35.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f124c01c620 con 0x7f125c103780 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f1252ffd700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12480692a0 0x7f124806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f124c088670 con 0x7f125c103780 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f126203b700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12480692a0 0x7f124806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f1252ffd700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] conn(0x7f124806ef70 0x7f1248071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 --> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1248071a30 con 0x7f124806ef70 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f124c08ac60 con 0x7f125c103780 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f126203b700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12480692a0 0x7f124806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1258009fd0 tx=0x7f125800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.392+0000 7f126283c700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] conn(0x7f124806ef70 0x7f1248071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.393+0000 7f126283c700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] conn(0x7f124806ef70 0x7f1248071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:35.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.394+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== osd.5 v2:172.21.15.203:6816/2488256923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1248071a30 con 0x7f124806ef70 2024-05-25T06:07:35.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.408+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 --> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f125c0fe530 con 0x7f124806ef70 2024-05-25T06:07:35.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f1252ffd700 1 -- 172.21.15.160:0/3104402455 <== osd.5 v2:172.21.15.203:6816/2488256923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f125c0fe530 con 0x7f124806ef70 2024-05-25T06:07:35.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] conn(0x7f124806ef70 msgr2=0x7f1248071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] conn(0x7f124806ef70 0x7f1248071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12480692a0 msgr2=0x7f124806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12480692a0 0x7f124806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1258009fd0 tx=0x7f125800b040 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 msgr2=0x7f125c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 0x7f125c113eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f124c0134d0 tx=0x7f124c009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 shutdown_connections 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:6816/2488256923,v1:172.21.15.203:6817/2488256923] conn(0x7f124806ef70 0x7f1248071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12480692a0 0x7f124806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125c102de0 0x7f125c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 --2- 172.21.15.160:0/3104402455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125c103780 0x7f125c113eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 >> 172.21.15.160:0/3104402455 conn(0x7f125c0f8990 msgr2=0x7f125c0f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:35.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 shutdown_connections 2024-05-25T06:07:35.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:35.409+0000 7f126429f700 1 -- 172.21.15.160:0/3104402455 wait complete. 2024-05-25T06:07:35.641 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:36.246 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:35 smithi160 ceph-mon[29041]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:35 smithi203 ceph-mon[36030]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:36.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 -- 172.21.15.160:0/1625936866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220103780 msgr2=0x7f72200fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 --2- 172.21.15.160:0/1625936866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220103780 0x7f72200fcdf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7210004940 tx=0x7f7210009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:36.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 -- 172.21.15.160:0/1625936866 shutdown_connections 2024-05-25T06:07:36.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 --2- 172.21.15.160:0/1625936866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220103780 0x7f72200fcdf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 --2- 172.21.15.160:0/1625936866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7220102de0 0x7f72201031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 -- 172.21.15.160:0/1625936866 >> 172.21.15.160:0/1625936866 conn(0x7f72200f8950 msgr2=0x7f72200fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 -- 172.21.15.160:0/1625936866 shutdown_connections 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.461+0000 7f72279d4700 1 -- 172.21.15.160:0/1625936866 wait complete. 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f72279d4700 1 Processor -- start 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f72279d4700 1 -- start start 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f72279d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 0x7f7220115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f72279d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7220103780 0x7f72201162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f72279d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201168f0 con 0x7f7220102de0 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f72279d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722010fd00 con 0x7f7220103780 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 0x7f7220115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 0x7f7220115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41758/0 (socket says 172.21.15.160:41758) 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 -- 172.21.15.160:0/1818229415 learned_addr learned my addr 172.21.15.160:0/1818229415 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7224f6f700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7220103780 0x7f72201162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 -- 172.21.15.160:0/1818229415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7220103780 msgr2=0x7f72201162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7220103780 0x7f72201162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 -- 172.21.15.160:0/1818229415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210005040 con 0x7f7220102de0 2024-05-25T06:07:36.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.462+0000 7f7225770700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 0x7f7220115d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f721c002a50 tx=0x7f721c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:36.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.463+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721c007cc0 con 0x7f7220102de0 2024-05-25T06:07:36.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.463+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f721c00f040 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.463+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722010ffe0 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.463+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721c00da30 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.463+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7220110570 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.464+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7220110a50 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.464+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f721c00db90 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7215ffb700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f720c0692a0 0x7f720c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f721c087690 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7224f6f700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f720c0692a0 0x7f720c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7215ffb700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] conn(0x7f720c06ef70 0x7f720c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 --> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f720c071a30 con 0x7f720c06ef70 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f721c087a70 con 0x7f7220102de0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7225f71700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] conn(0x7f720c06ef70 0x7f720c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7224f6f700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f720c0692a0 0x7f720c06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7210004890 tx=0x7f7210009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:36.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.465+0000 7f7225f71700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] conn(0x7f720c06ef70 0x7f720c071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:36.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.466+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== osd.1 v2:172.21.15.160:6810/3982025942 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f720c071a30 con 0x7f720c06ef70 2024-05-25T06:07:36.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.481+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 --> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72200fe530 con 0x7f720c06ef70 2024-05-25T06:07:36.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.481+0000 7f7215ffb700 1 -- 172.21.15.160:0/1818229415 <== osd.1 v2:172.21.15.160:6810/3982025942 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f72200fe530 con 0x7f720c06ef70 2024-05-25T06:07:36.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] conn(0x7f720c06ef70 msgr2=0x7f720c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] conn(0x7f720c06ef70 0x7f720c071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f720c0692a0 msgr2=0x7f720c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f720c0692a0 0x7f720c06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7210004890 tx=0x7f7210009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:36.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 msgr2=0x7f7220115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 0x7f7220115d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f721c002a50 tx=0x7f721c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 shutdown_connections 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6810/3982025942,v1:172.21.15.160:6811/3982025942] conn(0x7f720c06ef70 0x7f720c071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f720c0692a0 0x7f720c06b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7220102de0 0x7f7220115d60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 --2- 172.21.15.160:0/1818229415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7220103780 0x7f72201162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 >> 172.21.15.160:0/1818229415 conn(0x7f72200f8950 msgr2=0x7f7220071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 shutdown_connections 2024-05-25T06:07:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.482+0000 7f72279d4700 1 -- 172.21.15.160:0/1818229415 wait complete. 2024-05-25T06:07:36.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.696+0000 7f161f459700 1 -- 172.21.15.160:0/1527691046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 msgr2=0x7f16181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.696+0000 7f161f459700 1 --2- 172.21.15.160:0/1527691046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f16181031b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1614004940 tx=0x7f1614009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.696+0000 7f161f459700 1 -- 172.21.15.160:0/1527691046 shutdown_connections 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.696+0000 7f161f459700 1 --2- 172.21.15.160:0/1527691046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 0x7f16180fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.696+0000 7f161f459700 1 --2- 172.21.15.160:0/1527691046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f16181031b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.696+0000 7f161f459700 1 -- 172.21.15.160:0/1527691046 >> 172.21.15.160:0/1527691046 conn(0x7f16180f8990 msgr2=0x7f16180fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 -- 172.21.15.160:0/1527691046 shutdown_connections 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 -- 172.21.15.160:0/1527691046 wait complete. 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 Processor -- start 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 -- start start 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f1618070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 0x7f16180707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618072160 con 0x7f1618102de0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161f459700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618070ce0 con 0x7f1618103780 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161d1f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f1618070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161d1f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f1618070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41770/0 (socket says 172.21.15.160:41770) 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161d1f5700 1 -- 172.21.15.160:0/3285597000 learned_addr learned my addr 172.21.15.160:0/3285597000 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.697+0000 7f161c9f4700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 0x7f16180707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161d1f5700 1 -- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 msgr2=0x7f16180707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161d1f5700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 0x7f16180707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161d1f5700 1 -- 172.21.15.160:0/3285597000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614005040 con 0x7f1618102de0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161c9f4700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 0x7f16180707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161d1f5700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f1618070260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f161400b3b0 tx=0x7f1614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161400e040 con 0x7f1618102de0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1618070f60 con 0x7f1618102de0 2024-05-25T06:07:36.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f161400bbc0 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161401c3f0 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.698+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161811a110 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.699+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16180fe530 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f161400bd30 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f160dffb700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f16040692a0 0x7f160406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1614088b70 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f161c9f4700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f16040692a0 0x7f160406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f160dffb700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] conn(0x7f160406ef70 0x7f1604071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 --> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1604071a30 con 0x7f160406ef70 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1614088f50 con 0x7f1618102de0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f161c9f4700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f16040692a0 0x7f160406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1608009fd0 tx=0x7f1608009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.700+0000 7f161d9f6700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] conn(0x7f160406ef70 0x7f1604071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:36.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.701+0000 7f161d9f6700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] conn(0x7f160406ef70 0x7f1604071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.702+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== osd.4 v2:172.21.15.203:6808/2408351414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1604071a30 con 0x7f160406ef70 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.716+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 --> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f161805e5e0 con 0x7f160406ef70 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f160dffb700 1 -- 172.21.15.160:0/3285597000 <== osd.4 v2:172.21.15.203:6808/2408351414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f161805e5e0 con 0x7f160406ef70 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] conn(0x7f160406ef70 msgr2=0x7f1604071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] conn(0x7f160406ef70 0x7f1604071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f16040692a0 msgr2=0x7f160406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f16040692a0 0x7f160406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1608009fd0 tx=0x7f1608009ee0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 msgr2=0x7f1618070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f1618070260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f161400b3b0 tx=0x7f1614009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 shutdown_connections 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:6808/2408351414,v1:172.21.15.203:6809/2408351414] conn(0x7f160406ef70 0x7f1604071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f16040692a0 0x7f160406b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1618102de0 0x7f1618070260 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 --2- 172.21.15.160:0/3285597000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1618103780 0x7f16180707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 >> 172.21.15.160:0/3285597000 conn(0x7f16180f8990 msgr2=0x7f1618107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 shutdown_connections 2024-05-25T06:07:36.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:36.717+0000 7f161f459700 1 -- 172.21.15.160:0/3285597000 wait complete. 2024-05-25T06:07:37.794 INFO:teuthology.orchestra.run.smithi160.stdout:146028888078 2024-05-25T06:07:37.795 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-25T06:07:38.046 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:37 smithi160 ceph-mon[29041]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:38.196 INFO:teuthology.orchestra.run.smithi160.stdout:60129542175 2024-05-25T06:07:38.196 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-25T06:07:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:37 smithi203 ceph-mon[36030]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:39.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:39 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:39.833 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:40.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:40 smithi160 ceph-mon[29041]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:40.316 INFO:teuthology.orchestra.run.smithi160.stdout:167503724553 2024-05-25T06:07:40.317 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-25T06:07:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:40 smithi203 ceph-mon[36030]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:40.883 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:42 smithi160 ceph-mon[29041]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:42 smithi203 ceph-mon[36030]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:43.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.072+0000 7f555844a700 1 -- 172.21.15.160:0/1071864807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550065640 msgr2=0x7f5550065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.072+0000 7f555844a700 1 --2- 172.21.15.160:0/1071864807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550065640 0x7f5550065ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5540004940 tx=0x7f5540009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.072+0000 7f555844a700 1 -- 172.21.15.160:0/1071864807 shutdown_connections 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.072+0000 7f555844a700 1 --2- 172.21.15.160:0/1071864807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550065640 0x7f5550065ab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.072+0000 7f555844a700 1 --2- 172.21.15.160:0/1071864807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5550102e50 0x7f5550103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 -- 172.21.15.160:0/1071864807 >> 172.21.15.160:0/1071864807 conn(0x7f5550071d00 msgr2=0x7f5550072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 -- 172.21.15.160:0/1071864807 shutdown_connections 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 -- 172.21.15.160:0/1071864807 wait complete. 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 Processor -- start 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 -- start start 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5550065640 0x7f5550113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 0x7f55501140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5550114780 con 0x7f5550102e50 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.073+0000 7f555844a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555010dae0 con 0x7f5550065640 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 0x7f55501140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 0x7f55501140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41802/0 (socket says 172.21.15.160:41802) 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 -- 172.21.15.160:0/3239849048 learned_addr learned my addr 172.21.15.160:0/3239849048 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 -- 172.21.15.160:0/3239849048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5550065640 msgr2=0x7f5550113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55561e6700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5550065640 0x7f5550113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5550065640 0x7f5550113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 -- 172.21.15.160:0/3239849048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5540005040 con 0x7f5550102e50 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f55559e5700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 0x7f55501140a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f554000b3b0 tx=0x7f5540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554000e040 con 0x7f5550102e50 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f554000bbc0 con 0x7f5550102e50 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555010dd60 con 0x7f5550102e50 2024-05-25T06:07:43.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554001c3f0 con 0x7f5550102e50 2024-05-25T06:07:43.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.074+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555010e320 con 0x7f5550102e50 2024-05-25T06:07:43.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.075+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55500fe540 con 0x7f5550102e50 2024-05-25T06:07:43.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.076+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f554000bd30 con 0x7f5550102e50 2024-05-25T06:07:43.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.076+0000 7f5546ffd700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f553c0692a0 0x7f553c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.076+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5540088030 con 0x7f5550102e50 2024-05-25T06:07:43.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.076+0000 7f55561e6700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f553c0692a0 0x7f553c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.077+0000 7f55561e6700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f553c0692a0 0x7f553c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f554c006fd0 tx=0x7f554c008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:43.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.081+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f55400581e0 con 0x7f5550102e50 2024-05-25T06:07:43.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.272+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f555005f5c0 con 0x7f5550102e50 2024-05-25T06:07:43.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.272+0000 7f5546ffd700 1 -- 172.21.15.160:0/3239849048 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5540057d70 con 0x7f5550102e50 2024-05-25T06:07:43.273 INFO:teuthology.orchestra.run.smithi160.stdout:38654705701 2024-05-25T06:07:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f553c0692a0 msgr2=0x7f553c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f553c0692a0 0x7f553c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f554c006fd0 tx=0x7f554c008040 comp rx=0 tx=0).stop 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 msgr2=0x7f55501140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 0x7f55501140a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f554000b3b0 tx=0x7f5540009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 shutdown_connections 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f553c0692a0 0x7f553c06b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5550065640 0x7f5550113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 --2- 172.21.15.160:0/3239849048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5550102e50 0x7f55501140a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 >> 172.21.15.160:0/3239849048 conn(0x7f5550071d00 msgr2=0x7f55500f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 shutdown_connections 2024-05-25T06:07:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.275+0000 7f555844a700 1 -- 172.21.15.160:0/3239849048 wait complete. 2024-05-25T06:07:43.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 -- 172.21.15.160:0/2250614827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fbbe0 msgr2=0x7fc9f80fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 --2- 172.21.15.160:0/2250614827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fbbe0 0x7fc9f80fbfb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004940 tx=0x7fc9f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:43.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 -- 172.21.15.160:0/2250614827 shutdown_connections 2024-05-25T06:07:43.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 --2- 172.21.15.160:0/2250614827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f80fc580 0x7fc9f8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 --2- 172.21.15.160:0/2250614827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fbbe0 0x7fc9f80fbfb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 -- 172.21.15.160:0/2250614827 >> 172.21.15.160:0/2250614827 conn(0x7fc9f80f7860 msgr2=0x7fc9f80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 -- 172.21.15.160:0/2250614827 shutdown_connections 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.708+0000 7fca00196700 1 -- 172.21.15.160:0/2250614827 wait complete. 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fca00196700 1 Processor -- start 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fca00196700 1 -- start start 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fca00196700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f80fbbe0 0x7fc9f811a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fca00196700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 0x7fc9f811a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fca00196700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f811ada0 con 0x7fc9f80fc580 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fca00196700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81141c0 con 0x7fc9f80fbbe0 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 0x7fc9f811a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 0x7fc9f811a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41816/0 (socket says 172.21.15.160:41816) 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 -- 172.21.15.160:0/2106774499 learned_addr learned my addr 172.21.15.160:0/2106774499 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:43.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fdf32700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f80fbbe0 0x7fc9f811a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 -- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f80fbbe0 msgr2=0x7fc9f811a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f80fbbe0 0x7fc9f811a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 -- 172.21.15.160:0/2106774499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4005040 con 0x7fc9f80fc580 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.709+0000 7fc9fd731700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 0x7fc9f811a750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c830 tx=0x7fc9e8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.710+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e80106f0 con 0x7fc9f80fc580 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.710+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f81144a0 con 0x7fc9f80fc580 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.710+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9e8010d20 con 0x7fc9f80fc580 2024-05-25T06:07:43.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.710+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e80113a0 con 0x7fc9f80fc580 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.710+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f81149e0 con 0x7fc9f80fc580 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.711+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9f8114f20 con 0x7fc9f80fc580 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.711+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc9e8012400 con 0x7fc9f80fc580 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9ee7fc700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc9e40692a0 0x7fc9e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc9e8086e20 con 0x7fc9f80fc580 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9fdf32700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc9e40692a0 0x7fc9e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9ee7fc700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] conn(0x7fc9e406ef70 0x7fc9e4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 --> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9e4071a30 con 0x7fc9e406ef70 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e808c480 con 0x7fc9f80fc580 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9fe733700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] conn(0x7fc9e406ef70 0x7fc9e4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:43.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.712+0000 7fc9fdf32700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc9e40692a0 0x7fc9e406b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004f90 tx=0x7fc9f4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:43.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.713+0000 7fc9fe733700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] conn(0x7fc9e406ef70 0x7fc9e4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:43.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.713+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== osd.3 v2:172.21.15.203:6800/3678463726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc9e4071a30 con 0x7fc9e406ef70 2024-05-25T06:07:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.728+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 --> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc9f80f9830 con 0x7fc9e406ef70 2024-05-25T06:07:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.728+0000 7fc9ee7fc700 1 -- 172.21.15.160:0/2106774499 <== osd.3 v2:172.21.15.203:6800/3678463726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc9f80f9830 con 0x7fc9e406ef70 2024-05-25T06:07:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] conn(0x7fc9e406ef70 msgr2=0x7fc9e4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] conn(0x7fc9e406ef70 0x7fc9e4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc9e40692a0 msgr2=0x7fc9e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc9e40692a0 0x7fc9e406b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004f90 tx=0x7fc9f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 msgr2=0x7fc9f811a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 0x7fc9f811a750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c830 tx=0x7fc9e8004470 comp rx=0 tx=0).stop 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 shutdown_connections 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:6800/3678463726,v1:172.21.15.203:6801/3678463726] conn(0x7fc9e406ef70 0x7fc9e4071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc9e40692a0 0x7fc9e406b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f80fbbe0 0x7fc9f811a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 --2- 172.21.15.160:0/2106774499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f80fc580 0x7fc9f811a750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 >> 172.21.15.160:0/2106774499 conn(0x7fc9f80f7860 msgr2=0x7fc9f80f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 shutdown_connections 2024-05-25T06:07:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.729+0000 7fca00196700 1 -- 172.21.15.160:0/2106774499 wait complete. 2024-05-25T06:07:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:44 smithi160 ceph-mon[29041]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:44 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3239849048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-25T06:07:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:44 smithi203 ceph-mon[36030]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:44 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3239849048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-25T06:07:44.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-05-25T06:07:44.538 DEBUG:teuthology.parallel:result is None 2024-05-25T06:07:45.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 -- 172.21.15.160:0/2691459671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00103760 msgr2=0x7f9e000fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:45.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 --2- 172.21.15.160:0/2691459671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00103760 0x7f9e000fcdd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9df4004940 tx=0x7f9df4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 -- 172.21.15.160:0/2691459671 shutdown_connections 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 --2- 172.21.15.160:0/2691459671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00103760 0x7f9e000fcdd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 --2- 172.21.15.160:0/2691459671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e00102dc0 0x7f9e00103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 -- 172.21.15.160:0/2691459671 >> 172.21.15.160:0/2691459671 conn(0x7f9e000f8930 msgr2=0x7f9e000fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 -- 172.21.15.160:0/2691459671 shutdown_connections 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.915+0000 7f9e05391700 1 -- 172.21.15.160:0/2691459671 wait complete. 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.916+0000 7f9e05391700 1 Processor -- start 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.916+0000 7f9e05391700 1 -- start start 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.916+0000 7f9e05391700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 0x7f9e00113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.916+0000 7f9e05391700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e00103760 0x7f9e00113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.916+0000 7f9e05391700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001145b0 con 0x7f9e00102dc0 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.916+0000 7f9e05391700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0010db20 con 0x7f9e00103760 2024-05-25T06:07:45.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 0x7f9e00113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stdout:81604378650 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 0x7f9e00113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41840/0 (socket says 172.21.15.160:41840) 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 -- 172.21.15.160:0/1708463263 learned_addr learned my addr 172.21.15.160:0/1708463263 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfe7fc700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e00103760 0x7f9e00113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 -- 172.21.15.160:0/1708463263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e00103760 msgr2=0x7f9e00113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e00103760 0x7f9e00113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 -- 172.21.15.160:0/1708463263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4005040 con 0x7f9e00102dc0 2024-05-25T06:07:45.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9dfeffd700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 0x7f9e00113990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b790 tx=0x7f9df0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9df000f6f0 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9df000fd20 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0010de00 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9df00103a0 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.917+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0010e390 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.918+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e001000d0 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.919+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9df0010500 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.919+0000 7f9de77fe700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9de80692a0 0x7f9de806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.919+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9df0086c80 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.919+0000 7f9dfe7fc700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9de80692a0 0x7f9de806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.920+0000 7f9dfe7fc700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9de80692a0 0x7f9de806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9df4004890 tx=0x7f9df4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:43.924+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9df0056d80 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.116+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e0010efe0 con 0x7f9e00102dc0 2024-05-25T06:07:45.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.117+0000 7f9de77fe700 1 -- 172.21.15.160:0/1708463263 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9df0056910 con 0x7f9e00102dc0 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9de80692a0 msgr2=0x7f9de806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9de80692a0 0x7f9de806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9df4004890 tx=0x7f9df4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 msgr2=0x7f9e00113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 0x7f9e00113990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b790 tx=0x7f9df0009250 comp rx=0 tx=0).stop 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 shutdown_connections 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9de80692a0 0x7f9de806b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e00102dc0 0x7f9e00113990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 --2- 172.21.15.160:0/1708463263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e00103760 0x7f9e00113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 >> 172.21.15.160:0/1708463263 conn(0x7f9e000f8930 msgr2=0x7f9e000fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 shutdown_connections 2024-05-25T06:07:45.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:44.120+0000 7f9e05391700 1 -- 172.21.15.160:0/1708463263 wait complete. 2024-05-25T06:07:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:45 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1708463263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-25T06:07:45.300 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:45 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1708463263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-25T06:07:46.041 INFO:teuthology.orchestra.run.smithi160.stdout:111669149717 2024-05-25T06:07:46.041 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-25T06:07:46.085 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:46.090 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:46.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:46 smithi160 ceph-mon[29041]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:46 smithi203 ceph-mon[36030]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:46.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-05-25T06:07:46.615 DEBUG:teuthology.parallel:result is None 2024-05-25T06:07:48.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 -- 172.21.15.160:0/1628877692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc103780 msgr2=0x7f5afc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 --2- 172.21.15.160:0/1628877692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc103780 0x7f5afc0fcdf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009ce0 tx=0x7f5af0004d10 comp rx=0 tx=0).stop 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 -- 172.21.15.160:0/1628877692 shutdown_connections 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 --2- 172.21.15.160:0/1628877692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc103780 0x7f5afc0fcdf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 --2- 172.21.15.160:0/1628877692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5afc102de0 0x7f5afc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 -- 172.21.15.160:0/1628877692 >> 172.21.15.160:0/1628877692 conn(0x7f5afc0f8950 msgr2=0x7f5afc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 -- 172.21.15.160:0/1628877692 shutdown_connections 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.225+0000 7f5b021e4700 1 -- 172.21.15.160:0/1628877692 wait complete. 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5b021e4700 1 Processor -- start 2024-05-25T06:07:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5b021e4700 1 -- start start 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5b021e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 0x7f5afc113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5b021e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5afc103780 0x7f5afc114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5b021e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc114880 con 0x7f5afc102de0 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5b021e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc10dc50 con 0x7f5afc103780 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 0x7f5afc113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 0x7f5afc113cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58068/0 (socket says 172.21.15.160:58068) 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afb7fe700 1 -- 172.21.15.160:0/2112298808 learned_addr learned my addr 172.21.15.160:0/2112298808 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afaffd700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5afc103780 0x7f5afc114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afb7fe700 1 -- 172.21.15.160:0/2112298808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5afc103780 msgr2=0x7f5afc114230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afb7fe700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5afc103780 0x7f5afc114230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.226+0000 7f5afb7fe700 1 -- 172.21.15.160:0/2112298808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af0009940 con 0x7f5afc102de0 2024-05-25T06:07:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.227+0000 7f5afb7fe700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 0x7f5afc113cf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5af400c8c0 tx=0x7f5af40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.227+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5af400a980 con 0x7f5afc102de0 2024-05-25T06:07:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.227+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5af400ce60 con 0x7f5afc102de0 2024-05-25T06:07:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.227+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc10df30 con 0x7f5afc102de0 2024-05-25T06:07:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.227+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5af40115e0 con 0x7f5afc102de0 2024-05-25T06:07:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.227+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc10e4f0 con 0x7f5afc102de0 2024-05-25T06:07:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.228+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc0fe530 con 0x7f5afc102de0 2024-05-25T06:07:48.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.228+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5af4012400 con 0x7f5afc102de0 2024-05-25T06:07:48.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.229+0000 7f5ae3fff700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f5ae40692a0 0x7f5ae406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:48.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.229+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5af4086f60 con 0x7f5afc102de0 2024-05-25T06:07:48.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.229+0000 7f5afaffd700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f5ae40692a0 0x7f5ae406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:48.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.229+0000 7f5afaffd700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f5ae40692a0 0x7f5ae406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5afc10f590 tx=0x7f5af0004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:48.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.233+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5af4057060 con 0x7f5afc102de0 2024-05-25T06:07:48.311 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:48 smithi160 ceph-mon[29041]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:48 smithi203 ceph-mon[36030]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:48.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.424+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5afc05f5c0 con 0x7f5afc102de0 2024-05-25T06:07:48.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.424+0000 7f5ae3fff700 1 -- 172.21.15.160:0/2112298808 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5af40126b0 con 0x7f5afc102de0 2024-05-25T06:07:48.424 INFO:teuthology.orchestra.run.smithi160.stdout:60129542177 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f5ae40692a0 msgr2=0x7f5ae406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f5ae40692a0 0x7f5ae406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5afc10f590 tx=0x7f5af0004d10 comp rx=0 tx=0).stop 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 msgr2=0x7f5afc113cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 0x7f5afc113cf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5af400c8c0 tx=0x7f5af40044c0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 shutdown_connections 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f5ae40692a0 0x7f5ae406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5afc102de0 0x7f5afc113cf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 --2- 172.21.15.160:0/2112298808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5afc103780 0x7f5afc114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.426+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 >> 172.21.15.160:0/2112298808 conn(0x7f5afc0f8950 msgr2=0x7f5afc0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.427+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 shutdown_connections 2024-05-25T06:07:48.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:48.427+0000 7f5b021e4700 1 -- 172.21.15.160:0/2112298808 wait complete. 2024-05-25T06:07:49.199 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:49.316 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:49 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2112298808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-25T06:07:49.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:49 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2112298808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-25T06:07:49.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 -- 172.21.15.160:0/2625892593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fc620 msgr2=0x7f0e10100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:49.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 --2- 172.21.15.160:0/2625892593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fc620 0x7f0e10100c40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0e00004940 tx=0x7f0e00009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:49.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 -- 172.21.15.160:0/2625892593 shutdown_connections 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 --2- 172.21.15.160:0/2625892593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fc620 0x7f0e10100c40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 --2- 172.21.15.160:0/2625892593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fbc80 0x7f0e100fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 -- 172.21.15.160:0/2625892593 >> 172.21.15.160:0/2625892593 conn(0x7f0e100f7940 msgr2=0x7f0e100f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 -- 172.21.15.160:0/2625892593 shutdown_connections 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 -- 172.21.15.160:0/2625892593 wait complete. 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 Processor -- start 2024-05-25T06:07:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.635+0000 7f0e182ae700 1 -- start start 2024-05-25T06:07:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e182ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 0x7f0e10119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e182ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fc620 0x7f0e1011a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e182ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1011ac10 con 0x7f0e100fbc80 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e182ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e10114160 con 0x7f0e100fc620 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 0x7f0e10119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 0x7f0e10119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58076/0 (socket says 172.21.15.160:58076) 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 -- 172.21.15.160:0/1109175928 learned_addr learned my addr 172.21.15.160:0/1109175928 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e15849700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fc620 0x7f0e1011a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 -- 172.21.15.160:0/1109175928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fc620 msgr2=0x7f0e1011a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fc620 0x7f0e1011a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 -- 172.21.15.160:0/1109175928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00005040 con 0x7f0e100fbc80 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e15849700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fc620 0x7f0e1011a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e1604a700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 0x7f0e10119ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b790 tx=0x7f0e0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c00f770 con 0x7f0e100fbc80 2024-05-25T06:07:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0c00fda0 con 0x7f0e100fbc80 2024-05-25T06:07:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e101143e0 con 0x7f0e100fbc80 2024-05-25T06:07:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.636+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c00d310 con 0x7f0e100fbc80 2024-05-25T06:07:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.637+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e10114920 con 0x7f0e100fbc80 2024-05-25T06:07:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.637+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e10066b70 con 0x7f0e100fbc80 2024-05-25T06:07:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.638+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0e0c010420 con 0x7f0e100fbc80 2024-05-25T06:07:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.638+0000 7f0e06ffd700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0dfc0692a0 0x7f0dfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:49.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.639+0000 7f0e15849700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0dfc0692a0 0x7f0dfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:49.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.639+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0e0c086e80 con 0x7f0e100fbc80 2024-05-25T06:07:49.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.639+0000 7f0e15849700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0dfc0692a0 0x7f0dfc06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e10115890 tx=0x7f0e00009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.643+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0e0c056f80 con 0x7f0e100fbc80 2024-05-25T06:07:49.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.833+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0e10115640 con 0x7f0e100fbc80 2024-05-25T06:07:49.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.834+0000 7f0e06ffd700 1 -- 172.21.15.160:0/1109175928 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e0c056b10 con 0x7f0e100fbc80 2024-05-25T06:07:49.834 INFO:teuthology.orchestra.run.smithi160.stdout:146028888081 2024-05-25T06:07:49.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0dfc0692a0 msgr2=0x7f0dfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:49.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0dfc0692a0 0x7f0dfc06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e10115890 tx=0x7f0e00009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:49.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 msgr2=0x7f0e10119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:49.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 0x7f0e10119ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b790 tx=0x7f0e0c009250 comp rx=0 tx=0).stop 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 shutdown_connections 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0dfc0692a0 0x7f0dfc06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e100fbc80 0x7f0e10119ff0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 --2- 172.21.15.160:0/1109175928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100fc620 0x7f0e1011a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 >> 172.21.15.160:0/1109175928 conn(0x7f0e100f7940 msgr2=0x7f0e1010aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 shutdown_connections 2024-05-25T06:07:49.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:49.836+0000 7f0e182ae700 1 -- 172.21.15.160:0/1109175928 wait complete. 2024-05-25T06:07:50.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:50 smithi160 ceph-mon[29041]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:50.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:50 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1109175928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-25T06:07:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:50 smithi203 ceph-mon[36030]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:50 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1109175928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-25T06:07:51.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-05-25T06:07:51.197 DEBUG:teuthology.parallel:result is None 2024-05-25T06:07:52.027 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888081 for osd.4 2024-05-25T06:07:52.027 DEBUG:teuthology.parallel:result is None 2024-05-25T06:07:52.288 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:52 smithi160 ceph-mon[29041]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:52 smithi203 ceph-mon[36030]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:52.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 -- 172.21.15.160:0/2974650960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fbc00 msgr2=0x7faf9c0fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:52.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2974650960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fbc00 0x7faf9c0fbfd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf8c004940 tx=0x7faf8c009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:52.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 -- 172.21.15.160:0/2974650960 shutdown_connections 2024-05-25T06:07:52.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2974650960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2974650960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fbc00 0x7faf9c0fbfd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 -- 172.21.15.160:0/2974650960 >> 172.21.15.160:0/2974650960 conn(0x7faf9c0f7820 msgr2=0x7faf9c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 -- 172.21.15.160:0/2974650960 shutdown_connections 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.712+0000 7fafa10ff700 1 -- 172.21.15.160:0/2974650960 wait complete. 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7fafa10ff700 1 Processor -- start 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7fafa10ff700 1 -- start start 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7fafa10ff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c0fbc00 0x7faf9c11a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7fafa10ff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c11a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7fafa10ff700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c11aeb0 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7fafa10ff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c114490 con 0x7faf9c0fbc00 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.713+0000 7faf9a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c11a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c11a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58088/0 (socket says 172.21.15.160:58088) 2024-05-25T06:07:52.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9a59c700 1 -- 172.21.15.160:0/2621470502 learned_addr learned my addr 172.21.15.160:0/2621470502 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9ad9d700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c0fbc00 0x7faf9c11a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9a59c700 1 -- 172.21.15.160:0/2621470502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c0fbc00 msgr2=0x7faf9c11a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9a59c700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c0fbc00 0x7faf9c11a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9a59c700 1 -- 172.21.15.160:0/2621470502 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c005040 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf9a59c700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c11a860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faf90002a50 tx=0x7faf900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf90007cc0 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf9000f040 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c114770 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9000da30 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.714+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c114cb0 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.715+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c0fe2e0 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.716+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7faf9000db90 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.716+0000 7faf837fe700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faf840692a0 0x7faf8406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:52.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.716+0000 7faf9ad9d700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faf840692a0 0x7faf8406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:52.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.716+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faf90086c10 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.717+0000 7faf9ad9d700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faf840692a0 0x7faf8406b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf8c004f90 tx=0x7faf8c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:52.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.721+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faf90056c90 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.914+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf9c066b30 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.914+0000 7faf837fe700 1 -- 172.21.15.160:0/2621470502 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf90056820 con 0x7faf9c0fc5a0 2024-05-25T06:07:52.915 INFO:teuthology.orchestra.run.smithi160.stdout:111669149718 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faf840692a0 msgr2=0x7faf8406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faf840692a0 0x7faf8406b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf8c004f90 tx=0x7faf8c009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 msgr2=0x7faf9c11a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c11a860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faf90002a50 tx=0x7faf900046b0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 shutdown_connections 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faf840692a0 0x7faf8406b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c0fbc00 0x7faf9c11a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 --2- 172.21.15.160:0/2621470502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c0fc5a0 0x7faf9c11a860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 >> 172.21.15.160:0/2621470502 conn(0x7faf9c0f7820 msgr2=0x7faf9c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:52.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 shutdown_connections 2024-05-25T06:07:52.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.917+0000 7fafa10ff700 1 -- 172.21.15.160:0/2621470502 wait complete. 2024-05-25T06:07:52.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3564393236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0040fbd40 msgr2=0x7fb0040fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:52.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3564393236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0040fbd40 0x7fb0040fc1b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faff4004940 tx=0x7faff4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:52.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3564393236 shutdown_connections 2024-05-25T06:07:52.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3564393236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0040fbd40 0x7fb0040fc1b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3564393236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb004101d90 0x7fb004102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3564393236 >> 172.21.15.160:0/3564393236 conn(0x7fb004071d00 msgr2=0x7fb004072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:52.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3564393236 shutdown_connections 2024-05-25T06:07:52.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.931+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3564393236 wait complete. 2024-05-25T06:07:52.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.932+0000 7fb00c0f1700 1 Processor -- start 2024-05-25T06:07:52.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.932+0000 7fb00c0f1700 1 -- start start 2024-05-25T06:07:52.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.932+0000 7fb00c0f1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0040fbd40 0x7fb004112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00c0f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 0x7fb004112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00c0f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004113640 con 0x7fb004101d90 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00c0f1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00410ca30 con 0x7fb0040fbd40 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 0x7fb004112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 0x7fb004112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58114/0 (socket says 172.21.15.160:58114) 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 -- 172.21.15.160:0/3210822683 learned_addr learned my addr 172.21.15.160:0/3210822683 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb009e8d700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0040fbd40 0x7fb004112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 -- 172.21.15.160:0/3210822683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0040fbd40 msgr2=0x7fb004112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0040fbd40 0x7fb004112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:52.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 -- 172.21.15.160:0/3210822683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4005040 con 0x7fb004101d90 2024-05-25T06:07:52.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00968c700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 0x7fb004112ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faff400b3b0 tx=0x7faff4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:52.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff400e040 con 0x7fb004101d90 2024-05-25T06:07:52.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faff400bdd0 con 0x7fb004101d90 2024-05-25T06:07:52.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00410ccb0 con 0x7fb004101d90 2024-05-25T06:07:52.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.934+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00410d210 con 0x7fb004101d90 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.933+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff40136f0 con 0x7fb004101d90 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.935+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0040fd430 con 0x7fb004101d90 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.935+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7faff40138a0 con 0x7fb004101d90 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.936+0000 7faffa7fc700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faff00692a0 0x7faff006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.936+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faff408ad20 con 0x7fb004101d90 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.936+0000 7fb009e8d700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faff00692a0 0x7faff006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:52.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.936+0000 7fb009e8d700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faff00692a0 0x7faff006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb000004990 tx=0x7fb000009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:52.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:52.940+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faff405ada0 con 0x7fb004101d90 2024-05-25T06:07:53.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.138+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb004108df0 con 0x7fb004101d90 2024-05-25T06:07:53.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.138+0000 7faffa7fc700 1 -- 172.21.15.160:0/3210822683 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faff405a930 con 0x7fb004101d90 2024-05-25T06:07:53.139 INFO:teuthology.orchestra.run.smithi160.stdout:167503724556 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faff00692a0 msgr2=0x7faff006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faff00692a0 0x7faff006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb000004990 tx=0x7fb000009ee0 comp rx=0 tx=0).stop 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 msgr2=0x7fb004112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 0x7fb004112ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faff400b3b0 tx=0x7faff4009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 shutdown_connections 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7faff00692a0 0x7faff006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0040fbd40 0x7fb004112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:53.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 --2- 172.21.15.160:0/3210822683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb004101d90 0x7fb004112ff0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:53.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 >> 172.21.15.160:0/3210822683 conn(0x7fb004071d00 msgr2=0x7fb0040fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:53.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 shutdown_connections 2024-05-25T06:07:53.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:53.141+0000 7fb00c0f1700 1 -- 172.21.15.160:0/3210822683 wait complete. 2024-05-25T06:07:53.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:53 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2621470502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-25T06:07:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:53 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2621470502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-25T06:07:53.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149717 got 111669149718 for osd.3 2024-05-25T06:07:53.516 DEBUG:teuthology.parallel:result is None 2024-05-25T06:07:54.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:54 smithi160 ceph-mon[29041]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:54.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:54 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3210822683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-25T06:07:54.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:54 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:54.230 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724556 for osd.5 2024-05-25T06:07:54.231 DEBUG:teuthology.parallel:result is None 2024-05-25T06:07:54.231 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-25T06:07:54.231 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-25T06:07:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:54 smithi203 ceph-mon[36030]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:54 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3210822683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-25T06:07:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:54 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:07:54.492 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:56.131 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:56 smithi160 ceph-mon[29041]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:56.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 -- 172.21.15.160:0/3350414736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 msgr2=0x7f12b80fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:56.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 --2- 172.21.15.160:0/3350414736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b80fbc70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009ce0 tx=0x7f12ac004d10 comp rx=0 tx=0).stop 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 -- 172.21.15.160:0/3350414736 shutdown_connections 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 --2- 172.21.15.160:0/3350414736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b80fbc70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 --2- 172.21.15.160:0/3350414736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b8101c60 0x7f12b8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 -- 172.21.15.160:0/3350414736 >> 172.21.15.160:0/3350414736 conn(0x7f12b80f7810 msgr2=0x7f12b80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 -- 172.21.15.160:0/3350414736 shutdown_connections 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.231+0000 7f12c0685700 1 -- 172.21.15.160:0/3350414736 wait complete. 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12c0685700 1 Processor -- start 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12c0685700 1 -- start start 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12c0685700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b8101c60 0x7f12b810ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12c0685700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b810f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12c0685700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b810f860 con 0x7f12b8102600 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12c0685700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b810f9a0 con 0x7f12b8101c60 2024-05-25T06:07:56.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12bdc20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b810f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12bdc20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b810f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43914/0 (socket says 172.21.15.160:43914) 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12bdc20700 1 -- 172.21.15.160:0/198070368 learned_addr learned my addr 172.21.15.160:0/198070368 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12be421700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b8101c60 0x7f12b810ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12bdc20700 1 -- 172.21.15.160:0/198070368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b8101c60 msgr2=0x7f12b810ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12bdc20700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b8101c60 0x7f12b810ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.232+0000 7f12bdc20700 1 -- 172.21.15.160:0/198070368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac009940 con 0x7f12b8102600 2024-05-25T06:07:56.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.233+0000 7f12bdc20700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b810f320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00be40 tx=0x7f12ac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:56.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.233+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ac00e040 con 0x7f12b8102600 2024-05-25T06:07:56.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.233+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b8117c90 con 0x7f12b8102600 2024-05-25T06:07:56.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.233+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12ac004610 con 0x7f12b8102600 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.233+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ac012650 con 0x7f12b8102600 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.233+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b81181f0 con 0x7f12b8102600 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.234+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b80fd3b0 con 0x7f12b8102600 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.234+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f12ac007d10 con 0x7f12b8102600 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.234+0000 7f12b6ffd700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12a00692a0 0x7f12a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.235+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f12ac088a60 con 0x7f12b8102600 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.235+0000 7f12be421700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12a00692a0 0x7f12a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:56.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.235+0000 7f12be421700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12a00692a0 0x7f12a006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12a8004f40 tx=0x7f12a800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:56.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.239+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f12ac055050 con 0x7f12b8102600 2024-05-25T06:07:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:56 smithi203 ceph-mon[36030]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:56.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.419+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12b804bed0 con 0x7f12a00692a0 2024-05-25T06:07:56.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.421+0000 7f12b6ffd700 1 -- 172.21.15.160:0/198070368 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27986 (secure 0 0 0) 0x7f12b804bed0 con 0x7f12a00692a0 2024-05-25T06:07:56.421 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12a00692a0 msgr2=0x7f12a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12a00692a0 0x7f12a006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12a8004f40 tx=0x7f12a800b040 comp rx=0 tx=0).stop 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 msgr2=0x7f12b810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b810f320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00be40 tx=0x7f12ac004d10 comp rx=0 tx=0).stop 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 shutdown_connections 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f12a00692a0 0x7f12a006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:56.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b8101c60 0x7f12b810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:56.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 --2- 172.21.15.160:0/198070368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8102600 0x7f12b810f320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:56.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 >> 172.21.15.160:0/198070368 conn(0x7f12b80f7810 msgr2=0x7f12b80f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:56.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 shutdown_connections 2024-05-25T06:07:56.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:56.423+0000 7f12c0685700 1 -- 172.21.15.160:0/198070368 wait complete. 2024-05-25T06:07:56.424 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-25T06:07:57.103 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-05-25T06:07:54.631126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002228"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-25T06:07:03.233908+0000","last_change":"2024-05-25T06:06:37.338244+0000","last_active":"2024-05-25T06:07:03.233908+0000","last_peered":"2024-05-25T06:07:03.233908+0000","last_clean":"2024-05-25T06:07:03.233908+0000","last_became_active":"2024-05-25T06:06:37.337771+0000","last_became_peered":"2024-05-25T06:06:37.337771+0000","last_unstale":"2024-05-25T06:07:03.233908+0000","last_undegraded":"2024-05-25T06:07:03.233908+0000","last_fullsized":"2024-05-25T06:07:03.233908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-25T06:05:49.656846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-25T06:05:49.656846+0000","last_clean_scrub_stamp":"2024-05-25T06:05:49.656846+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-26T12:20:54.998416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46400000000000002}]},{"osd":1,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.437},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51300000000000001}]},{"osd":3,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.38500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.498}]},{"osd":1,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45200000000000001}]},{"osd":2,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.433},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.45800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.375,"15min":0.38100000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48199999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.27400000000000002}]},{"osd":1,"last update":"Sat May 25 06:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.307},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.52000000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.32300000000000001}]},{"osd":3,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat May 25 06:07:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 25 06:07:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.221,"5min":0.182,"15min":0.182},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20300000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat May 25 06:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.38300000000000001}]},{"osd":3,"last update":"Sat May 25 06:07:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.67000000000000004}]},{"osd":4,"last update":"Sat May 25 06:07:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.64300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:23 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.314},"min":{"1min":0.247,"5min":0.219,"15min":0.219},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.41199999999999998}]},{"osd":2,"last update":"Sat May 25 06:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.441}]},{"osd":3,"last update":"Sat May 25 06:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-25T06:07:57.104 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-25T06:07:57.361 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:07:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:58 smithi203 ceph-mon[36030]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-25T06:07:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:07:58 smithi203 ceph-mon[36030]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:58.424 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:58 smithi160 ceph-mon[29041]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-25T06:07:58.425 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:07:58 smithi160 ceph-mon[29041]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 -- 172.21.15.160:0/3764362782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728102de0 msgr2=0x7f87281031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 --2- 172.21.15.160:0/3764362782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728102de0 0x7f87281031b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8724004940 tx=0x7f8724009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 -- 172.21.15.160:0/3764362782 shutdown_connections 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 --2- 172.21.15.160:0/3764362782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8728103780 0x7f87280fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 --2- 172.21.15.160:0/3764362782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728102de0 0x7f87281031b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 -- 172.21.15.160:0/3764362782 >> 172.21.15.160:0/3764362782 conn(0x7f87280f8990 msgr2=0x7f87280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 -- 172.21.15.160:0/3764362782 shutdown_connections 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 -- 172.21.15.160:0/3764362782 wait complete. 2024-05-25T06:07:58.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.917+0000 7f8730060700 1 Processor -- start 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f8730060700 1 -- start start 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f8730060700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8728102de0 0x7f8728113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f8730060700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 0x7f8728113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f8730060700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87281145b0 con 0x7f8728103780 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f8730060700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872810db20 con 0x7f8728102de0 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 0x7f8728113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 0x7f8728113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43942/0 (socket says 172.21.15.160:43942) 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 -- 172.21.15.160:0/4118429836 learned_addr learned my addr 172.21.15.160:0/4118429836 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872ddfc700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8728102de0 0x7f8728113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 -- 172.21.15.160:0/4118429836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8728102de0 msgr2=0x7f8728113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8728102de0 0x7f8728113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 -- 172.21.15.160:0/4118429836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724005040 con 0x7f8728103780 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f872d5fb700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 0x7f8728113ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f871800b790 tx=0x7f8718009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:58.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f871800f6f0 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f871800fd20 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872810de00 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.918+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87180103a0 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.919+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872810e3c0 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.919+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87281000f0 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.920+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8718011400 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.920+0000 7f871e7fc700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f87140692a0 0x7f871406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.920+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8718086d20 con 0x7f8728103780 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.920+0000 7f872ddfc700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f87140692a0 0x7f871406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.921+0000 7f872ddfc700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f87140692a0 0x7f871406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f872400b3b0 tx=0x7f8724009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:07:58.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:58.925+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8718056da0 con 0x7f8728103780 2024-05-25T06:07:59.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.109+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f872810efe0 con 0x7f87140692a0 2024-05-25T06:07:59.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.110+0000 7f871e7fc700 1 -- 172.21.15.160:0/4118429836 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27955 (secure 0 0 0) 0x7f872810efe0 con 0x7f87140692a0 2024-05-25T06:07:59.112 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f87140692a0 msgr2=0x7f871406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f87140692a0 0x7f871406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f872400b3b0 tx=0x7f8724009e30 comp rx=0 tx=0).stop 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 msgr2=0x7f8728113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 0x7f8728113ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f871800b790 tx=0x7f8718009220 comp rx=0 tx=0).stop 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 shutdown_connections 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f87140692a0 0x7f871406b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8728102de0 0x7f8728113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 --2- 172.21.15.160:0/4118429836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8728103780 0x7f8728113ed0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.112+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 >> 172.21.15.160:0/4118429836 conn(0x7f87280f8990 msgr2=0x7f8728107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.113+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 shutdown_connections 2024-05-25T06:07:59.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:07:59.113+0000 7f8730060700 1 -- 172.21.15.160:0/4118429836 wait complete. 2024-05-25T06:07:59.115 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-25T06:07:59.714 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-05-25T06:07:58.631728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001942"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-25T06:07:03.233908+0000","last_change":"2024-05-25T06:06:37.338244+0000","last_active":"2024-05-25T06:07:03.233908+0000","last_peered":"2024-05-25T06:07:03.233908+0000","last_clean":"2024-05-25T06:07:03.233908+0000","last_became_active":"2024-05-25T06:06:37.337771+0000","last_became_peered":"2024-05-25T06:06:37.337771+0000","last_unstale":"2024-05-25T06:07:03.233908+0000","last_undegraded":"2024-05-25T06:07:03.233908+0000","last_fullsized":"2024-05-25T06:07:03.233908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-25T06:05:49.656846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-25T06:05:49.656846+0000","last_clean_scrub_stamp":"2024-05-25T06:05:49.656846+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-26T12:20:54.998416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.623},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60599999999999998}]},{"osd":3,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.371},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45900000000000002}]},{"osd":1,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38800000000000001}]},{"osd":4,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.45800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.375,"15min":0.38100000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48199999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.313}]},{"osd":1,"last update":"Sat May 25 06:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.52000000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.29799999999999999}]},{"osd":3,"last update":"Sat May 25 06:07:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Sat May 25 06:07:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 25 06:07:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.221,"5min":0.182,"15min":0.182},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.309},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20300000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29599999999999999}]},{"osd":2,"last update":"Sat May 25 06:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.498,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.27700000000000002}]},{"osd":3,"last update":"Sat May 25 06:07:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat May 25 06:07:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 25 06:07:23 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.314},"min":{"1min":0.247,"5min":0.219,"15min":0.219},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Sat May 25 06:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.49199999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.26900000000000002}]},{"osd":3,"last update":"Sat May 25 06:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat May 25 06:07:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.433},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-25T06:07:59.715 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-25T06:07:59.715 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-25T06:07:59.715 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-25T06:07:59.715 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-25T06:07:59.972 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:00.384 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:00 smithi160 ceph-mon[29041]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:00 smithi203 ceph-mon[36030]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:01.368 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:01 smithi160 ceph-mon[29041]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-25T06:08:01.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:01 smithi203 ceph-mon[36030]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-25T06:08:01.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 -- 172.21.15.160:0/360084952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 msgr2=0x7f90100fbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:01.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 --2- 172.21.15.160:0/360084952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f90100fbf90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f900c004940 tx=0x7f900c009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:01.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 -- 172.21.15.160:0/360084952 shutdown_connections 2024-05-25T06:08:01.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 --2- 172.21.15.160:0/360084952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100fc560 0x7f9010100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:01.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 --2- 172.21.15.160:0/360084952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f90100fbf90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:01.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 -- 172.21.15.160:0/360084952 >> 172.21.15.160:0/360084952 conn(0x7f90100f7860 msgr2=0x7f90100f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:01.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 -- 172.21.15.160:0/360084952 shutdown_connections 2024-05-25T06:08:01.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.612+0000 7f90174f0700 1 -- 172.21.15.160:0/360084952 wait complete. 2024-05-25T06:08:01.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f90174f0700 1 Processor -- start 2024-05-25T06:08:01.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f90174f0700 1 -- start start 2024-05-25T06:08:01.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f90174f0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f9010119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:01.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f90174f0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100fc560 0x7f901011a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:01.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f90174f0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901011abb0 con 0x7f90100fbbc0 2024-05-25T06:08:01.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f90174f0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010114100 con 0x7f90100fc560 2024-05-25T06:08:01.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f9010119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:01.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f9010119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43952/0 (socket says 172.21.15.160:43952) 2024-05-25T06:08:01.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 -- 172.21.15.160:0/753066449 learned_addr learned my addr 172.21.15.160:0/753066449 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:01.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 -- 172.21.15.160:0/753066449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100fc560 msgr2=0x7f901011a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:01.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f9014a8b700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100fc560 0x7f901011a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:01.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100fc560 0x7f901011a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:01.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 -- 172.21.15.160:0/753066449 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c005040 con 0x7f90100fbbc0 2024-05-25T06:08:01.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f901528c700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f9010119f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f900c00b3b0 tx=0x7f900c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:01.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.613+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900c00e040 con 0x7f90100fbbc0 2024-05-25T06:08:01.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.614+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f900c00bbc0 con 0x7f90100fbbc0 2024-05-25T06:08:01.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.614+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9010114320 con 0x7f90100fbbc0 2024-05-25T06:08:01.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.614+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900c01c3f0 con 0x7f90100fbbc0 2024-05-25T06:08:01.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.614+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9010114860 con 0x7f90100fbbc0 2024-05-25T06:08:01.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.615+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9010066b70 con 0x7f90100fbbc0 2024-05-25T06:08:01.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.615+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f900c00bd30 con 0x7f90100fbbc0 2024-05-25T06:08:01.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.615+0000 7f9005ffb700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ffc0692a0 0x7f8ffc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:01.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.616+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f900c088100 con 0x7f90100fbbc0 2024-05-25T06:08:01.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.616+0000 7f9014a8b700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ffc0692a0 0x7f8ffc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:01.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.616+0000 7f9014a8b700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ffc0692a0 0x7f8ffc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9000006fd0 tx=0x7f9000008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:01.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.620+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f900c058230 con 0x7f90100fbbc0 2024-05-25T06:08:01.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.848+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f901004c710 con 0x7f90100fbbc0 2024-05-25T06:08:01.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.849+0000 7f9005ffb700 1 -- 172.21.15.160:0/753066449 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f900c057dc0 con 0x7f90100fbbc0 2024-05-25T06:08:01.850 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:01.850 INFO:teuthology.orchestra.run.smithi160.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-25T06:08:01.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ffc0692a0 msgr2=0x7f8ffc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:01.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ffc0692a0 0x7f8ffc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9000006fd0 tx=0x7f9000008040 comp rx=0 tx=0).stop 2024-05-25T06:08:01.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 msgr2=0x7f9010119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:01.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f9010119f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f900c00b3b0 tx=0x7f900c009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:01.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 shutdown_connections 2024-05-25T06:08:01.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8ffc0692a0 0x7f8ffc06b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:01.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90100fbbc0 0x7f9010119f90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:01.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 --2- 172.21.15.160:0/753066449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100fc560 0x7f901011a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:01.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 >> 172.21.15.160:0/753066449 conn(0x7f90100f7860 msgr2=0x7f90100f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:01.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 shutdown_connections 2024-05-25T06:08:01.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:01.851+0000 7f90174f0700 1 -- 172.21.15.160:0/753066449 wait complete. 2024-05-25T06:08:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:02 smithi203 ceph-mon[36030]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/753066449' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-25T06:08:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:02 smithi160 ceph-mon[29041]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:02 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/753066449' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-25T06:08:02.574 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-25T06:08:02.575 INFO:tasks.cephadm:Setup complete, yielding 2024-05-25T06:08:02.575 INFO:teuthology.run_tasks:Running task print... 2024-05-25T06:08:02.584 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-05-25T06:08:02.584 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:02.595 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:02.596 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-25T06:08:02.865 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:04.271 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:04 smithi160 ceph-mon[29041]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:04 smithi203 ceph-mon[36030]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:04.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.475+0000 7fc69e1fb700 1 -- 172.21.15.160:0/288458134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 msgr2=0x7fc6980ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.475+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/288458134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980ffa20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc688004940 tx=0x7fc688009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.475+0000 7fc69e1fb700 1 -- 172.21.15.160:0/288458134 shutdown_connections 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.475+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/288458134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6980fff60 0x7fc698102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.475+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/288458134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980ffa20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.475+0000 7fc69e1fb700 1 -- 172.21.15.160:0/288458134 >> 172.21.15.160:0/288458134 conn(0x7fc6980f7430 msgr2=0x7fc6980f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 -- 172.21.15.160:0/288458134 shutdown_connections 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 -- 172.21.15.160:0/288458134 wait complete. 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 Processor -- start 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 -- start start 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6980fff60 0x7fc6980fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980fbed0 con 0x7fc698066240 2024-05-25T06:08:04.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc69e1fb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980fc040 con 0x7fc6980fff60 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc6977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc6977fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43962/0 (socket says 172.21.15.160:43962) 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc6977fe700 1 -- 172.21.15.160:0/1031651509 learned_addr learned my addr 172.21.15.160:0/1031651509 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.476+0000 7fc68ffff700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6980fff60 0x7fc6980fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc6977fe700 1 -- 172.21.15.160:0/1031651509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6980fff60 msgr2=0x7fc6980fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc6977fe700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6980fff60 0x7fc6980fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc6977fe700 1 -- 172.21.15.160:0/1031651509 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc688005040 con 0x7fc698066240 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc6977fe700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980fd450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc68800b3b0 tx=0x7fc688009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc68800e040 con 0x7fc698066240 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6980fc2c0 con 0x7fc698066240 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc68800bbf0 con 0x7fc698066240 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc68801c630 con 0x7fc698066240 2024-05-25T06:08:04.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.477+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc698115a80 con 0x7fc698066240 2024-05-25T06:08:04.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.478+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69805f5c0 con 0x7fc698066240 2024-05-25T06:08:04.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.478+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc68800bd60 con 0x7fc698066240 2024-05-25T06:08:04.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.479+0000 7fc694ff9700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc6840692a0 0x7fc68406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:04.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.479+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc688087b00 con 0x7fc698066240 2024-05-25T06:08:04.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.479+0000 7fc68ffff700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc6840692a0 0x7fc68406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:04.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.479+0000 7fc68ffff700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc6840692a0 0x7fc68406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6800102e0 tx=0x7fc68000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:04.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.484+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc688054170 con 0x7fc698066240 2024-05-25T06:08:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.669+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fc69804bed0 con 0x7fc698066240 2024-05-25T06:08:04.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.673+0000 7fc694ff9700 1 -- 172.21.15.160:0/1031651509 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc69804bed0 con 0x7fc698066240 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.675+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc6840692a0 msgr2=0x7fc68406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.675+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc6840692a0 0x7fc68406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6800102e0 tx=0x7fc68000b040 comp rx=0 tx=0).stop 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.675+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 msgr2=0x7fc6980fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.675+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980fd450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc68800b3b0 tx=0x7fc688009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 shutdown_connections 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fc6840692a0 0x7fc68406b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc698066240 0x7fc6980fd450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 --2- 172.21.15.160:0/1031651509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6980fff60 0x7fc6980fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 >> 172.21.15.160:0/1031651509 conn(0x7fc6980f7430 msgr2=0x7fc6980f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:04.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 shutdown_connections 2024-05-25T06:08:04.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:04.676+0000 7fc69e1fb700 1 -- 172.21.15.160:0/1031651509 wait complete. 2024-05-25T06:08:05.375 INFO:teuthology.run_tasks:Running task print... 2024-05-25T06:08:05.384 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-25T06:08:05.384 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:05.395 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:05.396 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-25T06:08:05.673 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:05.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:05 smithi160 ceph-mon[29041]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:05 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1031651509' entity='client.admin' 2024-05-25T06:08:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:05 smithi203 ceph-mon[36030]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:05 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1031651509' entity='client.admin' 2024-05-25T06:08:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:06.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:06.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 -- 172.21.15.160:0/4254330985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 msgr2=0x7f95e40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 --2- 172.21.15.160:0/4254330985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e40fd3d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f95d0004940 tx=0x7f95d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 -- 172.21.15.160:0/4254330985 shutdown_connections 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 --2- 172.21.15.160:0/4254330985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e40fd3d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 --2- 172.21.15.160:0/4254330985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e40fbcb0 0x7f95e40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 -- 172.21.15.160:0/4254330985 >> 172.21.15.160:0/4254330985 conn(0x7f95e40f7430 msgr2=0x7f95e40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 -- 172.21.15.160:0/4254330985 shutdown_connections 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.259+0000 7f95eac7a700 1 -- 172.21.15.160:0/4254330985 wait complete. 2024-05-25T06:08:07.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95eac7a700 1 Processor -- start 2024-05-25T06:08:07.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95eac7a700 1 -- start start 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95eac7a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e40fbcb0 0x7f95e410c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95eac7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e410ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95eac7a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e410d470 con 0x7f95e40fcf60 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95eac7a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e410dcc0 con 0x7f95e40fbcb0 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e410ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e410ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54362/0 (socket says 172.21.15.160:54362) 2024-05-25T06:08:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95e3fff700 1 -- 172.21.15.160:0/3340524461 learned_addr learned my addr 172.21.15.160:0/3340524461 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.260+0000 7f95e8a16700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e40fbcb0 0x7f95e410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e3fff700 1 -- 172.21.15.160:0/3340524461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e40fbcb0 msgr2=0x7f95e410c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e3fff700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e40fbcb0 0x7f95e410c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e3fff700 1 -- 172.21.15.160:0/3340524461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0005040 con 0x7f95e40fcf60 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e3fff700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e410ce70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009150 tx=0x7f95d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d000e040 con 0x7f95e40fcf60 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e410df40 con 0x7f95e40fcf60 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95d0004290 con 0x7f95e40fcf60 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d001c630 con 0x7f95e40fcf60 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.261+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e410e480 con 0x7f95e40fcf60 2024-05-25T06:08:07.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.262+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e405f5c0 con 0x7f95e40fcf60 2024-05-25T06:08:07.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.262+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f95d0013420 con 0x7f95e40fcf60 2024-05-25T06:08:07.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.263+0000 7f95e17fa700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f95d40692a0 0x7f95d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:07.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.263+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f95d0087a40 con 0x7f95e40fcf60 2024-05-25T06:08:07.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.263+0000 7f95e8a16700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f95d40692a0 0x7f95d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:07.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.263+0000 7f95e8a16700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f95d40692a0 0x7f95d406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95d80041d0 tx=0x7f95d80043c0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:07.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.267+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f95d0057be0 con 0x7f95e40fcf60 2024-05-25T06:08:07.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.466+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e410e800 con 0x7f95d40692a0 2024-05-25T06:08:07.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.467+0000 7f95e17fa700 1 -- 172.21.15.160:0/3340524461 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f95e410e800 con 0x7f95d40692a0 2024-05-25T06:08:07.468 INFO:teuthology.orchestra.run.smithi160.stdout:Backend: cephadm 2024-05-25T06:08:07.468 INFO:teuthology.orchestra.run.smithi160.stdout:Available: Yes 2024-05-25T06:08:07.468 INFO:teuthology.orchestra.run.smithi160.stdout:Paused: No 2024-05-25T06:08:07.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f95d40692a0 msgr2=0x7f95d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:07.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f95d40692a0 0x7f95d406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95d80041d0 tx=0x7f95d80043c0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 msgr2=0x7f95e410ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:07.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e410ce70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009150 tx=0x7f95d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:07.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 shutdown_connections 2024-05-25T06:08:07.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f95d40692a0 0x7f95d406b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e40fbcb0 0x7f95e410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 --2- 172.21.15.160:0/3340524461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e40fcf60 0x7f95e410ce70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:07.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 >> 172.21.15.160:0/3340524461 conn(0x7f95e40f7430 msgr2=0x7f95e40f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:07.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 shutdown_connections 2024-05-25T06:08:07.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:07.470+0000 7f95eac7a700 1 -- 172.21.15.160:0/3340524461 wait complete. 2024-05-25T06:08:07.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:07 smithi160 ceph-mon[29041]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:08.048 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-25T06:08:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:07 smithi203 ceph-mon[36030]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:08.305 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:08.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:08 smithi160 ceph-mon[29041]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:08.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:08 smithi203 ceph-mon[36030]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:09.956 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:09 smithi160 ceph-mon[29041]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:09.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 -- 172.21.15.160:0/3931883248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fe410 msgr2=0x7f6b5c0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:09.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 --2- 172.21.15.160:0/3931883248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fe820 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c004940 tx=0x7f6b4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:09.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 -- 172.21.15.160:0/3931883248 shutdown_connections 2024-05-25T06:08:09.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 --2- 172.21.15.160:0/3931883248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:09.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 --2- 172.21.15.160:0/3931883248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fe820 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:09.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 -- 172.21.15.160:0/3931883248 >> 172.21.15.160:0/3931883248 conn(0x7f6b5c0f6bd0 msgr2=0x7f6b5c0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:09.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 -- 172.21.15.160:0/3931883248 shutdown_connections 2024-05-25T06:08:09.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.973+0000 7f6b640cd700 1 -- 172.21.15.160:0/3931883248 wait complete. 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b640cd700 1 Processor -- start 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b640cd700 1 -- start start 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b640cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b640cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b640cd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0fe020 con 0x7f6b5c0fed60 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b640cd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0fc610 con 0x7f6b5c0fe410 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61668700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61668700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54366/0 (socket says 172.21.15.160:54366) 2024-05-25T06:08:09.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61668700 1 -- 172.21.15.160:0/4042080729 learned_addr learned my addr 172.21.15.160:0/4042080729 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61e69700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61668700 1 -- 172.21.15.160:0/4042080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fe410 msgr2=0x7f6b5c0fda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61668700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61668700 1 -- 172.21.15.160:0/4042080729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c005040 con 0x7f6b5c0fed60 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.974+0000 7f6b61e69700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.975+0000 7f6b61668700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0fc0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b58002a80 tx=0x7f6b580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.975+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b58007cc0 con 0x7f6b5c0fed60 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.975+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b5800f040 con 0x7f6b5c0fed60 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.975+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c0fc8c0 con 0x7f6b5c0fed60 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.975+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b580096f0 con 0x7f6b5c0fed60 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.975+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c0fce80 con 0x7f6b5c0fed60 2024-05-25T06:08:09.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.976+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c04c710 con 0x7f6b5c0fed60 2024-05-25T06:08:09.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.976+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6b5800d690 con 0x7f6b5c0fed60 2024-05-25T06:08:09.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.977+0000 7f6b527fc700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6b480692a0 0x7f6b4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:09.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.977+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6b58086620 con 0x7f6b5c0fed60 2024-05-25T06:08:09.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.977+0000 7f6b61e69700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6b480692a0 0x7f6b4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:09.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.977+0000 7f6b61e69700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6b480692a0 0x7f6b4806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00b3b0 tx=0x7f6b4c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:09.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:09.981+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6b5804d250 con 0x7f6b5c0fed60 2024-05-25T06:08:10.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:09 smithi203 ceph-mon[36030]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:10.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.175+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b5c102460 con 0x7f6b480692a0 2024-05-25T06:08:10.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.184+0000 7f6b527fc700 1 -- 172.21.15.160:0/4042080729 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2700 (secure 0 0 0) 0x7f6b5c102460 con 0x7f6b480692a0 2024-05-25T06:08:10.184 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (3m) 2m ago 5m 25.9M - 0.25.0 c8568f914cd2 86108650b602 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (5m) 2m ago 5m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (4m) 69s ago 4m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (3m) 2m ago 4m 79.8M - 9.4.7 954c08fa6188 59fd8b61d321 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:9283 running (6m) 2m ago 6m 473M - 17.2.7-958-g7b01852c baf1547092e7 fbeba8913b6d 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (4m) 69s ago 4m 433M - 17.2.7-958-g7b01852c baf1547092e7 c8ec43fcaa21 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (6m) 2m ago 6m 41.4M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (4m) 69s ago 4m 38.4M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (5m) 2m ago 5m 15.4M - 1.5.0 0da6a335fe13 3a8c0c37653a 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (4m) 69s ago 4m 15.8M - 1.5.0 0da6a335fe13 aec5d8f8dcff 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (3m) 2m ago 3m 39.9M 4288M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (2m) 2m ago 2m 37.5M 4288M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:08:10.185 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (2m) 2m ago 2m 38.8M 4288M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:08:10.186 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (2m) 69s ago 2m 38.6M 5312M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:08:10.186 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (100s) 69s ago 99s 37.0M 5312M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:08:10.186 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (74s) 69s ago 73s 13.2M 5312M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:08:10.186 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (3m) 2m ago 4m 35.9M - 2.43.0 a07b618ecd1d 6cc9a4b6650c 2024-05-25T06:08:10.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6b480692a0 msgr2=0x7f6b4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:10.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6b480692a0 0x7f6b4806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00b3b0 tx=0x7f6b4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 msgr2=0x7f6b5c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0fc0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b58002a80 tx=0x7f6b580046b0 comp rx=0 tx=0).stop 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 shutdown_connections 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6b480692a0 0x7f6b4806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c0fe410 0x7f6b5c0fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 --2- 172.21.15.160:0/4042080729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b5c0fed60 0x7f6b5c0fc0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 >> 172.21.15.160:0/4042080729 conn(0x7f6b5c0f6bd0 msgr2=0x7f6b5c0f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 shutdown_connections 2024-05-25T06:08:10.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:10.186+0000 7f6b640cd700 1 -- 172.21.15.160:0/4042080729 wait complete. 2024-05-25T06:08:10.825 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-25T06:08:11.086 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:11.842 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:11 smithi160 ceph-mon[29041]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:11.842 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:11 smithi160 ceph-mon[29041]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:12.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:11 smithi203 ceph-mon[36030]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:12.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:11 smithi203 ceph-mon[36030]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 -- 172.21.15.160:0/12529550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 msgr2=0x7f08dc0faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 --2- 172.21.15.160:0/12529550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc0faee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08cc004940 tx=0x7f08cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 -- 172.21.15.160:0/12529550 shutdown_connections 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 --2- 172.21.15.160:0/12529550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08dc0fb420 0x7f08dc1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 --2- 172.21.15.160:0/12529550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc0faee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 -- 172.21.15.160:0/12529550 >> 172.21.15.160:0/12529550 conn(0x7f08dc0f6740 msgr2=0x7f08dc0f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 -- 172.21.15.160:0/12529550 shutdown_connections 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 -- 172.21.15.160:0/12529550 wait complete. 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.586+0000 7f08e258a700 1 Processor -- start 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08e258a700 1 -- start start 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08e258a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08e258a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08dc0fb420 0x7f08dc10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08e258a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc10e440 con 0x7f08dc0faad0 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08e258a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc10e5b0 con 0x7f08dc0fb420 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc10d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54374/0 (socket says 172.21.15.160:54374) 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08dbfff700 1 -- 172.21.15.160:0/233336040 learned_addr learned my addr 172.21.15.160:0/233336040 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08db7fe700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08dc0fb420 0x7f08dc10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08dbfff700 1 -- 172.21.15.160:0/233336040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08dc0fb420 msgr2=0x7f08dc10de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08dbfff700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08dc0fb420 0x7f08dc10de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.587+0000 7f08dbfff700 1 -- 172.21.15.160:0/233336040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc005040 con 0x7f08dc0faad0 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.588+0000 7f08dbfff700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc10d900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b3b0 tx=0x7f08cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.588+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08cc00e040 con 0x7f08dc0faad0 2024-05-25T06:08:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.588+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08cc00bbf0 con 0x7f08dc0faad0 2024-05-25T06:08:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.588+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc112fb0 con 0x7f08dc0faad0 2024-05-25T06:08:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.588+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08cc01c630 con 0x7f08dc0faad0 2024-05-25T06:08:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.588+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc1134e0 con 0x7f08dc0faad0 2024-05-25T06:08:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.589+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc04c710 con 0x7f08dc0faad0 2024-05-25T06:08:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.589+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f08cc00bd60 con 0x7f08dc0faad0 2024-05-25T06:08:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.590+0000 7f08d8ff9700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f08c40692a0 0x7f08c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.590+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f08cc087b00 con 0x7f08dc0faad0 2024-05-25T06:08:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.590+0000 7f08db7fe700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f08c40692a0 0x7f08c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.590+0000 7f08db7fe700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f08c40692a0 0x7f08c406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08d0004f40 tx=0x7f08d000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:12.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.594+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f08cc057c00 con 0x7f08dc0faad0 2024-05-25T06:08:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.789+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f08dc101f00 con 0x7f08c40692a0 2024-05-25T06:08:12.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.794+0000 7f08d8ff9700 1 -- 172.21.15.160:0/233336040 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7f08dc101f00 con 0x7f08c40692a0 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:crash 2/2 2m ago 5m * 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:mgr 2/2 2m ago 5m count:2 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:mon 2/2 2m ago 5m smithi160:172.21.15.160=smithi160;smithi203:172.21.15.203=smithi203;count:2 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:osd 6 2m ago - 2024-05-25T06:08:12.794 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-05-25T06:08:12.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f08c40692a0 msgr2=0x7f08c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:12.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f08c40692a0 0x7f08c406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08d0004f40 tx=0x7f08d000b040 comp rx=0 tx=0).stop 2024-05-25T06:08:12.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 msgr2=0x7f08dc10d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:12.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc10d900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b3b0 tx=0x7f08cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 shutdown_connections 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f08c40692a0 0x7f08c406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08dc0faad0 0x7f08dc10d900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 --2- 172.21.15.160:0/233336040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08dc0fb420 0x7f08dc10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 >> 172.21.15.160:0/233336040 conn(0x7f08dc0f6740 msgr2=0x7f08dc1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 shutdown_connections 2024-05-25T06:08:12.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:12.796+0000 7f08e258a700 1 -- 172.21.15.160:0/233336040 wait complete. 2024-05-25T06:08:13.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-25T06:08:13.691 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:13.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:13 smithi160 ceph-mon[29041]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:13.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:13 smithi160 ceph-mon[29041]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:13 smithi203 ceph-mon[36030]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:13 smithi203 ceph-mon[36030]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:15.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.099+0000 7f47e92df700 1 -- 172.21.15.160:0/870917270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e4066f30 msgr2=0x7f47e40ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:15.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.099+0000 7f47e92df700 1 --2- 172.21.15.160:0/870917270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e4066f30 0x7f47e40ff9e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f47d4004940 tx=0x7f47d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- 172.21.15.160:0/870917270 shutdown_connections 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 --2- 172.21.15.160:0/870917270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e40fff20 0x7f47e4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 --2- 172.21.15.160:0/870917270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e4066f30 0x7f47e40ff9e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- 172.21.15.160:0/870917270 >> 172.21.15.160:0/870917270 conn(0x7f47e40f7450 msgr2=0x7f47e40f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- 172.21.15.160:0/870917270 shutdown_connections 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- 172.21.15.160:0/870917270 wait complete. 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 Processor -- start 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- start start 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e4066f30 0x7f47e410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 0x7f47e410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:15.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e410e4c0 con 0x7f47e40fff20 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.100+0000 7f47e92df700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e410e630 con 0x7f47e4066f30 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 0x7f47e410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 0x7f47e410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54386/0 (socket says 172.21.15.160:54386) 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 -- 172.21.15.160:0/3394526461 learned_addr learned my addr 172.21.15.160:0/3394526461 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e2ffd700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e4066f30 0x7f47e410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 -- 172.21.15.160:0/3394526461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e4066f30 msgr2=0x7f47e410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e4066f30 0x7f47e410d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 -- 172.21.15.160:0/3394526461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4005040 con 0x7f47e40fff20 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e27fc700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 0x7f47e410dec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c8c0 tx=0x7f47d80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d800a980 con 0x7f47e40fff20 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47d800ce60 con 0x7f47e40fff20 2024-05-25T06:08:15.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e4113040 con 0x7f47e40fff20 2024-05-25T06:08:15.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d80115e0 con 0x7f47e40fff20 2024-05-25T06:08:15.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.101+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e41135d0 con 0x7f47e40fff20 2024-05-25T06:08:15.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.102+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e404c710 con 0x7f47e40fff20 2024-05-25T06:08:15.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.103+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f47d8011740 con 0x7f47e40fff20 2024-05-25T06:08:15.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.103+0000 7f47cb7fe700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f47cc0692a0 0x7f47cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:15.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.103+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f47d8086f80 con 0x7f47e40fff20 2024-05-25T06:08:15.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.103+0000 7f47e2ffd700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f47cc0692a0 0x7f47cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:15.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.104+0000 7f47e2ffd700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f47cc0692a0 0x7f47cc06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b970 tx=0x7f47d4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:15.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.108+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f47d80570f0 con 0x7f47e40fff20 2024-05-25T06:08:15.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.299+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f47e40f8f90 con 0x7f47cc0692a0 2024-05-25T06:08:15.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.300+0000 7f47cb7fe700 1 -- 172.21.15.160:0/3394526461 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f47e40f8f90 con 0x7f47cc0692a0 2024-05-25T06:08:15.302 INFO:teuthology.orchestra.run.smithi160.stdout:HOST ADDR LABELS STATUS 2024-05-25T06:08:15.302 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160 172.21.15.160 2024-05-25T06:08:15.302 INFO:teuthology.orchestra.run.smithi160.stdout:smithi203 172.21.15.203 2024-05-25T06:08:15.302 INFO:teuthology.orchestra.run.smithi160.stdout:2 hosts in cluster 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f47cc0692a0 msgr2=0x7f47cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f47cc0692a0 0x7f47cc06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b970 tx=0x7f47d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 msgr2=0x7f47e410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 0x7f47e410dec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c8c0 tx=0x7f47d80044c0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 shutdown_connections 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f47cc0692a0 0x7f47cc06b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e4066f30 0x7f47e410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 --2- 172.21.15.160:0/3394526461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e40fff20 0x7f47e410dec0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 >> 172.21.15.160:0/3394526461 conn(0x7f47e40f7450 msgr2=0x7f47e40f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.302+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 shutdown_connections 2024-05-25T06:08:15.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:15.303+0000 7f47e92df700 1 -- 172.21.15.160:0/3394526461 wait complete. 2024-05-25T06:08:15.869 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:15 smithi160 ceph-mon[29041]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:15.895 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-25T06:08:15.951 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:15 smithi203 ceph-mon[36030]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:16.147 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:16.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:16 smithi160 ceph-mon[29041]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:17.145 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:16 smithi203 ceph-mon[36030]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:17.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 -- 172.21.15.160:0/3061590036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fbcb0 msgr2=0x7f53e00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 --2- 172.21.15.160:0/3061590036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fbcb0 0x7f53e00fc0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53d0004940 tx=0x7f53d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 -- 172.21.15.160:0/3061590036 shutdown_connections 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 --2- 172.21.15.160:0/3061590036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53e00fcf60 0x7f53e00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 --2- 172.21.15.160:0/3061590036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fbcb0 0x7f53e00fc0c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 -- 172.21.15.160:0/3061590036 >> 172.21.15.160:0/3061590036 conn(0x7f53e00f7430 msgr2=0x7f53e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 -- 172.21.15.160:0/3061590036 shutdown_connections 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 -- 172.21.15.160:0/3061590036 wait complete. 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.629+0000 7f53e64e9700 1 Processor -- start 2024-05-25T06:08:17.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53e64e9700 1 -- start start 2024-05-25T06:08:17.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53e64e9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53e00fbcb0 0x7f53e010da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:17.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53e64e9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 0x7f53e010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53e64e9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e010e5c0 con 0x7f53e00fcf60 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53e64e9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e010edc0 con 0x7f53e00fbcb0 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 0x7f53e010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 0x7f53e010dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47140/0 (socket says 172.21.15.160:47140) 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 -- 172.21.15.160:0/674454637 learned_addr learned my addr 172.21.15.160:0/674454637 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53dffff700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53e00fbcb0 0x7f53e010da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 -- 172.21.15.160:0/674454637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53e00fbcb0 msgr2=0x7f53e010da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53e00fbcb0 0x7f53e010da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 -- 172.21.15.160:0/674454637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0005040 con 0x7f53e00fcf60 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.630+0000 7f53df7fe700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 0x7f53e010dfc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c830 tx=0x7f53d4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.631+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d40106f0 con 0x7f53e00fcf60 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.631+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e010f0a0 con 0x7f53e00fcf60 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.631+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53d4010d20 con 0x7f53e00fcf60 2024-05-25T06:08:17.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.631+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d40113a0 con 0x7f53e00fcf60 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.631+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e010f5e0 con 0x7f53e00fcf60 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.632+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0101af0 con 0x7f53e00fcf60 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.632+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f53d4012400 con 0x7f53e00fcf60 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.632+0000 7f53dcff9700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f53c80692b0 0x7f53c806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.633+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f53d4086e20 con 0x7f53e00fcf60 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.633+0000 7f53dffff700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f53c80692b0 0x7f53c806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:17.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.633+0000 7f53dffff700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f53c80692b0 0x7f53c806b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b3b0 tx=0x7f53d0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:17.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.637+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f53d4056f20 con 0x7f53e00fcf60 2024-05-25T06:08:17.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.831+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f53e010f860 con 0x7f53c80692b0 2024-05-25T06:08:17.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.832+0000 7f53dcff9700 1 -- 172.21.15.160:0/674454637 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f53e010f860 con 0x7f53c80692b0 2024-05-25T06:08:17.832 INFO:teuthology.orchestra.run.smithi160.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-25T06:08:17.832 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-25T06:08:17.832 INFO:teuthology.orchestra.run.smithi160.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 73s ago Has a FileSystem, LVM detected 2024-05-25T06:08:17.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f53c80692b0 msgr2=0x7f53c806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f53c80692b0 0x7f53c806b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b3b0 tx=0x7f53d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 msgr2=0x7f53e010dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 0x7f53e010dfc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c830 tx=0x7f53d4004470 comp rx=0 tx=0).stop 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 shutdown_connections 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f53c80692b0 0x7f53c806b760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53e00fbcb0 0x7f53e010da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 --2- 172.21.15.160:0/674454637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e00fcf60 0x7f53e010dfc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.834+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 >> 172.21.15.160:0/674454637 conn(0x7f53e00f7430 msgr2=0x7f53e0065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.835+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 shutdown_connections 2024-05-25T06:08:17.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:17.835+0000 7f53e64e9700 1 -- 172.21.15.160:0/674454637 wait complete. 2024-05-25T06:08:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:17 smithi160 ceph-mon[29041]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:17 smithi203 ceph-mon[36030]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:18.598 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:18.607 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:18.607 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-25T06:08:18.865 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:18.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:18 smithi160 ceph-mon[29041]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:18 smithi203 ceph-mon[36030]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:19.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:19 smithi160 ceph-mon[29041]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:19 smithi203 ceph-mon[36030]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 -- 172.21.15.160:0/3991244312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 msgr2=0x7f02140fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 --2- 172.21.15.160:0/3991244312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f02140fc0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f020c009ce0 tx=0x7f020c004d10 comp rx=0 tx=0).stop 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 -- 172.21.15.160:0/3991244312 shutdown_connections 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 --2- 172.21.15.160:0/3991244312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02140fcf70 0x7f02140fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 --2- 172.21.15.160:0/3991244312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f02140fc0d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 -- 172.21.15.160:0/3991244312 >> 172.21.15.160:0/3991244312 conn(0x7f02140f7420 msgr2=0x7f02140f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:20.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 -- 172.21.15.160:0/3991244312 shutdown_connections 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.430+0000 7f0218a24700 1 -- 172.21.15.160:0/3991244312 wait complete. 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0218a24700 1 Processor -- start 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0218a24700 1 -- start start 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0218a24700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f021410dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0218a24700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02140fcf70 0x7f021410e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0218a24700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021410e6f0 con 0x7f02140fbcc0 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0218a24700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021410ef40 con 0x7f02140fcf70 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f021259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f021410dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f021259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f021410dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47156/0 (socket says 172.21.15.160:47156) 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f021259c700 1 -- 172.21.15.160:0/2039605757 learned_addr learned my addr 172.21.15.160:0/2039605757 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f0211d9b700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02140fcf70 0x7f021410e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f021259c700 1 -- 172.21.15.160:0/2039605757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02140fcf70 msgr2=0x7f021410e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:20.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f021259c700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02140fcf70 0x7f021410e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.431+0000 7f021259c700 1 -- 172.21.15.160:0/2039605757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c009940 con 0x7f02140fbcc0 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.432+0000 7f021259c700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f021410dbb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f020c004740 tx=0x7f020c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.432+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f020c00e040 con 0x7f02140fbcc0 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.432+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f020c0128e0 con 0x7f02140fbcc0 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.432+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021410f1c0 con 0x7f02140fbcc0 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.432+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f020c01c9a0 con 0x7f02140fbcc0 2024-05-25T06:08:20.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.432+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021410f700 con 0x7f02140fbcc0 2024-05-25T06:08:20.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.433+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0214101b00 con 0x7f02140fbcc0 2024-05-25T06:08:20.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.433+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f020c012a50 con 0x7f02140fbcc0 2024-05-25T06:08:20.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.434+0000 7f020affd700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f02000692a0 0x7f020006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:20.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.434+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f020c088a50 con 0x7f02140fbcc0 2024-05-25T06:08:20.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.434+0000 7f0211d9b700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f02000692a0 0x7f020006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:20.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.434+0000 7f0211d9b700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f02000692a0 0x7f020006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007fd0 tx=0x7f01fc009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:20.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.438+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f020c058b50 con 0x7f02140fbcc0 2024-05-25T06:08:20.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:20.659+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f02141048c0 con 0x7f02000692a0 2024-05-25T06:08:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:21 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-25T06:08:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:21 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-25T06:08:21.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.979+0000 7f020affd700 1 -- 172.21.15.160:0/2039605757 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f02141048c0 con 0x7f02000692a0 2024-05-25T06:08:21.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f02000692a0 msgr2=0x7f020006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f02000692a0 0x7f020006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007fd0 tx=0x7f01fc009040 comp rx=0 tx=0).stop 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 msgr2=0x7f021410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f021410dbb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f020c004740 tx=0x7f020c004d10 comp rx=0 tx=0).stop 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 shutdown_connections 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f02000692a0 0x7f020006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02140fbcc0 0x7f021410dbb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 --2- 172.21.15.160:0/2039605757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02140fcf70 0x7f021410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.982+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 >> 172.21.15.160:0/2039605757 conn(0x7f02140f7420 msgr2=0x7f0214066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.983+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 shutdown_connections 2024-05-25T06:08:21.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:21.983+0000 7f0218a24700 1 -- 172.21.15.160:0/2039605757 wait complete. 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:21 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[29037]: 2024-05-25T06:08:21.959+0000 7f0ca147c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: osdmap e41: 6 total, 6 up, 6 in 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-25T06:08:22.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: osdmap e42: 6 total, 6 up, 6 in 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: osdmap e43: 6 total, 6 up, 6 in 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: fsmap cephfs:0 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: Saving service mds.cephfs spec with placement count:4 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:22.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:22.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:22.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:22.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:22 smithi160 ceph-mon[29041]: Deploying daemon mds.cephfs.smithi160.rrhzig on smithi160 2024-05-25T06:08:22.537 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-25T06:08:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: osdmap e41: 6 total, 6 up, 6 in 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: osdmap e42: 6 total, 6 up, 6 in 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: osdmap e43: 6 total, 6 up, 6 in 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: fsmap cephfs:0 2024-05-25T06:08:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: Saving service mds.cephfs spec with placement count:4 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:22.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:22 smithi203 ceph-mon[36030]: Deploying daemon mds.cephfs.smithi160.rrhzig on smithi160 2024-05-25T06:08:23.832 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:23 smithi160 ceph-mon[29041]: pgmap v173: 65 pgs: 1 creating+activating, 9 creating+peering, 49 unknown, 6 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:23 smithi160 ceph-mon[29041]: osdmap e44: 6 total, 6 up, 6 in 2024-05-25T06:08:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:23 smithi203 ceph-mon[36030]: pgmap v173: 65 pgs: 1 creating+activating, 9 creating+peering, 49 unknown, 6 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:23 smithi203 ceph-mon[36030]: osdmap e44: 6 total, 6 up, 6 in 2024-05-25T06:08:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:25.140 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:24 smithi160 ceph-mon[29041]: osdmap e45: 6 total, 6 up, 6 in 2024-05-25T06:08:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:24 smithi203 ceph-mon[36030]: osdmap e45: 6 total, 6 up, 6 in 2024-05-25T06:08:26.137 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:25 smithi160 ceph-mon[29041]: pgmap v176: 65 pgs: 5 creating+activating, 9 creating+peering, 27 unknown, 24 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:26.137 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:26.137 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:25 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:26.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 -- 172.21.15.160:0/3929296679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 msgr2=0x7fab4c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:26.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 --2- 172.21.15.160:0/3929296679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c0fd3d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab34004940 tx=0x7fab34009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:26.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 -- 172.21.15.160:0/3929296679 shutdown_connections 2024-05-25T06:08:26.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 --2- 172.21.15.160:0/3929296679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c0fd3d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:26.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 --2- 172.21.15.160:0/3929296679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 0x7fab4c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 -- 172.21.15.160:0/3929296679 >> 172.21.15.160:0/3929296679 conn(0x7fab4c0f7430 msgr2=0x7fab4c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 -- 172.21.15.160:0/3929296679 shutdown_connections 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.136+0000 7fab51c8b700 1 -- 172.21.15.160:0/3929296679 wait complete. 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab51c8b700 1 Processor -- start 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab51c8b700 1 -- start start 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab51c8b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 0x7fab4c10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab51c8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab51c8b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c10e5c0 con 0x7fab4c0fcf60 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab51c8b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c10ed70 con 0x7fab4c0fbcb0 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab43fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab43fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37196/0 (socket says 172.21.15.160:37196) 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab43fff700 1 -- 172.21.15.160:0/98749663 learned_addr learned my addr 172.21.15.160:0/98749663 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:26.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab4b7fe700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 0x7fab4c10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab43fff700 1 -- 172.21.15.160:0/98749663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 msgr2=0x7fab4c10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab43fff700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 0x7fab4c10da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab43fff700 1 -- 172.21.15.160:0/98749663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab34005040 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.137+0000 7fab4b7fe700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 0x7fab4c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.138+0000 7fab43fff700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c10dfc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fab34013870 tx=0x7fab34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.138+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab34021070 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.138+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab34009430 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.138+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab3400e070 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.138+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c10eff0 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.138+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c10f530 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.139+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c101af0 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.139+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fab3401c750 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.140+0000 7fab48ff9700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fab38069230 0x7fab3806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.140+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fab34088460 con 0x7fab4c0fcf60 2024-05-25T06:08:26.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.140+0000 7fab4b7fe700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fab38069230 0x7fab3806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:26.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.140+0000 7fab4b7fe700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fab38069230 0x7fab3806b6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009b80 tx=0x7fab3c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:26.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.144+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fab34058220 con 0x7fab4c0fcf60 2024-05-25T06:08:26.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.377+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab4c05f5c0 con 0x7fab4c0fcf60 2024-05-25T06:08:26.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.377+0000 7fab48ff9700 1 -- 172.21.15.160:0/98749663 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1249 (secure 0 0 0) 0x7fab34057db0 con 0x7fab4c0fcf60 2024-05-25T06:08:26.377 INFO:teuthology.orchestra.run.smithi160.stdout:e4 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 4 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:08:25.978586+0000 2024-05-25T06:08:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 0 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 1 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:in 0 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14464} 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:08:26.379 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 0 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14464} state up:creating seq 1 addr [v2:172.21.15.160:6826/2072396647,v1:172.21.15.160:6827/2072396647] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:26.380 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fab38069230 msgr2=0x7fab3806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fab38069230 0x7fab3806b6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009b80 tx=0x7fab3c00b040 comp rx=0 tx=0).stop 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 msgr2=0x7fab4c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c10dfc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fab34013870 tx=0x7fab34009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 shutdown_connections 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fab38069230 0x7fab3806b6e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4c0fbcb0 0x7fab4c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 --2- 172.21.15.160:0/98749663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c0fcf60 0x7fab4c10dfc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:26.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 >> 172.21.15.160:0/98749663 conn(0x7fab4c0f7430 msgr2=0x7fab4c065b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:26.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 shutdown_connections 2024-05-25T06:08:26.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:26.379+0000 7fab51c8b700 1 -- 172.21.15.160:0/98749663 wait complete. 2024-05-25T06:08:26.382 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 4 2024-05-25T06:08:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:25 smithi203 ceph-mon[36030]: pgmap v176: 65 pgs: 5 creating+activating, 9 creating+peering, 27 unknown, 24 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-25T06:08:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:25 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:26.963 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:26.972 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:26.972 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6826/2072396647,v1:172.21.15.160:6827/2072396647] up:boot 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: daemon mds.cephfs.smithi160.rrhzig assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: Cluster is now healthy 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: fsmap cephfs:0 1 up:standby 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:creating} 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: daemon mds.cephfs.smithi160.rrhzig is now active in filesystem cephfs as rank 0 2024-05-25T06:08:27.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:27.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:27.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:27.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:27.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: Deploying daemon mds.cephfs.smithi203.yltymd on smithi203 2024-05-25T06:08:27.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:26 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/98749663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:08:27.232 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6826/2072396647,v1:172.21.15.160:6827/2072396647] up:boot 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: daemon mds.cephfs.smithi160.rrhzig assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: Cluster is now healthy 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: fsmap cephfs:0 1 up:standby 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:creating} 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: daemon mds.cephfs.smithi160.rrhzig is now active in filesystem cephfs as rank 0 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: Deploying daemon mds.cephfs.smithi203.yltymd on smithi203 2024-05-25T06:08:27.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:26 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/98749663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:08:28.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:27 smithi203 ceph-mon[36030]: pgmap v177: 65 pgs: 5 creating+activating, 60 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 655 B/s wr, 2 op/s 2024-05-25T06:08:28.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:27 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6826/2072396647,v1:172.21.15.160:6827/2072396647] up:active 2024-05-25T06:08:28.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:27 smithi203 ceph-mon[36030]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 2024-05-25T06:08:28.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:27 smithi160 ceph-mon[29041]: pgmap v177: 65 pgs: 5 creating+activating, 60 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 655 B/s wr, 2 op/s 2024-05-25T06:08:28.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:27 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6826/2072396647,v1:172.21.15.160:6827/2072396647] up:active 2024-05-25T06:08:28.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:27 smithi160 ceph-mon[29041]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 2024-05-25T06:08:28.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 -- 172.21.15.160:0/815644991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fcf70 msgr2=0x7fbb900fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:28.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 --2- 172.21.15.160:0/815644991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fcf70 0x7fbb900fd3e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb78004940 tx=0x7fbb78009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:28.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 -- 172.21.15.160:0/815644991 shutdown_connections 2024-05-25T06:08:28.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 --2- 172.21.15.160:0/815644991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fcf70 0x7fbb900fd3e0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb78004940 tx=0x7fbb78009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:28.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 --2- 172.21.15.160:0/815644991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb900fbcc0 0x7fbb900fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:28.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 -- 172.21.15.160:0/815644991 >> 172.21.15.160:0/815644991 conn(0x7fbb900f7420 msgr2=0x7fbb900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:28.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 -- 172.21.15.160:0/815644991 shutdown_connections 2024-05-25T06:08:28.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.954+0000 7fbb951d1700 1 -- 172.21.15.160:0/815644991 wait complete. 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb951d1700 1 Processor -- start 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb951d1700 1 -- start start 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb951d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 0x7fbb9010dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb951d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9010e0f0 0x7fbb9010ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb951d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9010e5b0 con 0x7fbb900fbcc0 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb951d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9010e6f0 con 0x7fbb9010e0f0 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb8ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 0x7fbb9010dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb8ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 0x7fbb9010dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37216/0 (socket says 172.21.15.160:37216) 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.955+0000 7fbb8ed9d700 1 -- 172.21.15.160:0/2936770237 learned_addr learned my addr 172.21.15.160:0/2936770237 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb8e59c700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9010e0f0 0x7fbb9010ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:28.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb8ed9d700 1 -- 172.21.15.160:0/2936770237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9010e0f0 msgr2=0x7fbb9010ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb8ed9d700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9010e0f0 0x7fbb9010ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb8ed9d700 1 -- 172.21.15.160:0/2936770237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78005040 con 0x7fbb900fbcc0 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb8ed9d700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 0x7fbb9010dbb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbb88002a50 tx=0x7fbb880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb88007cc0 con 0x7fbb900fbcc0 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9010f5a0 con 0x7fbb900fbcc0 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb8800f040 con 0x7fbb900fbcc0 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8800db70 con 0x7fbb900fbcc0 2024-05-25T06:08:28.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.956+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9010fb30 con 0x7fbb900fbcc0 2024-05-25T06:08:28.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.957+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb901060e0 con 0x7fbb900fbcc0 2024-05-25T06:08:28.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.958+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbb8800dd10 con 0x7fbb900fbcc0 2024-05-25T06:08:28.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.958+0000 7fbb877fe700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb7c0692a0 0x7fbb7c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:28.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.958+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbb88086910 con 0x7fbb900fbcc0 2024-05-25T06:08:28.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.958+0000 7fbb8e59c700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb7c0692a0 0x7fbb7c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:28.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.959+0000 7fbb8e59c700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb7c0692a0 0x7fbb7c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009040 tx=0x7fbb78009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:28.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:28.963+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbb88056650 con 0x7fbb900fbcc0 2024-05-25T06:08:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.203+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fbb90105a20 con 0x7fbb900fbcc0 2024-05-25T06:08:29.897 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:29.898 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:29.898 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-25T06:08:29.898 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:29.898 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:29.898 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: Deploying daemon mds.cephfs.smithi160.tlctof on smithi160 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] up:boot 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 1 up:standby 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 1 up:standby 2024-05-25T06:08:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:29 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2936770237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-25T06:08:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:29.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:29.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-25T06:08:29.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:29.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:29.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:29.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:29.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: Deploying daemon mds.cephfs.smithi160.tlctof on smithi160 2024-05-25T06:08:29.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] up:boot 2024-05-25T06:08:29.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 1 up:standby 2024-05-25T06:08:29.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:08:29.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 1 up:standby 2024-05-25T06:08:29.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:29 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2936770237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-25T06:08:29.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.994+0000 7fbb877fe700 1 -- 172.21.15.160:0/2936770237 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbb880561e0 con 0x7fbb900fbcc0 2024-05-25T06:08:29.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb7c0692a0 msgr2=0x7fbb7c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:29.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb7c0692a0 0x7fbb7c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009040 tx=0x7fbb78009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:29.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 msgr2=0x7fbb9010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:29.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 0x7fbb9010dbb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbb88002a50 tx=0x7fbb880046b0 comp rx=0 tx=0).stop 2024-05-25T06:08:29.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 shutdown_connections 2024-05-25T06:08:29.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fbb7c0692a0 0x7fbb7c06b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:29.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb900fbcc0 0x7fbb9010dbb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:29.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 --2- 172.21.15.160:0/2936770237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9010e0f0 0x7fbb9010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:29.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 >> 172.21.15.160:0/2936770237 conn(0x7fbb900f7420 msgr2=0x7fbb900f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:29.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 shutdown_connections 2024-05-25T06:08:29.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:29.997+0000 7fbb951d1700 1 -- 172.21.15.160:0/2936770237 wait complete. 2024-05-25T06:08:30.701 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-25T06:08:30.701 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2936770237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-25T06:08:30.701 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: daemon mds.cephfs.smithi203.yltymd assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-25T06:08:30.701 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-25T06:08:30.701 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-25T06:08:30.702 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 1 up:standby 2024-05-25T06:08:30.702 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:creating} 2024-05-25T06:08:30.702 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:30 smithi160 ceph-mon[29041]: daemon mds.cephfs.smithi203.yltymd is now active in filesystem cephfs as rank 1 2024-05-25T06:08:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-25T06:08:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2936770237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-25T06:08:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: daemon mds.cephfs.smithi203.yltymd assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-25T06:08:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-25T06:08:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-25T06:08:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: fsmap cephfs:1 {0=cephfs.smithi160.rrhzig=up:active} 1 up:standby 2024-05-25T06:08:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:creating} 2024-05-25T06:08:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:30 smithi203 ceph-mon[36030]: daemon mds.cephfs.smithi203.yltymd is now active in filesystem cephfs as rank 1 2024-05-25T06:08:30.979 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:30.995 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:30.996 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-25T06:08:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-25T06:08:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] up:active 2024-05-25T06:08:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 2024-05-25T06:08:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:32.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:32.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:32.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:32.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:32 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:32.358 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] up:active 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-25T06:08:32.359 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:32 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:33.151 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:33 smithi160 ceph-mon[29041]: Deploying daemon mds.cephfs.smithi203.tndnrb on smithi203 2024-05-25T06:08:33.151 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:33 smithi160 ceph-mon[29041]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-25T06:08:33.151 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:33 smithi160 ceph-mon[29041]: Cluster is now healthy 2024-05-25T06:08:33.151 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:33 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:boot 2024-05-25T06:08:33.151 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:33 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby 2024-05-25T06:08:33.152 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:33 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:08:33.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 -- 172.21.15.160:0/4203771226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 msgr2=0x7f59880fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 --2- 172.21.15.160:0/4203771226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f59880fc0d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f597c004940 tx=0x7f597c009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 -- 172.21.15.160:0/4203771226 shutdown_connections 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 --2- 172.21.15.160:0/4203771226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 0x7f59880fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 --2- 172.21.15.160:0/4203771226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f59880fc0d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 -- 172.21.15.160:0/4203771226 >> 172.21.15.160:0/4203771226 conn(0x7f59880f7440 msgr2=0x7f59880f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 -- 172.21.15.160:0/4203771226 shutdown_connections 2024-05-25T06:08:33.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.163+0000 7f598cbac700 1 -- 172.21.15.160:0/4203771226 wait complete. 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598cbac700 1 Processor -- start 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598cbac700 1 -- start start 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598cbac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f5988066c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598cbac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 0x7f59881126e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598cbac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988112c20 con 0x7f59880fbcc0 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598cbac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988112d60 con 0x7f59880fcf70 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f5988066c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f5988066c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37250/0 (socket says 172.21.15.160:37250) 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.164+0000 7f598659c700 1 -- 172.21.15.160:0/3507954385 learned_addr learned my addr 172.21.15.160:0/3507954385 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f5985d9b700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 0x7f59881126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:33.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f598659c700 1 -- 172.21.15.160:0/3507954385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 msgr2=0x7f59881126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f598659c700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 0x7f59881126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f598659c700 1 -- 172.21.15.160:0/3507954385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c005040 con 0x7f59880fbcc0 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f5985d9b700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 0x7f59881126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f598659c700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f5988066c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f597c013870 tx=0x7f597c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597c021070 con 0x7f59880fbcc0 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f597c01c400 con 0x7f59880fbcc0 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5988112f60 con 0x7f59880fbcc0 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597c00e070 con 0x7f59880fbcc0 2024-05-25T06:08:33.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.165+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5988113410 con 0x7f59880fbcc0 2024-05-25T06:08:33.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.166+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598804c710 con 0x7f59880fbcc0 2024-05-25T06:08:33.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.167+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f597c01ca20 con 0x7f59880fbcc0 2024-05-25T06:08:33.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.167+0000 7f5976ffd700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f59700692a0 0x7f597006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:33.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.167+0000 7f5985d9b700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f59700692a0 0x7f597006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:33.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.167+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f597c0885b0 con 0x7f59880fbcc0 2024-05-25T06:08:33.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.168+0000 7f5985d9b700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f59700692a0 0x7f597006b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5978004600 tx=0x7f597800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:33.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.172+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f597c058370 con 0x7f59880fbcc0 2024-05-25T06:08:33.284 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:33 smithi203 ceph-mon[36030]: Deploying daemon mds.cephfs.smithi203.tndnrb on smithi203 2024-05-25T06:08:33.284 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:33 smithi203 ceph-mon[36030]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-25T06:08:33.284 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:33 smithi203 ceph-mon[36030]: Cluster is now healthy 2024-05-25T06:08:33.285 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:33 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:boot 2024-05-25T06:08:33.285 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:33 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby 2024-05-25T06:08:33.285 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:33 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:08:33.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:33.410+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f59881047b0 con 0x7f59880fbcc0 2024-05-25T06:08:34.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.019+0000 7f5976ffd700 1 -- 172.21.15.160:0/3507954385 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v12) v1 ==== 121+0+0 (secure 0 0 0) 0x7f597c004490 con 0x7f59880fbcc0 2024-05-25T06:08:34.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f59700692a0 msgr2=0x7f597006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f59700692a0 0x7f597006b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5978004600 tx=0x7f597800b040 comp rx=0 tx=0).stop 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 msgr2=0x7f5988066c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f5988066c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f597c013870 tx=0x7f597c009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 shutdown_connections 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f59700692a0 0x7f597006b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59880fbcc0 0x7f5988066c80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 --2- 172.21.15.160:0/3507954385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59880fcf70 0x7f59881126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.021+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 >> 172.21.15.160:0/3507954385 conn(0x7f59880f7440 msgr2=0x7f59880f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.022+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 shutdown_connections 2024-05-25T06:08:34.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:34.022+0000 7f598cbac700 1 -- 172.21.15.160:0/3507954385 wait complete. 2024-05-25T06:08:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:34 smithi203 ceph-mon[36030]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 7 op/s 2024-05-25T06:08:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:34 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3507954385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-25T06:08:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:34 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:34.410 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:34 smithi160 ceph-mon[29041]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 7 op/s 2024-05-25T06:08:34.410 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:34 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3507954385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-25T06:08:34.410 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:34 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:34.714 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:34.723 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:34.724 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-05-25T06:08:35.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3507954385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-25T06:08:35.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby 2024-05-25T06:08:35.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 2024-05-25T06:08:35.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:35.462 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3507954385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:35.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:36 smithi160 ceph-mon[29041]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-25T06:08:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:36 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] up:boot 2024-05-25T06:08:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:36 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:36 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:08:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:36 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:08:36.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:36 smithi203 ceph-mon[36030]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-25T06:08:36.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:36 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] up:boot 2024-05-25T06:08:36.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:36 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:36.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:36 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:08:36.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:36 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 -- 172.21.15.160:0/2907651256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 msgr2=0x7f73f40fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 --2- 172.21.15.160:0/2907651256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f40fe840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73e8004940 tx=0x7f73e8009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 -- 172.21.15.160:0/2907651256 shutdown_connections 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 --2- 172.21.15.160:0/2907651256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40fed80 0x7f73f40faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 --2- 172.21.15.160:0/2907651256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f40fe840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 -- 172.21.15.160:0/2907651256 >> 172.21.15.160:0/2907651256 conn(0x7f73f40f6bd0 msgr2=0x7f73f40f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 -- 172.21.15.160:0/2907651256 shutdown_connections 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 -- 172.21.15.160:0/2907651256 wait complete. 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 Processor -- start 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.078+0000 7f73fb045700 1 -- start start 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73fb045700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73fb045700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40fed80 0x7f73f410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73fb045700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f410e530 con 0x7f73f40fe430 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73fb045700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f4112ed0 con 0x7f73f40fed80 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54394/0 (socket says 172.21.15.160:54394) 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 -- 172.21.15.160:0/1339894895 learned_addr learned my addr 172.21.15.160:0/1339894895 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f3fff700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40fed80 0x7f73f410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 -- 172.21.15.160:0/1339894895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40fed80 msgr2=0x7f73f410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40fed80 0x7f73f410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 -- 172.21.15.160:0/1339894895 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8005040 con 0x7f73f40fe430 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f8de1700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f410d960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b3b0 tx=0x7f73e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73e800e040 con 0x7f73f40fe430 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f41130d0 con 0x7f73f40fe430 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.079+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73e800bcc0 con 0x7f73f40fe430 2024-05-25T06:08:37.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.080+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73e801c630 con 0x7f73f40fe430 2024-05-25T06:08:37.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.080+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f4113640 con 0x7f73f40fe430 2024-05-25T06:08:37.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.080+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f404c710 con 0x7f73f40fe430 2024-05-25T06:08:37.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.081+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f73e8013980 con 0x7f73f40fe430 2024-05-25T06:08:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.081+0000 7f73f17fa700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f73e40692a0 0x7f73e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.082+0000 7f73f3fff700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f73e40692a0 0x7f73e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.082+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f73e8088020 con 0x7f73f40fe430 2024-05-25T06:08:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.082+0000 7f73f3fff700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f73e40692a0 0x7f73e406b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006fd0 tx=0x7f73e0008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:37.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.086+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f73e8057d10 con 0x7f73f40fe430 2024-05-25T06:08:37.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:37.325+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f73f4113920 con 0x7f73f40fe430 2024-05-25T06:08:37.450 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:standby-replay 2024-05-25T06:08:37.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: Dropping low affinity active daemon mds.cephfs.smithi160.rrhzig in favor of higher affinity standby. 2024-05-25T06:08:37.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: Replacing daemon mds.cephfs.smithi160.rrhzig as rank 0 with standby daemon mds.cephfs.smithi160.tlctof 2024-05-25T06:08:37.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-25T06:08:37.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:08:37.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: osdmap e46: 6 total, 6 up, 6 in 2024-05-25T06:08:37.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:replay,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 2024-05-25T06:08:37.452 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:37.452 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:37.452 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:37 smithi160 ceph-mon[29041]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:standby-replay 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: Dropping low affinity active daemon mds.cephfs.smithi160.rrhzig in favor of higher affinity standby. 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: Replacing daemon mds.cephfs.smithi160.rrhzig as rank 0 with standby daemon mds.cephfs.smithi160.tlctof 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: osdmap e46: 6 total, 6 up, 6 in 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:replay,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:37.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:37 smithi203 ceph-mon[36030]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-25T06:08:38.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.185+0000 7f73f17fa700 1 -- 172.21.15.160:0/1339894895 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7f73e80578a0 con 0x7f73f40fe430 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f73e40692a0 msgr2=0x7f73e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f73e40692a0 0x7f73e406b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006fd0 tx=0x7f73e0008040 comp rx=0 tx=0).stop 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 msgr2=0x7f73f410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f410d960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b3b0 tx=0x7f73e8009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 shutdown_connections 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f73e40692a0 0x7f73e406b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73f40fe430 0x7f73f410d960 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.187+0000 7f73fb045700 1 --2- 172.21.15.160:0/1339894895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40fed80 0x7f73f410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.188+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 >> 172.21.15.160:0/1339894895 conn(0x7f73f40f6bd0 msgr2=0x7f73f40f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.188+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 shutdown_connections 2024-05-25T06:08:38.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:38.188+0000 7f73fb045700 1 -- 172.21.15.160:0/1339894895 wait complete. 2024-05-25T06:08:38.189 INFO:teuthology.orchestra.run.smithi160.stderr:inline data disabled 2024-05-25T06:08:38.429 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:38 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] up:boot 2024-05-25T06:08:38.429 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:38 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:resolve 2024-05-25T06:08:38.429 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:38 smithi160 ceph-mon[29041]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:resolve,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:38.429 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:08:38.430 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:38 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1339894895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-05-25T06:08:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:38 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] up:boot 2024-05-25T06:08:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:38 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:resolve 2024-05-25T06:08:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:38 smithi203 ceph-mon[36030]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:resolve,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:38 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:08:38.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:38 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1339894895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-05-25T06:08:39.024 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:08:39.034 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:08:39.035 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-25T06:08:39.295 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:reconnect 2024-05-25T06:08:39.295 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1339894895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-05-25T06:08:39.295 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] up:active 2024-05-25T06:08:39.295 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] up:standby-replay 2024-05-25T06:08:39.295 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:reconnect,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:39.295 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-25T06:08:39.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:39.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:39.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:reconnect 2024-05-25T06:08:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1339894895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-05-25T06:08:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] up:active 2024-05-25T06:08:39.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] up:standby-replay 2024-05-25T06:08:39.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:reconnect,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:39.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-25T06:08:39.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:39.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:39.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:39 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:40.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:rejoin 2024-05-25T06:08:40.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:rejoin,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:40.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: daemon mds.cephfs.smithi160.tlctof is now active in filesystem cephfs as rank 0 2024-05-25T06:08:40.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-25T06:08:40.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: Cluster is now healthy 2024-05-25T06:08:40.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:active 2024-05-25T06:08:40.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:40.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:40 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:08:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:rejoin 2024-05-25T06:08:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: fsmap cephfs:2/2 {0=cephfs.smithi160.tlctof=up:rejoin,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: daemon mds.cephfs.smithi160.tlctof is now active in filesystem cephfs as rank 0 2024-05-25T06:08:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-25T06:08:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: Cluster is now healthy 2024-05-25T06:08:40.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: mds.? [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] up:active 2024-05-25T06:08:40.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 1 up:standby-replay 1 up:standby 2024-05-25T06:08:40.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:40 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:08:41.456 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:41 smithi160 ceph-mon[29041]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-25T06:08:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:41 smithi203 ceph-mon[36030]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 -- 172.21.15.160:0/3337254279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 msgr2=0x7fe77c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 --2- 172.21.15.160:0/3337254279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c0fd3d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe768004940 tx=0x7fe768009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 -- 172.21.15.160:0/3337254279 shutdown_connections 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 --2- 172.21.15.160:0/3337254279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c0fd3d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 --2- 172.21.15.160:0/3337254279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77c0fbcb0 0x7fe77c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 -- 172.21.15.160:0/3337254279 >> 172.21.15.160:0/3337254279 conn(0x7fe77c0f7430 msgr2=0x7fe77c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 -- 172.21.15.160:0/3337254279 shutdown_connections 2024-05-25T06:08:41.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.951+0000 7fe783240700 1 -- 172.21.15.160:0/3337254279 wait complete. 2024-05-25T06:08:41.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.952+0000 7fe783240700 1 Processor -- start 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.952+0000 7fe783240700 1 -- start start 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.952+0000 7fe783240700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77c0fbcb0 0x7fe77c110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.952+0000 7fe783240700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.952+0000 7fe783240700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c066350 con 0x7fe77c0fcf60 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.952+0000 7fe783240700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c066490 con 0x7fe77c0fbcb0 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54412/0 (socket says 172.21.15.160:54412) 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 -- 172.21.15.160:0/3367613415 learned_addr learned my addr 172.21.15.160:0/3367613415 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe780fdc700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77c0fbcb0 0x7fe77c110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 -- 172.21.15.160:0/3367613415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77c0fbcb0 msgr2=0x7fe77c110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77c0fbcb0 0x7fe77c110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 -- 172.21.15.160:0/3367613415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768005040 con 0x7fe77c0fcf60 2024-05-25T06:08:41.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe77bfff700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c112c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe76800b3b0 tx=0x7fe768009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76800e040 con 0x7fe77c0fcf60 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c113220 con 0x7fe77c0fcf60 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76800bbf0 con 0x7fe77c0fcf60 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76801c630 con 0x7fe77c0fcf60 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.953+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c1137b0 con 0x7fe77c0fcf60 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.955+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c05f5c0 con 0x7fe77c0fcf60 2024-05-25T06:08:41.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.955+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe76800bd60 con 0x7fe77c0fcf60 2024-05-25T06:08:41.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.956+0000 7fe7797fa700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe76c0692a0 0x7fe76c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:41.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.956+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe768087ee0 con 0x7fe77c0fcf60 2024-05-25T06:08:41.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.956+0000 7fe780fdc700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe76c0692a0 0x7fe76c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:41.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.956+0000 7fe780fdc700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe76c0692a0 0x7fe76c06b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe77c065fd0 tx=0x7fe77000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:41.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:41.960+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe7680541c0 con 0x7fe77c0fcf60 2024-05-25T06:08:42.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.209+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe77c04bed0 con 0x7fe77c0fcf60 2024-05-25T06:08:42.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.209+0000 7fe7797fa700 1 -- 172.21.15.160:0/3367613415 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe7680577e0 con 0x7fe77c0fcf60 2024-05-25T06:08:42.209 INFO:teuthology.orchestra.run.smithi160.stdout:e22 2024-05-25T06:08:42.209 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:08:42.209 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 22 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:08:40.188865+0000 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:08:42.210 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:08:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:08:42.212 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:08:42.212 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:08:42.212 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:08:42.212 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:08:42.212 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:42.212 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe76c0692a0 msgr2=0x7fe76c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe76c0692a0 0x7fe76c06b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe77c065fd0 tx=0x7fe77000b040 comp rx=0 tx=0).stop 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 msgr2=0x7fe77c112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c112c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe76800b3b0 tx=0x7fe768009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 shutdown_connections 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe76c0692a0 0x7fe76c06b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77c0fbcb0 0x7fe77c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 --2- 172.21.15.160:0/3367613415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe77c0fcf60 0x7fe77c112c80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 >> 172.21.15.160:0/3367613415 conn(0x7fe77c0f7430 msgr2=0x7fe77c0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 shutdown_connections 2024-05-25T06:08:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:42.213+0000 7fe783240700 1 -- 172.21.15.160:0/3367613415 wait complete. 2024-05-25T06:08:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 22 2024-05-25T06:08:42.904 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-25T06:08:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:42 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3367613415' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:08:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:43.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:43.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:43.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:08:43.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:42 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3367613415' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:08:43.816 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:44 smithi160 ceph-mon[29041]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.0 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-25T06:08:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:44 smithi203 ceph-mon[36030]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.0 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-25T06:08:45.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:45.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:45 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:45.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:45 smithi160 ceph-mon[29041]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 -- 172.21.15.160:0/1879037214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fbcc0 msgr2=0x7f0c000fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 --2- 172.21.15.160:0/1879037214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fbcc0 0x7f0c000fc0d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc004940 tx=0x7f0bfc009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 -- 172.21.15.160:0/1879037214 shutdown_connections 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 --2- 172.21.15.160:0/1879037214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000fcf70 0x7f0c000fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 --2- 172.21.15.160:0/1879037214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fbcc0 0x7f0c000fc0d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 -- 172.21.15.160:0/1879037214 >> 172.21.15.160:0/1879037214 conn(0x7f0c000f7420 msgr2=0x7f0c000f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 -- 172.21.15.160:0/1879037214 shutdown_connections 2024-05-25T06:08:45.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 -- 172.21.15.160:0/1879037214 wait complete. 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 Processor -- start 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.541+0000 7f0c0714f700 1 -- start start 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0c0714f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000fbcc0 0x7f0c00066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0c0714f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 0x7f0c00112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0c0714f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00112c60 con 0x7f0c000fcf70 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0c0714f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00112da0 con 0x7f0c000fbcc0 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 0x7f0c00112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 0x7f0c00112720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45138/0 (socket says 172.21.15.160:45138) 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 -- 172.21.15.160:0/4272884631 learned_addr learned my addr 172.21.15.160:0/4272884631 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0c04eeb700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000fbcc0 0x7f0c00066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 -- 172.21.15.160:0/4272884631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000fbcc0 msgr2=0x7f0c00066d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:45.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000fbcc0 0x7f0c00066d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 -- 172.21.15.160:0/4272884631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc005040 con 0x7f0c000fcf70 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf7fff700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 0x7f0c00112720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0002a80 tx=0x7f0bf00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.542+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf0007cc0 con 0x7f0c000fcf70 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.543+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c00113030 con 0x7f0c000fcf70 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.543+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bf000f040 con 0x7f0c000fcf70 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.543+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf000da60 con 0x7f0c000fcf70 2024-05-25T06:08:45.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.543+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c001135c0 con 0x7f0c000fcf70 2024-05-25T06:08:45.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.544+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0004c710 con 0x7f0c000fcf70 2024-05-25T06:08:45.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.544+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0bf000dc90 con 0x7f0c000fcf70 2024-05-25T06:08:45.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.544+0000 7f0bf57fa700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0be80692a0 0x7f0be806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:45.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.545+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0bf0086e30 con 0x7f0c000fcf70 2024-05-25T06:08:45.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.545+0000 7f0c04eeb700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0be80692a0 0x7f0be806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:45.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.545+0000 7f0c04eeb700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0be80692a0 0x7f0be806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc004f90 tx=0x7f0bfc009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:45.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.549+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0bf0056b20 con 0x7f0c000fcf70 2024-05-25T06:08:45.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:45 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:45.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:45 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:45.575 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:45 smithi203 ceph-mon[36030]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-25T06:08:45.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.781+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c00113810 con 0x7f0c000fcf70 2024-05-25T06:08:45.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.781+0000 7f0bf57fa700 1 -- 172.21.15.160:0/4272884631 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f0bf00566b0 con 0x7f0c000fcf70 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.783+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0be80692a0 msgr2=0x7f0be806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.783+0000 7f0c0714f700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0be80692a0 0x7f0be806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc004f90 tx=0x7f0bfc009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.783+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 msgr2=0x7f0c00112720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.783+0000 7f0c0714f700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 0x7f0c00112720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0002a80 tx=0x7f0bf00046b0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 shutdown_connections 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0be80692a0 0x7f0be806b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000fbcc0 0x7f0c00066d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 --2- 172.21.15.160:0/4272884631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c000fcf70 0x7f0c00112720 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 >> 172.21.15.160:0/4272884631 conn(0x7f0c000f7420 msgr2=0x7f0c000f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 shutdown_connections 2024-05-25T06:08:45.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:45.784+0000 7f0c0714f700 1 -- 172.21.15.160:0/4272884631 wait complete. 2024-05-25T06:08:45.785 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 22 2024-05-25T06:08:45.794 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:08:46.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:46 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/4272884631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-25T06:08:46.508 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-25T06:08:46.523 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:46 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/4272884631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-25T06:08:47.297 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:47.549 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:47 smithi160 ceph-mon[29041]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-25T06:08:47.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:47 smithi203 ceph-mon[36030]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-25T06:08:49.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:49 smithi160 ceph-mon[29041]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-25T06:08:50.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 -- 172.21.15.160:0/3948855371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fbcb0 msgr2=0x7f4d000fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:50.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3948855371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fbcb0 0x7f4d000fc0c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4004940 tx=0x7f4cf4009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 -- 172.21.15.160:0/3948855371 shutdown_connections 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3948855371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d000fcf60 0x7f4d000fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3948855371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fbcb0 0x7f4d000fc0c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 -- 172.21.15.160:0/3948855371 >> 172.21.15.160:0/3948855371 conn(0x7f4d000f7430 msgr2=0x7f4d000f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 -- 172.21.15.160:0/3948855371 shutdown_connections 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.087+0000 7f4d06db2700 1 -- 172.21.15.160:0/3948855371 wait complete. 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d06db2700 1 Processor -- start 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d06db2700 1 -- start start 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d06db2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d000fbcb0 0x7f4d00110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d06db2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 0x7f4d00112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d06db2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00066330 con 0x7f4d000fcf60 2024-05-25T06:08:50.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d06db2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00066470 con 0x7f4d000fbcb0 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4cfffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 0x7f4d00112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4cfffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 0x7f4d00112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45162/0 (socket says 172.21.15.160:45162) 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4cfffff700 1 -- 172.21.15.160:0/3375764070 learned_addr learned my addr 172.21.15.160:0/3375764070 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.088+0000 7f4d04b4e700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d000fbcb0 0x7f4d00110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfffff700 1 -- 172.21.15.160:0/3375764070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d000fbcb0 msgr2=0x7f4d00110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfffff700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d000fbcb0 0x7f4d00110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfffff700 1 -- 172.21.15.160:0/3375764070 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4005040 con 0x7f4d000fcf60 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfffff700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 0x7f4d00112c70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00c8c0 tx=0x7f4cec0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cec00a760 con 0x7f4d000fcf60 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cec00ad90 con 0x7f4d000fcf60 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d00113210 con 0x7f4d000fcf60 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cec0113a0 con 0x7f4d000fcf60 2024-05-25T06:08:50.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.089+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d001137d0 con 0x7f4d000fcf60 2024-05-25T06:08:50.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.090+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0005f5c0 con 0x7f4d000fcf60 2024-05-25T06:08:50.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.090+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4cec012400 con 0x7f4d000fcf60 2024-05-25T06:08:50.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.091+0000 7f4cfd7fa700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f4cf00692b0 0x7f4cf006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:50.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.091+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4cec087200 con 0x7f4d000fcf60 2024-05-25T06:08:50.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.091+0000 7f4d04b4e700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f4cf00692b0 0x7f4cf006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:50.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.091+0000 7f4d04b4e700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f4cf00692b0 0x7f4cf006b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:50.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.095+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4cec0534e0 con 0x7f4d000fcf60 2024-05-25T06:08:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:49 smithi203 ceph-mon[36030]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-25T06:08:50.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.392+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f4d00113aa0 con 0x7f4d000fcf60 2024-05-25T06:08:50.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.393+0000 7f4cfd7fa700 1 -- 172.21.15.160:0/3375764070 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v22) v1 ==== 78+0+99 (secure 0 0 0) 0x7f4cec056b00 con 0x7f4d000fcf60 2024-05-25T06:08:50.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f4cf00692b0 msgr2=0x7f4cf006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:50.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f4cf00692b0 0x7f4cf006b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:50.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 msgr2=0x7f4d00112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:50.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 0x7f4d00112c70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00c8c0 tx=0x7f4cec0044c0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 shutdown_connections 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f4cf00692b0 0x7f4cf006b760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d000fbcb0 0x7f4d00110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 --2- 172.21.15.160:0/3375764070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fcf60 0x7f4d00112c70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 >> 172.21.15.160:0/3375764070 conn(0x7f4d000f7430 msgr2=0x7f4d000f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 shutdown_connections 2024-05-25T06:08:50.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:50.396+0000 7f4d06db2700 1 -- 172.21.15.160:0/3375764070 wait complete. 2024-05-25T06:08:50.407 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:08:51.018 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-25T06:08:51.031 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 -- 172.21.15.160:0/619790123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fb090 msgr2=0x7f83380fb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 --2- 172.21.15.160:0/619790123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fb090 0x7f83380fb4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f832c0048d0 tx=0x7f832c009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 -- 172.21.15.160:0/619790123 shutdown_connections 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 --2- 172.21.15.160:0/619790123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83380fc360 0x7f83380fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 --2- 172.21.15.160:0/619790123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fb090 0x7f83380fb4a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 -- 172.21.15.160:0/619790123 >> 172.21.15.160:0/619790123 conn(0x7f83380f6850 msgr2=0x7f83380f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 -- 172.21.15.160:0/619790123 shutdown_connections 2024-05-25T06:08:51.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 -- 172.21.15.160:0/619790123 wait complete. 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 Processor -- start 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.142+0000 7f834079d640 1 -- start start 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f834079d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83380fb090 0x7f8338065f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f834079d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 0x7f8338113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f834079d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833810d8b0 con 0x7f83380fc360 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f834079d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338114410 con 0x7f83380fb090 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 0x7f8338113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 0x7f8338113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45190/0 (socket says 172.21.15.160:45190) 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 -- 172.21.15.160:0/1882672946 learned_addr learned my addr 172.21.15.160:0/1882672946 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833e539640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83380fb090 0x7f8338065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 -- 172.21.15.160:0/1882672946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83380fb090 msgr2=0x7f8338065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83380fb090 0x7f8338065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 -- 172.21.15.160:0/1882672946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c005040 con 0x7f83380fc360 2024-05-25T06:08:51.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f833dd38640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 0x7f8338113ed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8334002a50 tx=0x7f8334004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:51.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8334007c30 con 0x7f83380fc360 2024-05-25T06:08:51.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f833400f040 con 0x7f83380fc360 2024-05-25T06:08:51.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f833400db50 con 0x7f83380fc360 2024-05-25T06:08:51.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338114610 con 0x7f83380fc360 2024-05-25T06:08:51.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.143+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8338114ab0 con 0x7f83380fc360 2024-05-25T06:08:51.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.144+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338105590 con 0x7f83380fc360 2024-05-25T06:08:51.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.145+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f833401c400 con 0x7f83380fc360 2024-05-25T06:08:51.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.145+0000 7f832affd640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8324069210 0x7f832406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.145+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8334086a70 con 0x7f83380fc360 2024-05-25T06:08:51.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.145+0000 7f833e539640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8324069210 0x7f832406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.146+0000 7f833e539640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8324069210 0x7f832406b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f832c0317e0 tx=0x7f832c009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:51.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.149+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f833404d250 con 0x7f83380fc360 2024-05-25T06:08:51.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.320+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8338114f70 con 0x7f83380fc360 2024-05-25T06:08:51.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.321+0000 7f832affd640 1 -- 172.21.15.160:0/1882672946 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14462 (secure 0 0 0) 0x7f8334052d80 con 0x7f83380fc360 2024-05-25T06:08:51.321 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:51.321 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":46,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:08:36.175998+0000","last_up_change":"2024-05-25T06:07:03.230915+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-25T06:08:20.895353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-25T06:08:21.186881+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641643302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641643302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6809","nonce":2641643302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641643302},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641643302}]},"public_addr":"172.21.15.160:6803/2641643302","cluster_addr":"172.21.15.160:6805/2641643302","heartbeat_back_addr":"172.21.15.160:6809/2641643302","heartbeat_front_addr":"172.21.15.160:6807/2641643302","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6811","nonce":3982025942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6813","nonce":3982025942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6817","nonce":3982025942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3982025942},{"type":"v1","addr":"172.21.15.160:6815","nonce":3982025942}]},"public_addr":"172.21.15.160:6811/3982025942","cluster_addr":"172.21.15.160:6813/3982025942","heartbeat_back_addr":"172.21.15.160:6817/3982025942","heartbeat_front_addr":"172.21.15.160:6815/3982025942","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6819","nonce":1185286089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6821","nonce":1185286089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6825","nonce":1185286089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1185286089},{"type":"v1","addr":"172.21.15.160:6823","nonce":1185286089}]},"public_addr":"172.21.15.160:6819/1185286089","cluster_addr":"172.21.15.160:6821/1185286089","heartbeat_back_addr":"172.21.15.160:6825/1185286089","heartbeat_front_addr":"172.21.15.160:6823/1185286089","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6801","nonce":3678463726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6803","nonce":3678463726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6807","nonce":3678463726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3678463726},{"type":"v1","addr":"172.21.15.203:6805","nonce":3678463726}]},"public_addr":"172.21.15.203:6801/3678463726","cluster_addr":"172.21.15.203:6803/3678463726","heartbeat_back_addr":"172.21.15.203:6807/3678463726","heartbeat_front_addr":"172.21.15.203:6805/3678463726","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6809","nonce":2408351414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6811","nonce":2408351414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6815","nonce":2408351414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2408351414},{"type":"v1","addr":"172.21.15.203:6813","nonce":2408351414}]},"public_addr":"172.21.15.203:6809/2408351414","cluster_addr":"172.21.15.203:6811/2408351414","heartbeat_back_addr":"172.21.15.203:6815/2408351414","heartbeat_front_addr":"172.21.15.203:6813/2408351414","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6817","nonce":2488256923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6819","nonce":2488256923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6823","nonce":2488256923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2488256923},{"type":"v1","addr":"172.21.15.203:6821","nonce":2488256923}]},"public_addr":"172.21.15.203:6817/2488256923","cluster_addr":"172.21.15.203:6819/2488256923","heartbeat_back_addr":"172.21.15.203:6823/2488256923","heartbeat_front_addr":"172.21.15.203:6821/2488256923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6827/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6826/2072396647":"2024-05-26T06:08:36.175975+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8324069210 msgr2=0x7f832406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8324069210 0x7f832406b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f832c0317e0 tx=0x7f832c009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 msgr2=0x7f8338113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 0x7f8338113ed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8334002a50 tx=0x7f8334004600 comp rx=0 tx=0).stop 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 shutdown_connections 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f8324069210 0x7f832406b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83380fc360 0x7f8338113ed0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 --2- 172.21.15.160:0/1882672946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83380fb090 0x7f8338065f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 >> 172.21.15.160:0/1882672946 conn(0x7f83380f6850 msgr2=0x7f83380ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 shutdown_connections 2024-05-25T06:08:51.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.323+0000 7f834079d640 1 -- 172.21.15.160:0/1882672946 wait complete. 2024-05-25T06:08:51.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-25T06:05:48.910750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-25T06:08:20.895353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-25T06:08:21.186881+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-25T06:08:51.336 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-25T06:08:51.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 -- 172.21.15.160:0/353548675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 msgr2=0x7f01dc0fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 --2- 172.21.15.160:0/353548675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc0fc610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01d80048d0 tx=0x7f01d8009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 -- 172.21.15.160:0/353548675 shutdown_connections 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 --2- 172.21.15.160:0/353548675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc0fd430 0x7f01dc0fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 --2- 172.21.15.160:0/353548675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc0fc610 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 -- 172.21.15.160:0/353548675 >> 172.21.15.160:0/353548675 conn(0x7f01dc0f7920 msgr2=0x7f01dc0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 -- 172.21.15.160:0/353548675 shutdown_connections 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 -- 172.21.15.160:0/353548675 wait complete. 2024-05-25T06:08:51.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 Processor -- start 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.436+0000 7f01e3607640 1 -- start start 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e3607640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc10cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e3607640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc0fd430 0x7f01dc10d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e3607640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc10d890 con 0x7f01dc0fc200 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e3607640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc112250 con 0x7f01dc0fd430 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc10cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc10cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45210/0 (socket says 172.21.15.160:45210) 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 -- 172.21.15.160:0/1165676641 learned_addr learned my addr 172.21.15.160:0/1165676641 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e0ba2640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc0fd430 0x7f01dc10d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 -- 172.21.15.160:0/1165676641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc0fd430 msgr2=0x7f01dc10d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc0fd430 0x7f01dc10d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 -- 172.21.15.160:0/1165676641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8005040 con 0x7f01dc0fc200 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e13a3640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc10cd00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01d802bda0 tx=0x7f01d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d80094c0 con 0x7f01dc0fc200 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01d8009620 con 0x7f01dc0fc200 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc112390 con 0x7f01dc0fc200 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.437+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d803b640 con 0x7f01dc0fc200 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.438+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc112860 con 0x7f01dc0fc200 2024-05-25T06:08:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.438+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc04bcc0 con 0x7f01dc0fc200 2024-05-25T06:08:51.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.439+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f01d802f070 con 0x7f01dc0fc200 2024-05-25T06:08:51.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.439+0000 7f01d1ffb640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f01c8069210 0x7f01c806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.440+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f01d80a9730 con 0x7f01dc0fc200 2024-05-25T06:08:51.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.440+0000 7f01e0ba2640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f01c8069210 0x7f01c806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.440+0000 7f01e0ba2640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f01c8069210 0x7f01c806b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01cc004ed0 tx=0x7f01cc00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:51.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.444+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f01d8075990 con 0x7f01dc0fc200 2024-05-25T06:08:51.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.611+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01dc1129f0 con 0x7f01dc0fc200 2024-05-25T06:08:51.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.612+0000 7f01d1ffb640 1 -- 172.21.15.160:0/1165676641 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01d8078fb0 con 0x7f01dc0fc200 2024-05-25T06:08:51.612 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-25T06:08:51.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f01c8069210 msgr2=0x7f01c806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f01c8069210 0x7f01c806b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01cc004ed0 tx=0x7f01cc00b040 comp rx=0 tx=0).stop 2024-05-25T06:08:51.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 msgr2=0x7f01dc10cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc10cd00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01d802bda0 tx=0x7f01d8009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 shutdown_connections 2024-05-25T06:08:51.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f01c8069210 0x7f01c806b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc0fd430 0x7f01dc10d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 --2- 172.21.15.160:0/1165676641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01dc0fc200 0x7f01dc10cd00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 >> 172.21.15.160:0/1165676641 conn(0x7f01dc0f7920 msgr2=0x7f01dc0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:51.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 shutdown_connections 2024-05-25T06:08:51.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.614+0000 7f01e3607640 1 -- 172.21.15.160:0/1165676641 wait complete. 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3375764070' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-25T06:08:51.624 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:51 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1882672946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:08:51.625 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-25T06:08:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:08:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:08:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:51.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3375764070' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-25T06:08:51.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-25T06:08:51.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:51 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1882672946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 -- 172.21.15.160:0/22231940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fb0f0 msgr2=0x7f3d200fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/22231940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fb0f0 0x7f3d200fb500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3d100048d0 tx=0x7f3d10009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 -- 172.21.15.160:0/22231940 shutdown_connections 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/22231940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fc320 0x7f3d200fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/22231940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fb0f0 0x7f3d200fb500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 -- 172.21.15.160:0/22231940 >> 172.21.15.160:0/22231940 conn(0x7f3d200f6830 msgr2=0x7f3d200f8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 -- 172.21.15.160:0/22231940 shutdown_connections 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.723+0000 7f3d27c2e640 1 -- 172.21.15.160:0/22231940 wait complete. 2024-05-25T06:08:51.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d27c2e640 1 Processor -- start 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d27c2e640 1 -- start start 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d27c2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fb0f0 0x7f3d2010ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d27c2e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 0x7f3d2010d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d27c2e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2010d880 con 0x7f3d200fc320 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d27c2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20112240 con 0x7f3d200fb0f0 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d251c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 0x7f3d2010d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d251c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 0x7f3d2010d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45232/0 (socket says 172.21.15.160:45232) 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d251c9640 1 -- 172.21.15.160:0/877663774 learned_addr learned my addr 172.21.15.160:0/877663774 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d259ca640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fb0f0 0x7f3d2010ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d251c9640 1 -- 172.21.15.160:0/877663774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fb0f0 msgr2=0x7f3d2010ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d251c9640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fb0f0 0x7f3d2010ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d251c9640 1 -- 172.21.15.160:0/877663774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10005040 con 0x7f3d200fc320 2024-05-25T06:08:51.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.724+0000 7f3d259ca640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fb0f0 0x7f3d2010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:08:51.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.725+0000 7f3d251c9640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 0x7f3d2010d200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c002a80 tx=0x7f3d1c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:51.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.725+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1c007c30 con 0x7f3d200fc320 2024-05-25T06:08:51.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.725+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d1c00f040 con 0x7f3d200fc320 2024-05-25T06:08:51.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.725+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d201123e0 con 0x7f3d200fc320 2024-05-25T06:08:51.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.725+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1c009670 con 0x7f3d200fc320 2024-05-25T06:08:51.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.725+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d201128d0 con 0x7f3d200fc320 2024-05-25T06:08:51.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.726+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2004bcc0 con 0x7f3d200fc320 2024-05-25T06:08:51.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.726+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3d1c00d6b0 con 0x7f3d200fc320 2024-05-25T06:08:51.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.727+0000 7f3d167fc640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3d0c069210 0x7f3d0c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:51.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.727+0000 7f3d259ca640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3d0c069210 0x7f3d0c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:51.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.727+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3d1c087160 con 0x7f3d200fc320 2024-05-25T06:08:51.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.727+0000 7f3d259ca640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3d0c069210 0x7f3d0c06b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3d200fa380 tx=0x7f3d10009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:51.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.731+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3d1c056e50 con 0x7f3d200fc320 2024-05-25T06:08:51.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.895+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f3d20103bd0 con 0x7f3d200fc320 2024-05-25T06:08:51.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.896+0000 7f3d167fc640 1 -- 172.21.15.160:0/877663774 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3d1c0569e0 con 0x7f3d200fc320 2024-05-25T06:08:51.896 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3d0c069210 msgr2=0x7f3d0c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3d0c069210 0x7f3d0c06b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3d200fa380 tx=0x7f3d10009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 msgr2=0x7f3d2010d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 0x7f3d2010d200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c002a80 tx=0x7f3d1c004600 comp rx=0 tx=0).stop 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 shutdown_connections 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f3d0c069210 0x7f3d0c06b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d200fc320 0x7f3d2010d200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 --2- 172.21.15.160:0/877663774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d200fb0f0 0x7f3d2010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:51.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 >> 172.21.15.160:0/877663774 conn(0x7f3d200f6830 msgr2=0x7f3d20065530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:51.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 shutdown_connections 2024-05-25T06:08:51.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:51.898+0000 7f3d27c2e640 1 -- 172.21.15.160:0/877663774 wait complete. 2024-05-25T06:08:51.909 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 -- 172.21.15.160:0/1653473691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fb060 msgr2=0x7facc40fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 --2- 172.21.15.160:0/1653473691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fb060 0x7facc40fb470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7facb40048d0 tx=0x7facb4009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 -- 172.21.15.160:0/1653473691 shutdown_connections 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 --2- 172.21.15.160:0/1653473691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facc40fc330 0x7facc40fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 --2- 172.21.15.160:0/1653473691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fb060 0x7facc40fb470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 -- 172.21.15.160:0/1653473691 >> 172.21.15.160:0/1653473691 conn(0x7facc40f6820 msgr2=0x7facc40f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 -- 172.21.15.160:0/1653473691 shutdown_connections 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 -- 172.21.15.160:0/1653473691 wait complete. 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.008+0000 7facc9d54640 1 Processor -- start 2024-05-25T06:08:52.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc9d54640 1 -- start start 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc9d54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facc40fb060 0x7facc4065f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc9d54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 0x7facc4111b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc9d54640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc4112050 con 0x7facc40fc330 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc9d54640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc4112190 con 0x7facc40fb060 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 0x7facc4111b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 0x7facc4111b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45258/0 (socket says 172.21.15.160:45258) 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 -- 172.21.15.160:0/3736253377 learned_addr learned my addr 172.21.15.160:0/3736253377 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc37fe640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facc40fb060 0x7facc4065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 -- 172.21.15.160:0/3736253377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facc40fb060 msgr2=0x7facc4065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facc40fb060 0x7facc4065f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 -- 172.21.15.160:0/3736253377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4005040 con 0x7facc40fc330 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facc2ffd640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 0x7facc4111b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7facb800c790 tx=0x7facb80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.009+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb800a820 con 0x7facc40fc330 2024-05-25T06:08:52.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.010+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc41123f0 con 0x7facc40fc330 2024-05-25T06:08:52.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.010+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facb800ae50 con 0x7facc40fc330 2024-05-25T06:08:52.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.010+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb80115e0 con 0x7facc40fc330 2024-05-25T06:08:52.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.010+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc4112890 con 0x7facc40fc330 2024-05-25T06:08:52.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.011+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc4105560 con 0x7facc40fc330 2024-05-25T06:08:52.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.011+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7facb8012400 con 0x7facc40fc330 2024-05-25T06:08:52.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.011+0000 7facabfff640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7facac069210 0x7facac06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:52.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.012+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7facb8087340 con 0x7facc40fc330 2024-05-25T06:08:52.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.012+0000 7facc37fe640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7facac069210 0x7facac06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:52.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.012+0000 7facc37fe640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7facac069210 0x7facac06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7facb40317e0 tx=0x7facb4009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:52.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.016+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7facb8053620 con 0x7facc40fc330 2024-05-25T06:08:52.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.181+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7facc4103bd0 con 0x7facc40fc330 2024-05-25T06:08:52.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.182+0000 7facabfff640 1 -- 172.21.15.160:0/3736253377 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7facb8056c40 con 0x7facc40fc330 2024-05-25T06:08:52.182 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:08:52.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7facac069210 msgr2=0x7facac06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7facac069210 0x7facac06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7facb40317e0 tx=0x7facb4009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 msgr2=0x7facc4111b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 0x7facc4111b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7facb800c790 tx=0x7facb80043c0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 shutdown_connections 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7facac069210 0x7facac06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc40fc330 0x7facc4111b10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 --2- 172.21.15.160:0/3736253377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facc40fb060 0x7facc4065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 >> 172.21.15.160:0/3736253377 conn(0x7facc40f6820 msgr2=0x7facc40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:52.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 shutdown_connections 2024-05-25T06:08:52.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.184+0000 7facc9d54640 1 -- 172.21.15.160:0/3736253377 wait complete. 2024-05-25T06:08:52.195 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 -- 172.21.15.160:0/3257673679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 msgr2=0x7f99740fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 --2- 172.21.15.160:0/3257673679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f99740fdbd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99680048d0 tx=0x7f9968009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 -- 172.21.15.160:0/3257673679 shutdown_connections 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 --2- 172.21.15.160:0/3257673679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 0x7f99740fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 --2- 172.21.15.160:0/3257673679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f99740fdbd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 -- 172.21.15.160:0/3257673679 >> 172.21.15.160:0/3257673679 conn(0x7f99740f5fc0 msgr2=0x7f99740f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 -- 172.21.15.160:0/3257673679 shutdown_connections 2024-05-25T06:08:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.291+0000 7f997ab14640 1 -- 172.21.15.160:0/3257673679 wait complete. 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f997ab14640 1 Processor -- start 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f997ab14640 1 -- start start 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f997ab14640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f997410ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f997ab14640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 0x7f997410d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f997ab14640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997410d880 con 0x7f99740fd7c0 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f997ab14640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974112240 con 0x7f99740fe110 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f99788b0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f997410ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f99788b0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f997410ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45288/0 (socket says 172.21.15.160:45288) 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f99788b0640 1 -- 172.21.15.160:0/3206181101 learned_addr learned my addr 172.21.15.160:0/3206181101 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f9973fff640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 0x7f997410d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f99788b0640 1 -- 172.21.15.160:0/3206181101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 msgr2=0x7f997410d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f99788b0640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 0x7f997410d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f99788b0640 1 -- 172.21.15.160:0/3206181101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968005040 con 0x7f99740fd7c0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.292+0000 7f9973fff640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 0x7f997410d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.293+0000 7f99788b0640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f997410ccc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9968003030 tx=0x7f9968009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.293+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99680094c0 con 0x7f99740fd7c0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.293+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9968009620 con 0x7f99740fd7c0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.293+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9974112380 con 0x7f99740fd7c0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.293+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996803b640 con 0x7f99740fd7c0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.293+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9974112870 con 0x7f99740fd7c0 2024-05-25T06:08:52.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.294+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997404bcc0 con 0x7f99740fd7c0 2024-05-25T06:08:52.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.294+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f996802f070 con 0x7f99740fd7c0 2024-05-25T06:08:52.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.295+0000 7f99717fa640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9964069210 0x7f996406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:52.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.295+0000 7f9973fff640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9964069210 0x7f996406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:52.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.295+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f99680a97b0 con 0x7f99740fd7c0 2024-05-25T06:08:52.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.295+0000 7f9973fff640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9964069210 0x7f996406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f997410e1b0 tx=0x7f996000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:52.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.299+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9968075a10 con 0x7f99740fd7c0 2024-05-25T06:08:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:52 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1165676641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:08:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:52 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/877663774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:08:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:52 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3736253377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:08:52.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.503+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9974112ae0 con 0x7f99740fd7c0 2024-05-25T06:08:52.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.504+0000 7f99717fa640 1 -- 172.21.15.160:0/3206181101 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f9968079030 con 0x7f99740fd7c0 2024-05-25T06:08:52.504 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:08:52.505 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:08:40.188865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi160.tlctof","rank":0,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6829/3783773637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3783773637},{"type":"v1","addr":"172.21.15.160:6829","nonce":3783773637}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6827/2201510143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2201510143},{"type":"v1","addr":"172.21.15.160:6827","nonce":2201510143}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi203.yltymd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6825/3325193142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3325193142},{"type":"v1","addr":"172.21.15.203:6825","nonce":3325193142}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24283":{"gid":24283,"name":"cephfs.smithi203.tndnrb","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.203:6827/1913747634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1913747634},{"type":"v1","addr":"172.21.15.203:6827","nonce":1913747634}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-05-25T06:08:52.505 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 22 2024-05-25T06:08:52.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9964069210 msgr2=0x7f996406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9964069210 0x7f996406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f997410e1b0 tx=0x7f996000b040 comp rx=0 tx=0).stop 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 msgr2=0x7f997410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f997410ccc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9968003030 tx=0x7f9968009da0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 shutdown_connections 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f9964069210 0x7f996406b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740fe110 0x7f997410d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 --2- 172.21.15.160:0/3206181101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99740fd7c0 0x7f997410ccc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 >> 172.21.15.160:0/3206181101 conn(0x7f99740f5fc0 msgr2=0x7f99740f7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 shutdown_connections 2024-05-25T06:08:52.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:52.506+0000 7f997ab14640 1 -- 172.21.15.160:0/3206181101 wait complete. 2024-05-25T06:08:52.517 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 22, 'max_mds': 2, 'flags': 50} 2024-05-25T06:08:52.517 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-25T06:08:52.545 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-25T06:08:52.545 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-25T06:08:52.545 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-25T06:08:52.545 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-25T06:08:52.545 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-25T06:08:52.546 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-25T06:08:52.546 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-25T06:08:52.546 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-25T06:08:52.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:52.546 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-05-25T06:08:52.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:52.564 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2024-05-25T06:08:52.630 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2024-05-25T06:08:52.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:08:52.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:52.632 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-05-25T06:08:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:52 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1165676641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:08:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:52 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/877663774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:08:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:52 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3736253377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:08:52.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:52.653 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-05-25T06:08:52.727 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-05-25T06:08:52.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:08:52.728 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-25T06:08:52.729 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-25T06:08:52.983 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:53.443 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:53 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3206181101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-25T06:08:53.443 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:53 smithi160 ceph-mon[29041]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-25T06:08:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:53 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3206181101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-25T06:08:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:53 smithi203 ceph-mon[36030]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-25T06:08:54.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.504+0000 7fd2403fe700 1 -- 172.21.15.160:0/1999266520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 msgr2=0x7fd2380fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.504+0000 7fd2403fe700 1 --2- 172.21.15.160:0/1999266520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380fcdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd228004940 tx=0x7fd228009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- 172.21.15.160:0/1999266520 shutdown_connections 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 --2- 172.21.15.160:0/1999266520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380fcdd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 --2- 172.21.15.160:0/1999266520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd238102dc0 0x7fd238103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- 172.21.15.160:0/1999266520 >> 172.21.15.160:0/1999266520 conn(0x7fd2380f8930 msgr2=0x7fd2380fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- 172.21.15.160:0/1999266520 shutdown_connections 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- 172.21.15.160:0/1999266520 wait complete. 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 Processor -- start 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- start start 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd238102dc0 0x7fd238070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:54.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2380721b0 con 0x7fd238103760 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.505+0000 7fd2403fe700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238070ce0 con 0x7fd238102dc0 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45314/0 (socket says 172.21.15.160:45314) 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 -- 172.21.15.160:0/997959083 learned_addr learned my addr 172.21.15.160:0/997959083 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23e19a700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd238102dc0 0x7fd238070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 -- 172.21.15.160:0/997959083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd238102dc0 msgr2=0x7fd238070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd238102dc0 0x7fd238070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 -- 172.21.15.160:0/997959083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228005040 con 0x7fd238103760 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd23d999700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380707a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd228009100 tx=0x7fd228009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:54.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd22800e040 con 0x7fd238103760 2024-05-25T06:08:54.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd228009af0 con 0x7fd238103760 2024-05-25T06:08:54.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd238070f60 con 0x7fd238103760 2024-05-25T06:08:54.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd22801c3f0 con 0x7fd238103760 2024-05-25T06:08:54.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.506+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23811a0a0 con 0x7fd238103760 2024-05-25T06:08:54.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.507+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2380fe510 con 0x7fd238103760 2024-05-25T06:08:54.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.508+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd22800baf0 con 0x7fd238103760 2024-05-25T06:08:54.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.508+0000 7fd22effd700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd2240692a0 0x7fd22406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:54.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.508+0000 7fd23e19a700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd2240692a0 0x7fd22406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:54.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.508+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd228088440 con 0x7fd238103760 2024-05-25T06:08:54.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.508+0000 7fd23e19a700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd2240692a0 0x7fd22406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd234004550 tx=0x7fd234007ed0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:54.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.512+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd228054720 con 0x7fd238103760 2024-05-25T06:08:54.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.748+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd23811a290 con 0x7fd238103760 2024-05-25T06:08:54.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.748+0000 7fd22effd700 1 -- 172.21.15.160:0/997959083 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+83 (secure 0 0 0) 0x7fd228057d40 con 0x7fd238103760 2024-05-25T06:08:54.748 INFO:teuthology.orchestra.run.smithi160.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-25T06:08:54.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd2240692a0 msgr2=0x7fd22406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:54.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd2240692a0 0x7fd22406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd234004550 tx=0x7fd234007ed0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 msgr2=0x7fd2380707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380707a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd228009100 tx=0x7fd228009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 shutdown_connections 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fd2240692a0 0x7fd22406b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd238102dc0 0x7fd238070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 --2- 172.21.15.160:0/997959083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd238103760 0x7fd2380707a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 >> 172.21.15.160:0/997959083 conn(0x7fd2380f8930 msgr2=0x7fd2380f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 shutdown_connections 2024-05-25T06:08:54.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:54.750+0000 7fd2403fe700 1 -- 172.21.15.160:0/997959083 wait complete. 2024-05-25T06:08:54.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:54 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:54.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:54 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:54.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:54 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:08:54.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:54 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-25T06:08:55.414 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-25T06:08:55.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:55.415 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2024-05-25T06:08:55.431 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-25T06:08:55.431 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-25T06:08:55.431 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2024-05-25T06:08:55.431 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-25T06:08:55.431 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f0 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f1 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-25T06:08:55.432 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42247sec preferred_lft 42247sec 2024-05-25T06:08:55.433 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2024-05-25T06:08:55.433 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-25T06:08:55.433 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add name ceph-brx type bridge 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr flush dev ceph-brx 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set ceph-brx up 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-25T06:08:55.433 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-25T06:08:55.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:08:55.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:08:55.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:55.663 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-25T06:08:55.688 INFO:teuthology.orchestra.run.smithi160.stdout:1 2024-05-25T06:08:55.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:55.690 DEBUG:teuthology.orchestra.run.smithi160:> route 2024-05-25T06:08:55.762 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2024-05-25T06:08:55.762 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-25T06:08:55.763 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-25T06:08:55.763 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-25T06:08:55.763 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-25T06:08:55.763 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:08:55.763 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-25T06:08:55.763 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-25T06:08:55.763 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-25T06:08:55.763 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-25T06:08:55.764 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-25T06:08:55.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:08:55.961 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:55 smithi160 ceph-mon[29041]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-25T06:08:55.961 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:55 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/997959083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-25T06:08:56.016 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:55 smithi203 ceph-mon[36030]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-25T06:08:56.017 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:55 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/997959083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-25T06:08:56.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:08:56.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:56.067 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-05-25T06:08:56.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:56.084 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2024-05-25T06:08:56.139 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:08:56.139 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-25T06:08:56.139 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-25T06:08:56.139 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-25T06:08:56.139 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-25T06:08:56.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:08:56.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:08:56.257 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-25T06:08:56.258 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-25T06:08:56.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:08:56.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:08:56.385 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:08:56.385 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-25T06:08:56.385 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.0 up 2024-05-25T06:08:56.385 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.0 master ceph-brx 2024-05-25T06:08:56.385 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-25T06:08:56.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:08:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:08:56.463 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-25T06:08:56.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-25T06:08:56.463 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-25T06:08:56.478 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-25T06:08:56.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-25T06:08:56.478 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-25T06:08:56.533 DEBUG:teuthology.orchestra.run.smithi160:> sudo modprobe fuse 2024-05-25T06:08:56.598 DEBUG:teuthology.orchestra.run.smithi160:> cat /proc/self/mounts | awk '{print $2}' 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/proc 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/sys 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/dev 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/security 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/shm 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/pts 2024-05-25T06:08:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/run 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/cgroup 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/pstore 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/bpf 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/ 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/selinux 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/mqueue 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/debug 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/hugepages 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/tracing 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/fuse/connections 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/config 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-sysctl.service 2024-05-25T06:08:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/nfs/rpc_pipefs 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/1000 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/392280ee06daf717eaff1ac3eebc2c9ca71014016d600c5251dbf4e95443804b/merged 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/e787375a5a554c8b1c3b079a992e8f663a3cbd4d82eff2f59aa559893e17357b/merged 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/0 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/ad73a128d296dd983c427798b6b622a72f8537ccbe2a904a1c3c18a0875c6532/merged 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/ba0d2171d041070af07d1a367f0d557d4fa1058d64f19ecb8f6dd791f5efe046/merged 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/c851468cdd7338db5b0e5a6898b30297ff67e270977cfd382059acdfb9008566/merged 2024-05-25T06:08:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/f0379164b4067be880beaa32f5c695a7dcc884b05a0ac5bfe6223de5a7cd6a00/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/dcc666dc1f5a4458f1efddb5de94c5ea9293fe1ca2f88f1f019d114b2b2e167f/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/00bd41ca15a076a12ef27e9919c29e8c94946d7f9f08837d68e3f8e89dd611a8/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/129524c7318013e407cf993694960ab5465afd7da4c1859980a98832375be27c/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/5d117040339fbe33e8bea295595db52a228b89070ca39a1c2b74fc43187d7474/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/8c945a0a2e3aeb1f8fa9a5c53e8a415c6105085176b4eaa470b9c83216decc1e/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/c9a220776e59ac47df95577bfebc1f5dfa4462f0c8294cadf8eac6bdc4c0d90e/merged 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-25T06:08:56.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:56.659 DEBUG:teuthology.orchestra.run.smithi160:> ls /sys/fs/fuse/connections 2024-05-25T06:08:56.710 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-25T06:08:56.711 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-25T06:08:56.753 DEBUG:teuthology.orchestra.run.smithi160:> sudo modprobe fuse 2024-05-25T06:08:56.777 DEBUG:teuthology.orchestra.run.smithi160:> cat /proc/self/mounts | awk '{print $2}' 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/proc 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/sys 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/dev 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/security 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/shm 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/pts 2024-05-25T06:08:56.834 INFO:teuthology.orchestra.run.smithi160.stdout:/run 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/cgroup 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/pstore 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/bpf 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/ 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/selinux 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/mqueue 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/debug 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/hugepages 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/tracing 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/fuse/connections 2024-05-25T06:08:56.835 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/config 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-sysctl.service 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/nfs/rpc_pipefs 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/1000 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/392280ee06daf717eaff1ac3eebc2c9ca71014016d600c5251dbf4e95443804b/merged 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/e787375a5a554c8b1c3b079a992e8f663a3cbd4d82eff2f59aa559893e17357b/merged 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/0 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/ad73a128d296dd983c427798b6b622a72f8537ccbe2a904a1c3c18a0875c6532/merged 2024-05-25T06:08:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/ba0d2171d041070af07d1a367f0d557d4fa1058d64f19ecb8f6dd791f5efe046/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/c851468cdd7338db5b0e5a6898b30297ff67e270977cfd382059acdfb9008566/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/f0379164b4067be880beaa32f5c695a7dcc884b05a0ac5bfe6223de5a7cd6a00/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/dcc666dc1f5a4458f1efddb5de94c5ea9293fe1ca2f88f1f019d114b2b2e167f/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/00bd41ca15a076a12ef27e9919c29e8c94946d7f9f08837d68e3f8e89dd611a8/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/129524c7318013e407cf993694960ab5465afd7da4c1859980a98832375be27c/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/5d117040339fbe33e8bea295595db52a228b89070ca39a1c2b74fc43187d7474/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/8c945a0a2e3aeb1f8fa9a5c53e8a415c6105085176b4eaa470b9c83216decc1e/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/containers/storage/overlay/c9a220776e59ac47df95577bfebc1f5dfa4462f0c8294cadf8eac6bdc4c0d90e/merged 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-25T06:08:56.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:56.838 DEBUG:teuthology.orchestra.run.smithi160:> ls /sys/fs/fuse/connections 2024-05-25T06:08:56.870 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:2024-05-25T06:08:56.870+0000 7ff8e0ac3300 -1 init, newargv = 0x56366fd9d770 newargc=15 2024-05-25T06:08:56.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:ceph-fuse[76154]: starting ceph client 2024-05-25T06:08:56.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:ceph-fuse[76154]: starting fuse 2024-05-25T06:08:56.886 INFO:teuthology.orchestra.run.smithi160.stdout:152 2024-05-25T06:08:56.886 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [152] 2024-05-25T06:08:56.890 DEBUG:teuthology.orchestra.run.smithi160:> sudo stdin-killer -- python3 -c ' 2024-05-25T06:08:56.890 DEBUG:teuthology.orchestra.run.smithi160:> import glob 2024-05-25T06:08:56.890 DEBUG:teuthology.orchestra.run.smithi160:> import re 2024-05-25T06:08:56.890 DEBUG:teuthology.orchestra.run.smithi160:> import os 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> import subprocess 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> def _find_admin_socket(client_name): 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> files = glob.glob(asok_path) 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-25T06:08:56.891 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> # Given a non-glob path, it better be there 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> if "*" not in asok_path: 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> assert(len(files) == 1) 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> return files[0] 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> for f in files: 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-25T06:08:56.892 DEBUG:teuthology.orchestra.run.smithi160:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-25T06:08:56.893 DEBUG:teuthology.orchestra.run.smithi160:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-25T06:08:56.893 DEBUG:teuthology.orchestra.run.smithi160:> contents = proc_f.read() 2024-05-25T06:08:56.893 DEBUG:teuthology.orchestra.run.smithi160:> if mountpoint in contents: 2024-05-25T06:08:56.893 DEBUG:teuthology.orchestra.run.smithi160:> return f 2024-05-25T06:08:56.894 DEBUG:teuthology.orchestra.run.smithi160:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-25T06:08:56.894 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-25T06:08:56.894 DEBUG:teuthology.orchestra.run.smithi160:> print(_find_admin_socket("client.0")) 2024-05-25T06:08:56.894 DEBUG:teuthology.orchestra.run.smithi160:> ' 2024-05-25T06:08:57.004 INFO:teuthology.orchestra.run.smithi160.stdout:/var/run/ceph/ceph-client.0.76154.asok 2024-05-25T06:08:57.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:08:57.015 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76154.asok 2024-05-25T06:08:57.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:57.015 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76154.asok status 2024-05-25T06:08:57.148 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:08:57.148 INFO:teuthology.orchestra.run.smithi160.stdout: "metadata": { 2024-05-25T06:08:57.148 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_sha1": "7b01852c69b416ba2d4f5e12d093f476d16043d8", 2024-05-25T06:08:57.148 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)", 2024-05-25T06:08:57.148 INFO:teuthology.orchestra.run.smithi160.stdout: "entity_id": "0", 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": "smithi160", 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "pid": "76154", 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "root": "/" 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "dentry_count": 0, 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "dentry_pinned_count": 0, 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 14534, 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "inst": { 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "name": { 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "client", 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "num": 14534 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": { 2024-05-25T06:08:57.149 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "192.168.0.1:0", 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 1579019524 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": { 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "192.168.0.1:0", 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 1579019524 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "inst_str": "client.14534 192.168.0.1:0/1579019524", 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "addr_str": "192.168.0.1:0/1579019524", 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "inode_count": 1, 2024-05-25T06:08:57.150 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_epoch": 22, 2024-05-25T06:08:57.151 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_epoch": 46, 2024-05-25T06:08:57.151 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_epoch_barrier": 0, 2024-05-25T06:08:57.151 INFO:teuthology.orchestra.run.smithi160.stdout: "blocklisted": false, 2024-05-25T06:08:57.151 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs" 2024-05-25T06:08:57.151 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:08:57.157 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-25T06:08:57.414 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:08:57.853 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:57 smithi160 ceph-mon[29041]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-25T06:08:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:57 smithi203 ceph-mon[36030]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-25T06:08:59.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 -- 172.21.15.160:0/531013688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc1026a0 msgr2=0x7f00fc0fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:59.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 --2- 172.21.15.160:0/531013688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc1026a0 0x7f00fc0fbd10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00f0004940 tx=0x7f00f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:59.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 -- 172.21.15.160:0/531013688 shutdown_connections 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 --2- 172.21.15.160:0/531013688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc1026a0 0x7f00fc0fbd10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 --2- 172.21.15.160:0/531013688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc101d00 0x7f00fc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 -- 172.21.15.160:0/531013688 >> 172.21.15.160:0/531013688 conn(0x7f00fc0f7870 msgr2=0x7f00fc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 -- 172.21.15.160:0/531013688 shutdown_connections 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 -- 172.21.15.160:0/531013688 wait complete. 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.058+0000 7f0101e3e700 1 Processor -- start 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f0101e3e700 1 -- start start 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f0101e3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 0x7f00fc06a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f0101e3e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc1026a0 0x7f00fc06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f0101e3e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc06b200 con 0x7f00fc101d00 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f0101e3e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc11ef30 con 0x7f00fc1026a0 2024-05-25T06:08:59.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00fb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 0x7f00fc06a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00fb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 0x7f00fc06a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45940/0 (socket says 172.21.15.160:45940) 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00fb7fe700 1 -- 172.21.15.160:0/2791733482 learned_addr learned my addr 172.21.15.160:0/2791733482 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00faffd700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc1026a0 0x7f00fc06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00fb7fe700 1 -- 172.21.15.160:0/2791733482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc1026a0 msgr2=0x7f00fc06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00fb7fe700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc1026a0 0x7f00fc06abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00fb7fe700 1 -- 172.21.15.160:0/2791733482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f0005040 con 0x7f00fc101d00 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.059+0000 7f00faffd700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc1026a0 0x7f00fc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.060+0000 7f00fb7fe700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 0x7f00fc06a650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00c830 tx=0x7f00ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.060+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00ec00a760 con 0x7f00fc101d00 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.060+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00ec00ad90 con 0x7f00fc101d00 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.060+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc11f130 con 0x7f00fc101d00 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.060+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00ec0113a0 con 0x7f00fc101d00 2024-05-25T06:08:59.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.060+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc11f620 con 0x7f00fc101d00 2024-05-25T06:08:59.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.061+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc0fd450 con 0x7f00fc101d00 2024-05-25T06:08:59.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.061+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f00ec011610 con 0x7f00fc101d00 2024-05-25T06:08:59.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.062+0000 7f00e3fff700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f00e40692a0 0x7f00e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:08:59.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.062+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f00ec087260 con 0x7f00fc101d00 2024-05-25T06:08:59.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.062+0000 7f00faffd700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f00e40692a0 0x7f00e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:08:59.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.062+0000 7f00faffd700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f00e40692a0 0x7f00e406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00fc06bbc0 tx=0x7f00f0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:08:59.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.066+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f00ec056fd0 con 0x7f00fc101d00 2024-05-25T06:08:59.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.296+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f00fc11f8a0 con 0x7f00fc101d00 2024-05-25T06:08:59.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.296+0000 7f00e3fff700 1 -- 172.21.15.160:0/2791733482 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+83 (secure 0 0 0) 0x7f00ec056b60 con 0x7f00fc101d00 2024-05-25T06:08:59.297 INFO:teuthology.orchestra.run.smithi160.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-25T06:08:59.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f00e40692a0 msgr2=0x7f00e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f00e40692a0 0x7f00e406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00fc06bbc0 tx=0x7f00f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 msgr2=0x7f00fc06a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 0x7f00fc06a650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00c830 tx=0x7f00ec004470 comp rx=0 tx=0).stop 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 shutdown_connections 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f00e40692a0 0x7f00e406b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00fc101d00 0x7f00fc06a650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 --2- 172.21.15.160:0/2791733482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc1026a0 0x7f00fc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.298+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 >> 172.21.15.160:0/2791733482 conn(0x7f00fc0f7870 msgr2=0x7f00fc0f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.299+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 shutdown_connections 2024-05-25T06:08:59.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:08:59.299+0000 7f0101e3e700 1 -- 172.21.15.160:0/2791733482 wait complete. 2024-05-25T06:08:59.888 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:59 smithi160 ceph-mon[29041]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:08:59.888 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:08:59 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2791733482' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-25T06:08:59.915 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-25T06:08:59.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:08:59.916 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-05-25T06:08:59.935 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-25T06:08:59.935 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-25T06:08:59.935 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-05-25T06:08:59.935 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-05-25T06:08:59.935 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-05-25T06:08:59.935 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42249sec preferred_lft 42249sec 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-05-25T06:08:59.936 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-05-25T06:08:59.937 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-25T06:08:59.937 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:08:59.937 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-05-25T06:08:59.937 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2024-05-25T06:08:59.937 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2024-05-25T06:08:59.937 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2024-05-25T06:08:59.938 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-25T06:08:59.938 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-05-25T06:09:00.033 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:09:00.042 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:59 smithi203 ceph-mon[36030]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:09:00.042 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:08:59 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2791733482' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-25T06:09:00.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:09:00.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:00.150 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-25T06:09:00.179 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-05-25T06:09:00.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:00.180 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-05-25T06:09:00.244 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-05-25T06:09:00.244 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-25T06:09:00.244 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-25T06:09:00.244 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-25T06:09:00.244 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-25T06:09:00.244 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:09:00.244 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-05-25T06:09:00.245 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-25T06:09:00.245 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-25T06:09:00.245 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-25T06:09:00.245 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-05-25T06:09:00.332 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:09:00.534 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:09:00.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:00.541 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-05-25T06:09:00.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:00.557 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-05-25T06:09:00.613 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:09:00.613 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-05-25T06:09:00.613 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:00.613 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-25T06:09:00.613 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-05-25T06:09:00.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:09:00.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:09:00.734 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-25T06:09:00.734 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:09:00.734 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-05-25T06:09:00.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-25T06:09:00.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-25T06:09:00.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-25T06:09:00.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-25T06:09:00.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-25T06:09:00.735 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-05-25T06:09:00.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:09:00.858 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:09:00.864 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-25T06:09:00.864 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-05-25T06:09:00.864 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-05-25T06:09:00.864 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2024-05-25T06:09:00.864 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-05-25T06:09:00.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-25T06:09:00.935 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:09:00.942 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-25T06:09:00.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-25T06:09:00.942 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-25T06:09:00.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-25T06:09:00.959 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:01.013 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-05-25T06:09:01.080 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-05-25T06:09:01.136 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-05-25T06:09:01.137 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-05-25T06:09:01.138 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-05-25T06:09:01.139 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-05-25T06:09:01.139 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/0 2024-05-25T06:09:01.139 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:09:01.139 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay 2024-05-25T06:09:01.139 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/d6d3d333847f7ebdefe2d8accf5121381303bce7e1292ec520c6865d8d68435d/merged 2024-05-25T06:09:01.139 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/5d5a0be17b581b7cf5e138df5460b7acb1ac1d069bf6b8dd01d052a077dcea62/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/522a20c919ac941a5d072dee88ee915657f84c5e21599d0fda1f503926c9313b/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/e59da2f912900bb37ebbb8b54d5daa2010016d467eaee75c51fb3a73c3f919d8/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/2872f961082dfd118dfe5d1895be432e80b3ad7293c3ed8d8250987af6758097/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/2d2ccf5be046f068f560568ae72b00e2fe67328ad2eef0bf359e0a3d92df66d2/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/80f2299afccfe80edc29d3b3ac090f9636d03e63a921988851ae09e9b570bb6a/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/b4bd915aeefdea07f98742e9e525ca1a88520812a5fd2bd02d841552af9732a5/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/a9b8c22b6a87ec4e9a744075f909ffc638e4d71ccbd9332aad60629729afea58/merged 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:01.140 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:01.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:01.141 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-05-25T06:09:01.197 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-25T06:09:01.197 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-25T06:09:01.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-05-25T06:09:01.264 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-05-25T06:09:01.319 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-05-25T06:09:01.320 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/0 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/d6d3d333847f7ebdefe2d8accf5121381303bce7e1292ec520c6865d8d68435d/merged 2024-05-25T06:09:01.321 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/5d5a0be17b581b7cf5e138df5460b7acb1ac1d069bf6b8dd01d052a077dcea62/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/522a20c919ac941a5d072dee88ee915657f84c5e21599d0fda1f503926c9313b/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/e59da2f912900bb37ebbb8b54d5daa2010016d467eaee75c51fb3a73c3f919d8/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/2872f961082dfd118dfe5d1895be432e80b3ad7293c3ed8d8250987af6758097/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/2d2ccf5be046f068f560568ae72b00e2fe67328ad2eef0bf359e0a3d92df66d2/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/80f2299afccfe80edc29d3b3ac090f9636d03e63a921988851ae09e9b570bb6a/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/b4bd915aeefdea07f98742e9e525ca1a88520812a5fd2bd02d841552af9732a5/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/a9b8c22b6a87ec4e9a744075f909ffc638e4d71ccbd9332aad60629729afea58/merged 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:01.322 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:01.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:01.323 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-05-25T06:09:01.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:2024-05-25T06:09:01.379+0000 7f357e4b1300 -1 init, newargv = 0x559211d61850 newargc=15 2024-05-25T06:09:01.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[61779]: starting ceph client 2024-05-25T06:09:01.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[61779]: starting fuse 2024-05-25T06:09:01.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:01 smithi203 ceph-mon[36030]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 3 op/s 2024-05-25T06:09:01.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:01 smithi160 ceph-mon[29041]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 3 op/s 2024-05-25T06:09:02.481 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-05-25T06:09:02.514 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-05-25T06:09:02.573 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-05-25T06:09:02.573 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-05-25T06:09:02.573 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-05-25T06:09:02.574 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-05-25T06:09:02.574 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-05-25T06:09:02.574 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-05-25T06:09:02.574 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-05-25T06:09:02.574 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-05-25T06:09:02.574 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-05-25T06:09:02.575 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-05-25T06:09:02.575 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-05-25T06:09:02.575 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-05-25T06:09:02.575 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:09:02.575 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-05-25T06:09:02.576 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-05-25T06:09:02.576 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-05-25T06:09:02.576 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-05-25T06:09:02.576 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-25T06:09:02.576 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-05-25T06:09:02.577 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-05-25T06:09:02.577 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-05-25T06:09:02.577 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-25T06:09:02.577 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-05-25T06:09:02.577 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-05-25T06:09:02.578 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-05-25T06:09:02.578 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/0 2024-05-25T06:09:02.578 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-05-25T06:09:02.578 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay 2024-05-25T06:09:02.578 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/d6d3d333847f7ebdefe2d8accf5121381303bce7e1292ec520c6865d8d68435d/merged 2024-05-25T06:09:02.579 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/5d5a0be17b581b7cf5e138df5460b7acb1ac1d069bf6b8dd01d052a077dcea62/merged 2024-05-25T06:09:02.579 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/522a20c919ac941a5d072dee88ee915657f84c5e21599d0fda1f503926c9313b/merged 2024-05-25T06:09:02.579 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/e59da2f912900bb37ebbb8b54d5daa2010016d467eaee75c51fb3a73c3f919d8/merged 2024-05-25T06:09:02.579 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/2872f961082dfd118dfe5d1895be432e80b3ad7293c3ed8d8250987af6758097/merged 2024-05-25T06:09:02.579 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/2d2ccf5be046f068f560568ae72b00e2fe67328ad2eef0bf359e0a3d92df66d2/merged 2024-05-25T06:09:02.579 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/80f2299afccfe80edc29d3b3ac090f9636d03e63a921988851ae09e9b570bb6a/merged 2024-05-25T06:09:02.580 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/b4bd915aeefdea07f98742e9e525ca1a88520812a5fd2bd02d841552af9732a5/merged 2024-05-25T06:09:02.580 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/containers/storage/overlay/a9b8c22b6a87ec4e9a744075f909ffc638e4d71ccbd9332aad60629729afea58/merged 2024-05-25T06:09:02.580 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-05-25T06:09:02.580 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:02.580 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-25T06:09:02.580 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:02.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:02.581 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-05-25T06:09:02.631 INFO:teuthology.orchestra.run.smithi203.stdout:116 2024-05-25T06:09:02.631 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [116] 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> import re 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> import os 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2024-05-25T06:09:02.632 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> return f 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> 2024-05-25T06:09:02.633 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.1")) 2024-05-25T06:09:02.634 DEBUG:teuthology.orchestra.run.smithi203:> ' 2024-05-25T06:09:02.755 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.1.61779.asok 2024-05-25T06:09:02.758 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-25T06:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-25T06:09:02.765 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61779.asok 2024-05-25T06:09:02.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:02.766 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61779.asok status 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "7b01852c69b416ba2d4f5e12d093f476d16043d8", 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)", 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "1", 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-25T06:09:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "61779", 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 24325, 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 24325 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2024-05-25T06:09:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 2204786649 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 2204786649 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.24325 192.168.0.1:0/2204786649", 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.1:0/2204786649", 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 22, 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 46, 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2024-05-25T06:09:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2024-05-25T06:09:02.899 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2024-05-25T06:09:02.899 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-05-25T06:09:02.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:02.904 DEBUG:teuthology.orchestra.run.smithi160:> stat --file-system '--printf=%T 2024-05-25T06:09:02.904 DEBUG:teuthology.orchestra.run.smithi160:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-25T06:09:02.922 INFO:teuthology.orchestra.run.smithi160.stdout:fuseblk 2024-05-25T06:09:02.923 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-25T06:09:02.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:02.923 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-25T06:09:02.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:02.993 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2024-05-25T06:09:02.993 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:03.009 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2024-05-25T06:09:03.009 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:03.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:09:03.009 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:03.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:03 smithi160 ceph-mon[29041]: pgmap v196: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-25T06:09:04.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:03 smithi203 ceph-mon[36030]: pgmap v196: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-25T06:09:05.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:05 smithi160 ceph-mon[29041]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-25T06:09:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:05 smithi203 ceph-mon[36030]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-25T06:09:06.461 INFO:teuthology.run_tasks:Running task print... 2024-05-25T06:09:06.471 INFO:teuthology.task.print:**** done client 2024-05-25T06:09:06.472 INFO:teuthology.run_tasks:Running task parallel... 2024-05-25T06:09:06.561 INFO:teuthology.task.parallel:starting parallel... 2024-05-25T06:09:06.561 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-25T06:09:06.562 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-25T06:09:06.562 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:09:06.562 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-25T06:09:06.563 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-25T06:09:06.563 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-25T06:09:06.567 INFO:tasks.workunit:Pulling workunits from ref 17c123e524b17eb0cf38bdb0a0ca89d80b36c131 2024-05-25T06:09:06.567 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-25T06:09:06.567 INFO:tasks.workunit:timeout=3h 2024-05-25T06:09:06.568 INFO:tasks.workunit:cleanup=True 2024-05-25T06:09:06.568 DEBUG:teuthology.orchestra.run.smithi160:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 98h/152d Inode: 1 Links: 2 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-25 06:08:25.982755687 +0000 2024-05-25T06:09:06.589 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-25 06:09:02.990387068 +0000 2024-05-25T06:09:06.590 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-25T06:09:06.590 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-25T06:09:06.590 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-25T06:09:06.662 DEBUG:teuthology.orchestra.run.smithi203:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 74h/116d Inode: 1 Links: 3 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-25 06:09:06.656801813 +0000 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-25 06:09:06.656801813 +0000 2024-05-25T06:09:06.684 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2024-05-25T06:09:06.685 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-25T06:09:06.685 DEBUG:teuthology.orchestra.run.smithi203:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-25T06:09:06.760 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 17c123e524b17eb0cf38bdb0a0ca89d80b36c131 2024-05-25T06:09:06.761 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 17c123e524b17eb0cf38bdb0a0ca89d80b36c131 2024-05-25T06:09:06.777 INFO:tasks.workunit.client.0.smithi160.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-25T06:09:06.819 INFO:tasks.workunit.client.1.smithi203.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-25T06:09:06.846 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:09:07.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:07 smithi160 ceph-mon[29041]: pgmap v198: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-25T06:09:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:07 smithi203 ceph-mon[36030]: pgmap v198: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-25T06:09:08.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 -- 172.21.15.160:0/1487740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 msgr2=0x7fe8a00fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 --2- 172.21.15.160:0/1487740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a00fd280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe890004940 tx=0x7fe890009e30 comp rx=0 tx=0).stop 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 -- 172.21.15.160:0/1487740018 shutdown_connections 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 --2- 172.21.15.160:0/1487740018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 0x7fe8a00fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 --2- 172.21.15.160:0/1487740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a00fd280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 -- 172.21.15.160:0/1487740018 >> 172.21.15.160:0/1487740018 conn(0x7fe8a00f8510 msgr2=0x7fe8a00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 -- 172.21.15.160:0/1487740018 shutdown_connections 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 -- 172.21.15.160:0/1487740018 wait complete. 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 Processor -- start 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.471+0000 7fe8a556e700 1 -- start start 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe8a556e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a010da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe8a556e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 0x7fe8a010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:08.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe8a556e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a010e5b0 con 0x7fe8a00fce70 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe8a556e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a010edb0 con 0x7fe8a00fdfe0 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a010da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a010da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57480/0 (socket says 172.21.15.160:57480) 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 -- 172.21.15.160:0/4053867621 learned_addr learned my addr 172.21.15.160:0/4053867621 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89e7fc700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 0x7fe8a010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 -- 172.21.15.160:0/4053867621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 msgr2=0x7fe8a010dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 0x7fe8a010dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 -- 172.21.15.160:0/4053867621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890005040 con 0x7fe8a00fce70 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89e7fc700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 0x7fe8a010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe89effd700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a010da70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe89000b3b0 tx=0x7fe890009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.472+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe89000e040 con 0x7fe8a00fce70 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.473+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a010f030 con 0x7fe8a00fce70 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.473+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe89000bcc0 con 0x7fe8a00fce70 2024-05-25T06:09:08.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.473+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe89001c630 con 0x7fe8a00fce70 2024-05-25T06:09:08.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.473+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a010f570 con 0x7fe8a00fce70 2024-05-25T06:09:08.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.474+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a005f5c0 con 0x7fe8a00fce70 2024-05-25T06:09:08.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.474+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe890013980 con 0x7fe8a00fce70 2024-05-25T06:09:08.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.474+0000 7fe8877fe700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8880692a0 0x7fe88806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:08.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.475+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe890088020 con 0x7fe8a00fce70 2024-05-25T06:09:08.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.475+0000 7fe89e7fc700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8880692a0 0x7fe88806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:08.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.475+0000 7fe89e7fc700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8880692a0 0x7fe88806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe894009fd0 tx=0x7fe894009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:08.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.479+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe890057d10 con 0x7fe8a00fce70 2024-05-25T06:09:08.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.669+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fe8a004bed0 con 0x7fe8a00fce70 2024-05-25T06:09:08.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.670+0000 7fe8877fe700 1 -- 172.21.15.160:0/4053867621 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe8900578a0 con 0x7fe8a00fce70 2024-05-25T06:09:08.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8880692a0 msgr2=0x7fe88806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:08.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8880692a0 0x7fe88806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe894009fd0 tx=0x7fe894009ee0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 msgr2=0x7fe8a010da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:08.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a010da70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe89000b3b0 tx=0x7fe890009e30 comp rx=0 tx=0).stop 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 shutdown_connections 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fe8880692a0 0x7fe88806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a00fce70 0x7fe8a010da70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 --2- 172.21.15.160:0/4053867621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a00fdfe0 0x7fe8a010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 >> 172.21.15.160:0/4053867621 conn(0x7fe8a00f8510 msgr2=0x7fe8a00f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 shutdown_connections 2024-05-25T06:09:08.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:08.673+0000 7fe8a556e700 1 -- 172.21.15.160:0/4053867621 wait complete. 2024-05-25T06:09:08.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:09.352 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-25T06:09:09.654 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:09:09.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:09 smithi160 ceph-mon[29041]: pgmap v199: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:10.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:09 smithi203 ceph-mon[36030]: pgmap v199: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:11.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2917917238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 msgr2=0x7f0ae00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:11.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2917917238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae00fd3d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0acc004940 tx=0x7f0acc009e30 comp rx=0 tx=0).stop 2024-05-25T06:09:11.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2917917238 shutdown_connections 2024-05-25T06:09:11.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2917917238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae00fd3d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:11.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2917917238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 0x7f0ae00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2917917238 >> 172.21.15.160:0/2917917238 conn(0x7f0ae00f7430 msgr2=0x7f0ae00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2917917238 shutdown_connections 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.307+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2917917238 wait complete. 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae6cf8700 1 Processor -- start 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae6cf8700 1 -- start start 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae6cf8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 0x7f0ae010c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae6cf8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae010ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae6cf8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae010d4d0 con 0x7f0ae00fcf60 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae6cf8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae010dcc0 con 0x7f0ae00fbcb0 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0adffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae010ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0adffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae010ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57490/0 (socket says 172.21.15.160:57490) 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0adffff700 1 -- 172.21.15.160:0/2531790754 learned_addr learned my addr 172.21.15.160:0/2531790754 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae4a94700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 0x7f0ae010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:11.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0adffff700 1 -- 172.21.15.160:0/2531790754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 msgr2=0x7f0ae010c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0adffff700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 0x7f0ae010c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0adffff700 1 -- 172.21.15.160:0/2531790754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc005040 con 0x7f0ae00fcf60 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.308+0000 7f0ae4a94700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 0x7f0ae010c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.309+0000 7f0adffff700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae010ce40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009150 tx=0x7f0acc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.309+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc00e040 con 0x7f0ae00fcf60 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.309+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0acc00bcc0 con 0x7f0ae00fcf60 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.309+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae010df40 con 0x7f0ae00fcf60 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.309+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc01c630 con 0x7f0ae00fcf60 2024-05-25T06:09:11.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.309+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae010e500 con 0x7f0ae00fcf60 2024-05-25T06:09:11.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.310+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae005f5c0 con 0x7f0ae00fcf60 2024-05-25T06:09:11.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.311+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0acc013420 con 0x7f0ae00fcf60 2024-05-25T06:09:11.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.311+0000 7f0add7fa700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0ad00692a0 0x7f0ad006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:11.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.311+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0acc087fc0 con 0x7f0ae00fcf60 2024-05-25T06:09:11.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.311+0000 7f0ae4a94700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0ad00692a0 0x7f0ad006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:11.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.311+0000 7f0ae4a94700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0ad00692a0 0x7f0ad006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4006fd0 tx=0x7f0ad4008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:11.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.315+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0acc057d30 con 0x7f0ae00fcf60 2024-05-25T06:09:11.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.505+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f0ae004bed0 con 0x7f0ae00fcf60 2024-05-25T06:09:11.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.505+0000 7f0add7fa700 1 -- 172.21.15.160:0/2531790754 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0acc0578c0 con 0x7f0ae00fcf60 2024-05-25T06:09:11.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0ad00692a0 msgr2=0x7f0ad006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:11.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0ad00692a0 0x7f0ad006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4006fd0 tx=0x7f0ad4008040 comp rx=0 tx=0).stop 2024-05-25T06:09:11.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 msgr2=0x7f0ae010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:11.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae010ce40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009150 tx=0x7f0acc009e30 comp rx=0 tx=0).stop 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 shutdown_connections 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f0ad00692a0 0x7f0ad006b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ae00fbcb0 0x7f0ae010c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 --2- 172.21.15.160:0/2531790754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ae00fcf60 0x7f0ae010ce40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 >> 172.21.15.160:0/2531790754 conn(0x7f0ae00f7430 msgr2=0x7f0ae00f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 shutdown_connections 2024-05-25T06:09:11.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:11.507+0000 7f0ae6cf8700 1 -- 172.21.15.160:0/2531790754 wait complete. 2024-05-25T06:09:11.865 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:11 smithi160 ceph-mon[29041]: pgmap v200: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:12.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:11 smithi203 ceph-mon[36030]: pgmap v200: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:12.285 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-25T06:09:12.549 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:09:13.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:13 smithi160 ceph-mon[29041]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-25T06:09:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:13 smithi203 ceph-mon[36030]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-25T06:09:14.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 -- 172.21.15.160:0/2524119254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fbcc0 msgr2=0x7f90500fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 --2- 172.21.15.160:0/2524119254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fbcc0 0x7f90500fc0d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9044004940 tx=0x7f9044009e30 comp rx=0 tx=0).stop 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 -- 172.21.15.160:0/2524119254 shutdown_connections 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 --2- 172.21.15.160:0/2524119254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90500fcf70 0x7f90500fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 --2- 172.21.15.160:0/2524119254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fbcc0 0x7f90500fc0d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 -- 172.21.15.160:0/2524119254 >> 172.21.15.160:0/2524119254 conn(0x7f90500f7420 msgr2=0x7f90500f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 -- 172.21.15.160:0/2524119254 shutdown_connections 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.284+0000 7f9056d77700 1 -- 172.21.15.160:0/2524119254 wait complete. 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9056d77700 1 Processor -- start 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9056d77700 1 -- start start 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9056d77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90500fbcc0 0x7f90500ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9056d77700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 0x7f90500fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:14.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9056d77700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500fe7f0 con 0x7f90500fcf70 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9056d77700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500fe930 con 0x7f90500fbcc0 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f904ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 0x7f90500fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f904ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 0x7f90500fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57500/0 (socket says 172.21.15.160:57500) 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f904ffff700 1 -- 172.21.15.160:0/1810923738 learned_addr learned my addr 172.21.15.160:0/1810923738 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f9054b13700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90500fbcc0 0x7f90500ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f904ffff700 1 -- 172.21.15.160:0/1810923738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90500fbcc0 msgr2=0x7f90500ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f904ffff700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90500fbcc0 0x7f90500ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.285+0000 7f904ffff700 1 -- 172.21.15.160:0/1810923738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044005040 con 0x7f90500fcf70 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.286+0000 7f904ffff700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 0x7f90500fe2b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f903c002a50 tx=0x7f903c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.286+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903c007cc0 con 0x7f90500fcf70 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.286+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f903c00f040 con 0x7f90500fcf70 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.286+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90500fec10 con 0x7f90500fcf70 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.286+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903c00da30 con 0x7f90500fcf70 2024-05-25T06:09:14.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.286+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9050115af0 con 0x7f90500fcf70 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.287+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050101b00 con 0x7f90500fcf70 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.287+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f903c00db90 con 0x7f90500fcf70 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.288+0000 7f904d7fa700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f90400692a0 0x7f904006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.288+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f903c086ec0 con 0x7f90500fcf70 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.288+0000 7f9054b13700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f90400692a0 0x7f904006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.288+0000 7f9054b13700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f90400692a0 0x7f904006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9044004890 tx=0x7f9044009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:14.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.292+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f903c0531a0 con 0x7f90500fcf70 2024-05-25T06:09:14.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.482+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f905005f5c0 con 0x7f90500fcf70 2024-05-25T06:09:14.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.482+0000 7f904d7fa700 1 -- 172.21.15.160:0/1810923738 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f903c0567c0 con 0x7f90500fcf70 2024-05-25T06:09:14.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f90400692a0 msgr2=0x7f904006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:14.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f90400692a0 0x7f904006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9044004890 tx=0x7f9044009e30 comp rx=0 tx=0).stop 2024-05-25T06:09:14.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 msgr2=0x7f90500fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:14.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 0x7f90500fe2b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f903c002a50 tx=0x7f903c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 shutdown_connections 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f90400692a0 0x7f904006b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90500fbcc0 0x7f90500ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 --2- 172.21.15.160:0/1810923738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90500fcf70 0x7f90500fe2b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 >> 172.21.15.160:0/1810923738 conn(0x7f90500f7420 msgr2=0x7f9050066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 shutdown_connections 2024-05-25T06:09:14.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:14.485+0000 7f9056d77700 1 -- 172.21.15.160:0/1810923738 wait complete. 2024-05-25T06:09:15.133 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-05-25T06:09:15.388 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:09:15.784 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:15 smithi160 ceph-mon[29041]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 4 op/s 2024-05-25T06:09:15.951 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:15 smithi203 ceph-mon[36030]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 4 op/s 2024-05-25T06:09:16.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 -- 172.21.15.160:0/105348159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 msgr2=0x7fb2580fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:16.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 --2- 172.21.15.160:0/105348159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb2580fe7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb254009ce0 tx=0x7fb254004d10 comp rx=0 tx=0).stop 2024-05-25T06:09:16.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 -- 172.21.15.160:0/105348159 shutdown_connections 2024-05-25T06:09:16.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 --2- 172.21.15.160:0/105348159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580fed30 0x7fb2580fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:16.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 --2- 172.21.15.160:0/105348159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb2580fe7f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:16.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 -- 172.21.15.160:0/105348159 >> 172.21.15.160:0/105348159 conn(0x7fb2580f6b80 msgr2=0x7fb2580f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 -- 172.21.15.160:0/105348159 shutdown_connections 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.979+0000 7fb260068700 1 -- 172.21.15.160:0/105348159 wait complete. 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb260068700 1 Processor -- start 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb260068700 1 -- start start 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb260068700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb25810dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb260068700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580fed30 0x7fb25810e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb260068700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25810e620 con 0x7fb2580fe3e0 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb260068700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25810ee70 con 0x7fb2580fed30 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb25de04700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb25810dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb25de04700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb25810dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55490/0 (socket says 172.21.15.160:55490) 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb25de04700 1 -- 172.21.15.160:0/2244923746 learned_addr learned my addr 172.21.15.160:0/2244923746 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:09:16.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.980+0000 7fb25d603700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580fed30 0x7fb25810e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb25de04700 1 -- 172.21.15.160:0/2244923746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580fed30 msgr2=0x7fb25810e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb25de04700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580fed30 0x7fb25810e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb25de04700 1 -- 172.21.15.160:0/2244923746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009940 con 0x7fb2580fe3e0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb25de04700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb25810dae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb254009380 tx=0x7fb254004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb25400e040 con 0x7fb2580fe3e0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2540128e0 con 0x7fb2580fe3e0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb25401c930 con 0x7fb2580fe3e0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25810f0f0 con 0x7fb2580fe3e0 2024-05-25T06:09:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.981+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25810f630 con 0x7fb2580fe3e0 2024-05-25T06:09:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.982+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25805f5c0 con 0x7fb2580fe3e0 2024-05-25T06:09:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.983+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb254012a50 con 0x7fb2580fe3e0 2024-05-25T06:09:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.983+0000 7fb24e7fc700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2440692a0 0x7fb24406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:09:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.983+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb254088c90 con 0x7fb2580fe3e0 2024-05-25T06:09:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.983+0000 7fb25d603700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2440692a0 0x7fb24406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:09:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.983+0000 7fb25d603700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2440692a0 0x7fb24406b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb248009fd0 tx=0x7fb24800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:09:16.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:16.987+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb254058a00 con 0x7fb2580fe3e0 2024-05-25T06:09:17.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:17.188+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fb25810fa10 con 0x7fb2440692a0 2024-05-25T06:09:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:17 smithi160 ceph-mon[29041]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-25T06:09:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:17 smithi203 ceph-mon[36030]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-25T06:09:18.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:18 smithi160 ceph-mon[29041]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:09:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:18 smithi203 ceph-mon[36030]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:09:19.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:19 smithi160 ceph-mon[29041]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:19 smithi203 ceph-mon[36030]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:21.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:21 smithi160 ceph-mon[29041]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:21 smithi203 ceph-mon[36030]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:23.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:23 smithi160 ceph-mon[29041]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-25T06:09:23.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:23 smithi203 ceph-mon[36030]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-25T06:09:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:25.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:25 smithi160 ceph-mon[29041]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:26.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:25 smithi203 ceph-mon[36030]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:09:27.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:27 smithi160 ceph-mon[29041]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-25T06:09:28.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:27 smithi203 ceph-mon[36030]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-25T06:09:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:29 smithi160 ceph-mon[29041]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:09:30.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:29 smithi203 ceph-mon[36030]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:09:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:31 smithi160 ceph-mon[29041]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-25T06:09:32.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:31 smithi203 ceph-mon[36030]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-25T06:09:33.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:33 smithi160 ceph-mon[29041]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-25T06:09:34.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:33 smithi203 ceph-mon[36030]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-25T06:09:35.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:35 smithi160 ceph-mon[29041]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:36.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:35 smithi203 ceph-mon[36030]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:37.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:37 smithi160 ceph-mon[29041]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:37 smithi203 ceph-mon[36030]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:38.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:38 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:39.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:38 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:39.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:39 smithi160 ceph-mon[29041]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:09:40.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:39 smithi203 ceph-mon[36030]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:09:41.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:41 smithi160 ceph-mon[29041]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:42.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:41 smithi203 ceph-mon[36030]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:44.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:43 smithi203 ceph-mon[36030]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:43 smithi160 ceph-mon[29041]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:45 smithi203 ceph-mon[36030]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:45 smithi160 ceph-mon[29041]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:48.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:47 smithi203 ceph-mon[36030]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:47 smithi160 ceph-mon[29041]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:49 smithi203 ceph-mon[36030]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:09:50.199 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:49 smithi160 ceph-mon[29041]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:09:51.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:50 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:09:51.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:50 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:09:52.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:51 smithi203 ceph-mon[36030]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:52.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:51 smithi160 ceph-mon[29041]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:54.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:53 smithi203 ceph-mon[36030]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:54.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:53 smithi160 ceph-mon[29041]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:09:56.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:55 smithi203 ceph-mon[36030]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:55 smithi160 ceph-mon[29041]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:09:57.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:57 smithi160 ceph-mon[29041]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:57.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:09:57.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:09:57.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:57 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:09:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:57 smithi203 ceph-mon[36030]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:09:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:09:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:09:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:57 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:09:58.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.839+0000 7fb24e7fc700 1 -- 172.21.15.160:0/2244923746 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb25810fa10 con 0x7fb2440692a0 2024-05-25T06:09:58.840 INFO:teuthology.orchestra.run.smithi160.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2440692a0 msgr2=0x7fb24406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2440692a0 0x7fb24406b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb248009fd0 tx=0x7fb24800b040 comp rx=0 tx=0).stop 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 msgr2=0x7fb25810dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb25810dae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb254009380 tx=0x7fb254004d10 comp rx=0 tx=0).stop 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 shutdown_connections 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb2440692a0 0x7fb24406b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2580fe3e0 0x7fb25810dae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:58.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 --2- 172.21.15.160:0/2244923746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580fed30 0x7fb25810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:09:58.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 >> 172.21.15.160:0/2244923746 conn(0x7fb2580f6b80 msgr2=0x7fb2580f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:09:58.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 shutdown_connections 2024-05-25T06:09:58.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:09:58.842+0000 7fb260068700 1 -- 172.21.15.160:0/2244923746 wait complete. 2024-05-25T06:09:59.535 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-05-25T06:09:59.857 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:10:00.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:10:00.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:10:00.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:00.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:10:00.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:10:00.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:10:00.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:00.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:09:59 smithi203 ceph-mon[36030]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:10:00.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:00.201 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:09:59 smithi160 ceph-mon[29041]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:10:01.115 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:00 smithi160 ceph-mon[29041]: overall HEALTH_OK 2024-05-25T06:10:01.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:00 smithi203 ceph-mon[36030]: overall HEALTH_OK 2024-05-25T06:10:01.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:01 smithi160 ceph-mon[29041]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:01 smithi203 ceph-mon[36030]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.240+0000 7f6cb45af700 1 -- 172.21.15.160:0/314410940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 msgr2=0x7f6cac0fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.240+0000 7f6cb45af700 1 --2- 172.21.15.160:0/314410940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac0fe7f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8004940 tx=0x7f6ca8009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.241+0000 7f6cb45af700 1 -- 172.21.15.160:0/314410940 shutdown_connections 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.241+0000 7f6cb45af700 1 --2- 172.21.15.160:0/314410940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cac0fed30 0x7f6cac0fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.241+0000 7f6cb45af700 1 --2- 172.21.15.160:0/314410940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac0fe7f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.241+0000 7f6cb45af700 1 -- 172.21.15.160:0/314410940 >> 172.21.15.160:0/314410940 conn(0x7f6cac0f6b80 msgr2=0x7f6cac0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.241+0000 7f6cb45af700 1 -- 172.21.15.160:0/314410940 shutdown_connections 2024-05-25T06:10:02.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.241+0000 7f6cb45af700 1 -- 172.21.15.160:0/314410940 wait complete. 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb45af700 1 Processor -- start 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb45af700 1 -- start start 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb45af700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac10d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb45af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cac0fed30 0x7f6cac10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb45af700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac10e320 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb45af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac10ead0 con 0x7f6cac0fed30 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb234b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac10d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb234b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac10d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54724/0 (socket says 172.21.15.160:54724) 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.242+0000 7f6cb234b700 1 -- 172.21.15.160:0/138061812 learned_addr learned my addr 172.21.15.160:0/138061812 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6cb1b4a700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cac0fed30 0x7f6cac10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6cb234b700 1 -- 172.21.15.160:0/138061812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cac0fed30 msgr2=0x7f6cac10dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6cb234b700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cac0fed30 0x7f6cac10dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6cb234b700 1 -- 172.21.15.160:0/138061812 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8005040 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6cb234b700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac10d7e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800b0d0 tx=0x7f6ca8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ca8021070 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ca801c8d0 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ca800e070 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.243+0000 7f6cb45af700 1 -- 172.21.15.160:0/138061812 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac10ed50 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.244+0000 7f6cb45af700 1 -- 172.21.15.160:0/138061812 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac10f230 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.245+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c940052b0 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.247+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6ca801c400 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.247+0000 7f6ca2ffd700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6c9806d780 0x7f6c9806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.248+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6ca8088ba0 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.248+0000 7f6cb1b4a700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6c9806d780 0x7f6c9806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.248+0000 7f6cb1b4a700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6c9806d780 0x7f6c9806fc30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009b80 tx=0x7f6c9c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:02.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.251+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6ca80588f0 con 0x7f6cac0fe3e0 2024-05-25T06:10:02.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.471+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c94000bc0 con 0x7f6c9806d780 2024-05-25T06:10:02.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.473+0000 7f6ca2ffd700 1 -- 172.21.15.160:0/138061812 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f6c94000bc0 con 0x7f6c9806d780 2024-05-25T06:10:02.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6c9806d780 msgr2=0x7f6c9806fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6c9806d780 0x7f6c9806fc30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009b80 tx=0x7f6c9c00b040 comp rx=0 tx=0).stop 2024-05-25T06:10:02.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 msgr2=0x7f6cac10d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac10d7e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800b0d0 tx=0x7f6ca8009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 shutdown_connections 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f6c9806d780 0x7f6c9806fc30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0fe3e0 0x7f6cac10d7e0 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800b0d0 tx=0x7f6ca8009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 --2- 172.21.15.160:0/138061812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cac0fed30 0x7f6cac10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 >> 172.21.15.160:0/138061812 conn(0x7f6cac0f6b80 msgr2=0x7f6cac0f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.476+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 shutdown_connections 2024-05-25T06:10:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.477+0000 7f6ca0ff9700 1 -- 172.21.15.160:0/138061812 wait complete. 2024-05-25T06:10:02.488 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:10:02.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.593+0000 7facad02c700 1 -- 172.21.15.160:0/2090025083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 msgr2=0x7faca80ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.593+0000 7facad02c700 1 --2- 172.21.15.160:0/2090025083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca80ff3d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac98004940 tx=0x7fac98009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:02.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.594+0000 7facad02c700 1 -- 172.21.15.160:0/2090025083 shutdown_connections 2024-05-25T06:10:02.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.594+0000 7facad02c700 1 --2- 172.21.15.160:0/2090025083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 0x7faca81006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.594+0000 7facad02c700 1 --2- 172.21.15.160:0/2090025083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca80ff3d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.594+0000 7facad02c700 1 -- 172.21.15.160:0/2090025083 >> 172.21.15.160:0/2090025083 conn(0x7faca80fa8f0 msgr2=0x7faca80fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:02.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.594+0000 7facad02c700 1 -- 172.21.15.160:0/2090025083 shutdown_connections 2024-05-25T06:10:02.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.594+0000 7facad02c700 1 -- 172.21.15.160:0/2090025083 wait complete. 2024-05-25T06:10:02.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.595+0000 7facad02c700 1 Processor -- start 2024-05-25T06:10:02.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.595+0000 7facad02c700 1 -- start start 2024-05-25T06:10:02.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7facad02c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:02.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7facad02c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 0x7faca8110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:02.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7faca6d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:02.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7faca6d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca810fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54740/0 (socket says 172.21.15.160:54740) 2024-05-25T06:10:02.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7faca6d9d700 1 -- 172.21.15.160:0/3559328641 learned_addr learned my addr 172.21.15.160:0/3559328641 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:02.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8110690 con 0x7faca80fefc0 2024-05-25T06:10:02.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8110ee0 con 0x7faca8100270 2024-05-25T06:10:02.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.596+0000 7faca659c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 0x7faca8110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.597+0000 7faca659c700 1 -- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 msgr2=0x7faca810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.597+0000 7faca659c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.597+0000 7faca659c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98005040 con 0x7faca8100270 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.597+0000 7faca6d9d700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.597+0000 7faca659c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 0x7faca8110090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009fd0 tx=0x7fac9c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.597+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c004e90 con 0x7faca8100270 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.598+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca8111160 con 0x7faca8100270 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.598+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac9c00f040 con 0x7faca8100270 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.598+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca81116a0 con 0x7faca8100270 2024-05-25T06:10:02.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.598+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c00ca50 con 0x7faca8100270 2024-05-25T06:10:02.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.600+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac940052e0 con 0x7faca8100270 2024-05-25T06:10:02.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.600+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fac9c00cbb0 con 0x7faca8100270 2024-05-25T06:10:02.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.600+0000 7fac8f7fe700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fac90069370 0x7fac9006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:02.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.601+0000 7faca6d9d700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fac90069370 0x7fac9006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:02.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.601+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fac9c086b50 con 0x7faca8100270 2024-05-25T06:10:02.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.601+0000 7faca6d9d700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fac90069370 0x7fac9006b820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac98004560 tx=0x7fac98009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:02.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.609+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fac9c04e250 con 0x7faca8100270 2024-05-25T06:10:02.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.848+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac94000bf0 con 0x7fac90069370 2024-05-25T06:10:02.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.850+0000 7fac8f7fe700 1 -- 172.21.15.160:0/3559328641 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fac94000bf0 con 0x7fac90069370 2024-05-25T06:10:02.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.852+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fac90069370 msgr2=0x7fac9006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.852+0000 7facad02c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fac90069370 0x7fac9006b820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac98004560 tx=0x7fac98009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:02.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.852+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 msgr2=0x7faca8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:02.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.852+0000 7facad02c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 0x7faca8110090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009fd0 tx=0x7fac9c00b040 comp rx=0 tx=0).stop 2024-05-25T06:10:02.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 shutdown_connections 2024-05-25T06:10:02.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fac90069370 0x7fac9006b820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faca80fefc0 0x7faca810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 --2- 172.21.15.160:0/3559328641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8100270 0x7faca8110090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:02.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 >> 172.21.15.160:0/3559328641 conn(0x7faca80fa8f0 msgr2=0x7faca80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:02.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 shutdown_connections 2024-05-25T06:10:02.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.853+0000 7facad02c700 1 -- 172.21.15.160:0/3559328641 wait complete. 2024-05-25T06:10:03.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.999+0000 7fb81683b700 1 -- 172.21.15.160:0/1645820298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 msgr2=0x7fb8100fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.999+0000 7fb81683b700 1 --2- 172.21.15.160:0/1645820298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 0x7fb8100fd3d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb804004940 tx=0x7fb804009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:03.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.999+0000 7fb81683b700 1 -- 172.21.15.160:0/1645820298 shutdown_connections 2024-05-25T06:10:03.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.999+0000 7fb81683b700 1 --2- 172.21.15.160:0/1645820298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 0x7fb8100fd3d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.999+0000 7fb81683b700 1 --2- 172.21.15.160:0/1645820298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb8100fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:02.999+0000 7fb81683b700 1 -- 172.21.15.160:0/1645820298 >> 172.21.15.160:0/1645820298 conn(0x7fb8100f7430 msgr2=0x7fb8100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:03.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.000+0000 7fb81683b700 1 -- 172.21.15.160:0/1645820298 shutdown_connections 2024-05-25T06:10:03.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.000+0000 7fb81683b700 1 -- 172.21.15.160:0/1645820298 wait complete. 2024-05-25T06:10:03.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.000+0000 7fb81683b700 1 Processor -- start 2024-05-25T06:10:03.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.001+0000 7fb81683b700 1 -- start start 2024-05-25T06:10:03.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.001+0000 7fb81683b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb81010fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.001+0000 7fb81683b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 0x7fb810110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.001+0000 7fb80ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb81010fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.001+0000 7fb80ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb81010fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:53598/0 (socket says 172.21.15.160:53598) 2024-05-25T06:10:03.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb80ffff700 1 -- 172.21.15.160:0/2172788893 learned_addr learned my addr 172.21.15.160:0/2172788893 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:03.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb80f7fe700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 0x7fb810110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810110780 con 0x7fb8100fcf60 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810110fd0 con 0x7fb8100fbcb0 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb80ffff700 1 -- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 msgr2=0x7fb810110180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb80ffff700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 0x7fb810110180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.002+0000 7fb80ffff700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804005040 con 0x7fb8100fbcb0 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.003+0000 7fb80ffff700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb81010fc40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb800002a50 tx=0x7fb8000046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.003+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb800007cc0 con 0x7fb8100fbcb0 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.003+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8101112b0 con 0x7fb8100fbcb0 2024-05-25T06:10:03.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.003+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb80000f040 con 0x7fb8100fbcb0 2024-05-25T06:10:03.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.003+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80000da00 con 0x7fb8100fbcb0 2024-05-25T06:10:03.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.005+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb810111730 con 0x7fb8100fbcb0 2024-05-25T06:10:03.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.005+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb800009710 con 0x7fb8100fbcb0 2024-05-25T06:10:03.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.006+0000 7fb80cff9700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb7f80692a0 0x7fb7f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.007+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8101018f0 con 0x7fb8100fbcb0 2024-05-25T06:10:03.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.007+0000 7fb80f7fe700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb7f80692a0 0x7fb7f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.007+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb8000112f0 con 0x7fb8100fbcb0 2024-05-25T06:10:03.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.007+0000 7fb80f7fe700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb7f80692a0 0x7fb7f806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b3b0 tx=0x7fb804009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:03.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.013+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb800052920 con 0x7fb8100fbcb0 2024-05-25T06:10:03.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.248+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8101119e0 con 0x7fb7f80692a0 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (5m) 72s ago 6m 24.9M - 0.25.0 c8568f914cd2 86108650b602 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (7m) 72s ago 7m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (6m) 79s ago 6m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (5m) 72s ago 6m 82.7M - 9.4.7 954c08fa6188 59fd8b61d321 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (98s) 72s ago 97s 11.3M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (92s) 72s ago 91s 17.1M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:10:03.261 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (89s) 79s ago 89s 13.0M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (95s) 79s ago 94s 17.1M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:9283 running (8m) 72s ago 8m 482M - 17.2.7-958-g7b01852c baf1547092e7 fbeba8913b6d 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (6m) 79s ago 6m 434M - 17.2.7-958-g7b01852c baf1547092e7 c8ec43fcaa21 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (8m) 72s ago 8m 46.0M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (6m) 79s ago 6m 36.3M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (7m) 72s ago 7m 15.9M - 1.5.0 0da6a335fe13 3a8c0c37653a 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (6m) 79s ago 6m 15.0M - 1.5.0 0da6a335fe13 aec5d8f8dcff 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (5m) 72s ago 5m 46.5M 4288M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (4m) 72s ago 4m 45.8M 4288M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (4m) 72s ago 4m 44.9M 4288M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (3m) 79s ago 3m 45.3M 5312M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (3m) 79s ago 3m 42.6M 5312M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (3m) 79s ago 3m 44.3M 5312M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:10:03.262 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (5m) 72s ago 6m 57.1M - 2.43.0 a07b618ecd1d 6cc9a4b6650c 2024-05-25T06:10:03.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.260+0000 7fb80cff9700 1 -- 172.21.15.160:0/2172788893 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fb8101119e0 con 0x7fb7f80692a0 2024-05-25T06:10:03.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.262+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb7f80692a0 msgr2=0x7fb7f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.262+0000 7fb81683b700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb7f80692a0 0x7fb7f806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b3b0 tx=0x7fb804009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 msgr2=0x7fb81010fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb81010fc40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb800002a50 tx=0x7fb8000046b0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 shutdown_connections 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fb7f80692a0 0x7fb7f806b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8100fbcb0 0x7fb81010fc40 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb800002a50 tx=0x7fb8000046b0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 --2- 172.21.15.160:0/2172788893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8100fcf60 0x7fb810110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.263+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 >> 172.21.15.160:0/2172788893 conn(0x7fb8100f7430 msgr2=0x7fb810065bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.264+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 shutdown_connections 2024-05-25T06:10:03.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.264+0000 7fb81683b700 1 -- 172.21.15.160:0/2172788893 wait complete. 2024-05-25T06:10:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 -- 172.21.15.160:0/540664564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c0fd9c0 msgr2=0x7ff82c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 --2- 172.21.15.160:0/540664564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c0ffda0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff828004940 tx=0x7ff828009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 -- 172.21.15.160:0/540664564 shutdown_connections 2024-05-25T06:10:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 --2- 172.21.15.160:0/540664564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c1002e0 0x7ff82c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 --2- 172.21.15.160:0/540664564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c0ffda0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 -- 172.21.15.160:0/540664564 >> 172.21.15.160:0/540664564 conn(0x7ff82c0f7440 msgr2=0x7ff82c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 -- 172.21.15.160:0/540664564 shutdown_connections 2024-05-25T06:10:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.423+0000 7ff833536700 1 -- 172.21.15.160:0/540664564 wait complete. 2024-05-25T06:10:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.424+0000 7ff833536700 1 Processor -- start 2024-05-25T06:10:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.424+0000 7ff833536700 1 -- start start 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff833536700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff833536700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 0x7ff82c10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff833536700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c10e550 con 0x7ff82c1002e0 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff833536700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c10ed50 con 0x7ff82c0fd9c0 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff830ad1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 0x7ff82c10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff8312d2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff830ad1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 0x7ff82c10df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54772/0 (socket says 172.21.15.160:54772) 2024-05-25T06:10:03.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff830ad1700 1 -- 172.21.15.160:0/4149650377 learned_addr learned my addr 172.21.15.160:0/4149650377 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:03.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff830ad1700 1 -- 172.21.15.160:0/4149650377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c0fd9c0 msgr2=0x7ff82c10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff830ad1700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.425+0000 7ff830ad1700 1 -- 172.21.15.160:0/4149650377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828005040 con 0x7ff82c1002e0 2024-05-25T06:10:03.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff8312d2700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:10:03.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff830ad1700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 0x7ff82c10df50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eb40 tx=0x7ff81c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81c007cc0 con 0x7ff82c1002e0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff81c00f040 con 0x7ff82c1002e0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff833536700 1 -- 172.21.15.160:0/4149650377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c10efd0 con 0x7ff82c1002e0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81c009620 con 0x7ff82c1002e0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.426+0000 7ff833536700 1 -- 172.21.15.160:0/4149650377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c10f510 con 0x7ff82c1002e0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.428+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c05f5c0 con 0x7ff82c1002e0 2024-05-25T06:10:03.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.429+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff81c009780 con 0x7ff82c1002e0 2024-05-25T06:10:03.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.429+0000 7ff821ffb700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff818069300 0x7ff81806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.429+0000 7ff8312d2700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff818069300 0x7ff81806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.430+0000 7ff8312d2700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff818069300 0x7ff81806b7b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff828004890 tx=0x7ff828009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:03.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.433+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff81c057320 con 0x7ff82c1002e0 2024-05-25T06:10:03.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.438+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff81c08c060 con 0x7ff82c1002e0 2024-05-25T06:10:03.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.773+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff82c105970 con 0x7ff82c1002e0 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.774+0000 7ff821ffb700 1 -- 172.21.15.160:0/4149650377 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7ff81c0569f0 con 0x7ff82c1002e0 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:10:03.774 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:10:03.775 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 14 2024-05-25T06:10:03.776 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:10:03.776 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:10:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff818069300 msgr2=0x7ff81806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff818069300 0x7ff81806b7b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff828004890 tx=0x7ff828009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 msgr2=0x7ff82c10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 0x7ff82c10df50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eb40 tx=0x7ff81c00b040 comp rx=0 tx=0).stop 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 shutdown_connections 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7ff818069300 0x7ff81806b7b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff82c0fd9c0 0x7ff82c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 --2- 172.21.15.160:0/4149650377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff82c1002e0 0x7ff82c10df50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 >> 172.21.15.160:0/4149650377 conn(0x7ff82c0f7440 msgr2=0x7ff82c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 shutdown_connections 2024-05-25T06:10:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.777+0000 7ff8177fe700 1 -- 172.21.15.160:0/4149650377 wait complete. 2024-05-25T06:10:03.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 -- 172.21.15.160:0/372494674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 msgr2=0x7f58600fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 --2- 172.21.15.160:0/372494674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58600fc0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5850004940 tx=0x7f5850009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:03.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 -- 172.21.15.160:0/372494674 shutdown_connections 2024-05-25T06:10:03.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 --2- 172.21.15.160:0/372494674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600fcf60 0x7f58600fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 --2- 172.21.15.160:0/372494674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58600fc0c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 -- 172.21.15.160:0/372494674 >> 172.21.15.160:0/372494674 conn(0x7f58600f7430 msgr2=0x7f58600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:03.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 -- 172.21.15.160:0/372494674 shutdown_connections 2024-05-25T06:10:03.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.885+0000 7f586521c700 1 -- 172.21.15.160:0/372494674 wait complete. 2024-05-25T06:10:03.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.886+0000 7f586521c700 1 Processor -- start 2024-05-25T06:10:03.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.886+0000 7f586521c700 1 -- start start 2024-05-25T06:10:03.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f586521c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58601106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f586521c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600fcf60 0x7f5860112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f586521c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5860066300 con 0x7f58600fbcb0 2024-05-25T06:10:03.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f586521c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5860066440 con 0x7f58600fcf60 2024-05-25T06:10:03.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58601106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58601106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54788/0 (socket says 172.21.15.160:54788) 2024-05-25T06:10:03.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 -- 172.21.15.160:0/1401218534 learned_addr learned my addr 172.21.15.160:0/1401218534 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:03.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585e59c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600fcf60 0x7f5860112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 -- 172.21.15.160:0/1401218534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600fcf60 msgr2=0x7f5860112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:03.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600fcf60 0x7f5860112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:03.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 -- 172.21.15.160:0/1401218534 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850005040 con 0x7f58600fbcb0 2024-05-25T06:10:03.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f585ed9d700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58601106c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5850009150 tx=0x7f5850009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:03.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585000e040 con 0x7f58600fbcb0 2024-05-25T06:10:03.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5860113150 con 0x7f58600fbcb0 2024-05-25T06:10:03.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5850004290 con 0x7f58600fbcb0 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.887+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585001c630 con 0x7f58600fbcb0 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.888+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586010c7a0 con 0x7f58600fbcb0 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.889+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586005f5c0 con 0x7f58600fbcb0 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.889+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f585000bbf0 con 0x7f58600fbcb0 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.890+0000 7f58477fe700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f58480692a0 0x7f584806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.890+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5850087ec0 con 0x7f58600fbcb0 2024-05-25T06:10:03.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.890+0000 7f585e59c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f58480692a0 0x7f584806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:03.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.891+0000 7f585e59c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f58480692a0 0x7f584806b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5854009fd0 tx=0x7f585400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:03.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:03.895+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f58500541a0 con 0x7f58600fbcb0 2024-05-25T06:10:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.105+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5860101250 con 0x7f58480692a0 2024-05-25T06:10:04.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:04.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:04.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:04.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:04.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: Upgrade: Target is version 19.0.0-2843-gea0684c0 (unknown) 2024-05-25T06:10:04.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee'] 2024-05-25T06:10:04.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:10:04.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:04.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: Upgrade: Need to upgrade myself (mgr.smithi160.nalzlg) 2024-05-25T06:10:04.107 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:03 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/4149650377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:10:04.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.107+0000 7f58477fe700 1 -- 172.21.15.160:0/1401218534 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5860101250 con 0x7f58480692a0 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "0/2 daemons upgraded", 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee image on host smithi203", 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:10:04.108 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:10:04.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f58480692a0 msgr2=0x7f584806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:04.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f58480692a0 0x7f584806b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5854009fd0 tx=0x7f585400b040 comp rx=0 tx=0).stop 2024-05-25T06:10:04.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 msgr2=0x7f58601106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:04.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58601106c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5850009150 tx=0x7f5850009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:04.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 shutdown_connections 2024-05-25T06:10:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f58480692a0 0x7f584806b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58600fbcb0 0x7f58601106c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 --2- 172.21.15.160:0/1401218534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600fcf60 0x7f5860112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 >> 172.21.15.160:0/1401218534 conn(0x7f58600f7430 msgr2=0x7f58600f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 shutdown_connections 2024-05-25T06:10:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:04.110+0000 7f586521c700 1 -- 172.21.15.160:0/1401218534 wait complete. 2024-05-25T06:10:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: Upgrade: Target is version 19.0.0-2843-gea0684c0 (unknown) 2024-05-25T06:10:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee'] 2024-05-25T06:10:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:10:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: Upgrade: Need to upgrade myself (mgr.smithi160.nalzlg) 2024-05-25T06:10:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:03 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/4149650377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:10:05.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:04 smithi160 ceph-mon[29041]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:05.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:04 smithi160 ceph-mon[29041]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee on smithi203 2024-05-25T06:10:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:04 smithi203 ceph-mon[36030]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:04 smithi203 ceph-mon[36030]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee on smithi203 2024-05-25T06:10:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:05 smithi160 ceph-mon[29041]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:05 smithi160 ceph-mon[29041]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:05 smithi203 ceph-mon[36030]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:05 smithi203 ceph-mon[36030]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:07 smithi160 ceph-mon[29041]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:07 smithi203 ceph-mon[36030]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:09.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:08 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:08 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:10.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:09 smithi160 ceph-mon[29041]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:10:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:09 smithi203 ceph-mon[36030]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:10:12.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:11 smithi160 ceph-mon[29041]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:11 smithi203 ceph-mon[36030]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:13 smithi160 ceph-mon[29041]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:13 smithi203 ceph-mon[36030]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:15 smithi160 ceph-mon[29041]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:15 smithi203 ceph-mon[36030]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:17 smithi160 ceph-mon[29041]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:17 smithi203 ceph-mon[36030]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:19 smithi160 ceph-mon[29041]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:10:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:19 smithi203 ceph-mon[36030]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:10:22.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:21 smithi160 ceph-mon[29041]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:21 smithi203 ceph-mon[36030]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:23.142 INFO:tasks.workunit.client.1.smithi203.stderr:Updating files: 88% (11179/12563) Updating files: 89% (11182/12563) Updating files: 90% (11307/12563) Updating files: 91% (11433/12563) Updating files: 92% (11558/12563) Updating files: 93% (11684/12563) Updating files: 94% (11810/12563) Updating files: 95% (11935/12563) Updating files: 96% (12061/12563) Updating files: 97% (12187/12563) Updating files: 98% (12312/12563) Updating files: 99% (12438/12563) Updating files: 100% (12563/12563) Updating files: 100% (12563/12563), done. 2024-05-25T06:10:23.470 INFO:tasks.workunit.client.1.smithi203.stderr:Note: switching to '17c123e524b17eb0cf38bdb0a0ca89d80b36c131'. 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr:state without impacting any branches by switching back to a branch. 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr: git switch -c 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.471 INFO:tasks.workunit.client.1.smithi203.stderr:Or undo this operation with: 2024-05-25T06:10:23.472 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.472 INFO:tasks.workunit.client.1.smithi203.stderr: git switch - 2024-05-25T06:10:23.472 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.472 INFO:tasks.workunit.client.1.smithi203.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-25T06:10:23.472 INFO:tasks.workunit.client.1.smithi203.stderr: 2024-05-25T06:10:23.472 INFO:tasks.workunit.client.1.smithi203.stderr:HEAD is now at 17c123e524b Merge pull request #57653 from petrutlucian94/wip-66204-squid 2024-05-25T06:10:23.477 DEBUG:teuthology.orchestra.run.smithi203:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-25T06:10:23.540 INFO:tasks.workunit.client.1.smithi203.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-25T06:10:23.542 INFO:tasks.workunit.client.1.smithi203.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-25T06:10:23.542 INFO:tasks.workunit.client.1.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-25T06:10:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:23 smithi160 ceph-mon[29041]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:23 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:24.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:23 smithi203 ceph-mon[36030]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:10:24.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:23 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:24.497 INFO:tasks.workunit.client.1.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-25T06:10:24.553 INFO:tasks.workunit.client.1.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-25T06:10:24.595 INFO:tasks.workunit.client.1.smithi203.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-25T06:10:24.597 INFO:tasks.workunit.client.1.smithi203.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-25T06:10:24.597 INFO:tasks.workunit.client.1.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-25T06:10:24.609 INFO:tasks.workunit.client.0.smithi160.stderr:Note: switching to '17c123e524b17eb0cf38bdb0a0ca89d80b36c131'. 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr:state without impacting any branches by switching back to a branch. 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr: git switch -c 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr:Or undo this operation with: 2024-05-25T06:10:24.610 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.611 INFO:tasks.workunit.client.0.smithi160.stderr: git switch - 2024-05-25T06:10:24.611 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.611 INFO:tasks.workunit.client.0.smithi160.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-25T06:10:24.611 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-25T06:10:24.611 INFO:tasks.workunit.client.0.smithi160.stderr:HEAD is now at 17c123e524b Merge pull request #57653 from petrutlucian94/wip-66204-squid 2024-05-25T06:10:24.615 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-25T06:10:24.637 INFO:tasks.workunit.client.1.smithi203.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-25T06:10:24.640 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-25T06:10:24.640 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-25T06:10:24.672 INFO:tasks.workunit.client.0.smithi160.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-25T06:10:24.674 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-25T06:10:24.674 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-25T06:10:24.697 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-25T06:10:24.698 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-25T06:10:24.699 DEBUG:teuthology.orchestra.run.smithi203:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=17c123e524b17eb0cf38bdb0a0ca89d80b36c131 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-25T06:10:24.764 INFO:tasks.workunit.client.1.smithi203.stderr:+ mkdir -p fsstress 2024-05-25T06:10:24.766 INFO:tasks.workunit.client.1.smithi203.stderr:+ pushd fsstress 2024-05-25T06:10:24.767 INFO:tasks.workunit.client.1.smithi203.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-25T06:10:24.768 INFO:tasks.workunit.client.1.smithi203.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-25T06:10:25.475 INFO:tasks.workunit.client.1.smithi203.stderr:+ tar xzf ltp-full.tgz 2024-05-25T06:10:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:25 smithi203 ceph-mon[36030]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:25 smithi160 ceph-mon[29041]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:10:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:27 smithi203 ceph-mon[36030]: pgmap v238: 65 pgs: 65 active+clean; 7.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 562 KiB/s wr, 12 op/s 2024-05-25T06:10:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:27 smithi160 ceph-mon[29041]: pgmap v238: 65 pgs: 65 active+clean; 7.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 562 KiB/s wr, 12 op/s 2024-05-25T06:10:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:29 smithi203 ceph-mon[36030]: pgmap v239: 65 pgs: 65 active+clean; 8.2 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 666 KiB/s wr, 31 op/s 2024-05-25T06:10:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:29 smithi160 ceph-mon[29041]: pgmap v239: 65 pgs: 65 active+clean; 8.2 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 666 KiB/s wr, 31 op/s 2024-05-25T06:10:31.550 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-25T06:10:31.675 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-25T06:10:31.722 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-25T06:10:31.723 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-25T06:10:31.724 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-25T06:10:31.769 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-25T06:10:31.773 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:10:31.773 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-25T06:10:31.791 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-25T06:10:31.792 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-25T06:10:31.792 DEBUG:teuthology.orchestra.run.smithi160:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=17c123e524b17eb0cf38bdb0a0ca89d80b36c131 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-25T06:10:31.857 INFO:tasks.workunit.client.0.smithi160.stderr:+ mkdir -p fsstress 2024-05-25T06:10:31.859 INFO:tasks.workunit.client.0.smithi160.stderr:+ pushd fsstress 2024-05-25T06:10:31.860 INFO:tasks.workunit.client.0.smithi160.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-25T06:10:31.860 INFO:tasks.workunit.client.0.smithi160.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-25T06:10:32.271 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:31 smithi160 ceph-mon[29041]: pgmap v240: 65 pgs: 65 active+clean; 14 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-05-25T06:10:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:31 smithi203 ceph-mon[36030]: pgmap v240: 65 pgs: 65 active+clean; 14 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-05-25T06:10:33.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ tar xzf ltp-full.tgz 2024-05-25T06:10:34.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:33 smithi160 ceph-mon[29041]: pgmap v241: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-25T06:10:34.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 -- 172.21.15.160:0/3370671572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fcf60 msgr2=0x7f39180fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:34.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3370671572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fcf60 0x7f39180fd3d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3904004940 tx=0x7f3904009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:34.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 -- 172.21.15.160:0/3370671572 shutdown_connections 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3370671572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fcf60 0x7f39180fd3d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3370671572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fbcb0 0x7f39180fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 -- 172.21.15.160:0/3370671572 >> 172.21.15.160:0/3370671572 conn(0x7f39180f7430 msgr2=0x7f39180f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 -- 172.21.15.160:0/3370671572 shutdown_connections 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.261+0000 7f391ecc4700 1 -- 172.21.15.160:0/3370671572 wait complete. 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.262+0000 7f391ecc4700 1 Processor -- start 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.262+0000 7f391ecc4700 1 -- start start 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.262+0000 7f391ecc4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fbcb0 0x7f391810c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.262+0000 7f391ecc4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 0x7f391810ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.262+0000 7f391ecc4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391810d2b0 con 0x7f39180fbcb0 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.262+0000 7f391ecc4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391810d420 con 0x7f39180fcf60 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.263+0000 7f3917fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 0x7f391810ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:34.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.263+0000 7f3917fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 0x7f391810ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:56750/0 (socket says 172.21.15.160:56750) 2024-05-25T06:10:34.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.263+0000 7f3917fff700 1 -- 172.21.15.160:0/3952699871 learned_addr learned my addr 172.21.15.160:0/3952699871 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:34.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.263+0000 7f3917fff700 1 -- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fbcb0 msgr2=0x7f391810c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:34.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.263+0000 7f3917fff700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fbcb0 0x7f391810c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.263+0000 7f3917fff700 1 -- 172.21.15.160:0/3952699871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904005040 con 0x7f39180fcf60 2024-05-25T06:10:34.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.264+0000 7f3917fff700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 0x7f391810ccb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3904013870 tx=0x7f3904009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:34.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.264+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3904021070 con 0x7f39180fcf60 2024-05-25T06:10:34.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.264+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3918111e20 con 0x7f39180fcf60 2024-05-25T06:10:34.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.264+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f390401c400 con 0x7f39180fcf60 2024-05-25T06:10:34.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.264+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390400e070 con 0x7f39180fcf60 2024-05-25T06:10:34.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.264+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3918112330 con 0x7f39180fcf60 2024-05-25T06:10:34.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.265+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391804c710 con 0x7f39180fcf60 2024-05-25T06:10:34.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.266+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f390401ca20 con 0x7f39180fcf60 2024-05-25T06:10:34.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.266+0000 7f39157fa700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f39080692a0 0x7f390806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:34.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.266+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f39040884a0 con 0x7f39180fcf60 2024-05-25T06:10:34.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.267+0000 7f391ca60700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f39080692a0 0x7f390806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:34.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.267+0000 7f391ca60700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f39080692a0 0x7f390806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f390c009fd0 tx=0x7f390c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:34.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.272+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f39040582b0 con 0x7f39180fcf60 2024-05-25T06:10:34.302 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:33 smithi203 ceph-mon[36030]: pgmap v241: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-05-25T06:10:34.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.576+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3918112570 con 0x7f39080692a0 2024-05-25T06:10:34.578 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.578+0000 7f39157fa700 1 -- 172.21.15.160:0/3952699871 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f3918112570 con 0x7f39080692a0 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.580+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f39080692a0 msgr2=0x7f390806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.580+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f39080692a0 0x7f390806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f390c009fd0 tx=0x7f390c00b040 comp rx=0 tx=0).stop 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 msgr2=0x7f391810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 0x7f391810ccb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3904013870 tx=0x7f3904009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 shutdown_connections 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f39080692a0 0x7f390806b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39180fbcb0 0x7f391810c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 --2- 172.21.15.160:0/3952699871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39180fcf60 0x7f391810ccb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 >> 172.21.15.160:0/3952699871 conn(0x7f39180f7430 msgr2=0x7f39180f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:34.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 shutdown_connections 2024-05-25T06:10:34.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.581+0000 7f391ecc4700 1 -- 172.21.15.160:0/3952699871 wait complete. 2024-05-25T06:10:34.594 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:10:34.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.725+0000 7fec818a7700 1 -- 172.21.15.160:0/1301020885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fedf0 msgr2=0x7fec7c0faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:34.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.725+0000 7fec818a7700 1 --2- 172.21.15.160:0/1301020885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fedf0 0x7fec7c0faf30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec70004880 tx=0x7fec70009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:34.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.725+0000 7fec818a7700 1 -- 172.21.15.160:0/1301020885 shutdown_connections 2024-05-25T06:10:34.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.725+0000 7fec818a7700 1 --2- 172.21.15.160:0/1301020885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fedf0 0x7fec7c0faf30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.725+0000 7fec818a7700 1 --2- 172.21.15.160:0/1301020885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c0fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.725+0000 7fec818a7700 1 -- 172.21.15.160:0/1301020885 >> 172.21.15.160:0/1301020885 conn(0x7fec7c0f6b80 msgr2=0x7fec7c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:34.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.726+0000 7fec818a7700 1 -- 172.21.15.160:0/1301020885 shutdown_connections 2024-05-25T06:10:34.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.726+0000 7fec818a7700 1 -- 172.21.15.160:0/1301020885 wait complete. 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.726+0000 7fec818a7700 1 Processor -- start 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec818a7700 1 -- start start 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec818a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c10d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec818a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec7c0fedf0 0x7fec7c10dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec818a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c10e340 con 0x7fec7c0fedf0 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c10d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c10d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:56764/0 (socket says 172.21.15.160:56764) 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7affd700 1 -- 172.21.15.160:0/3085588379 learned_addr learned my addr 172.21.15.160:0/3085588379 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7a7fc700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec7c0fedf0 0x7fec7c10dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c10eb40 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7affd700 1 -- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec7c0fedf0 msgr2=0x7fec7c10dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7affd700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec7c0fedf0 0x7fec7c10dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:34.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.727+0000 7fec7affd700 1 -- 172.21.15.160:0/3085588379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70005040 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.728+0000 7fec7affd700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c10d800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0041d0 tx=0x7fec6c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:34.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.728+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6c007cc0 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.728+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c10edc0 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.728+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec6c00f040 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.729+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6c00cb50 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.729+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c10f300 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.730+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fec6c00ccb0 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.731+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c05f5c0 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.731+0000 7fec637fe700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fec64069300 0x7fec6406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:34.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.732+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fec6c086dd0 con 0x7fec7c0fe4a0 2024-05-25T06:10:34.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.732+0000 7fec7a7fc700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fec64069300 0x7fec6406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:34.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.733+0000 7fec7a7fc700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fec64069300 0x7fec6406b7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec70009fd0 tx=0x7fec70009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:34.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:34.736+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fec6c04e250 con 0x7fec7c0fe4a0 2024-05-25T06:10:35.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.067+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec7c10f680 con 0x7fec64069300 2024-05-25T06:10:35.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.069+0000 7fec637fe700 1 -- 172.21.15.160:0/3085588379 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fec7c10f680 con 0x7fec64069300 2024-05-25T06:10:35.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fec64069300 msgr2=0x7fec6406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fec64069300 0x7fec6406b7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec70009fd0 tx=0x7fec70009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 msgr2=0x7fec7c10d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c10d800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0041d0 tx=0x7fec6c00b040 comp rx=0 tx=0).stop 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 shutdown_connections 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7fec64069300 0x7fec6406b7b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec7c0fe4a0 0x7fec7c10d800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 --2- 172.21.15.160:0/3085588379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec7c0fedf0 0x7fec7c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 >> 172.21.15.160:0/3085588379 conn(0x7fec7c0f6b80 msgr2=0x7fec7c0f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 shutdown_connections 2024-05-25T06:10:35.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.072+0000 7fec818a7700 1 -- 172.21.15.160:0/3085588379 wait complete. 2024-05-25T06:10:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.219+0000 7f8200888700 1 -- 172.21.15.160:0/2262849803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fbd40 msgr2=0x7f81fc0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.219+0000 7f8200888700 1 --2- 172.21.15.160:0/2262849803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fbd40 0x7f81fc0fc150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f81ec004940 tx=0x7f81ec009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.219+0000 7f8200888700 1 -- 172.21.15.160:0/2262849803 shutdown_connections 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.219+0000 7f8200888700 1 --2- 172.21.15.160:0/2262849803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fcff0 0x7f81fc0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.219+0000 7f8200888700 1 --2- 172.21.15.160:0/2262849803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fbd40 0x7f81fc0fc150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.219+0000 7f8200888700 1 -- 172.21.15.160:0/2262849803 >> 172.21.15.160:0/2262849803 conn(0x7f81fc0f74c0 msgr2=0x7f81fc0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.220+0000 7f8200888700 1 -- 172.21.15.160:0/2262849803 shutdown_connections 2024-05-25T06:10:35.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.220+0000 7f8200888700 1 -- 172.21.15.160:0/2262849803 wait complete. 2024-05-25T06:10:35.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.220+0000 7f8200888700 1 Processor -- start 2024-05-25T06:10:35.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f8200888700 1 -- start start 2024-05-25T06:10:35.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f8200888700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fbd40 0x7f81fc112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:35.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f8200888700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 0x7f81fc115000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:35.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f8200888700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc066ab0 con 0x7f81fc0fcff0 2024-05-25T06:10:35.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f8200888700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc066bf0 con 0x7f81fc0fbd40 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81f9d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 0x7f81fc115000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81f9d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 0x7f81fc115000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51606/0 (socket says 172.21.15.160:51606) 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81f9d9b700 1 -- 172.21.15.160:0/1924806683 learned_addr learned my addr 172.21.15.160:0/1924806683 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81fa59c700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fbd40 0x7f81fc112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81f9d9b700 1 -- 172.21.15.160:0/1924806683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fbd40 msgr2=0x7f81fc112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81f9d9b700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fbd40 0x7f81fc112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81f9d9b700 1 -- 172.21.15.160:0/1924806683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec005040 con 0x7f81fc0fcff0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.221+0000 7f81fa59c700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fbd40 0x7f81fc112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.222+0000 7f81f9d9b700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 0x7f81fc115000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f81f000eb40 tx=0x7f81f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.222+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81f0007cc0 con 0x7f81fc0fcff0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.222+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc115540 con 0x7f81fc0fcff0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.222+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81f000f040 con 0x7f81fc0fcff0 2024-05-25T06:10:35.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.222+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81f000ca10 con 0x7f81fc0fcff0 2024-05-25T06:10:35.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.222+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc115890 con 0x7f81fc0fcff0 2024-05-25T06:10:35.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.223+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc05f5c0 con 0x7f81fc0fcff0 2024-05-25T06:10:35.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.224+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f81f000cb70 con 0x7f81fc0fcff0 2024-05-25T06:10:35.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.225+0000 7f81eaffd700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f81e40692a0 0x7f81e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:35.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.225+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f81f0086d30 con 0x7f81fc0fcff0 2024-05-25T06:10:35.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.225+0000 7f81fa59c700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f81e40692a0 0x7f81e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:35.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.226+0000 7f81fa59c700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f81e40692a0 0x7f81e406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81ec004890 tx=0x7f81ec009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:35.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.231+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f81f004e250 con 0x7f81fc0fcff0 2024-05-25T06:10:35.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.565+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f81fc101290 con 0x7f81e40692a0 2024-05-25T06:10:35.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.580+0000 7f81eaffd700 1 -- 172.21.15.160:0/1924806683 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7f81fc101290 con 0x7f81e40692a0 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (6m) 105s ago 7m 24.9M - 0.25.0 c8568f914cd2 86108650b602 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (7m) 105s ago 7m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (6m) 111s ago 6m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (6m) 105s ago 7m 82.7M - 9.4.7 954c08fa6188 59fd8b61d321 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (2m) 105s ago 2m 11.3M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (2m) 105s ago 2m 17.1M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (2m) 111s ago 2m 13.0M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (2m) 111s ago 2m 17.1M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:9283 running (9m) 105s ago 9m 482M - 17.2.7-958-g7b01852c baf1547092e7 fbeba8913b6d 2024-05-25T06:10:35.581 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (6m) 111s ago 6m 434M - 17.2.7-958-g7b01852c baf1547092e7 c8ec43fcaa21 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (9m) 105s ago 9m 46.0M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (6m) 111s ago 6m 36.3M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (7m) 105s ago 7m 15.9M - 1.5.0 0da6a335fe13 3a8c0c37653a 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (6m) 111s ago 6m 15.0M - 1.5.0 0da6a335fe13 aec5d8f8dcff 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (5m) 105s ago 5m 46.5M 4288M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (5m) 105s ago 5m 45.8M 4288M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (4m) 105s ago 4m 44.9M 4288M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (4m) 111s ago 4m 45.3M 5312M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:10:35.582 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (4m) 111s ago 4m 42.6M 5312M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:10:35.583 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (3m) 111s ago 3m 44.3M 5312M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:10:35.583 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (5m) 105s ago 7m 57.1M - 2.43.0 a07b618ecd1d 6cc9a4b6650c 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f81e40692a0 msgr2=0x7f81e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f81e40692a0 0x7f81e406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81ec004890 tx=0x7f81ec009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 msgr2=0x7f81fc115000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 0x7f81fc115000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f81f000eb40 tx=0x7f81f000b040 comp rx=0 tx=0).stop 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 shutdown_connections 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f81e40692a0 0x7f81e406b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81fc0fbd40 0x7f81fc112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 --2- 172.21.15.160:0/1924806683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81fc0fcff0 0x7f81fc115000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 >> 172.21.15.160:0/1924806683 conn(0x7f81fc0f74c0 msgr2=0x7f81fc0f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 shutdown_connections 2024-05-25T06:10:35.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.583+0000 7f8200888700 1 -- 172.21.15.160:0/1924806683 wait complete. 2024-05-25T06:10:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: Upgrade: Updating mgr.smithi203.rotqex 2024-05-25T06:10:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:10:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:10:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:10:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: Deploying daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:10:35.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:35.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: pgmap v242: 65 pgs: 65 active+clean; 31 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 2.7 MiB/s wr, 193 op/s 2024-05-25T06:10:35.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:35 smithi203 ceph-mon[36030]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:35.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: Upgrade: Updating mgr.smithi203.rotqex 2024-05-25T06:10:35.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:35.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:10:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:10:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:10:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: Deploying daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:10:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: pgmap v242: 65 pgs: 65 active+clean; 31 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 2.7 MiB/s wr, 193 op/s 2024-05-25T06:10:35.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:35 smithi160 ceph-mon[29041]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:35.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 -- 172.21.15.160:0/1440836638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 msgr2=0x7f730c1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 --2- 172.21.15.160:0/1440836638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1046d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72fc004940 tx=0x7f72fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:35.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 -- 172.21.15.160:0/1440836638 shutdown_connections 2024-05-25T06:10:35.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 --2- 172.21.15.160:0/1440836638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1046d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 --2- 172.21.15.160:0/1440836638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 0x7f730c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 -- 172.21.15.160:0/1440836638 >> 172.21.15.160:0/1440836638 conn(0x7f730c0f7420 msgr2=0x7f730c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:35.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 -- 172.21.15.160:0/1440836638 shutdown_connections 2024-05-25T06:10:35.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 -- 172.21.15.160:0/1440836638 wait complete. 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.728+0000 7f7314061700 1 Processor -- start 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7314061700 1 -- start start 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7314061700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 0x7f730c10fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7314061700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7314061700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c1107a0 con 0x7f730c1022f0 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7314061700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c110ee0 con 0x7f730c066b50 2024-05-25T06:10:35.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f73115fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7311dfd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 0x7f730c10fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f73115fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55928/0 (socket says 172.21.15.160:55928) 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f73115fc700 1 -- 172.21.15.160:0/1138907730 learned_addr learned my addr 172.21.15.160:0/1138907730 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7311dfd700 1 -- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 msgr2=0x7f730c1101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7311dfd700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f7311dfd700 1 -- 172.21.15.160:0/1138907730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc005040 con 0x7f730c066b50 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.729+0000 7f73115fc700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.730+0000 7f7311dfd700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 0x7f730c10fc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f730c066890 tx=0x7f730800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.730+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7308004e90 con 0x7f730c066b50 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.730+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f730800f040 con 0x7f730c066b50 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.730+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c111160 con 0x7f730c066b50 2024-05-25T06:10:35.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.730+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c1116a0 con 0x7f730c066b50 2024-05-25T06:10:35.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.730+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f730800ca20 con 0x7f730c066b50 2024-05-25T06:10:35.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.731+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c05f5c0 con 0x7f730c066b50 2024-05-25T06:10:35.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.732+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f730800cbd0 con 0x7f730c066b50 2024-05-25T06:10:35.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.732+0000 7f73027fc700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f72f80692a0 0x7f72f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:35.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.732+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7308086be0 con 0x7f730c066b50 2024-05-25T06:10:35.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.732+0000 7f73115fc700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f72f80692a0 0x7f72f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:35.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.733+0000 7f73115fc700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f72f80692a0 0x7f72f806b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00b0d0 tx=0x7f72fc009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:35.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:35.737+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f730804e250 con 0x7f730c066b50 2024-05-25T06:10:36.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.148+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f730c111910 con 0x7f730c066b50 2024-05-25T06:10:36.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.148+0000 7f73027fc700 1 -- 172.21.15.160:0/1138907730 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f7308056980 con 0x7f730c066b50 2024-05-25T06:10:36.148 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:10:36.149 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:10:36.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:10:36.150 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:10:36.150 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 14 2024-05-25T06:10:36.150 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:10:36.150 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f72f80692a0 msgr2=0x7f72f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f72f80692a0 0x7f72f806b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00b0d0 tx=0x7f72fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 msgr2=0x7f730c10fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 0x7f730c10fc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f730c066890 tx=0x7f730800b040 comp rx=0 tx=0).stop 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 shutdown_connections 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f72f80692a0 0x7f72f806b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f730c066b50 0x7f730c10fc60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 --2- 172.21.15.160:0/1138907730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f730c1022f0 0x7f730c1101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 >> 172.21.15.160:0/1138907730 conn(0x7f730c0f7420 msgr2=0x7f730c0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:36.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 shutdown_connections 2024-05-25T06:10:36.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.152+0000 7f7314061700 1 -- 172.21.15.160:0/1138907730 wait complete. 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 -- 172.21.15.160:0/2725231258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fdff0 msgr2=0x7f26a00fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/2725231258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fdff0 0x7f26a00fe460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2694004940 tx=0x7f2694009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 -- 172.21.15.160:0/2725231258 shutdown_connections 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/2725231258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fdff0 0x7f26a00fe460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/2725231258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a00fce80 0x7f26a00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 -- 172.21.15.160:0/2725231258 >> 172.21.15.160:0/2725231258 conn(0x7f26a00f8520 msgr2=0x7f26a00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 -- 172.21.15.160:0/2725231258 shutdown_connections 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.288+0000 7f26a6a5c700 1 -- 172.21.15.160:0/2725231258 wait complete. 2024-05-25T06:10:36.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.289+0000 7f26a6a5c700 1 Processor -- start 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.289+0000 7f26a6a5c700 1 -- start start 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.289+0000 7f26a6a5c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 0x7f26a010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.289+0000 7f26a6a5c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a00fdff0 0x7f26a010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.289+0000 7f26a6a5c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a010e4d0 con 0x7f26a00fce80 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.289+0000 7f26a6a5c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a010ed20 con 0x7f26a00fdff0 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 0x7f26a010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 0x7f26a010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55940/0 (socket says 172.21.15.160:55940) 2024-05-25T06:10:36.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 -- 172.21.15.160:0/3055838936 learned_addr learned my addr 172.21.15.160:0/3055838936 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269f7fe700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a00fdff0 0x7f26a010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 -- 172.21.15.160:0/3055838936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a00fdff0 msgr2=0x7f26a010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a00fdff0 0x7f26a010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 -- 172.21.15.160:0/3055838936 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694005040 con 0x7f26a00fce80 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269ffff700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 0x7f26a010d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f269000eb40 tx=0x7f269000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2690007cc0 con 0x7f26a00fce80 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a010efa0 con 0x7f26a00fce80 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a010f4e0 con 0x7f26a00fce80 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.290+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f269000f040 con 0x7f26a00fce80 2024-05-25T06:10:36.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.291+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f269000c930 con 0x7f26a00fce80 2024-05-25T06:10:36.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.292+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c0052e0 con 0x7f26a00fce80 2024-05-25T06:10:36.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.292+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f26900090d0 con 0x7f26a00fce80 2024-05-25T06:10:36.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.293+0000 7f269cff9700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f26880692a0 0x7f268806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:10:36.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.293+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2690086df0 con 0x7f26a00fce80 2024-05-25T06:10:36.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.293+0000 7f269f7fe700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f26880692a0 0x7f268806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:10:36.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.294+0000 7f269f7fe700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f26880692a0 0x7f268806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2694009b30 tx=0x7f2694009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:10:36.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.298+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f269004e250 con 0x7f26a00fce80 2024-05-25T06:10:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:36 smithi160 ceph-mon[29041]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:36 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1138907730' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:10:36.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.606+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 --> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f268c000bf0 con 0x7f26880692a0 2024-05-25T06:10:36.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.607+0000 7f269cff9700 1 -- 172.21.15.160:0/3055838936 <== mgr.14210 v2:172.21.15.160:6800/4089125744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f268c000bf0 con 0x7f26880692a0 2024-05-25T06:10:36.607 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:10:36.607 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "0/2 daemons upgraded", 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mgr daemons", 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:10:36.608 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.610+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f26880692a0 msgr2=0x7f268806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.610+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f26880692a0 0x7f268806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2694009b30 tx=0x7f2694009e30 comp rx=0 tx=0).stop 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 msgr2=0x7f26a010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 0x7f26a010d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f269000eb40 tx=0x7f269000b040 comp rx=0 tx=0).stop 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 shutdown_connections 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:6800/4089125744,v1:172.21.15.160:6801/4089125744] conn(0x7f26880692a0 0x7f268806b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26a00fce80 0x7f26a010d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 --2- 172.21.15.160:0/3055838936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a00fdff0 0x7f26a010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 >> 172.21.15.160:0/3055838936 conn(0x7f26a00f8520 msgr2=0x7f26a00f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:10:36.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 shutdown_connections 2024-05-25T06:10:36.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:10:36.611+0000 7f26a6a5c700 1 -- 172.21.15.160:0/3055838936 wait complete. 2024-05-25T06:10:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:36 smithi203 ceph-mon[36030]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:36 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1138907730' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:10:37.642 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:37 smithi203 ceph-mon[36030]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:37.642 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:37 smithi203 ceph-mon[36030]: pgmap v243: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 684 KiB/s rd, 4.0 MiB/s wr, 278 op/s 2024-05-25T06:10:37.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:37 smithi160 ceph-mon[29041]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:10:37.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:37 smithi160 ceph-mon[29041]: pgmap v243: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 684 KiB/s rd, 4.0 MiB/s wr, 278 op/s 2024-05-25T06:10:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:38 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:10:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:38 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:10:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:39 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:39 smithi203 ceph-mon[36030]: pgmap v244: 65 pgs: 65 active+clean; 53 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 911 KiB/s rd, 4.0 MiB/s wr, 326 op/s 2024-05-25T06:10:39.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:39 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:39.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:39 smithi160 ceph-mon[29041]: pgmap v244: 65 pgs: 65 active+clean; 53 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 911 KiB/s rd, 4.0 MiB/s wr, 326 op/s 2024-05-25T06:10:41.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:41 smithi160 ceph-mon[29041]: pgmap v245: 65 pgs: 65 active+clean; 59 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 911 KiB/s rd, 4.4 MiB/s wr, 371 op/s 2024-05-25T06:10:42.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:41 smithi203 ceph-mon[36030]: pgmap v245: 65 pgs: 65 active+clean; 59 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 911 KiB/s rd, 4.4 MiB/s wr, 371 op/s 2024-05-25T06:10:43.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:43 smithi160 ceph-mon[29041]: pgmap v246: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 413 op/s 2024-05-25T06:10:44.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:43 smithi203 ceph-mon[36030]: pgmap v246: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 413 op/s 2024-05-25T06:10:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:45 smithi203 ceph-mon[36030]: pgmap v247: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-05-25T06:10:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:45 smithi160 ceph-mon[29041]: pgmap v247: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-05-25T06:10:48.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:47 smithi203 ceph-mon[36030]: pgmap v248: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 448 op/s 2024-05-25T06:10:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:47 smithi160 ceph-mon[29041]: pgmap v248: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 448 op/s 2024-05-25T06:10:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:49 smithi203 ceph-mon[36030]: pgmap v249: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-05-25T06:10:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:50.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:49 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:10:50.199 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:49 smithi160 ceph-mon[29041]: pgmap v249: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-05-25T06:10:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:10:50.200 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:49 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:10:51.779 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:51 smithi203 ceph-mon[36030]: pgmap v250: 65 pgs: 65 active+clean; 88 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2024-05-25T06:10:52.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:51 smithi160 ceph-mon[29041]: pgmap v250: 65 pgs: 65 active+clean; 88 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2024-05-25T06:10:53.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:53 smithi203 ceph-mon[36030]: pgmap v251: 65 pgs: 65 active+clean; 105 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 459 op/s 2024-05-25T06:10:53.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:53 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:53.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:53 smithi160 ceph-mon[29041]: pgmap v251: 65 pgs: 65 active+clean; 105 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 459 op/s 2024-05-25T06:10:53.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:53 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:10:55.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:55 smithi203 ceph-mon[36030]: pgmap v252: 65 pgs: 65 active+clean; 112 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 466 op/s 2024-05-25T06:10:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:55 smithi160 ceph-mon[29041]: pgmap v252: 65 pgs: 65 active+clean; 112 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 466 op/s 2024-05-25T06:10:58.127 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: pgmap v253: 65 pgs: 65 active+clean; 124 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 477 op/s 2024-05-25T06:10:58.127 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex restarted 2024-05-25T06:10:58.128 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:10:58.128 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:10:58.128 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:10:58.128 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:10:58.128 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:57 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:10:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: pgmap v253: 65 pgs: 65 active+clean; 124 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 477 op/s 2024-05-25T06:10:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: Standby manager daemon smithi203.rotqex restarted 2024-05-25T06:10:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:10:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:10:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:10:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:10:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:57 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:10:59.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:58 smithi203 ceph-mon[36030]: mgrmap e20: smithi160.nalzlg(active, since 7m), standbys: smithi203.rotqex 2024-05-25T06:10:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:58 smithi160 ceph-mon[29041]: mgrmap e20: smithi160.nalzlg(active, since 7m), standbys: smithi203.rotqex 2024-05-25T06:11:00.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:59 smithi160 ceph-mon[29041]: pgmap v254: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-05-25T06:11:00.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:10:59 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.364 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:59 smithi203 ceph-mon[36030]: pgmap v254: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-05-25T06:11:00.365 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.365 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.365 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:00.365 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:10:59 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:02.049 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:01 smithi203 ceph-mon[36030]: pgmap v255: 65 pgs: 65 active+clean; 137 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 471 op/s 2024-05-25T06:11:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:01 smithi160 ceph-mon[29041]: pgmap v255: 65 pgs: 65 active+clean; 137 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 471 op/s 2024-05-25T06:11:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:03 smithi160 ceph-mon[29041]: pgmap v256: 65 pgs: 65 active+clean; 140 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-05-25T06:11:04.322 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:03 smithi203 ceph-mon[36030]: pgmap v256: 65 pgs: 65 active+clean; 140 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-05-25T06:11:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr fail", "who": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: Activating manager daemon smithi203.rotqex 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: osdmap e47: 6 total, 6 up, 6 in 2024-05-25T06:11:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "mgr fail", "who": "smithi160.nalzlg"}]': finished 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: mgrmap e21: smithi203.rotqex(active, starting, since 0.00745111s) 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:11:05.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: Manager daemon smithi203.rotqex is now available 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/trash_purge_schedule"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/trash_purge_schedule"}]: dispatch 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: Deploying cephadm binary to smithi160 2024-05-25T06:11:05.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:05 smithi203 ceph-mon[36030]: Deploying cephadm binary to smithi203 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr fail", "who": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: Activating manager daemon smithi203.rotqex 2024-05-25T06:11:05.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: osdmap e47: 6 total, 6 up, 6 in 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.14210 172.21.15.160:0/1126066602' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "mgr fail", "who": "smithi160.nalzlg"}]': finished 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: mgrmap e21: smithi203.rotqex(active, starting, since 0.00745111s) 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:11:05.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: Manager daemon smithi203.rotqex is now available 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/trash_purge_schedule"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi203.rotqex/trash_purge_schedule"}]: dispatch 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: Deploying cephadm binary to smithi160 2024-05-25T06:11:05.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:05 smithi160 ceph-mon[29041]: Deploying cephadm binary to smithi203 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.796+0000 7f0908bb5700 1 -- 172.21.15.160:0/3872625559 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0900012410 con 0x7f09040fefd0 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.796+0000 7f090be1b700 1 -- 172.21.15.160:0/3872625559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09040fefd0 msgr2=0x7f09040ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.796+0000 7f090be1b700 1 --2- 172.21.15.160:0/3872625559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09040fefd0 0x7f09040ff3e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0900009ce0 tx=0x7f0900004d10 comp rx=0 tx=0).stop 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.797+0000 7f090be1b700 1 -- 172.21.15.160:0/3872625559 shutdown_connections 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.797+0000 7f090be1b700 1 --2- 172.21.15.160:0/3872625559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0904100280 0x7f09041006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.797+0000 7f090be1b700 1 --2- 172.21.15.160:0/3872625559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09040fefd0 0x7f09040ff3e0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0900009ce0 tx=0x7f0900004d10 comp rx=0 tx=0).stop 2024-05-25T06:11:06.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.797+0000 7f090be1b700 1 -- 172.21.15.160:0/3872625559 >> 172.21.15.160:0/3872625559 conn(0x7f09040fa8f0 msgr2=0x7f09040fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:06.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.797+0000 7f090be1b700 1 -- 172.21.15.160:0/3872625559 shutdown_connections 2024-05-25T06:11:06.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.797+0000 7f090be1b700 1 -- 172.21.15.160:0/3872625559 wait complete. 2024-05-25T06:11:06.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.798+0000 7f090be1b700 1 Processor -- start 2024-05-25T06:11:06.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f090be1b700 1 -- start start 2024-05-25T06:11:06.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:06 smithi160 ceph-mon[29041]: [25/May/2024:06:11:05] ENGINE Bus STARTING 2024-05-25T06:11:06.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:06 smithi160 ceph-mon[29041]: [25/May/2024:06:11:05] ENGINE Serving on http://172.21.15.203:8765 2024-05-25T06:11:06.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:06 smithi160 ceph-mon[29041]: mgrmap e22: smithi203.rotqex(active, since 1.0097s) 2024-05-25T06:11:06.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:06 smithi160 ceph-mon[29041]: pgmap v3: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:06.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:06 smithi160 ceph-mon[29041]: [25/May/2024:06:11:05] ENGINE Serving on https://172.21.15.203:7150 2024-05-25T06:11:06.799 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:06 smithi160 ceph-mon[29041]: [25/May/2024:06:11:05] ENGINE Bus STARTED 2024-05-25T06:11:06.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f090be1b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0904100280 0x7f090410fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:06.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f090be1b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09041103a0 0x7f0904111110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:06.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f090be1b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904110860 con 0x7f0904100280 2024-05-25T06:11:06.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f090be1b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041109d0 con 0x7f09041103a0 2024-05-25T06:11:06.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f09093b6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09041103a0 0x7f0904111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:06.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f09093b6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09041103a0 0x7f0904111110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:33110/0 (socket says 172.21.15.160:33110) 2024-05-25T06:11:06.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f09093b6700 1 -- 172.21.15.160:0/438243606 learned_addr learned my addr 172.21.15.160:0/438243606 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:06.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.799+0000 7f0909bb7700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0904100280 0x7f090410fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:06.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.800+0000 7f0909bb7700 1 -- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09041103a0 msgr2=0x7f0904111110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:06.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.800+0000 7f0909bb7700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09041103a0 0x7f0904111110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:06.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.800+0000 7f0909bb7700 1 -- 172.21.15.160:0/438243606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0900009940 con 0x7f0904100280 2024-05-25T06:11:06.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.800+0000 7f0909bb7700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0904100280 0x7f090410fe60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0900008000 tx=0x7f0900004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:06.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.801+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090000e030 con 0x7f0904100280 2024-05-25T06:11:06.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.802+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0900007e80 con 0x7f0904100280 2024-05-25T06:11:06.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.802+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09000128d0 con 0x7f0904100280 2024-05-25T06:11:06.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.802+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0904111790 con 0x7f0904100280 2024-05-25T06:11:06.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.802+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0904111d20 con 0x7f0904100280 2024-05-25T06:11:06.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.803+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 23) v1 ==== 46241+0+0 (secure 0 0 0) 0x7f0900019070 con 0x7f0904100280 2024-05-25T06:11:06.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.804+0000 7f08fa7fc700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f08f0038f10 0x7f08f003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:06.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.804+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f090004e980 con 0x7f0904100280 2024-05-25T06:11:06.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.804+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09041082d0 con 0x7f0904100280 2024-05-25T06:11:06.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.804+0000 7f09093b6700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f08f0038f10 0x7f08f003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:06.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.806+0000 7f09093b6700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f08f0038f10 0x7f08f003b3c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f08f4008fd0 tx=0x7f08f400b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:06.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:06.813+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f0900020080 con 0x7f0904100280 2024-05-25T06:11:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:06 smithi203 ceph-mon[36030]: [25/May/2024:06:11:05] ENGINE Bus STARTING 2024-05-25T06:11:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:06 smithi203 ceph-mon[36030]: [25/May/2024:06:11:05] ENGINE Serving on http://172.21.15.203:8765 2024-05-25T06:11:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:06 smithi203 ceph-mon[36030]: mgrmap e22: smithi203.rotqex(active, since 1.0097s) 2024-05-25T06:11:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:06 smithi203 ceph-mon[36030]: pgmap v3: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:06 smithi203 ceph-mon[36030]: [25/May/2024:06:11:05] ENGINE Serving on https://172.21.15.203:7150 2024-05-25T06:11:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:06 smithi203 ceph-mon[36030]: [25/May/2024:06:11:05] ENGINE Bus STARTED 2024-05-25T06:11:07.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.192+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0904107bd0 con 0x7f08f0038f10 2024-05-25T06:11:07.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.208+0000 7f08fa7fc700 1 -- 172.21.15.160:0/438243606 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0904107bd0 con 0x7f08f0038f10 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f08f0038f10 msgr2=0x7f08f003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f08f0038f10 0x7f08f003b3c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f08f4008fd0 tx=0x7f08f400b040 comp rx=0 tx=0).stop 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0904100280 msgr2=0x7f090410fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0904100280 0x7f090410fe60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0900008000 tx=0x7f0900004d10 comp rx=0 tx=0).stop 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 shutdown_connections 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f08f0038f10 0x7f08f003b3c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0904100280 0x7f090410fe60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 --2- 172.21.15.160:0/438243606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09041103a0 0x7f0904111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 >> 172.21.15.160:0/438243606 conn(0x7f09040fa8f0 msgr2=0x7f09040fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:07.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 shutdown_connections 2024-05-25T06:11:07.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.212+0000 7f090be1b700 1 -- 172.21.15.160:0/438243606 wait complete. 2024-05-25T06:11:07.226 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:11:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f963dd9b700 1 -- 172.21.15.160:0/3508561768 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963400bcb0 con 0x7f96400fdfc0 2024-05-25T06:11:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f9645249700 1 -- 172.21.15.160:0/3508561768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 msgr2=0x7f96400fe430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f9645249700 1 --2- 172.21.15.160:0/3508561768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f96400fe430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9634004940 tx=0x7f9634009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f9645249700 1 -- 172.21.15.160:0/3508561768 shutdown_connections 2024-05-25T06:11:07.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f9645249700 1 --2- 172.21.15.160:0/3508561768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f96400fe430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f9645249700 1 --2- 172.21.15.160:0/3508561768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f96400fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.374+0000 7f9645249700 1 -- 172.21.15.160:0/3508561768 >> 172.21.15.160:0/3508561768 conn(0x7f96400f8510 msgr2=0x7f96400fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.375+0000 7f9645249700 1 -- 172.21.15.160:0/3508561768 shutdown_connections 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.375+0000 7f9645249700 1 -- 172.21.15.160:0/3508561768 wait complete. 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.375+0000 7f9645249700 1 Processor -- start 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.375+0000 7f9645249700 1 -- start start 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f9645249700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f964010fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f9645249700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f9640110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:07.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f9645249700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9640110660 con 0x7f96400fdfc0 2024-05-25T06:11:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f9645249700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96401107d0 con 0x7f96400fce50 2024-05-25T06:11:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f964010fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f964010fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:33132/0 (socket says 172.21.15.160:33132) 2024-05-25T06:11:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963ed9d700 1 -- 172.21.15.160:0/2375058604 learned_addr learned my addr 172.21.15.160:0/2375058604 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963e59c700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f9640110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963ed9d700 1 -- 172.21.15.160:0/2375058604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 msgr2=0x7f9640110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963ed9d700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f9640110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963ed9d700 1 -- 172.21.15.160:0/2375058604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634005040 con 0x7f96400fce50 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.376+0000 7f963e59c700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f9640110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.377+0000 7f963ed9d700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f964010fb20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9630002a50 tx=0x7f96300046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.377+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9630007cc0 con 0x7f96400fce50 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.377+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9640115230 con 0x7f96400fce50 2024-05-25T06:11:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.377+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f963000f040 con 0x7f96400fce50 2024-05-25T06:11:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.377+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401157f0 con 0x7f96400fce50 2024-05-25T06:11:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.378+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9630009640 con 0x7f96400fce50 2024-05-25T06:11:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.379+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) v1 ==== 46241+0+0 (secure 0 0 0) 0x7f9630029570 con 0x7f96400fce50 2024-05-25T06:11:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.379+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9640101950 con 0x7f96400fce50 2024-05-25T06:11:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.379+0000 7f96277fe700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f9628038ed0 0x7f962803b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:07.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.380+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f963004d1d0 con 0x7f96400fce50 2024-05-25T06:11:07.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.380+0000 7f963e59c700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f9628038ed0 0x7f962803b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:07.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.381+0000 7f963e59c700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f9628038ed0 0x7f962803b380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f963400b3b0 tx=0x7f9634009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:07.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.386+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f96300192b0 con 0x7f96400fce50 2024-05-25T06:11:07.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.779+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f964002b450 con 0x7f9628038ed0 2024-05-25T06:11:07.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.790+0000 7f96277fe700 1 -- 172.21.15.160:0/2375058604 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f964002b450 con 0x7f9628038ed0 2024-05-25T06:11:07.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f9628038ed0 msgr2=0x7f962803b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f9628038ed0 0x7f962803b380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f963400b3b0 tx=0x7f9634009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 msgr2=0x7f964010fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f964010fb20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9630002a50 tx=0x7f96300046b0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 shutdown_connections 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96400fce50 0x7f964010fb20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f9628038ed0 0x7f962803b380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 --2- 172.21.15.160:0/2375058604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96400fdfc0 0x7f9640110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.793+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 >> 172.21.15.160:0/2375058604 conn(0x7f96400f8510 msgr2=0x7f96400f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.794+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 shutdown_connections 2024-05-25T06:11:07.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.794+0000 7f9645249700 1 -- 172.21.15.160:0/2375058604 wait complete. 2024-05-25T06:11:07.886 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:07 smithi160 ceph-mon[29041]: pgmap v4: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:07.886 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:07 smithi160 ceph-mon[29041]: mgrmap e23: smithi203.rotqex(active, since 2s) 2024-05-25T06:11:07.886 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:07 smithi160 ceph-mon[29041]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:07.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:07 smithi203 ceph-mon[36030]: pgmap v4: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:07.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:07 smithi203 ceph-mon[36030]: mgrmap e23: smithi203.rotqex(active, since 2s) 2024-05-25T06:11:07.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:07 smithi203 ceph-mon[36030]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 -- 172.21.15.160:0/3176875967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fe090 msgr2=0x7f21f40fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 --2- 172.21.15.160:0/3176875967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fe090 0x7f21f40fe500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f21e8004940 tx=0x7f21e8009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 -- 172.21.15.160:0/3176875967 shutdown_connections 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 --2- 172.21.15.160:0/3176875967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fe090 0x7f21f40fe500 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 --2- 172.21.15.160:0/3176875967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f40fce90 0x7f21f40fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 -- 172.21.15.160:0/3176875967 >> 172.21.15.160:0/3176875967 conn(0x7f21f40f85c0 msgr2=0x7f21f40fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 -- 172.21.15.160:0/3176875967 shutdown_connections 2024-05-25T06:11:07.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.947+0000 7f21f99a6700 1 -- 172.21.15.160:0/3176875967 wait complete. 2024-05-25T06:11:07.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f99a6700 1 Processor -- start 2024-05-25T06:11:07.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f99a6700 1 -- start start 2024-05-25T06:11:07.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f99a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 0x7f21f4113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f99a6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f40fe090 0x7f21f41144e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f99a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4114ae0 con 0x7f21f40fce90 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f99a6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4115330 con 0x7f21f40fe090 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 0x7f21f4113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 0x7f21f4113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38416/0 (socket says 172.21.15.160:38416) 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f2ffd700 1 -- 172.21.15.160:0/611201583 learned_addr learned my addr 172.21.15.160:0/611201583 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.948+0000 7f21f27fc700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f40fe090 0x7f21f41144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.949+0000 7f21f2ffd700 1 -- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f40fe090 msgr2=0x7f21f41144e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.949+0000 7f21f2ffd700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f40fe090 0x7f21f41144e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.949+0000 7f21f2ffd700 1 -- 172.21.15.160:0/611201583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8005040 con 0x7f21f40fce90 2024-05-25T06:11:07.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.949+0000 7f21f2ffd700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 0x7f21f4113fa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f21e400eb70 tx=0x7f21e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:07.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.950+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e4007cc0 con 0x7f21f40fce90 2024-05-25T06:11:07.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.950+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21e400f040 con 0x7f21f40fce90 2024-05-25T06:11:07.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.950+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e400caa0 con 0x7f21f40fce90 2024-05-25T06:11:07.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.950+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f41155b0 con 0x7f21f40fce90 2024-05-25T06:11:07.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.950+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f4115af0 con 0x7f21f40fce90 2024-05-25T06:11:07.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.951+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 23) v1 ==== 46241+0+0 (secure 0 0 0) 0x7f21e400cc00 con 0x7f21f40fce90 2024-05-25T06:11:07.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.951+0000 7f21db7fe700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f21dc038ed0 0x7f21dc03b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:07.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.951+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f21e404c530 con 0x7f21f40fce90 2024-05-25T06:11:07.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.951+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f4101a20 con 0x7f21f40fce90 2024-05-25T06:11:07.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.952+0000 7f21f27fc700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f21dc038ed0 0x7f21dc03b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:07.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.953+0000 7f21f27fc700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f21dc038ed0 0x7f21dc03b380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21e8000c00 tx=0x7f21e8009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:07.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:07.958+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f21e4019080 con 0x7f21f40fce90 2024-05-25T06:11:08.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.299+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f21f4115d60 con 0x7f21dc038ed0 2024-05-25T06:11:08.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.310+0000 7f21db7fe700 1 -- 172.21.15.160:0/611201583 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f21f4115d60 con 0x7f21dc038ed0 2024-05-25T06:11:08.310 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:11:08.310 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (6m) 2m ago 8m 24.9M - 0.25.0 c8568f914cd2 86108650b602 2024-05-25T06:11:08.310 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (8m) 2m ago 8m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:11:08.310 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (7m) 9s ago 7m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (6m) 2m ago 7m 82.7M - 9.4.7 954c08fa6188 59fd8b61d321 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (2m) 2m ago 2m 11.3M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (2m) 2m ago 2m 17.1M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (2m) 9s ago 2m 40.6M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (2m) 9s ago 2m 73.9M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:9283 running (9m) 2m ago 9m 482M - 17.2.7-958-g7b01852c baf1547092e7 fbeba8913b6d 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (23s) 9s ago 7m 88.3M - 19.0.0-2843-gea0684c0 f9a9809cafd6 fbc025680ca5 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (9m) 2m ago 9m 46.0M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (7m) 9s ago 7m 36.7M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (8m) 2m ago 8m 15.9M - 1.5.0 0da6a335fe13 3a8c0c37653a 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (7m) 9s ago 7m 15.5M - 1.5.0 0da6a335fe13 aec5d8f8dcff 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (6m) 2m ago 6m 46.5M 4288M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:11:08.311 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (5m) 2m ago 5m 45.8M 4288M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:11:08.312 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (5m) 2m ago 5m 44.9M 4288M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:11:08.312 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (5m) 9s ago 5m 92.1M 5312M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:11:08.312 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (4m) 9s ago 4m 88.1M 5312M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:11:08.312 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (4m) 9s ago 4m 87.2M 5312M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:11:08.312 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (6m) 2m ago 7m 57.1M - 2.43.0 a07b618ecd1d 6cc9a4b6650c 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.313+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f21dc038ed0 msgr2=0x7f21dc03b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.313+0000 7f21f99a6700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f21dc038ed0 0x7f21dc03b380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21e8000c00 tx=0x7f21e8009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 msgr2=0x7f21f4113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 0x7f21f4113fa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f21e400eb70 tx=0x7f21e400b040 comp rx=0 tx=0).stop 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 shutdown_connections 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f21dc038ed0 0x7f21dc03b380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21f40fce90 0x7f21f4113fa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 --2- 172.21.15.160:0/611201583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f40fe090 0x7f21f41144e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 >> 172.21.15.160:0/611201583 conn(0x7f21f40f85c0 msgr2=0x7f21f40f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.314+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 shutdown_connections 2024-05-25T06:11:08.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.317+0000 7f21f99a6700 1 -- 172.21.15.160:0/611201583 wait complete. 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2724029183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fbcc0 msgr2=0x7fe5f40fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2724029183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f40fc0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4004940 tx=0x7fe5e4009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2724029183 shutdown_connections 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2724029183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fcf70 0x7fe5f40fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2724029183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f40fc0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2724029183 >> 172.21.15.160:0/2724029183 conn(0x7fe5f40f7440 msgr2=0x7fe5f40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2724029183 shutdown_connections 2024-05-25T06:11:08.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.435+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2724029183 wait complete. 2024-05-25T06:11:08.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.436+0000 7fe5fa5c1700 1 Processor -- start 2024-05-25T06:11:08.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.436+0000 7fe5fa5c1700 1 -- start start 2024-05-25T06:11:08.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.436+0000 7fe5fa5c1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f410c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:08.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.436+0000 7fe5fa5c1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fcf70 0x7fe5f410cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5fa5c1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f410d3f0 con 0x7fe5f40fcf70 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5fa5c1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f410d560 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f410c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f410c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:33158/0 (socket says 172.21.15.160:33158) 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f3fff700 1 -- 172.21.15.160:0/2577304385 learned_addr learned my addr 172.21.15.160:0/2577304385 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f37fe700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fcf70 0x7fe5f410cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f3fff700 1 -- 172.21.15.160:0/2577304385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fcf70 msgr2=0x7fe5f410cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f3fff700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fcf70 0x7fe5f410cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.437+0000 7fe5f3fff700 1 -- 172.21.15.160:0/2577304385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e4005040 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.438+0000 7fe5f3fff700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f410c8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009150 tx=0x7fe5e4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.438+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e400e040 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.438+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5e400be00 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.438+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e4013780 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.438+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f4066270 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.438+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f40667d0 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.439+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f404c710 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.439+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) v1 ==== 46241+0+0 (secure 0 0 0) 0x7fe5e401eb80 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.440+0000 7fe5f0ff9700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fe5dc038e80 0x7fe5dc03b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:08.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.440+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe5e40508f0 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.440+0000 7fe5f37fe700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fe5dc038e80 0x7fe5dc03b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:08.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.441+0000 7fe5f37fe700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fe5dc038e80 0x7fe5dc03b330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8004f60 tx=0x7fe5e800b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:08.447 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.447+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fe5e401e420 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.560+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 24) v1 ==== 46297+0+0 (secure 0 0 0) 0x7fe5e402ed40 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:08 smithi203 ceph-mon[36030]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:08.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:08 smithi203 ceph-mon[36030]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:08.903 INFO:tasks.workunit.client.1.smithi203.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-25T06:11:08.909 INFO:tasks.workunit.client.1.smithi203.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-25T06:11:08.910 INFO:tasks.workunit.client.1.smithi203.stderr:+ make 2024-05-25T06:11:08.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.928+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe5f4066a40 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.929 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:08 smithi160 ceph-mon[29041]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:08.929 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:08 smithi160 ceph-mon[29041]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:08.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.929+0000 7fe5f0ff9700 1 -- 172.21.15.160:0/2577304385 <== mon.1 v2:172.21.15.203:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7fe5e4011550 con 0x7fe5f40fbcc0 2024-05-25T06:11:08.929 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:11:08.929 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:11:08.930 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:08.931 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:11:08.931 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 12, 2024-05-25T06:11:08.931 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:11:08.931 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:11:08.931 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:11:08.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.932+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fe5dc038e80 msgr2=0x7fe5dc03b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:08.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.932+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fe5dc038e80 0x7fe5dc03b330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8004f60 tx=0x7fe5e800b040 comp rx=0 tx=0).stop 2024-05-25T06:11:08.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.932+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 msgr2=0x7fe5f410c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:08.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.932+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f410c8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009150 tx=0x7fe5e4009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:08.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 shutdown_connections 2024-05-25T06:11:08.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5f40fbcc0 0x7fe5f410c8b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fe5dc038e80 0x7fe5dc03b330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 --2- 172.21.15.160:0/2577304385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f40fcf70 0x7fe5f410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:08.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 >> 172.21.15.160:0/2577304385 conn(0x7fe5f40f7440 msgr2=0x7fe5f40f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:08.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 shutdown_connections 2024-05-25T06:11:08.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:08.933+0000 7fe5fa5c1700 1 -- 172.21.15.160:0/2577304385 wait complete. 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.094+0000 7fcbedb25700 1 -- 172.21.15.160:0/2596641240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 msgr2=0x7fcbe80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.094+0000 7fcbedb25700 1 --2- 172.21.15.160:0/2596641240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe80fc0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8004940 tx=0x7fcbd8009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.094+0000 7fcbedb25700 1 -- 172.21.15.160:0/2596641240 shutdown_connections 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.094+0000 7fcbedb25700 1 --2- 172.21.15.160:0/2596641240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 0x7fcbe80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.094+0000 7fcbedb25700 1 --2- 172.21.15.160:0/2596641240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe80fc0c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.094+0000 7fcbedb25700 1 -- 172.21.15.160:0/2596641240 >> 172.21.15.160:0/2596641240 conn(0x7fcbe80f7430 msgr2=0x7fcbe80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.095+0000 7fcbedb25700 1 -- 172.21.15.160:0/2596641240 shutdown_connections 2024-05-25T06:11:09.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.095+0000 7fcbedb25700 1 -- 172.21.15.160:0/2596641240 wait complete. 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.095+0000 7fcbedb25700 1 Processor -- start 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.096+0000 7fcbedb25700 1 -- start start 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.096+0000 7fcbedb25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.096+0000 7fcbedb25700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 0x7fcbe810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.096+0000 7fcbe77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.096+0000 7fcbe77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38446/0 (socket says 172.21.15.160:38446) 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.096+0000 7fcbe77fe700 1 -- 172.21.15.160:0/518571492 learned_addr learned my addr 172.21.15.160:0/518571492 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbe6ffd700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 0x7fcbe810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:09.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe810e4b0 con 0x7fcbe80fbcb0 2024-05-25T06:11:09.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe810ed00 con 0x7fcbe80fcf60 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbe6ffd700 1 -- 172.21.15.160:0/518571492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 msgr2=0x7fcbe810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbe6ffd700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe810d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbe6ffd700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc005040 con 0x7fcbe80fcf60 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.097+0000 7fcbe77fe700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.098+0000 7fcbe6ffd700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 0x7fcbe810deb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0094a0 tx=0x7fcbdc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.098+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbdc00e600 con 0x7fcbe80fcf60 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.098+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbdc00f040 con 0x7fcbe80fcf60 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.098+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8005040 con 0x7fcbe80fcf60 2024-05-25T06:11:09.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.098+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbdc013c40 con 0x7fcbe80fcf60 2024-05-25T06:11:09.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.098+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe810f330 con 0x7fcbe80fcf60 2024-05-25T06:11:09.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.099+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) v1 ==== 46297+0+0 (secure 0 0 0) 0x7fcbdc01c410 con 0x7fcbe80fcf60 2024-05-25T06:11:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.099+0000 7fcbcffff700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fcbd0038f10 0x7fcbd003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.099+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fcbdc04c8e0 con 0x7fcbe80fcf60 2024-05-25T06:11:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.100+0000 7fcbe77fe700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fcbd0038f10 0x7fcbd003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.100+0000 7fcbe77fe700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fcbd0038f10 0x7fcbd003b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800b0d0 tx=0x7fcbd8009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.100+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe8107340 con 0x7fcbe80fcf60 2024-05-25T06:11:09.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.107+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fcbdc01b370 con 0x7fcbe80fcf60 2024-05-25T06:11:09.293 INFO:tasks.workunit.client.1.smithi203.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-25T06:11:09.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.429+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbe810f550 con 0x7fcbd0038f10 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.435+0000 7fcbcffff700 1 -- 172.21.15.160:0/518571492 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fcbe810f550 con 0x7fcbd0038f10 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "1/2 daemons upgraded", 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "", 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:11:09.435 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fcbd0038f10 msgr2=0x7fcbd003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fcbd0038f10 0x7fcbd003b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800b0d0 tx=0x7fcbd8009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 msgr2=0x7fcbe810deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 0x7fcbe810deb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0094a0 tx=0x7fcbdc009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 shutdown_connections 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fcbd0038f10 0x7fcbd003b3c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe80fbcb0 0x7fcbe810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:09.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 --2- 172.21.15.160:0/518571492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80fcf60 0x7fcbe810deb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:09.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 >> 172.21.15.160:0/518571492 conn(0x7fcbe80f7430 msgr2=0x7fcbe80f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:09.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 shutdown_connections 2024-05-25T06:11:09.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:09.438+0000 7fcbedb25700 1 -- 172.21.15.160:0/518571492 wait complete. 2024-05-25T06:11:09.694 INFO:tasks.workunit.client.1.smithi203.stderr:++ readlink -f fsstress 2024-05-25T06:11:09.696 INFO:tasks.workunit.client.1.smithi203.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-25T06:11:09.696 INFO:tasks.workunit.client.1.smithi203.stderr:+ popd 2024-05-25T06:11:09.697 INFO:tasks.workunit.client.1.smithi203.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-25T06:11:09.697 INFO:tasks.workunit.client.1.smithi203.stderr:+ popd 2024-05-25T06:11:09.697 INFO:tasks.workunit.client.1.smithi203.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-25T06:11:09.698 INFO:tasks.workunit.client.1.smithi203.stderr:++ mktemp -d -p . 2024-05-25T06:11:09.702 INFO:tasks.workunit.client.1.smithi203.stderr:+ T=./tmp.Gkhzq4SuAA 2024-05-25T06:11:09.702 INFO:tasks.workunit.client.1.smithi203.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Gkhzq4SuAA -l 1 -n 1000 -p 10 -v 2024-05-25T06:11:09.706 INFO:tasks.workunit.client.1.smithi203.stdout:seed = 1717259206 2024-05-25T06:11:09.712 INFO:tasks.workunit.client.1.smithi203.stdout:0/0: mkdir d0 0 2024-05-25T06:11:09.713 INFO:tasks.workunit.client.1.smithi203.stdout:0/1: rename d0 to d0/d1 22 2024-05-25T06:11:09.714 INFO:tasks.workunit.client.1.smithi203.stdout:2/0: chown . 22103 1 2024-05-25T06:11:09.716 INFO:tasks.workunit.client.1.smithi203.stdout:0/2: rmdir d0 0 2024-05-25T06:11:09.716 INFO:tasks.workunit.client.1.smithi203.stdout:0/3: unlink - no file 2024-05-25T06:11:09.718 INFO:tasks.workunit.client.1.smithi203.stdout:2/1: symlink l0 0 2024-05-25T06:11:09.718 INFO:tasks.workunit.client.1.smithi203.stdout:2/2: truncate - no filename 2024-05-25T06:11:09.719 INFO:tasks.workunit.client.1.smithi203.stdout:0/4: creat f2 x:0 0 0 2024-05-25T06:11:09.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:09 smithi160 ceph-mon[29041]: pgmap v5: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:09.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:09 smithi160 ceph-mon[29041]: mgrmap e24: smithi203.rotqex(active, since 4s) 2024-05-25T06:11:09.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:09 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2577304385' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:11:09.721 INFO:tasks.workunit.client.1.smithi203.stdout:3/0: creat f0 x:0 0 0 2024-05-25T06:11:09.722 INFO:tasks.workunit.client.1.smithi203.stdout:0/5: symlink l3 0 2024-05-25T06:11:09.722 INFO:tasks.workunit.client.1.smithi203.stdout:0/6: fdatasync f2 0 2024-05-25T06:11:09.723 INFO:tasks.workunit.client.1.smithi203.stdout:5/0: symlink l0 0 2024-05-25T06:11:09.723 INFO:tasks.workunit.client.1.smithi203.stdout:3/1: creat f1 x:0 0 0 2024-05-25T06:11:09.723 INFO:tasks.workunit.client.1.smithi203.stdout:3/2: rmdir - no directory 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/0: stat - no entries 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/1: fdatasync - no filename 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/2: fdatasync - no filename 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/3: dread - no filename 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/4: stat - no entries 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/5: rmdir - no directory 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/6: rename - no filename 2024-05-25T06:11:09.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/7: fsync - no filename 2024-05-25T06:11:09.726 INFO:tasks.workunit.client.1.smithi203.stdout:0/7: creat f4 x:0 0 0 2024-05-25T06:11:09.726 INFO:tasks.workunit.client.1.smithi203.stdout:0/8: truncate f4 642901 0 2024-05-25T06:11:09.726 INFO:tasks.workunit.client.1.smithi203.stdout:5/1: creat f1 x:0 0 0 2024-05-25T06:11:09.726 INFO:tasks.workunit.client.1.smithi203.stdout:0/9: read - f2 zero size 2024-05-25T06:11:09.727 INFO:tasks.workunit.client.1.smithi203.stdout:3/3: mknod c2 0 2024-05-25T06:11:09.727 INFO:tasks.workunit.client.1.smithi203.stdout:3/4: dread - f1 zero size 2024-05-25T06:11:09.728 INFO:tasks.workunit.client.1.smithi203.stdout:2/3: link l0 l1 0 2024-05-25T06:11:09.729 INFO:tasks.workunit.client.1.smithi203.stdout:4/0: mkdir d0 0 2024-05-25T06:11:09.730 INFO:tasks.workunit.client.1.smithi203.stdout:5/2: mknod c2 0 2024-05-25T06:11:09.731 INFO:tasks.workunit.client.1.smithi203.stdout:5/3: read - f1 zero size 2024-05-25T06:11:09.731 INFO:tasks.workunit.client.1.smithi203.stdout:2/4: mkdir d2 0 2024-05-25T06:11:09.732 INFO:tasks.workunit.client.1.smithi203.stdout:2/5: read - no filename 2024-05-25T06:11:09.732 INFO:tasks.workunit.client.1.smithi203.stdout:2/6: fsync - no filename 2024-05-25T06:11:09.732 INFO:tasks.workunit.client.1.smithi203.stdout:5/4: chown l0 82401 1 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/0: write - no filename 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/1: link - no file 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/2: dwrite - no filename 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/3: dread - no filename 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/4: dread - no filename 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/5: dwrite - no filename 2024-05-25T06:11:09.734 INFO:tasks.workunit.client.1.smithi203.stdout:6/8: creat f0 x:0 0 0 2024-05-25T06:11:09.735 INFO:tasks.workunit.client.1.smithi203.stdout:5/5: creat f3 x:0 0 0 2024-05-25T06:11:09.735 INFO:tasks.workunit.client.1.smithi203.stdout:6/9: dread - f0 zero size 2024-05-25T06:11:09.737 INFO:tasks.workunit.client.1.smithi203.stdout:9/0: dwrite - no filename 2024-05-25T06:11:09.737 INFO:tasks.workunit.client.1.smithi203.stdout:7/0: creat f0 x:0 0 0 2024-05-25T06:11:09.738 INFO:tasks.workunit.client.1.smithi203.stdout:7/1: stat f0 0 2024-05-25T06:11:09.738 INFO:tasks.workunit.client.1.smithi203.stdout:2/7: getdents d2 0 2024-05-25T06:11:09.741 INFO:tasks.workunit.client.1.smithi203.stdout:3/5: dwrite f1 [0,4194304] 0 2024-05-25T06:11:09.741 INFO:tasks.workunit.client.1.smithi203.stdout:5/6: symlink l4 0 2024-05-25T06:11:09.742 INFO:tasks.workunit.client.1.smithi203.stdout:3/6: write f1 [651922,19711] 0 2024-05-25T06:11:09.742 INFO:tasks.workunit.client.1.smithi203.stdout:8/6: creat f0 x:0 0 0 2024-05-25T06:11:09.742 INFO:tasks.workunit.client.1.smithi203.stdout:5/7: chown c2 796681 1 2024-05-25T06:11:09.742 INFO:tasks.workunit.client.1.smithi203.stdout:5/8: dread - f3 zero size 2024-05-25T06:11:09.742 INFO:tasks.workunit.client.1.smithi203.stdout:3/7: write f0 [747636,25915] 0 2024-05-25T06:11:09.743 INFO:tasks.workunit.client.1.smithi203.stdout:7/2: mkdir d1 0 2024-05-25T06:11:09.745 INFO:tasks.workunit.client.1.smithi203.stdout:7/3: truncate f0 461092 0 2024-05-25T06:11:09.745 INFO:tasks.workunit.client.1.smithi203.stdout:9/1: mkdir d0 0 2024-05-25T06:11:09.746 INFO:tasks.workunit.client.1.smithi203.stdout:7/4: truncate f0 850612 0 2024-05-25T06:11:09.747 INFO:tasks.workunit.client.1.smithi203.stdout:8/7: creat f1 x:0 0 0 2024-05-25T06:11:09.748 INFO:tasks.workunit.client.1.smithi203.stdout:8/8: chown f1 169 1 2024-05-25T06:11:09.748 INFO:tasks.workunit.client.1.smithi203.stdout:2/8: unlink l0 0 2024-05-25T06:11:09.750 INFO:tasks.workunit.client.1.smithi203.stdout:5/9: mknod c5 0 2024-05-25T06:11:09.751 INFO:tasks.workunit.client.1.smithi203.stdout:3/8: symlink l3 0 2024-05-25T06:11:09.752 INFO:tasks.workunit.client.1.smithi203.stdout:5/10: write f3 [281126,6948] 0 2024-05-25T06:11:09.754 INFO:tasks.workunit.client.1.smithi203.stdout:5/11: write f1 [1032370,4188] 0 2024-05-25T06:11:09.755 INFO:tasks.workunit.client.1.smithi203.stdout:9/2: symlink d0/l1 0 2024-05-25T06:11:09.755 INFO:tasks.workunit.client.1.smithi203.stdout:9/3: write - no filename 2024-05-25T06:11:09.755 INFO:tasks.workunit.client.1.smithi203.stdout:9/4: fdatasync - no filename 2024-05-25T06:11:09.755 INFO:tasks.workunit.client.1.smithi203.stdout:9/5: dwrite - no filename 2024-05-25T06:11:09.756 INFO:tasks.workunit.client.1.smithi203.stdout:7/5: creat d1/f2 x:0 0 0 2024-05-25T06:11:09.758 INFO:tasks.workunit.client.1.smithi203.stdout:9/6: chown d0/l1 979498815 1 2024-05-25T06:11:09.758 INFO:tasks.workunit.client.1.smithi203.stdout:9/7: read - no filename 2024-05-25T06:11:09.758 INFO:tasks.workunit.client.1.smithi203.stdout:9/8: write - no filename 2024-05-25T06:11:09.759 INFO:tasks.workunit.client.1.smithi203.stdout:9/9: stat d0/l1 0 2024-05-25T06:11:09.759 INFO:tasks.workunit.client.1.smithi203.stdout:2/9: stat l1 0 2024-05-25T06:11:09.760 INFO:tasks.workunit.client.1.smithi203.stdout:9/10: readlink d0/l1 0 2024-05-25T06:11:09.760 INFO:tasks.workunit.client.1.smithi203.stdout:9/11: dwrite - no filename 2024-05-25T06:11:09.761 INFO:tasks.workunit.client.1.smithi203.stdout:8/9: dwrite f1 [0,4194304] 0 2024-05-25T06:11:09.761 INFO:tasks.workunit.client.1.smithi203.stdout:9/12: chown d0 109530 1 2024-05-25T06:11:09.762 INFO:tasks.workunit.client.1.smithi203.stdout:7/6: creat d1/f3 x:0 0 0 2024-05-25T06:11:09.764 INFO:tasks.workunit.client.1.smithi203.stdout:5/12: link f1 f6 0 2024-05-25T06:11:09.766 INFO:tasks.workunit.client.1.smithi203.stdout:9/13: symlink d0/l2 0 2024-05-25T06:11:09.766 INFO:tasks.workunit.client.1.smithi203.stdout:9/14: dread - no filename 2024-05-25T06:11:09.768 INFO:tasks.workunit.client.1.smithi203.stdout:2/10: creat d2/f3 x:0 0 0 2024-05-25T06:11:09.769 INFO:tasks.workunit.client.1.smithi203.stdout:7/7: symlink d1/l4 0 2024-05-25T06:11:09.770 INFO:tasks.workunit.client.1.smithi203.stdout:7/8: dread - d1/f2 zero size 2024-05-25T06:11:09.771 INFO:tasks.workunit.client.1.smithi203.stdout:5/13: dread f3 [0,4194304] 0 2024-05-25T06:11:09.772 INFO:tasks.workunit.client.1.smithi203.stdout:9/15: symlink d0/l3 0 2024-05-25T06:11:09.773 INFO:tasks.workunit.client.1.smithi203.stdout:2/11: mkdir d2/d4 0 2024-05-25T06:11:09.774 INFO:tasks.workunit.client.1.smithi203.stdout:8/10: dwrite f0 [0,4194304] 0 2024-05-25T06:11:09.775 INFO:tasks.workunit.client.1.smithi203.stdout:5/14: dread f3 [0,4194304] 0 2024-05-25T06:11:09.776 INFO:tasks.workunit.client.1.smithi203.stdout:9/16: creat d0/f4 x:0 0 0 2024-05-25T06:11:09.777 INFO:tasks.workunit.client.1.smithi203.stdout:2/12: truncate d2/f3 562565 0 2024-05-25T06:11:09.778 INFO:tasks.workunit.client.1.smithi203.stdout:8/11: mkdir d2 0 2024-05-25T06:11:09.778 INFO:tasks.workunit.client.1.smithi203.stdout:9/17: readlink d0/l3 0 2024-05-25T06:11:09.778 INFO:tasks.workunit.client.1.smithi203.stdout:5/15: read f3 [145358,66073] 0 2024-05-25T06:11:09.780 INFO:tasks.workunit.client.1.smithi203.stdout:8/12: chown f0 10002 1 2024-05-25T06:11:09.782 INFO:tasks.workunit.client.1.smithi203.stdout:5/16: mknod c7 0 2024-05-25T06:11:09.784 INFO:tasks.workunit.client.1.smithi203.stdout:2/13: symlink d2/d4/l5 0 2024-05-25T06:11:09.784 INFO:tasks.workunit.client.1.smithi203.stdout:7/9: dwrite f0 [0,4194304] 0 2024-05-25T06:11:09.786 INFO:tasks.workunit.client.1.smithi203.stdout:7/10: write f0 [2104392,102512] 0 2024-05-25T06:11:09.789 INFO:tasks.workunit.client.1.smithi203.stdout:8/13: rename f1 to d2/f3 0 2024-05-25T06:11:09.790 INFO:tasks.workunit.client.1.smithi203.stdout:5/17: write f3 [994760,47573] 0 2024-05-25T06:11:09.792 INFO:tasks.workunit.client.1.smithi203.stdout:2/14: mkdir d2/d6 0 2024-05-25T06:11:09.795 INFO:tasks.workunit.client.1.smithi203.stdout:7/11: creat d1/f5 x:0 0 0 2024-05-25T06:11:09.797 INFO:tasks.workunit.client.1.smithi203.stdout:8/14: unlink f0 0 2024-05-25T06:11:09.798 INFO:tasks.workunit.client.1.smithi203.stdout:9/18: link d0/l3 d0/l5 0 2024-05-25T06:11:09.801 INFO:tasks.workunit.client.1.smithi203.stdout:2/15: symlink d2/d4/l7 0 2024-05-25T06:11:09.802 INFO:tasks.workunit.client.1.smithi203.stdout:2/16: write d2/f3 [577364,5620] 0 2024-05-25T06:11:09.802 INFO:tasks.workunit.client.1.smithi203.stdout:7/12: mknod d1/c6 0 2024-05-25T06:11:09.804 INFO:tasks.workunit.client.1.smithi203.stdout:5/18: dread f3 [0,4194304] 0 2024-05-25T06:11:09.804 INFO:tasks.workunit.client.1.smithi203.stdout:7/13: write d1/f5 [763135,8945] 0 2024-05-25T06:11:09.805 INFO:tasks.workunit.client.1.smithi203.stdout:7/14: dread - d1/f3 zero size 2024-05-25T06:11:09.808 INFO:tasks.workunit.client.1.smithi203.stdout:2/17: creat d2/f8 x:0 0 0 2024-05-25T06:11:09.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/15: link d2/f3 d2/f4 0 2024-05-25T06:11:09.811 INFO:tasks.workunit.client.1.smithi203.stdout:7/15: creat d1/f7 x:0 0 0 2024-05-25T06:11:09.812 INFO:tasks.workunit.client.1.smithi203.stdout:2/18: rmdir d2/d4 39 2024-05-25T06:11:09.814 INFO:tasks.workunit.client.1.smithi203.stdout:7/16: write d1/f2 [245386,80807] 0 2024-05-25T06:11:09.815 INFO:tasks.workunit.client.1.smithi203.stdout:8/16: mkdir d2/d5 0 2024-05-25T06:11:09.815 INFO:tasks.workunit.client.1.smithi203.stdout:2/19: write d2/f3 [281014,116248] 0 2024-05-25T06:11:09.817 INFO:tasks.workunit.client.1.smithi203.stdout:8/17: chown d2/f3 0 1 2024-05-25T06:11:09.820 INFO:tasks.workunit.client.1.smithi203.stdout:8/18: stat d2/d5 0 2024-05-25T06:11:09.820 INFO:tasks.workunit.client.1.smithi203.stdout:5/19: dwrite f3 [0,4194304] 0 2024-05-25T06:11:09.825 INFO:tasks.workunit.client.1.smithi203.stdout:8/19: read d2/f4 [517588,101077] 0 2024-05-25T06:11:09.829 INFO:tasks.workunit.client.1.smithi203.stdout:2/20: dwrite d2/f3 [0,4194304] 0 2024-05-25T06:11:09.830 INFO:tasks.workunit.client.1.smithi203.stdout:5/20: dread f1 [0,4194304] 0 2024-05-25T06:11:09.831 INFO:tasks.workunit.client.1.smithi203.stdout:8/20: symlink d2/d5/l6 0 2024-05-25T06:11:09.836 INFO:tasks.workunit.client.1.smithi203.stdout:8/21: symlink d2/l7 0 2024-05-25T06:11:09.839 INFO:tasks.workunit.client.1.smithi203.stdout:8/22: link d2/f3 d2/d5/f8 0 2024-05-25T06:11:09.841 INFO:tasks.workunit.client.1.smithi203.stdout:8/23: mkdir d2/d5/d9 0 2024-05-25T06:11:09.844 INFO:tasks.workunit.client.1.smithi203.stdout:8/24: write d2/f3 [2529208,2718] 0 2024-05-25T06:11:09.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:09 smithi203 ceph-mon[36030]: pgmap v5: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:09.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:09 smithi203 ceph-mon[36030]: mgrmap e24: smithi203.rotqex(active, since 4s) 2024-05-25T06:11:09.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:09 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2577304385' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:11:09.904 INFO:tasks.workunit.client.1.smithi203.stdout:3/9: fsync f1 0 2024-05-25T06:11:09.906 INFO:tasks.workunit.client.1.smithi203.stdout:0/10: getdents . 0 2024-05-25T06:11:09.908 INFO:tasks.workunit.client.1.smithi203.stdout:3/10: dread f1 [0,4194304] 0 2024-05-25T06:11:09.909 INFO:tasks.workunit.client.1.smithi203.stdout:0/11: creat f5 x:0 0 0 2024-05-25T06:11:09.909 INFO:tasks.workunit.client.1.smithi203.stdout:0/12: write f5 [671712,72522] 0 2024-05-25T06:11:09.910 INFO:tasks.workunit.client.1.smithi203.stdout:0/13: readlink l3 0 2024-05-25T06:11:09.910 INFO:tasks.workunit.client.1.smithi203.stdout:3/11: creat f4 x:0 0 0 2024-05-25T06:11:09.912 INFO:tasks.workunit.client.1.smithi203.stdout:0/14: creat f6 x:0 0 0 2024-05-25T06:11:09.914 INFO:tasks.workunit.client.1.smithi203.stdout:3/12: rename f4 to f5 0 2024-05-25T06:11:09.914 INFO:tasks.workunit.client.1.smithi203.stdout:3/13: chown c2 166429 1 2024-05-25T06:11:09.916 INFO:tasks.workunit.client.1.smithi203.stdout:0/15: symlink l7 0 2024-05-25T06:11:09.916 INFO:tasks.workunit.client.1.smithi203.stdout:6/10: fsync f0 0 2024-05-25T06:11:09.917 INFO:tasks.workunit.client.1.smithi203.stdout:0/16: write f6 [146184,127438] 0 2024-05-25T06:11:09.917 INFO:tasks.workunit.client.1.smithi203.stdout:6/11: write f0 [895992,22768] 0 2024-05-25T06:11:09.921 INFO:tasks.workunit.client.1.smithi203.stdout:6/12: creat f1 x:0 0 0 2024-05-25T06:11:09.924 INFO:tasks.workunit.client.1.smithi203.stdout:6/13: write f0 [1498619,84391] 0 2024-05-25T06:11:09.927 INFO:tasks.workunit.client.1.smithi203.stdout:6/14: fsync f0 0 2024-05-25T06:11:09.929 INFO:tasks.workunit.client.1.smithi203.stdout:0/17: dwrite f5 [0,4194304] 0 2024-05-25T06:11:09.930 INFO:tasks.workunit.client.1.smithi203.stdout:0/18: rmdir - no directory 2024-05-25T06:11:09.931 INFO:tasks.workunit.client.1.smithi203.stdout:3/14: dwrite f5 [0,4194304] 0 2024-05-25T06:11:09.933 INFO:tasks.workunit.client.1.smithi203.stdout:0/19: truncate f6 1049800 0 2024-05-25T06:11:09.935 INFO:tasks.workunit.client.1.smithi203.stdout:3/15: write f0 [1471689,20855] 0 2024-05-25T06:11:09.936 INFO:tasks.workunit.client.1.smithi203.stdout:0/20: stat f5 0 2024-05-25T06:11:09.944 INFO:tasks.workunit.client.1.smithi203.stdout:6/15: dwrite f1 [0,4194304] 0 2024-05-25T06:11:09.948 INFO:tasks.workunit.client.1.smithi203.stdout:9/19: getdents d0 0 2024-05-25T06:11:09.952 INFO:tasks.workunit.client.1.smithi203.stdout:9/20: creat d0/f6 x:0 0 0 2024-05-25T06:11:09.953 INFO:tasks.workunit.client.1.smithi203.stdout:3/16: dwrite f1 [0,4194304] 0 2024-05-25T06:11:09.953 INFO:tasks.workunit.client.1.smithi203.stdout:0/21: dwrite f6 [0,4194304] 0 2024-05-25T06:11:09.954 INFO:tasks.workunit.client.1.smithi203.stdout:6/16: link f1 f2 0 2024-05-25T06:11:09.956 INFO:tasks.workunit.client.1.smithi203.stdout:9/21: write d0/f4 [347277,57684] 0 2024-05-25T06:11:09.959 INFO:tasks.workunit.client.1.smithi203.stdout:0/22: write f4 [1281429,103176] 0 2024-05-25T06:11:09.959 INFO:tasks.workunit.client.1.smithi203.stdout:6/17: creat f3 x:0 0 0 2024-05-25T06:11:09.961 INFO:tasks.workunit.client.1.smithi203.stdout:9/22: write d0/f6 [148475,46237] 0 2024-05-25T06:11:09.962 INFO:tasks.workunit.client.1.smithi203.stdout:7/17: getdents d1 0 2024-05-25T06:11:09.963 INFO:tasks.workunit.client.1.smithi203.stdout:3/17: mknod c6 0 2024-05-25T06:11:09.963 INFO:tasks.workunit.client.1.smithi203.stdout:6/18: write f0 [796535,107095] 0 2024-05-25T06:11:09.966 INFO:tasks.workunit.client.1.smithi203.stdout:9/23: creat d0/f7 x:0 0 0 2024-05-25T06:11:09.967 INFO:tasks.workunit.client.1.smithi203.stdout:7/18: mknod d1/c8 0 2024-05-25T06:11:09.968 INFO:tasks.workunit.client.1.smithi203.stdout:3/18: dread f1 [0,4194304] 0 2024-05-25T06:11:09.976 INFO:tasks.workunit.client.1.smithi203.stdout:3/19: symlink l7 0 2024-05-25T06:11:09.977 INFO:tasks.workunit.client.1.smithi203.stdout:7/19: dwrite d1/f3 [0,4194304] 0 2024-05-25T06:11:09.981 INFO:tasks.workunit.client.1.smithi203.stdout:6/19: link f0 f4 0 2024-05-25T06:11:09.982 INFO:tasks.workunit.client.1.smithi203.stdout:7/20: dread d1/f3 [0,4194304] 0 2024-05-25T06:11:09.982 INFO:tasks.workunit.client.1.smithi203.stdout:7/21: fdatasync f0 0 2024-05-25T06:11:09.984 INFO:tasks.workunit.client.1.smithi203.stdout:9/24: rename d0/l5 to d0/l8 0 2024-05-25T06:11:09.991 INFO:tasks.workunit.client.1.smithi203.stdout:7/22: mkdir d1/d9 0 2024-05-25T06:11:09.992 INFO:tasks.workunit.client.1.smithi203.stdout:6/20: dwrite f3 [0,4194304] 0 2024-05-25T06:11:09.992 INFO:tasks.workunit.client.1.smithi203.stdout:9/25: mknod d0/c9 0 2024-05-25T06:11:09.995 INFO:tasks.workunit.client.1.smithi203.stdout:9/26: write d0/f7 [939501,76722] 0 2024-05-25T06:11:09.998 INFO:tasks.workunit.client.1.smithi203.stdout:3/20: link f1 f8 0 2024-05-25T06:11:10.000 INFO:tasks.workunit.client.1.smithi203.stdout:6/21: symlink l5 0 2024-05-25T06:11:10.002 INFO:tasks.workunit.client.1.smithi203.stdout:3/21: dread f1 [0,4194304] 0 2024-05-25T06:11:10.004 INFO:tasks.workunit.client.1.smithi203.stdout:9/27: creat d0/fa x:0 0 0 2024-05-25T06:11:10.004 INFO:tasks.workunit.client.1.smithi203.stdout:9/28: fsync d0/f6 0 2024-05-25T06:11:10.005 INFO:tasks.workunit.client.1.smithi203.stdout:6/22: creat f6 x:0 0 0 2024-05-25T06:11:10.005 INFO:tasks.workunit.client.1.smithi203.stdout:6/23: stat l5 0 2024-05-25T06:11:10.008 INFO:tasks.workunit.client.1.smithi203.stdout:3/22: symlink l9 0 2024-05-25T06:11:10.008 INFO:tasks.workunit.client.1.smithi203.stdout:3/23: fsync f0 0 2024-05-25T06:11:10.011 INFO:tasks.workunit.client.1.smithi203.stdout:6/24: symlink l7 0 2024-05-25T06:11:10.013 INFO:tasks.workunit.client.1.smithi203.stdout:9/29: dwrite d0/f7 [0,4194304] 0 2024-05-25T06:11:10.013 INFO:tasks.workunit.client.1.smithi203.stdout:6/25: mkdir d8 0 2024-05-25T06:11:10.015 INFO:tasks.workunit.client.1.smithi203.stdout:7/23: truncate d1/f3 3794665 0 2024-05-25T06:11:10.017 INFO:tasks.workunit.client.1.smithi203.stdout:6/26: unlink f3 0 2024-05-25T06:11:10.018 INFO:tasks.workunit.client.1.smithi203.stdout:6/27: dread - f6 zero size 2024-05-25T06:11:10.019 INFO:tasks.workunit.client.1.smithi203.stdout:6/28: read f0 [133212,52406] 0 2024-05-25T06:11:10.019 INFO:tasks.workunit.client.1.smithi203.stdout:6/29: chown f6 1084507 1 2024-05-25T06:11:10.022 INFO:tasks.workunit.client.1.smithi203.stdout:7/24: creat d1/fa x:0 0 0 2024-05-25T06:11:10.023 INFO:tasks.workunit.client.1.smithi203.stdout:6/30: write f0 [2217721,115579] 0 2024-05-25T06:11:10.026 INFO:tasks.workunit.client.1.smithi203.stdout:7/25: creat d1/d9/fb x:0 0 0 2024-05-25T06:11:10.027 INFO:tasks.workunit.client.1.smithi203.stdout:7/26: creat d1/d9/fc x:0 0 0 2024-05-25T06:11:10.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:10 smithi160 ceph-mon[29041]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:10.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:10 smithi203 ceph-mon[36030]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:11.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:11 smithi203 ceph-mon[36030]: pgmap v6: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:11.900 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:11 smithi160 ceph-mon[29041]: pgmap v6: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:11:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:13 smithi160 ceph-mon[29041]: pgmap v7: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 343 op/s 2024-05-25T06:11:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:13 smithi160 ceph-mon[29041]: Standby manager daemon smithi160.nalzlg started 2024-05-25T06:11:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:13 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/crt"}]: dispatch 2024-05-25T06:11:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:13 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:11:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:13 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/key"}]: dispatch 2024-05-25T06:11:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:13 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:11:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:13 smithi203 ceph-mon[36030]: pgmap v7: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 343 op/s 2024-05-25T06:11:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:13 smithi203 ceph-mon[36030]: Standby manager daemon smithi160.nalzlg started 2024-05-25T06:11:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:13 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/crt"}]: dispatch 2024-05-25T06:11:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:13 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:11:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:13 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/key"}]: dispatch 2024-05-25T06:11:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:13 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/1705474012' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:11:14.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:14 smithi203 ceph-mon[36030]: mgrmap e25: smithi203.rotqex(active, since 9s), standbys: smithi160.nalzlg 2024-05-25T06:11:14.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:14 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:11:14.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:14 smithi160 ceph-mon[29041]: mgrmap e25: smithi203.rotqex(active, since 9s), standbys: smithi160.nalzlg 2024-05-25T06:11:14.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:14 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:11:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:15 smithi203 ceph-mon[36030]: pgmap v8: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 9.3 MiB/s wr, 267 op/s 2024-05-25T06:11:15.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:15 smithi160 ceph-mon[29041]: pgmap v8: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 9.3 MiB/s wr, 267 op/s 2024-05-25T06:11:17.670 INFO:tasks.workunit.client.1.smithi203.stdout:1/0: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:1/1: write - no filename 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:4/1: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:4/2: dread - no filename 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:4/3: dwrite - no filename 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:6/31: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:0/23: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:8/25: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:3/24: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:5/21: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:9/30: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:7/27: sync 2024-05-25T06:11:17.671 INFO:tasks.workunit.client.1.smithi203.stdout:2/21: sync 2024-05-25T06:11:17.672 INFO:tasks.workunit.client.1.smithi203.stdout:6/32: chown f4 1485 1 2024-05-25T06:11:17.672 INFO:tasks.workunit.client.1.smithi203.stdout:1/2: creat f0 x:0 0 0 2024-05-25T06:11:17.672 INFO:tasks.workunit.client.1.smithi203.stdout:1/3: chown f0 3820853 1 2024-05-25T06:11:17.673 INFO:tasks.workunit.client.1.smithi203.stdout:1/4: write f0 [220414,25034] 0 2024-05-25T06:11:17.674 INFO:tasks.workunit.client.1.smithi203.stdout:4/4: symlink d0/l1 0 2024-05-25T06:11:17.674 INFO:tasks.workunit.client.1.smithi203.stdout:4/5: fdatasync - no filename 2024-05-25T06:11:17.676 INFO:tasks.workunit.client.1.smithi203.stdout:8/26: read d2/d5/f8 [3593103,89456] 0 2024-05-25T06:11:17.678 INFO:tasks.workunit.client.1.smithi203.stdout:9/31: creat d0/fb x:0 0 0 2024-05-25T06:11:17.678 INFO:tasks.workunit.client.1.smithi203.stdout:0/24: rename l3 to l8 0 2024-05-25T06:11:17.679 INFO:tasks.workunit.client.1.smithi203.stdout:3/25: dread f5 [0,4194304] 0 2024-05-25T06:11:17.680 INFO:tasks.workunit.client.1.smithi203.stdout:3/26: chown f0 2100 1 2024-05-25T06:11:17.682 INFO:tasks.workunit.client.1.smithi203.stdout:8/27: dread d2/f4 [0,4194304] 0 2024-05-25T06:11:17.682 INFO:tasks.workunit.client.1.smithi203.stdout:4/6: rename d0/l1 to d0/l2 0 2024-05-25T06:11:17.682 INFO:tasks.workunit.client.1.smithi203.stdout:4/7: write - no filename 2024-05-25T06:11:17.682 INFO:tasks.workunit.client.1.smithi203.stdout:4/8: dread - no filename 2024-05-25T06:11:17.682 INFO:tasks.workunit.client.1.smithi203.stdout:4/9: dwrite - no filename 2024-05-25T06:11:17.683 INFO:tasks.workunit.client.1.smithi203.stdout:5/22: write f1 [2006754,86206] 0 2024-05-25T06:11:17.684 INFO:tasks.workunit.client.1.smithi203.stdout:6/33: creat d8/f9 x:0 0 0 2024-05-25T06:11:17.685 INFO:tasks.workunit.client.1.smithi203.stdout:6/34: readlink l5 0 2024-05-25T06:11:17.686 INFO:tasks.workunit.client.1.smithi203.stdout:0/25: unlink f2 0 2024-05-25T06:11:17.687 INFO:tasks.workunit.client.1.smithi203.stdout:2/22: write d2/f8 [680430,79023] 0 2024-05-25T06:11:17.689 INFO:tasks.workunit.client.1.smithi203.stdout:3/27: symlink la 0 2024-05-25T06:11:17.692 INFO:tasks.workunit.client.1.smithi203.stdout:4/10: creat d0/f3 x:0 0 0 2024-05-25T06:11:17.694 INFO:tasks.workunit.client.1.smithi203.stdout:7/28: dwrite d1/d9/fb [0,4194304] 0 2024-05-25T06:11:17.695 INFO:tasks.workunit.client.1.smithi203.stdout:9/32: link d0/fb d0/fc 0 2024-05-25T06:11:17.698 INFO:tasks.workunit.client.1.smithi203.stdout:1/5: truncate f0 158983 0 2024-05-25T06:11:17.699 INFO:tasks.workunit.client.1.smithi203.stdout:2/23: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:17.701 INFO:tasks.workunit.client.1.smithi203.stdout:5/23: dwrite f1 [0,4194304] 0 2024-05-25T06:11:17.701 INFO:tasks.workunit.client.1.smithi203.stdout:3/28: rename c6 to cb 0 2024-05-25T06:11:17.702 INFO:tasks.workunit.client.1.smithi203.stdout:8/28: write d2/f3 [3305533,77669] 0 2024-05-25T06:11:17.704 INFO:tasks.workunit.client.1.smithi203.stdout:4/11: creat d0/f4 x:0 0 0 2024-05-25T06:11:17.705 INFO:tasks.workunit.client.1.smithi203.stdout:0/26: dread f5 [0,4194304] 0 2024-05-25T06:11:17.708 INFO:tasks.workunit.client.1.smithi203.stdout:8/29: fsync d2/d5/f8 0 2024-05-25T06:11:17.708 INFO:tasks.workunit.client.1.smithi203.stdout:4/12: chown d0/f4 50893740 1 2024-05-25T06:11:17.708 INFO:tasks.workunit.client.1.smithi203.stdout:6/35: write f4 [2114767,108208] 0 2024-05-25T06:11:17.708 INFO:tasks.workunit.client.1.smithi203.stdout:9/33: chown d0/l3 25916 1 2024-05-25T06:11:17.710 INFO:tasks.workunit.client.1.smithi203.stdout:4/13: fdatasync d0/f3 0 2024-05-25T06:11:17.713 INFO:tasks.workunit.client.1.smithi203.stdout:7/29: creat d1/d9/fd x:0 0 0 2024-05-25T06:11:17.714 INFO:tasks.workunit.client.1.smithi203.stdout:5/24: creat f8 x:0 0 0 2024-05-25T06:11:17.714 INFO:tasks.workunit.client.1.smithi203.stdout:0/27: rename f4 to f9 0 2024-05-25T06:11:17.716 INFO:tasks.workunit.client.1.smithi203.stdout:9/34: creat d0/fd x:0 0 0 2024-05-25T06:11:17.718 INFO:tasks.workunit.client.1.smithi203.stdout:4/14: readlink d0/l2 0 2024-05-25T06:11:17.718 INFO:tasks.workunit.client.1.smithi203.stdout:2/24: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:17.718 INFO:tasks.workunit.client.1.smithi203.stdout:5/25: symlink l9 0 2024-05-25T06:11:17.720 INFO:tasks.workunit.client.1.smithi203.stdout:8/30: dread d2/f4 [0,4194304] 0 2024-05-25T06:11:17.724 INFO:tasks.workunit.client.1.smithi203.stdout:6/36: getdents d8 0 2024-05-25T06:11:17.724 INFO:tasks.workunit.client.1.smithi203.stdout:1/6: read f0 [92384,4539] 0 2024-05-25T06:11:17.725 INFO:tasks.workunit.client.1.smithi203.stdout:2/25: symlink d2/d6/l9 0 2024-05-25T06:11:17.727 INFO:tasks.workunit.client.1.smithi203.stdout:8/31: creat d2/fa x:0 0 0 2024-05-25T06:11:17.728 INFO:tasks.workunit.client.1.smithi203.stdout:3/29: write f1 [2279069,14341] 0 2024-05-25T06:11:17.729 INFO:tasks.workunit.client.1.smithi203.stdout:6/37: write f4 [1115195,20848] 0 2024-05-25T06:11:17.729 INFO:tasks.workunit.client.1.smithi203.stdout:5/26: fsync f8 0 2024-05-25T06:11:17.731 INFO:tasks.workunit.client.1.smithi203.stdout:1/7: mknod c1 0 2024-05-25T06:11:17.733 INFO:tasks.workunit.client.1.smithi203.stdout:1/8: stat f0 0 2024-05-25T06:11:17.734 INFO:tasks.workunit.client.1.smithi203.stdout:2/26: mkdir d2/d6/da 0 2024-05-25T06:11:17.734 INFO:tasks.workunit.client.1.smithi203.stdout:7/30: dwrite d1/d9/fd [0,4194304] 0 2024-05-25T06:11:17.735 INFO:tasks.workunit.client.1.smithi203.stdout:3/30: symlink lc 0 2024-05-25T06:11:17.736 INFO:tasks.workunit.client.1.smithi203.stdout:9/35: write d0/fa [860366,124041] 0 2024-05-25T06:11:17.736 INFO:tasks.workunit.client.1.smithi203.stdout:8/32: write d2/f3 [4456936,12403] 0 2024-05-25T06:11:17.741 INFO:tasks.workunit.client.1.smithi203.stdout:4/15: dwrite d0/f4 [0,4194304] 0 2024-05-25T06:11:17.741 INFO:tasks.workunit.client.1.smithi203.stdout:1/9: symlink l2 0 2024-05-25T06:11:17.741 INFO:tasks.workunit.client.1.smithi203.stdout:9/36: chown d0/l3 146037 1 2024-05-25T06:11:17.743 INFO:tasks.workunit.client.1.smithi203.stdout:2/27: read d2/f8 [568188,102546] 0 2024-05-25T06:11:17.743 INFO:tasks.workunit.client.1.smithi203.stdout:6/38: write f2 [3038784,21707] 0 2024-05-25T06:11:17.743 INFO:tasks.workunit.client.1.smithi203.stdout:5/27: write f1 [127307,74096] 0 2024-05-25T06:11:17.744 INFO:tasks.workunit.client.1.smithi203.stdout:7/31: rmdir d1/d9 39 2024-05-25T06:11:17.748 INFO:tasks.workunit.client.1.smithi203.stdout:6/39: chown d8 1754123 1 2024-05-25T06:11:17.748 INFO:tasks.workunit.client.1.smithi203.stdout:8/33: symlink d2/d5/d9/lb 0 2024-05-25T06:11:17.750 INFO:tasks.workunit.client.1.smithi203.stdout:2/28: read d2/f3 [1053574,66641] 0 2024-05-25T06:11:17.750 INFO:tasks.workunit.client.1.smithi203.stdout:0/28: dread f9 [0,4194304] 0 2024-05-25T06:11:17.753 INFO:tasks.workunit.client.1.smithi203.stdout:1/10: dread f0 [0,4194304] 0 2024-05-25T06:11:17.753 INFO:tasks.workunit.client.1.smithi203.stdout:4/16: write d0/f3 [452710,21213] 0 2024-05-25T06:11:17.756 INFO:tasks.workunit.client.1.smithi203.stdout:6/40: symlink d8/la 0 2024-05-25T06:11:17.756 INFO:tasks.workunit.client.1.smithi203.stdout:0/29: fdatasync f5 0 2024-05-25T06:11:17.756 INFO:tasks.workunit.client.1.smithi203.stdout:3/31: dread f5 [0,4194304] 0 2024-05-25T06:11:17.758 INFO:tasks.workunit.client.1.smithi203.stdout:4/17: write d0/f3 [707574,101926] 0 2024-05-25T06:11:17.758 INFO:tasks.workunit.client.1.smithi203.stdout:1/11: mkdir d3 0 2024-05-25T06:11:17.759 INFO:tasks.workunit.client.1.smithi203.stdout:6/41: creat d8/fb x:0 0 0 2024-05-25T06:11:17.763 INFO:tasks.workunit.client.1.smithi203.stdout:0/30: symlink la 0 2024-05-25T06:11:17.763 INFO:tasks.workunit.client.1.smithi203.stdout:7/32: dread d1/d9/fb [0,4194304] 0 2024-05-25T06:11:17.764 INFO:tasks.workunit.client.1.smithi203.stdout:1/12: unlink c1 0 2024-05-25T06:11:17.765 INFO:tasks.workunit.client.1.smithi203.stdout:4/18: creat d0/f5 x:0 0 0 2024-05-25T06:11:17.767 INFO:tasks.workunit.client.1.smithi203.stdout:5/28: getdents . 0 2024-05-25T06:11:17.767 INFO:tasks.workunit.client.1.smithi203.stdout:5/29: read f1 [1824520,71926] 0 2024-05-25T06:11:17.768 INFO:tasks.workunit.client.1.smithi203.stdout:6/42: unlink d8/la 0 2024-05-25T06:11:17.770 INFO:tasks.workunit.client.1.smithi203.stdout:8/34: getdents d2 0 2024-05-25T06:11:17.771 INFO:tasks.workunit.client.1.smithi203.stdout:0/31: mknod cb 0 2024-05-25T06:11:17.771 INFO:tasks.workunit.client.1.smithi203.stdout:2/29: write d2/f8 [625197,20779] 0 2024-05-25T06:11:17.773 INFO:tasks.workunit.client.1.smithi203.stdout:7/33: dread - d1/fa zero size 2024-05-25T06:11:17.775 INFO:tasks.workunit.client.1.smithi203.stdout:4/19: dread d0/f4 [0,4194304] 0 2024-05-25T06:11:17.775 INFO:tasks.workunit.client.1.smithi203.stdout:3/32: link l7 ld 0 2024-05-25T06:11:17.775 INFO:tasks.workunit.client.1.smithi203.stdout:3/33: rmdir - no directory 2024-05-25T06:11:17.775 INFO:tasks.workunit.client.1.smithi203.stdout:2/30: stat d2/d6/l9 0 2024-05-25T06:11:17.779 INFO:tasks.workunit.client.1.smithi203.stdout:9/37: dwrite d0/f6 [0,4194304] 0 2024-05-25T06:11:17.779 INFO:tasks.workunit.client.1.smithi203.stdout:8/35: rmdir d2 39 2024-05-25T06:11:17.784 INFO:tasks.workunit.client.1.smithi203.stdout:1/13: write f0 [145618,86237] 0 2024-05-25T06:11:17.788 INFO:tasks.workunit.client.1.smithi203.stdout:4/20: symlink d0/l6 0 2024-05-25T06:11:17.788 INFO:tasks.workunit.client.1.smithi203.stdout:0/32: write f6 [2883208,83764] 0 2024-05-25T06:11:17.790 INFO:tasks.workunit.client.1.smithi203.stdout:2/31: creat d2/d6/fb x:0 0 0 2024-05-25T06:11:17.790 INFO:tasks.workunit.client.1.smithi203.stdout:5/30: getdents . 0 2024-05-25T06:11:17.791 INFO:tasks.workunit.client.1.smithi203.stdout:5/31: fsync f8 0 2024-05-25T06:11:17.795 INFO:tasks.workunit.client.1.smithi203.stdout:3/34: rename ld to le 0 2024-05-25T06:11:17.795 INFO:tasks.workunit.client.1.smithi203.stdout:5/32: write f8 [399733,126484] 0 2024-05-25T06:11:17.796 INFO:tasks.workunit.client.1.smithi203.stdout:1/14: dwrite f0 [0,4194304] 0 2024-05-25T06:11:17.797 INFO:tasks.workunit.client.1.smithi203.stdout:1/15: readlink l2 0 2024-05-25T06:11:17.797 INFO:tasks.workunit.client.1.smithi203.stdout:5/33: chown f6 431 1 2024-05-25T06:11:17.798 INFO:tasks.workunit.client.1.smithi203.stdout:1/16: write f0 [1777499,79277] 0 2024-05-25T06:11:17.799 INFO:tasks.workunit.client.1.smithi203.stdout:4/21: unlink d0/l6 0 2024-05-25T06:11:17.800 INFO:tasks.workunit.client.1.smithi203.stdout:6/43: link f6 d8/fc 0 2024-05-25T06:11:17.800 INFO:tasks.workunit.client.1.smithi203.stdout:7/34: link d1/f5 d1/d9/fe 0 2024-05-25T06:11:17.801 INFO:tasks.workunit.client.1.smithi203.stdout:4/22: truncate d0/f5 806402 0 2024-05-25T06:11:17.801 INFO:tasks.workunit.client.1.smithi203.stdout:7/35: readlink d1/l4 0 2024-05-25T06:11:17.802 INFO:tasks.workunit.client.1.smithi203.stdout:8/36: creat d2/d5/fc x:0 0 0 2024-05-25T06:11:17.803 INFO:tasks.workunit.client.1.smithi203.stdout:8/37: stat d2/d5 0 2024-05-25T06:11:17.803 INFO:tasks.workunit.client.1.smithi203.stdout:9/38: dread d0/f4 [0,4194304] 0 2024-05-25T06:11:17.806 INFO:tasks.workunit.client.1.smithi203.stdout:5/34: mknod ca 0 2024-05-25T06:11:17.810 INFO:tasks.workunit.client.1.smithi203.stdout:1/17: unlink l2 0 2024-05-25T06:11:17.811 INFO:tasks.workunit.client.1.smithi203.stdout:0/33: write f9 [1117428,112840] 0 2024-05-25T06:11:17.814 INFO:tasks.workunit.client.1.smithi203.stdout:4/23: mknod d0/c7 0 2024-05-25T06:11:17.814 INFO:tasks.workunit.client.1.smithi203.stdout:1/18: truncate f0 4862938 0 2024-05-25T06:11:17.815 INFO:tasks.workunit.client.1.smithi203.stdout:8/38: dwrite d2/fa [0,4194304] 0 2024-05-25T06:11:17.815 INFO:tasks.workunit.client.1.smithi203.stdout:7/36: symlink d1/d9/lf 0 2024-05-25T06:11:17.817 INFO:tasks.workunit.client.1.smithi203.stdout:8/39: fdatasync d2/f3 0 2024-05-25T06:11:17.820 INFO:tasks.workunit.client.1.smithi203.stdout:6/44: creat d8/fd x:0 0 0 2024-05-25T06:11:17.821 INFO:tasks.workunit.client.1.smithi203.stdout:4/24: dread d0/f4 [0,4194304] 0 2024-05-25T06:11:17.822 INFO:tasks.workunit.client.1.smithi203.stdout:6/45: chown d8/fb 2051792 1 2024-05-25T06:11:17.822 INFO:tasks.workunit.client.1.smithi203.stdout:2/32: rename l1 to d2/d4/lc 0 2024-05-25T06:11:17.824 INFO:tasks.workunit.client.1.smithi203.stdout:7/37: mknod d1/d9/c10 0 2024-05-25T06:11:17.827 INFO:tasks.workunit.client.1.smithi203.stdout:9/39: write d0/f4 [67824,8504] 0 2024-05-25T06:11:17.829 INFO:tasks.workunit.client.1.smithi203.stdout:8/40: unlink d2/f4 0 2024-05-25T06:11:17.830 INFO:tasks.workunit.client.1.smithi203.stdout:5/35: write f3 [4996768,97479] 0 2024-05-25T06:11:17.830 INFO:tasks.workunit.client.1.smithi203.stdout:3/35: link le lf 0 2024-05-25T06:11:17.831 INFO:tasks.workunit.client.1.smithi203.stdout:4/25: dwrite d0/f3 [0,4194304] 0 2024-05-25T06:11:17.833 INFO:tasks.workunit.client.1.smithi203.stdout:0/34: write f5 [3399005,57155] 0 2024-05-25T06:11:17.833 INFO:tasks.workunit.client.1.smithi203.stdout:6/46: mknod d8/ce 0 2024-05-25T06:11:17.833 INFO:tasks.workunit.client.1.smithi203.stdout:4/26: write d0/f3 [3950674,28951] 0 2024-05-25T06:11:17.836 INFO:tasks.workunit.client.1.smithi203.stdout:9/40: creat d0/fe x:0 0 0 2024-05-25T06:11:17.838 INFO:tasks.workunit.client.1.smithi203.stdout:3/36: mkdir d10 0 2024-05-25T06:11:17.838 INFO:tasks.workunit.client.1.smithi203.stdout:9/41: chown d0/f7 22 1 2024-05-25T06:11:17.839 INFO:tasks.workunit.client.1.smithi203.stdout:5/36: creat fb x:0 0 0 2024-05-25T06:11:17.840 INFO:tasks.workunit.client.1.smithi203.stdout:8/41: creat d2/d5/d9/fd x:0 0 0 2024-05-25T06:11:17.840 INFO:tasks.workunit.client.1.smithi203.stdout:5/37: readlink l4 0 2024-05-25T06:11:17.843 INFO:tasks.workunit.client.1.smithi203.stdout:0/35: mknod cc 0 2024-05-25T06:11:17.845 INFO:tasks.workunit.client.1.smithi203.stdout:6/47: creat d8/ff x:0 0 0 2024-05-25T06:11:17.846 INFO:tasks.workunit.client.1.smithi203.stdout:1/19: rmdir d3 0 2024-05-25T06:11:17.848 INFO:tasks.workunit.client.1.smithi203.stdout:6/48: write d8/fd [813544,40938] 0 2024-05-25T06:11:17.848 INFO:tasks.workunit.client.1.smithi203.stdout:7/38: dwrite d1/d9/fb [0,4194304] 0 2024-05-25T06:11:17.848 INFO:tasks.workunit.client.1.smithi203.stdout:4/27: mkdir d0/d8 0 2024-05-25T06:11:17.851 INFO:tasks.workunit.client.1.smithi203.stdout:9/42: mkdir d0/df 0 2024-05-25T06:11:17.852 INFO:tasks.workunit.client.1.smithi203.stdout:2/33: write d2/f3 [2462706,86969] 0 2024-05-25T06:11:17.853 INFO:tasks.workunit.client.1.smithi203.stdout:8/42: mkdir d2/d5/d9/de 0 2024-05-25T06:11:17.854 INFO:tasks.workunit.client.1.smithi203.stdout:3/37: chown cb 110 1 2024-05-25T06:11:17.854 INFO:tasks.workunit.client.1.smithi203.stdout:3/38: chown lc 0 1 2024-05-25T06:11:17.856 INFO:tasks.workunit.client.1.smithi203.stdout:5/38: dwrite fb [0,4194304] 0 2024-05-25T06:11:17.861 INFO:tasks.workunit.client.1.smithi203.stdout:7/39: truncate d1/d9/fd 4690020 0 2024-05-25T06:11:17.861 INFO:tasks.workunit.client.1.smithi203.stdout:8/43: chown d2/d5/d9/de 152 1 2024-05-25T06:11:17.863 INFO:tasks.workunit.client.1.smithi203.stdout:4/28: mkdir d0/d9 0 2024-05-25T06:11:17.863 INFO:tasks.workunit.client.1.smithi203.stdout:6/49: creat d8/f10 x:0 0 0 2024-05-25T06:11:17.864 INFO:tasks.workunit.client.1.smithi203.stdout:9/43: unlink d0/l8 0 2024-05-25T06:11:17.864 INFO:tasks.workunit.client.1.smithi203.stdout:0/36: dread f5 [0,4194304] 0 2024-05-25T06:11:17.868 INFO:tasks.workunit.client.1.smithi203.stdout:9/44: stat d0/fe 0 2024-05-25T06:11:17.870 INFO:tasks.workunit.client.1.smithi203.stdout:0/37: write f9 [1473828,93377] 0 2024-05-25T06:11:17.871 INFO:tasks.workunit.client.1.smithi203.stdout:2/34: mknod d2/cd 0 2024-05-25T06:11:17.872 INFO:tasks.workunit.client.1.smithi203.stdout:6/50: mkdir d8/d11 0 2024-05-25T06:11:17.874 INFO:tasks.workunit.client.1.smithi203.stdout:3/39: dwrite f1 [0,4194304] 0 2024-05-25T06:11:17.874 INFO:tasks.workunit.client.1.smithi203.stdout:0/38: chown f6 41356 1 2024-05-25T06:11:17.876 INFO:tasks.workunit.client.1.smithi203.stdout:4/29: dread d0/f5 [0,4194304] 0 2024-05-25T06:11:17.876 INFO:tasks.workunit.client.1.smithi203.stdout:7/40: mkdir d1/d11 0 2024-05-25T06:11:17.878 INFO:tasks.workunit.client.1.smithi203.stdout:9/45: rename d0/c9 to d0/c10 0 2024-05-25T06:11:17.879 INFO:tasks.workunit.client.1.smithi203.stdout:6/51: creat d8/f12 x:0 0 0 2024-05-25T06:11:17.881 INFO:tasks.workunit.client.1.smithi203.stdout:1/20: getdents . 0 2024-05-25T06:11:17.883 INFO:tasks.workunit.client.1.smithi203.stdout:0/39: unlink cc 0 2024-05-25T06:11:17.885 INFO:tasks.workunit.client.1.smithi203.stdout:8/44: dwrite d2/fa [0,4194304] 0 2024-05-25T06:11:17.885 INFO:tasks.workunit.client.1.smithi203.stdout:3/40: rename cb to d10/c11 0 2024-05-25T06:11:17.887 INFO:tasks.workunit.client.1.smithi203.stdout:9/46: readlink d0/l3 0 2024-05-25T06:11:17.888 INFO:tasks.workunit.client.1.smithi203.stdout:8/45: read d2/fa [770274,50857] 0 2024-05-25T06:11:17.890 INFO:tasks.workunit.client.1.smithi203.stdout:4/30: write d0/f4 [4528294,79454] 0 2024-05-25T06:11:17.891 INFO:tasks.workunit.client.1.smithi203.stdout:1/21: mknod c4 0 2024-05-25T06:11:17.893 INFO:tasks.workunit.client.1.smithi203.stdout:2/35: dwrite d2/d6/fb [0,4194304] 0 2024-05-25T06:11:17.894 INFO:tasks.workunit.client.1.smithi203.stdout:1/22: fdatasync f0 0 2024-05-25T06:11:17.895 INFO:tasks.workunit.client.1.smithi203.stdout:7/41: mknod d1/c12 0 2024-05-25T06:11:17.896 INFO:tasks.workunit.client.1.smithi203.stdout:1/23: stat f0 0 2024-05-25T06:11:17.897 INFO:tasks.workunit.client.1.smithi203.stdout:0/40: mkdir dd 0 2024-05-25T06:11:17.898 INFO:tasks.workunit.client.1.smithi203.stdout:3/41: symlink d10/l12 0 2024-05-25T06:11:17.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:17 smithi203 ceph-mon[36030]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 8.7 MiB/s wr, 316 op/s 2024-05-25T06:11:17.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:17 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:17.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:17 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:17.900 INFO:tasks.workunit.client.1.smithi203.stdout:8/46: rmdir d2/d5/d9 39 2024-05-25T06:11:17.903 INFO:tasks.workunit.client.1.smithi203.stdout:5/39: dwrite f6 [0,4194304] 0 2024-05-25T06:11:17.903 INFO:tasks.workunit.client.1.smithi203.stdout:6/52: truncate f6 702573 0 2024-05-25T06:11:17.907 INFO:tasks.workunit.client.1.smithi203.stdout:9/47: creat d0/df/f11 x:0 0 0 2024-05-25T06:11:17.909 INFO:tasks.workunit.client.1.smithi203.stdout:9/48: readlink d0/l1 0 2024-05-25T06:11:17.910 INFO:tasks.workunit.client.1.smithi203.stdout:0/41: write f5 [4219703,20206] 0 2024-05-25T06:11:17.913 INFO:tasks.workunit.client.1.smithi203.stdout:5/40: mkdir dc 0 2024-05-25T06:11:17.915 INFO:tasks.workunit.client.1.smithi203.stdout:0/42: chown cb 1641748445 1 2024-05-25T06:11:17.915 INFO:tasks.workunit.client.1.smithi203.stdout:1/24: dwrite f0 [0,4194304] 0 2024-05-25T06:11:17.917 INFO:tasks.workunit.client.1.smithi203.stdout:2/36: unlink d2/d4/lc 0 2024-05-25T06:11:17.919 INFO:tasks.workunit.client.1.smithi203.stdout:7/42: symlink d1/d11/l13 0 2024-05-25T06:11:17.919 INFO:tasks.workunit.client.1.smithi203.stdout:0/43: truncate f9 2308045 0 2024-05-25T06:11:17.921 INFO:tasks.workunit.client.1.smithi203.stdout:4/31: dwrite d0/f5 [0,4194304] 0 2024-05-25T06:11:17.922 INFO:tasks.workunit.client.1.smithi203.stdout:9/49: unlink d0/fa 0 2024-05-25T06:11:17.922 INFO:tasks.workunit.client.1.smithi203.stdout:6/53: rename l7 to d8/l13 0 2024-05-25T06:11:17.924 INFO:tasks.workunit.client.1.smithi203.stdout:1/25: unlink f0 0 2024-05-25T06:11:17.924 INFO:tasks.workunit.client.1.smithi203.stdout:4/32: write d0/f3 [3305403,12006] 0 2024-05-25T06:11:17.929 INFO:tasks.workunit.client.1.smithi203.stdout:0/44: dread f9 [0,4194304] 0 2024-05-25T06:11:17.931 INFO:tasks.workunit.client.1.smithi203.stdout:0/45: read f9 [169801,5601] 0 2024-05-25T06:11:17.931 INFO:tasks.workunit.client.1.smithi203.stdout:0/46: fsync f6 0 2024-05-25T06:11:17.933 INFO:tasks.workunit.client.1.smithi203.stdout:9/50: mknod d0/df/c12 0 2024-05-25T06:11:17.934 INFO:tasks.workunit.client.1.smithi203.stdout:3/42: dwrite f5 [0,4194304] 0 2024-05-25T06:11:17.938 INFO:tasks.workunit.client.1.smithi203.stdout:8/47: creat d2/d5/d9/de/ff x:0 0 0 2024-05-25T06:11:17.939 INFO:tasks.workunit.client.1.smithi203.stdout:8/48: stat d2/d5/d9/fd 0 2024-05-25T06:11:17.939 INFO:tasks.workunit.client.1.smithi203.stdout:0/47: dread f6 [0,4194304] 0 2024-05-25T06:11:17.941 INFO:tasks.workunit.client.1.smithi203.stdout:0/48: read f6 [2870514,107845] 0 2024-05-25T06:11:17.941 INFO:tasks.workunit.client.1.smithi203.stdout:5/41: rename ca to dc/cd 0 2024-05-25T06:11:17.943 INFO:tasks.workunit.client.1.smithi203.stdout:3/43: dwrite f1 [0,4194304] 0 2024-05-25T06:11:17.950 INFO:tasks.workunit.client.1.smithi203.stdout:9/51: rename d0/f6 to d0/f13 0 2024-05-25T06:11:17.952 INFO:tasks.workunit.client.1.smithi203.stdout:9/52: read - d0/fb zero size 2024-05-25T06:11:17.953 INFO:tasks.workunit.client.1.smithi203.stdout:3/44: dread f5 [0,4194304] 0 2024-05-25T06:11:17.953 INFO:tasks.workunit.client.1.smithi203.stdout:5/42: rmdir dc 39 2024-05-25T06:11:17.953 INFO:tasks.workunit.client.1.smithi203.stdout:9/53: dread - d0/df/f11 zero size 2024-05-25T06:11:17.955 INFO:tasks.workunit.client.1.smithi203.stdout:4/33: mknod d0/d8/ca 0 2024-05-25T06:11:17.956 INFO:tasks.workunit.client.1.smithi203.stdout:2/37: link d2/f3 d2/fe 0 2024-05-25T06:11:17.957 INFO:tasks.workunit.client.1.smithi203.stdout:3/45: dread f5 [0,4194304] 0 2024-05-25T06:11:17.958 INFO:tasks.workunit.client.1.smithi203.stdout:6/54: truncate d8/fc 1722489 0 2024-05-25T06:11:17.959 INFO:tasks.workunit.client.1.smithi203.stdout:8/49: mkdir d2/d5/d10 0 2024-05-25T06:11:17.961 INFO:tasks.workunit.client.1.smithi203.stdout:0/49: creat dd/fe x:0 0 0 2024-05-25T06:11:17.961 INFO:tasks.workunit.client.1.smithi203.stdout:1/26: link c4 c5 0 2024-05-25T06:11:17.961 INFO:tasks.workunit.client.1.smithi203.stdout:1/27: dwrite - no filename 2024-05-25T06:11:17.963 INFO:tasks.workunit.client.1.smithi203.stdout:9/54: creat d0/df/f14 x:0 0 0 2024-05-25T06:11:17.965 INFO:tasks.workunit.client.1.smithi203.stdout:4/34: mknod d0/d8/cb 0 2024-05-25T06:11:17.965 INFO:tasks.workunit.client.1.smithi203.stdout:7/43: getdents d1 0 2024-05-25T06:11:17.967 INFO:tasks.workunit.client.1.smithi203.stdout:3/46: dwrite f8 [0,4194304] 0 2024-05-25T06:11:17.968 INFO:tasks.workunit.client.1.smithi203.stdout:4/35: write d0/f4 [983419,106589] 0 2024-05-25T06:11:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:17 smithi160 ceph-mon[29041]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 8.7 MiB/s wr, 316 op/s 2024-05-25T06:11:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:17 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:17 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:17.970 INFO:tasks.workunit.client.1.smithi203.stdout:4/36: stat d0 0 2024-05-25T06:11:17.974 INFO:tasks.workunit.client.1.smithi203.stdout:8/50: rename d2/d5/d9 to d2/d5/d10/d11 0 2024-05-25T06:11:17.976 INFO:tasks.workunit.client.1.smithi203.stdout:5/43: creat dc/fe x:0 0 0 2024-05-25T06:11:17.977 INFO:tasks.workunit.client.1.smithi203.stdout:9/55: creat d0/df/f15 x:0 0 0 2024-05-25T06:11:17.980 INFO:tasks.workunit.client.1.smithi203.stdout:3/47: dwrite f1 [4194304,4194304] 0 2024-05-25T06:11:17.983 INFO:tasks.workunit.client.1.smithi203.stdout:2/38: fsync d2/f3 0 2024-05-25T06:11:17.983 INFO:tasks.workunit.client.1.smithi203.stdout:3/48: write f1 [1393874,72292] 0 2024-05-25T06:11:17.987 INFO:tasks.workunit.client.1.smithi203.stdout:5/44: dread fb [0,4194304] 0 2024-05-25T06:11:17.988 INFO:tasks.workunit.client.1.smithi203.stdout:6/55: unlink f6 0 2024-05-25T06:11:17.989 INFO:tasks.workunit.client.1.smithi203.stdout:4/37: symlink d0/d8/lc 0 2024-05-25T06:11:17.993 INFO:tasks.workunit.client.1.smithi203.stdout:8/51: mknod d2/d5/d10/d11/de/c12 0 2024-05-25T06:11:17.995 INFO:tasks.workunit.client.1.smithi203.stdout:2/39: dread d2/d6/fb [0,4194304] 0 2024-05-25T06:11:17.996 INFO:tasks.workunit.client.1.smithi203.stdout:9/56: unlink d0/fd 0 2024-05-25T06:11:18.000 INFO:tasks.workunit.client.1.smithi203.stdout:5/45: dread f1 [0,4194304] 0 2024-05-25T06:11:18.001 INFO:tasks.workunit.client.1.smithi203.stdout:3/49: fdatasync f0 0 2024-05-25T06:11:18.002 INFO:tasks.workunit.client.1.smithi203.stdout:1/28: link c4 c6 0 2024-05-25T06:11:18.002 INFO:tasks.workunit.client.1.smithi203.stdout:1/29: dwrite - no filename 2024-05-25T06:11:18.002 INFO:tasks.workunit.client.1.smithi203.stdout:1/30: write - no filename 2024-05-25T06:11:18.002 INFO:tasks.workunit.client.1.smithi203.stdout:1/31: write - no filename 2024-05-25T06:11:18.002 INFO:tasks.workunit.client.1.smithi203.stdout:6/56: dwrite f4 [0,4194304] 0 2024-05-25T06:11:18.005 INFO:tasks.workunit.client.1.smithi203.stdout:7/44: write d1/d9/fc [524777,103327] 0 2024-05-25T06:11:18.005 INFO:tasks.workunit.client.1.smithi203.stdout:5/46: chown fb 189418 1 2024-05-25T06:11:18.005 INFO:tasks.workunit.client.1.smithi203.stdout:4/38: dread d0/f5 [0,4194304] 0 2024-05-25T06:11:18.008 INFO:tasks.workunit.client.1.smithi203.stdout:0/50: rename l8 to dd/lf 0 2024-05-25T06:11:18.008 INFO:tasks.workunit.client.1.smithi203.stdout:9/57: truncate d0/df/f11 692162 0 2024-05-25T06:11:18.009 INFO:tasks.workunit.client.1.smithi203.stdout:7/45: dread - d1/f7 zero size 2024-05-25T06:11:18.011 INFO:tasks.workunit.client.1.smithi203.stdout:2/40: dread d2/d6/fb [0,4194304] 0 2024-05-25T06:11:18.012 INFO:tasks.workunit.client.1.smithi203.stdout:7/46: write d1/d9/fc [1503267,126208] 0 2024-05-25T06:11:18.013 INFO:tasks.workunit.client.1.smithi203.stdout:7/47: dread - d1/f7 zero size 2024-05-25T06:11:18.015 INFO:tasks.workunit.client.1.smithi203.stdout:3/50: rmdir d10 39 2024-05-25T06:11:18.017 INFO:tasks.workunit.client.1.smithi203.stdout:1/32: stat c4 0 2024-05-25T06:11:18.017 INFO:tasks.workunit.client.1.smithi203.stdout:1/33: write - no filename 2024-05-25T06:11:18.021 INFO:tasks.workunit.client.1.smithi203.stdout:5/47: read fb [2213398,111321] 0 2024-05-25T06:11:18.023 INFO:tasks.workunit.client.1.smithi203.stdout:4/39: mkdir d0/d8/dd 0 2024-05-25T06:11:18.025 INFO:tasks.workunit.client.1.smithi203.stdout:5/48: dread f8 [0,4194304] 0 2024-05-25T06:11:18.025 INFO:tasks.workunit.client.1.smithi203.stdout:8/52: rename d2/d5/d10/d11/fd to d2/d5/d10/d11/de/f13 0 2024-05-25T06:11:18.025 INFO:tasks.workunit.client.1.smithi203.stdout:8/53: chown d2/d5 96838 1 2024-05-25T06:11:18.026 INFO:tasks.workunit.client.1.smithi203.stdout:0/51: mkdir dd/d10 0 2024-05-25T06:11:18.026 INFO:tasks.workunit.client.1.smithi203.stdout:0/52: fdatasync dd/fe 0 2024-05-25T06:11:18.029 INFO:tasks.workunit.client.1.smithi203.stdout:6/57: symlink d8/l14 0 2024-05-25T06:11:18.030 INFO:tasks.workunit.client.1.smithi203.stdout:9/58: write d0/fc [786277,114548] 0 2024-05-25T06:11:18.031 INFO:tasks.workunit.client.1.smithi203.stdout:9/59: chown d0/df/f14 45925212 1 2024-05-25T06:11:18.032 INFO:tasks.workunit.client.1.smithi203.stdout:3/51: write f5 [4968267,72473] 0 2024-05-25T06:11:18.038 INFO:tasks.workunit.client.1.smithi203.stdout:5/49: mkdir dc/df 0 2024-05-25T06:11:18.039 INFO:tasks.workunit.client.1.smithi203.stdout:8/54: write d2/fa [1045714,69452] 0 2024-05-25T06:11:18.039 INFO:tasks.workunit.client.1.smithi203.stdout:0/53: truncate f9 3160040 0 2024-05-25T06:11:18.044 INFO:tasks.workunit.client.1.smithi203.stdout:4/40: symlink d0/d9/le 0 2024-05-25T06:11:18.045 INFO:tasks.workunit.client.1.smithi203.stdout:5/50: creat dc/f10 x:0 0 0 2024-05-25T06:11:18.046 INFO:tasks.workunit.client.1.smithi203.stdout:5/51: truncate dc/f10 115487 0 2024-05-25T06:11:18.046 INFO:tasks.workunit.client.1.smithi203.stdout:8/55: creat d2/f14 x:0 0 0 2024-05-25T06:11:18.047 INFO:tasks.workunit.client.1.smithi203.stdout:5/52: fsync dc/fe 0 2024-05-25T06:11:18.048 INFO:tasks.workunit.client.1.smithi203.stdout:0/54: creat dd/f11 x:0 0 0 2024-05-25T06:11:18.048 INFO:tasks.workunit.client.1.smithi203.stdout:2/41: truncate d2/d6/fb 289717 0 2024-05-25T06:11:18.049 INFO:tasks.workunit.client.1.smithi203.stdout:3/52: write f0 [616108,92357] 0 2024-05-25T06:11:18.050 INFO:tasks.workunit.client.1.smithi203.stdout:9/60: link d0/fb d0/f16 0 2024-05-25T06:11:18.051 INFO:tasks.workunit.client.1.smithi203.stdout:9/61: chown d0/l2 3600 1 2024-05-25T06:11:18.051 INFO:tasks.workunit.client.1.smithi203.stdout:1/34: link c5 c7 0 2024-05-25T06:11:18.052 INFO:tasks.workunit.client.1.smithi203.stdout:5/53: mkdir dc/d11 0 2024-05-25T06:11:18.054 INFO:tasks.workunit.client.1.smithi203.stdout:8/56: symlink d2/d5/d10/d11/de/l15 0 2024-05-25T06:11:18.054 INFO:tasks.workunit.client.1.smithi203.stdout:7/48: getdents d1 0 2024-05-25T06:11:18.055 INFO:tasks.workunit.client.1.smithi203.stdout:2/42: truncate d2/f3 4221757 0 2024-05-25T06:11:18.060 INFO:tasks.workunit.client.1.smithi203.stdout:6/58: mkdir d8/d11/d15 0 2024-05-25T06:11:18.060 INFO:tasks.workunit.client.1.smithi203.stdout:9/62: write d0/f16 [1527439,102487] 0 2024-05-25T06:11:18.062 INFO:tasks.workunit.client.1.smithi203.stdout:0/55: symlink dd/d10/l12 0 2024-05-25T06:11:18.062 INFO:tasks.workunit.client.1.smithi203.stdout:4/41: dwrite d0/f5 [0,4194304] 0 2024-05-25T06:11:18.062 INFO:tasks.workunit.client.1.smithi203.stdout:2/43: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:18.064 INFO:tasks.workunit.client.1.smithi203.stdout:0/56: read - dd/fe zero size 2024-05-25T06:11:18.066 INFO:tasks.workunit.client.1.smithi203.stdout:4/42: fdatasync d0/f4 0 2024-05-25T06:11:18.068 INFO:tasks.workunit.client.1.smithi203.stdout:4/43: chown d0/f3 0 1 2024-05-25T06:11:18.071 INFO:tasks.workunit.client.1.smithi203.stdout:1/35: rename c5 to c8 0 2024-05-25T06:11:18.071 INFO:tasks.workunit.client.1.smithi203.stdout:1/36: write - no filename 2024-05-25T06:11:18.074 INFO:tasks.workunit.client.1.smithi203.stdout:5/54: creat dc/d11/f12 x:0 0 0 2024-05-25T06:11:18.077 INFO:tasks.workunit.client.1.smithi203.stdout:5/55: read f3 [926079,13945] 0 2024-05-25T06:11:18.077 INFO:tasks.workunit.client.1.smithi203.stdout:9/63: dwrite d0/fb [0,4194304] 0 2024-05-25T06:11:18.079 INFO:tasks.workunit.client.1.smithi203.stdout:0/57: chown f9 10664457 1 2024-05-25T06:11:18.079 INFO:tasks.workunit.client.1.smithi203.stdout:8/57: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:18.079 INFO:tasks.workunit.client.1.smithi203.stdout:5/56: write f3 [2062328,787] 0 2024-05-25T06:11:18.080 INFO:tasks.workunit.client.1.smithi203.stdout:1/37: creat f9 x:0 0 0 2024-05-25T06:11:18.084 INFO:tasks.workunit.client.1.smithi203.stdout:0/58: write f6 [827056,109136] 0 2024-05-25T06:11:18.084 INFO:tasks.workunit.client.1.smithi203.stdout:5/57: read f3 [11426,97662] 0 2024-05-25T06:11:18.084 INFO:tasks.workunit.client.1.smithi203.stdout:6/59: symlink d8/l16 0 2024-05-25T06:11:18.086 INFO:tasks.workunit.client.1.smithi203.stdout:7/49: write d1/f5 [1140895,74827] 0 2024-05-25T06:11:18.086 INFO:tasks.workunit.client.1.smithi203.stdout:9/64: rmdir d0 39 2024-05-25T06:11:18.088 INFO:tasks.workunit.client.1.smithi203.stdout:0/59: mknod dd/c13 0 2024-05-25T06:11:18.088 INFO:tasks.workunit.client.1.smithi203.stdout:2/44: getdents d2/d6/da 0 2024-05-25T06:11:18.090 INFO:tasks.workunit.client.1.smithi203.stdout:8/58: creat d2/d5/d10/d11/de/f16 x:0 0 0 2024-05-25T06:11:18.090 INFO:tasks.workunit.client.1.smithi203.stdout:5/58: creat dc/d11/f13 x:0 0 0 2024-05-25T06:11:18.092 INFO:tasks.workunit.client.1.smithi203.stdout:7/50: creat d1/d9/f14 x:0 0 0 2024-05-25T06:11:18.094 INFO:tasks.workunit.client.1.smithi203.stdout:5/59: stat dc/d11/f13 0 2024-05-25T06:11:18.095 INFO:tasks.workunit.client.1.smithi203.stdout:0/60: write dd/f11 [877232,68713] 0 2024-05-25T06:11:18.102 INFO:tasks.workunit.client.1.smithi203.stdout:9/65: write d0/f13 [2968366,129996] 0 2024-05-25T06:11:18.105 INFO:tasks.workunit.client.1.smithi203.stdout:6/60: dwrite f4 [4194304,4194304] 0 2024-05-25T06:11:18.109 INFO:tasks.workunit.client.1.smithi203.stdout:2/45: rename d2/d4/l5 to d2/d6/da/lf 0 2024-05-25T06:11:18.109 INFO:tasks.workunit.client.1.smithi203.stdout:6/61: write f0 [4663391,3558] 0 2024-05-25T06:11:18.113 INFO:tasks.workunit.client.1.smithi203.stdout:0/61: symlink dd/d10/l14 0 2024-05-25T06:11:18.115 INFO:tasks.workunit.client.1.smithi203.stdout:1/38: dwrite f9 [0,4194304] 0 2024-05-25T06:11:18.117 INFO:tasks.workunit.client.1.smithi203.stdout:5/60: write f6 [1782079,63137] 0 2024-05-25T06:11:18.121 INFO:tasks.workunit.client.1.smithi203.stdout:0/62: creat dd/f15 x:0 0 0 2024-05-25T06:11:18.125 INFO:tasks.workunit.client.1.smithi203.stdout:1/39: rename f9 to fa 0 2024-05-25T06:11:18.130 INFO:tasks.workunit.client.1.smithi203.stdout:8/59: dwrite d2/d5/d10/d11/de/ff [0,4194304] 0 2024-05-25T06:11:18.132 INFO:tasks.workunit.client.1.smithi203.stdout:1/40: creat fb x:0 0 0 2024-05-25T06:11:18.133 INFO:tasks.workunit.client.1.smithi203.stdout:9/66: dread d0/fc [0,4194304] 0 2024-05-25T06:11:18.134 INFO:tasks.workunit.client.1.smithi203.stdout:2/46: write d2/f3 [4409592,5123] 0 2024-05-25T06:11:18.140 INFO:tasks.workunit.client.1.smithi203.stdout:1/41: creat fc x:0 0 0 2024-05-25T06:11:18.140 INFO:tasks.workunit.client.1.smithi203.stdout:8/60: dread - d2/f14 zero size 2024-05-25T06:11:18.141 INFO:tasks.workunit.client.1.smithi203.stdout:5/61: dread f8 [0,4194304] 0 2024-05-25T06:11:18.143 INFO:tasks.workunit.client.1.smithi203.stdout:7/51: dwrite d1/f7 [0,4194304] 0 2024-05-25T06:11:18.145 INFO:tasks.workunit.client.1.smithi203.stdout:0/63: getdents dd/d10 0 2024-05-25T06:11:18.150 INFO:tasks.workunit.client.1.smithi203.stdout:8/61: read - d2/f14 zero size 2024-05-25T06:11:18.151 INFO:tasks.workunit.client.1.smithi203.stdout:1/42: write fb [819747,112270] 0 2024-05-25T06:11:18.151 INFO:tasks.workunit.client.1.smithi203.stdout:7/52: readlink d1/d9/lf 0 2024-05-25T06:11:18.151 INFO:tasks.workunit.client.1.smithi203.stdout:1/43: rmdir - no directory 2024-05-25T06:11:18.152 INFO:tasks.workunit.client.1.smithi203.stdout:5/62: link f6 dc/d11/f14 0 2024-05-25T06:11:18.153 INFO:tasks.workunit.client.1.smithi203.stdout:6/62: dwrite d8/ff [0,4194304] 0 2024-05-25T06:11:18.153 INFO:tasks.workunit.client.1.smithi203.stdout:2/47: mkdir d2/d4/d10 0 2024-05-25T06:11:18.156 INFO:tasks.workunit.client.1.smithi203.stdout:1/44: dread - fc zero size 2024-05-25T06:11:18.159 INFO:tasks.workunit.client.1.smithi203.stdout:5/63: read dc/f10 [93071,15102] 0 2024-05-25T06:11:18.159 INFO:tasks.workunit.client.1.smithi203.stdout:6/63: rmdir d8/d11 39 2024-05-25T06:11:18.159 INFO:tasks.workunit.client.1.smithi203.stdout:0/64: readlink dd/lf 0 2024-05-25T06:11:18.160 INFO:tasks.workunit.client.1.smithi203.stdout:1/45: write fb [640873,31796] 0 2024-05-25T06:11:18.161 INFO:tasks.workunit.client.1.smithi203.stdout:7/53: truncate d1/d9/fe 1424005 0 2024-05-25T06:11:18.164 INFO:tasks.workunit.client.1.smithi203.stdout:0/65: read f9 [170235,105843] 0 2024-05-25T06:11:18.165 INFO:tasks.workunit.client.1.smithi203.stdout:8/62: symlink d2/d5/d10/d11/de/l17 0 2024-05-25T06:11:18.166 INFO:tasks.workunit.client.1.smithi203.stdout:0/66: stat dd 0 2024-05-25T06:11:18.166 INFO:tasks.workunit.client.1.smithi203.stdout:7/54: fdatasync d1/f7 0 2024-05-25T06:11:18.167 INFO:tasks.workunit.client.1.smithi203.stdout:1/46: write fa [62297,12146] 0 2024-05-25T06:11:18.171 INFO:tasks.workunit.client.1.smithi203.stdout:0/67: dread - dd/fe zero size 2024-05-25T06:11:18.172 INFO:tasks.workunit.client.1.smithi203.stdout:2/48: creat d2/d4/f11 x:0 0 0 2024-05-25T06:11:18.173 INFO:tasks.workunit.client.1.smithi203.stdout:9/67: dwrite d0/df/f15 [0,4194304] 0 2024-05-25T06:11:18.174 INFO:tasks.workunit.client.1.smithi203.stdout:1/47: mknod cd 0 2024-05-25T06:11:18.178 INFO:tasks.workunit.client.1.smithi203.stdout:0/68: mknod dd/d10/c16 0 2024-05-25T06:11:18.179 INFO:tasks.workunit.client.1.smithi203.stdout:8/63: write d2/d5/d10/d11/de/f13 [619655,127828] 0 2024-05-25T06:11:18.181 INFO:tasks.workunit.client.1.smithi203.stdout:2/49: read - d2/d4/f11 zero size 2024-05-25T06:11:18.185 INFO:tasks.workunit.client.1.smithi203.stdout:5/64: creat dc/df/f15 x:0 0 0 2024-05-25T06:11:18.187 INFO:tasks.workunit.client.1.smithi203.stdout:0/69: creat dd/f17 x:0 0 0 2024-05-25T06:11:18.189 INFO:tasks.workunit.client.1.smithi203.stdout:2/50: symlink d2/d4/l12 0 2024-05-25T06:11:18.191 INFO:tasks.workunit.client.1.smithi203.stdout:7/55: link d1/d9/lf d1/d9/l15 0 2024-05-25T06:11:18.192 INFO:tasks.workunit.client.1.smithi203.stdout:6/64: dwrite d8/f10 [0,4194304] 0 2024-05-25T06:11:18.194 INFO:tasks.workunit.client.1.smithi203.stdout:6/65: stat l5 0 2024-05-25T06:11:18.194 INFO:tasks.workunit.client.1.smithi203.stdout:0/70: chown dd/d10/c16 9041449 1 2024-05-25T06:11:18.200 INFO:tasks.workunit.client.1.smithi203.stdout:1/48: getdents . 0 2024-05-25T06:11:18.202 INFO:tasks.workunit.client.1.smithi203.stdout:5/65: getdents dc/d11 0 2024-05-25T06:11:18.203 INFO:tasks.workunit.client.1.smithi203.stdout:6/66: read f2 [3853834,10090] 0 2024-05-25T06:11:18.204 INFO:tasks.workunit.client.1.smithi203.stdout:1/49: mknod ce 0 2024-05-25T06:11:18.209 INFO:tasks.workunit.client.1.smithi203.stdout:8/64: dwrite d2/fa [0,4194304] 0 2024-05-25T06:11:18.211 INFO:tasks.workunit.client.1.smithi203.stdout:9/68: dwrite d0/df/f11 [0,4194304] 0 2024-05-25T06:11:18.211 INFO:tasks.workunit.client.1.smithi203.stdout:2/51: write d2/d6/fb [893458,8691] 0 2024-05-25T06:11:18.212 INFO:tasks.workunit.client.1.smithi203.stdout:6/67: rmdir d8 39 2024-05-25T06:11:18.214 INFO:tasks.workunit.client.1.smithi203.stdout:2/52: chown d2 6966 1 2024-05-25T06:11:18.215 INFO:tasks.workunit.client.1.smithi203.stdout:7/56: getdents d1/d11 0 2024-05-25T06:11:18.218 INFO:tasks.workunit.client.1.smithi203.stdout:7/57: write d1/f5 [1123006,80539] 0 2024-05-25T06:11:18.223 INFO:tasks.workunit.client.1.smithi203.stdout:6/68: write d8/fb [255849,68424] 0 2024-05-25T06:11:18.224 INFO:tasks.workunit.client.1.smithi203.stdout:5/66: dread dc/d11/f14 [0,4194304] 0 2024-05-25T06:11:18.226 INFO:tasks.workunit.client.1.smithi203.stdout:8/65: write d2/f3 [3736063,32318] 0 2024-05-25T06:11:18.226 INFO:tasks.workunit.client.1.smithi203.stdout:1/50: dwrite fc [0,4194304] 0 2024-05-25T06:11:18.234 INFO:tasks.workunit.client.1.smithi203.stdout:8/66: truncate d2/f14 45517 0 2024-05-25T06:11:18.238 INFO:tasks.workunit.client.1.smithi203.stdout:8/67: mkdir d2/d5/d18 0 2024-05-25T06:11:18.242 INFO:tasks.workunit.client.1.smithi203.stdout:7/58: dwrite d1/d9/fd [0,4194304] 0 2024-05-25T06:11:18.242 INFO:tasks.workunit.client.1.smithi203.stdout:8/68: mknod d2/c19 0 2024-05-25T06:11:18.242 INFO:tasks.workunit.client.1.smithi203.stdout:5/67: dwrite dc/d11/f14 [0,4194304] 0 2024-05-25T06:11:18.244 INFO:tasks.workunit.client.1.smithi203.stdout:8/69: chown d2/d5/d10/d11/de 420498 1 2024-05-25T06:11:18.246 INFO:tasks.workunit.client.1.smithi203.stdout:8/70: write d2/fa [5099822,82172] 0 2024-05-25T06:11:18.248 INFO:tasks.workunit.client.1.smithi203.stdout:7/59: symlink d1/d9/l16 0 2024-05-25T06:11:18.248 INFO:tasks.workunit.client.1.smithi203.stdout:6/69: write d8/fc [484896,14806] 0 2024-05-25T06:11:18.248 INFO:tasks.workunit.client.1.smithi203.stdout:5/68: write dc/d11/f14 [4567566,122338] 0 2024-05-25T06:11:18.252 INFO:tasks.workunit.client.1.smithi203.stdout:1/51: truncate fb 601462 0 2024-05-25T06:11:18.254 INFO:tasks.workunit.client.1.smithi203.stdout:6/70: chown d8/fb 204 1 2024-05-25T06:11:18.259 INFO:tasks.workunit.client.1.smithi203.stdout:5/69: getdents dc/df 0 2024-05-25T06:11:18.259 INFO:tasks.workunit.client.1.smithi203.stdout:1/52: rename c6 to cf 0 2024-05-25T06:11:18.261 INFO:tasks.workunit.client.1.smithi203.stdout:5/70: rmdir dc/d11 39 2024-05-25T06:11:18.261 INFO:tasks.workunit.client.1.smithi203.stdout:8/71: dwrite d2/f3 [0,4194304] 0 2024-05-25T06:11:18.262 INFO:tasks.workunit.client.1.smithi203.stdout:1/53: creat f10 x:0 0 0 2024-05-25T06:11:18.263 INFO:tasks.workunit.client.1.smithi203.stdout:1/54: write fa [4252219,100500] 0 2024-05-25T06:11:18.263 INFO:tasks.workunit.client.1.smithi203.stdout:1/55: rmdir - no directory 2024-05-25T06:11:18.267 INFO:tasks.workunit.client.1.smithi203.stdout:6/71: dread f1 [0,4194304] 0 2024-05-25T06:11:18.270 INFO:tasks.workunit.client.1.smithi203.stdout:5/71: getdents dc/d11 0 2024-05-25T06:11:18.271 INFO:tasks.workunit.client.1.smithi203.stdout:1/56: dread fa [0,4194304] 0 2024-05-25T06:11:18.272 INFO:tasks.workunit.client.1.smithi203.stdout:8/72: creat d2/d5/d10/d11/de/f1a x:0 0 0 2024-05-25T06:11:18.274 INFO:tasks.workunit.client.1.smithi203.stdout:1/57: write f10 [588627,37487] 0 2024-05-25T06:11:18.318 INFO:tasks.workunit.client.1.smithi203.stdout:1/58: mknod c11 0 2024-05-25T06:11:18.319 INFO:tasks.workunit.client.1.smithi203.stdout:7/60: truncate d1/f3 3151391 0 2024-05-25T06:11:18.319 INFO:tasks.workunit.client.1.smithi203.stdout:1/59: stat c11 0 2024-05-25T06:11:18.319 INFO:tasks.workunit.client.1.smithi203.stdout:6/72: creat d8/d11/d15/f17 x:0 0 0 2024-05-25T06:11:18.321 INFO:tasks.workunit.client.1.smithi203.stdout:8/73: dread d2/d5/d10/d11/de/f13 [0,4194304] 0 2024-05-25T06:11:18.321 INFO:tasks.workunit.client.1.smithi203.stdout:7/61: chown d1/d11/l13 414 1 2024-05-25T06:11:18.322 INFO:tasks.workunit.client.1.smithi203.stdout:5/72: dread f1 [0,4194304] 0 2024-05-25T06:11:18.322 INFO:tasks.workunit.client.1.smithi203.stdout:7/62: stat d1/l4 0 2024-05-25T06:11:18.322 INFO:tasks.workunit.client.1.smithi203.stdout:1/60: chown c7 52409 1 2024-05-25T06:11:18.323 INFO:tasks.workunit.client.1.smithi203.stdout:6/73: creat d8/d11/d15/f18 x:0 0 0 2024-05-25T06:11:18.323 INFO:tasks.workunit.client.1.smithi203.stdout:5/73: rmdir dc/df 39 2024-05-25T06:11:18.324 INFO:tasks.workunit.client.1.smithi203.stdout:1/61: creat f12 x:0 0 0 2024-05-25T06:11:18.324 INFO:tasks.workunit.client.1.smithi203.stdout:8/74: truncate d2/d5/d10/d11/de/f1a 171200 0 2024-05-25T06:11:18.325 INFO:tasks.workunit.client.1.smithi203.stdout:7/63: truncate d1/f2 1345385 0 2024-05-25T06:11:18.326 INFO:tasks.workunit.client.1.smithi203.stdout:6/74: creat d8/d11/f19 x:0 0 0 2024-05-25T06:11:18.328 INFO:tasks.workunit.client.1.smithi203.stdout:7/64: rename d1/f5 to d1/d11/f17 0 2024-05-25T06:11:18.328 INFO:tasks.workunit.client.1.smithi203.stdout:1/62: dread f10 [0,4194304] 0 2024-05-25T06:11:18.329 INFO:tasks.workunit.client.1.smithi203.stdout:8/75: creat d2/d5/d18/f1b x:0 0 0 2024-05-25T06:11:18.329 INFO:tasks.workunit.client.1.smithi203.stdout:7/65: stat d1/d9/fb 0 2024-05-25T06:11:18.330 INFO:tasks.workunit.client.1.smithi203.stdout:6/75: stat d8/f9 0 2024-05-25T06:11:18.331 INFO:tasks.workunit.client.1.smithi203.stdout:8/76: dread - d2/d5/fc zero size 2024-05-25T06:11:18.331 INFO:tasks.workunit.client.1.smithi203.stdout:1/63: rename ce to c13 0 2024-05-25T06:11:18.332 INFO:tasks.workunit.client.1.smithi203.stdout:7/66: creat d1/d9/f18 x:0 0 0 2024-05-25T06:11:18.334 INFO:tasks.workunit.client.1.smithi203.stdout:5/74: dwrite f1 [0,4194304] 0 2024-05-25T06:11:18.334 INFO:tasks.workunit.client.1.smithi203.stdout:8/77: chown d2/d5/d10/d11/de/f13 61 1 2024-05-25T06:11:18.335 INFO:tasks.workunit.client.1.smithi203.stdout:8/78: dread - d2/d5/d10/d11/de/f16 zero size 2024-05-25T06:11:18.341 INFO:tasks.workunit.client.1.smithi203.stdout:5/75: symlink dc/l16 0 2024-05-25T06:11:18.344 INFO:tasks.workunit.client.1.smithi203.stdout:5/76: chown dc/df/f15 5 1 2024-05-25T06:11:18.345 INFO:tasks.workunit.client.1.smithi203.stdout:8/79: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:18.346 INFO:tasks.workunit.client.1.smithi203.stdout:7/67: write d1/f2 [1326300,58810] 0 2024-05-25T06:11:18.348 INFO:tasks.workunit.client.1.smithi203.stdout:5/77: mkdir dc/d11/d17 0 2024-05-25T06:11:18.352 INFO:tasks.workunit.client.1.smithi203.stdout:5/78: mkdir dc/d11/d18 0 2024-05-25T06:11:18.353 INFO:tasks.workunit.client.1.smithi203.stdout:6/76: dwrite f1 [0,4194304] 0 2024-05-25T06:11:18.355 INFO:tasks.workunit.client.1.smithi203.stdout:8/80: rename d2/d5/d10/d11/de/l15 to d2/l1c 0 2024-05-25T06:11:18.356 INFO:tasks.workunit.client.1.smithi203.stdout:6/77: mkdir d8/d11/d1a 0 2024-05-25T06:11:18.357 INFO:tasks.workunit.client.1.smithi203.stdout:8/81: truncate d2/d5/fc 434533 0 2024-05-25T06:11:18.357 INFO:tasks.workunit.client.1.smithi203.stdout:7/68: dread d1/d9/fc [0,4194304] 0 2024-05-25T06:11:18.358 INFO:tasks.workunit.client.1.smithi203.stdout:6/78: write f0 [8866231,94960] 0 2024-05-25T06:11:18.358 INFO:tasks.workunit.client.1.smithi203.stdout:5/79: creat dc/f19 x:0 0 0 2024-05-25T06:11:18.362 INFO:tasks.workunit.client.1.smithi203.stdout:6/79: mkdir d8/d11/d1b 0 2024-05-25T06:11:18.364 INFO:tasks.workunit.client.1.smithi203.stdout:8/82: rename d2/fa to d2/d5/d10/d11/de/f1d 0 2024-05-25T06:11:18.366 INFO:tasks.workunit.client.1.smithi203.stdout:7/69: dread d1/d9/fd [0,4194304] 0 2024-05-25T06:11:18.368 INFO:tasks.workunit.client.1.smithi203.stdout:7/70: write d1/f7 [3361243,70996] 0 2024-05-25T06:11:18.368 INFO:tasks.workunit.client.1.smithi203.stdout:6/80: unlink f4 0 2024-05-25T06:11:18.374 INFO:tasks.workunit.client.1.smithi203.stdout:7/71: unlink d1/f7 0 2024-05-25T06:11:18.376 INFO:tasks.workunit.client.1.smithi203.stdout:8/83: dwrite d2/f14 [0,4194304] 0 2024-05-25T06:11:18.376 INFO:tasks.workunit.client.1.smithi203.stdout:6/81: mknod d8/d11/d1a/c1c 0 2024-05-25T06:11:18.377 INFO:tasks.workunit.client.1.smithi203.stdout:7/72: symlink d1/d9/l19 0 2024-05-25T06:11:18.379 INFO:tasks.workunit.client.1.smithi203.stdout:7/73: write d1/d9/f14 [935887,55181] 0 2024-05-25T06:11:18.381 INFO:tasks.workunit.client.1.smithi203.stdout:8/84: creat d2/d5/d10/d11/de/f1e x:0 0 0 2024-05-25T06:11:18.382 INFO:tasks.workunit.client.1.smithi203.stdout:6/82: creat d8/d11/d1a/f1d x:0 0 0 2024-05-25T06:11:18.387 INFO:tasks.workunit.client.1.smithi203.stdout:5/80: dwrite dc/f10 [0,4194304] 0 2024-05-25T06:11:18.389 INFO:tasks.workunit.client.1.smithi203.stdout:8/85: mkdir d2/d5/d10/d11/d1f 0 2024-05-25T06:11:18.390 INFO:tasks.workunit.client.1.smithi203.stdout:6/83: creat d8/d11/f1e x:0 0 0 2024-05-25T06:11:18.391 INFO:tasks.workunit.client.1.smithi203.stdout:7/74: rename d1/d9/lf to d1/d11/l1a 0 2024-05-25T06:11:18.393 INFO:tasks.workunit.client.1.smithi203.stdout:6/84: symlink d8/d11/d1a/l1f 0 2024-05-25T06:11:18.394 INFO:tasks.workunit.client.1.smithi203.stdout:8/86: symlink d2/d5/d10/d11/l20 0 2024-05-25T06:11:18.395 INFO:tasks.workunit.client.1.smithi203.stdout:5/81: mkdir dc/d11/d17/d1a 0 2024-05-25T06:11:18.399 INFO:tasks.workunit.client.1.smithi203.stdout:6/85: mknod d8/d11/d1b/c20 0 2024-05-25T06:11:18.399 INFO:tasks.workunit.client.1.smithi203.stdout:6/86: dread - d8/d11/f1e zero size 2024-05-25T06:11:18.400 INFO:tasks.workunit.client.1.smithi203.stdout:8/87: rename d2/l1c to d2/l21 0 2024-05-25T06:11:18.401 INFO:tasks.workunit.client.1.smithi203.stdout:8/88: read - d2/d5/d10/d11/de/f16 zero size 2024-05-25T06:11:18.402 INFO:tasks.workunit.client.1.smithi203.stdout:5/82: rename c2 to dc/c1b 0 2024-05-25T06:11:18.406 INFO:tasks.workunit.client.1.smithi203.stdout:5/83: creat dc/d11/f1c x:0 0 0 2024-05-25T06:11:18.408 INFO:tasks.workunit.client.1.smithi203.stdout:7/75: rmdir d1 39 2024-05-25T06:11:18.409 INFO:tasks.workunit.client.1.smithi203.stdout:8/89: symlink d2/d5/d18/l22 0 2024-05-25T06:11:18.410 INFO:tasks.workunit.client.1.smithi203.stdout:5/84: chown dc/fe 842 1 2024-05-25T06:11:18.410 INFO:tasks.workunit.client.1.smithi203.stdout:5/85: readlink l4 0 2024-05-25T06:11:18.412 INFO:tasks.workunit.client.1.smithi203.stdout:5/86: fsync dc/d11/f12 0 2024-05-25T06:11:18.413 INFO:tasks.workunit.client.1.smithi203.stdout:6/87: dread f0 [4194304,4194304] 0 2024-05-25T06:11:18.415 INFO:tasks.workunit.client.1.smithi203.stdout:5/87: truncate dc/d11/f12 573226 0 2024-05-25T06:11:18.415 INFO:tasks.workunit.client.1.smithi203.stdout:6/88: write d8/d11/f1e [628245,122103] 0 2024-05-25T06:11:18.415 INFO:tasks.workunit.client.1.smithi203.stdout:8/90: dread d2/f14 [0,4194304] 0 2024-05-25T06:11:18.416 INFO:tasks.workunit.client.1.smithi203.stdout:6/89: chown f0 1 1 2024-05-25T06:11:18.419 INFO:tasks.workunit.client.1.smithi203.stdout:5/88: mknod dc/d11/d18/c1d 0 2024-05-25T06:11:18.420 INFO:tasks.workunit.client.1.smithi203.stdout:6/90: symlink d8/d11/l21 0 2024-05-25T06:11:18.421 INFO:tasks.workunit.client.1.smithi203.stdout:8/91: write d2/d5/d10/d11/de/f13 [1155740,120580] 0 2024-05-25T06:11:18.422 INFO:tasks.workunit.client.1.smithi203.stdout:5/89: fsync f8 0 2024-05-25T06:11:18.423 INFO:tasks.workunit.client.1.smithi203.stdout:6/91: rmdir d8/d11/d1b 39 2024-05-25T06:11:18.424 INFO:tasks.workunit.client.1.smithi203.stdout:8/92: write d2/f3 [2523735,80207] 0 2024-05-25T06:11:18.424 INFO:tasks.workunit.client.1.smithi203.stdout:5/90: stat dc/d11/d17 0 2024-05-25T06:11:18.425 INFO:tasks.workunit.client.1.smithi203.stdout:6/92: truncate f0 6837407 0 2024-05-25T06:11:18.425 INFO:tasks.workunit.client.1.smithi203.stdout:8/93: write d2/d5/f8 [24911,75254] 0 2024-05-25T06:11:18.427 INFO:tasks.workunit.client.1.smithi203.stdout:6/93: write f1 [4753216,3321] 0 2024-05-25T06:11:18.427 INFO:tasks.workunit.client.1.smithi203.stdout:5/91: mknod dc/d11/c1e 0 2024-05-25T06:11:18.429 INFO:tasks.workunit.client.1.smithi203.stdout:6/94: rename d8/d11/d1a to d8/d11/d1a/d22 22 2024-05-25T06:11:18.429 INFO:tasks.workunit.client.1.smithi203.stdout:8/94: read - d2/d5/d10/d11/de/f1e zero size 2024-05-25T06:11:18.430 INFO:tasks.workunit.client.1.smithi203.stdout:5/92: write dc/d11/f13 [730522,38997] 0 2024-05-25T06:11:18.431 INFO:tasks.workunit.client.1.smithi203.stdout:6/95: write d8/ff [3227499,82968] 0 2024-05-25T06:11:18.433 INFO:tasks.workunit.client.1.smithi203.stdout:5/93: dread - dc/df/f15 zero size 2024-05-25T06:11:18.433 INFO:tasks.workunit.client.1.smithi203.stdout:8/95: mknod d2/d5/c23 0 2024-05-25T06:11:18.434 INFO:tasks.workunit.client.1.smithi203.stdout:5/94: chown dc/d11/f14 0 1 2024-05-25T06:11:18.435 INFO:tasks.workunit.client.1.smithi203.stdout:8/96: write d2/f3 [3589957,39644] 0 2024-05-25T06:11:18.439 INFO:tasks.workunit.client.1.smithi203.stdout:6/96: creat d8/d11/f23 x:0 0 0 2024-05-25T06:11:18.440 INFO:tasks.workunit.client.1.smithi203.stdout:5/95: symlink dc/l1f 0 2024-05-25T06:11:18.443 INFO:tasks.workunit.client.1.smithi203.stdout:8/97: link d2/l7 d2/d5/d10/l24 0 2024-05-25T06:11:18.443 INFO:tasks.workunit.client.1.smithi203.stdout:5/96: getdents dc/df 0 2024-05-25T06:11:18.446 INFO:tasks.workunit.client.1.smithi203.stdout:6/97: dwrite f1 [0,4194304] 0 2024-05-25T06:11:18.446 INFO:tasks.workunit.client.1.smithi203.stdout:8/98: symlink d2/d5/d10/d11/d1f/l25 0 2024-05-25T06:11:18.447 INFO:tasks.workunit.client.1.smithi203.stdout:5/97: truncate dc/d11/f12 1026212 0 2024-05-25T06:11:18.448 INFO:tasks.workunit.client.1.smithi203.stdout:8/99: truncate d2/f3 4616469 0 2024-05-25T06:11:18.449 INFO:tasks.workunit.client.1.smithi203.stdout:5/98: symlink dc/df/l20 0 2024-05-25T06:11:18.450 INFO:tasks.workunit.client.1.smithi203.stdout:6/98: mkdir d8/d11/d1b/d24 0 2024-05-25T06:11:18.451 INFO:tasks.workunit.client.1.smithi203.stdout:8/100: symlink d2/l26 0 2024-05-25T06:11:18.453 INFO:tasks.workunit.client.1.smithi203.stdout:6/99: fsync d8/fd 0 2024-05-25T06:11:18.459 INFO:tasks.workunit.client.1.smithi203.stdout:6/100: mknod d8/d11/d1b/c25 0 2024-05-25T06:11:18.460 INFO:tasks.workunit.client.1.smithi203.stdout:5/99: dwrite dc/d11/f12 [0,4194304] 0 2024-05-25T06:11:18.464 INFO:tasks.workunit.client.1.smithi203.stdout:8/101: dwrite d2/d5/d18/f1b [0,4194304] 0 2024-05-25T06:11:18.466 INFO:tasks.workunit.client.1.smithi203.stdout:6/101: creat d8/d11/d1b/f26 x:0 0 0 2024-05-25T06:11:18.468 INFO:tasks.workunit.client.1.smithi203.stdout:8/102: write d2/d5/d10/d11/de/f1a [985391,87441] 0 2024-05-25T06:11:18.471 INFO:tasks.workunit.client.1.smithi203.stdout:5/100: rename f3 to dc/f21 0 2024-05-25T06:11:18.477 INFO:tasks.workunit.client.1.smithi203.stdout:6/102: rename d8/f10 to d8/d11/d15/f27 0 2024-05-25T06:11:18.482 INFO:tasks.workunit.client.1.smithi203.stdout:5/101: rename dc/df to dc/d11/d17/d1a/d22 0 2024-05-25T06:11:18.486 INFO:tasks.workunit.client.1.smithi203.stdout:6/103: rename d8/d11/d1a to d8/d11/d15/d28 0 2024-05-25T06:11:18.486 INFO:tasks.workunit.client.1.smithi203.stdout:8/103: getdents d2/d5/d10/d11/d1f 0 2024-05-25T06:11:18.488 INFO:tasks.workunit.client.1.smithi203.stdout:5/102: mknod dc/d11/d18/c23 0 2024-05-25T06:11:18.488 INFO:tasks.workunit.client.1.smithi203.stdout:5/103: readlink dc/l1f 0 2024-05-25T06:11:18.491 INFO:tasks.workunit.client.1.smithi203.stdout:8/104: symlink d2/d5/d10/d11/de/l27 0 2024-05-25T06:11:18.491 INFO:tasks.workunit.client.1.smithi203.stdout:6/104: mkdir d8/d29 0 2024-05-25T06:11:18.494 INFO:tasks.workunit.client.1.smithi203.stdout:5/104: creat dc/d11/d17/d1a/d22/f24 x:0 0 0 2024-05-25T06:11:18.611 INFO:tasks.workunit.client.1.smithi203.stdout:6/105: dwrite f2 [4194304,4194304] 0 2024-05-25T06:11:18.614 INFO:tasks.workunit.client.1.smithi203.stdout:8/105: dwrite d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:18.616 INFO:tasks.workunit.client.1.smithi203.stdout:5/105: dwrite dc/d11/f14 [0,4194304] 0 2024-05-25T06:11:18.620 INFO:tasks.workunit.client.1.smithi203.stdout:8/106: mkdir d2/d5/d28 0 2024-05-25T06:11:18.625 INFO:tasks.workunit.client.1.smithi203.stdout:6/106: dwrite d8/ff [0,4194304] 0 2024-05-25T06:11:18.627 INFO:tasks.workunit.client.1.smithi203.stdout:6/107: chown d8/fb 2015036704 1 2024-05-25T06:11:18.630 INFO:tasks.workunit.client.1.smithi203.stdout:5/106: dwrite dc/f19 [0,4194304] 0 2024-05-25T06:11:18.632 INFO:tasks.workunit.client.1.smithi203.stdout:5/107: fsync dc/d11/f13 0 2024-05-25T06:11:18.633 INFO:tasks.workunit.client.1.smithi203.stdout:8/107: creat d2/d5/f29 x:0 0 0 2024-05-25T06:11:18.635 INFO:tasks.workunit.client.1.smithi203.stdout:6/108: dread d8/ff [0,4194304] 0 2024-05-25T06:11:18.636 INFO:tasks.workunit.client.1.smithi203.stdout:5/108: mknod dc/d11/c25 0 2024-05-25T06:11:18.637 INFO:tasks.workunit.client.1.smithi203.stdout:8/108: creat d2/f2a x:0 0 0 2024-05-25T06:11:18.638 INFO:tasks.workunit.client.1.smithi203.stdout:5/109: write dc/fe [733281,118894] 0 2024-05-25T06:11:18.638 INFO:tasks.workunit.client.1.smithi203.stdout:8/109: stat d2/d5/d28 0 2024-05-25T06:11:18.639 INFO:tasks.workunit.client.1.smithi203.stdout:6/109: mkdir d8/d2a 0 2024-05-25T06:11:18.639 INFO:tasks.workunit.client.1.smithi203.stdout:8/110: chown d2/d5/d10 304 1 2024-05-25T06:11:18.643 INFO:tasks.workunit.client.1.smithi203.stdout:6/110: unlink d8/d11/d15/d28/l1f 0 2024-05-25T06:11:18.643 INFO:tasks.workunit.client.1.smithi203.stdout:8/111: readlink d2/d5/d10/d11/l20 0 2024-05-25T06:11:18.650 INFO:tasks.workunit.client.1.smithi203.stdout:5/110: dwrite dc/d11/f14 [0,4194304] 0 2024-05-25T06:11:18.653 INFO:tasks.workunit.client.1.smithi203.stdout:8/112: rename d2/l7 to d2/d5/l2b 0 2024-05-25T06:11:18.654 INFO:tasks.workunit.client.1.smithi203.stdout:6/111: dread d8/ff [0,4194304] 0 2024-05-25T06:11:18.657 INFO:tasks.workunit.client.1.smithi203.stdout:6/112: creat d8/d11/d15/f2b x:0 0 0 2024-05-25T06:11:18.657 INFO:tasks.workunit.client.1.smithi203.stdout:8/113: unlink d2/d5/d10/d11/de/l27 0 2024-05-25T06:11:18.660 INFO:tasks.workunit.client.1.smithi203.stdout:5/111: dread dc/f10 [0,4194304] 0 2024-05-25T06:11:18.663 INFO:tasks.workunit.client.1.smithi203.stdout:5/112: creat dc/d11/d17/f26 x:0 0 0 2024-05-25T06:11:18.665 INFO:tasks.workunit.client.1.smithi203.stdout:8/114: dread d2/f3 [0,4194304] 0 2024-05-25T06:11:18.665 INFO:tasks.workunit.client.1.smithi203.stdout:6/113: creat d8/f2c x:0 0 0 2024-05-25T06:11:18.666 INFO:tasks.workunit.client.1.smithi203.stdout:6/114: readlink d8/d11/l21 0 2024-05-25T06:11:18.667 INFO:tasks.workunit.client.1.smithi203.stdout:5/113: creat dc/d11/d17/d1a/f27 x:0 0 0 2024-05-25T06:11:18.668 INFO:tasks.workunit.client.1.smithi203.stdout:8/115: mknod d2/d5/d10/d11/de/c2c 0 2024-05-25T06:11:18.673 INFO:tasks.workunit.client.1.smithi203.stdout:5/114: symlink dc/l28 0 2024-05-25T06:11:18.674 INFO:tasks.workunit.client.1.smithi203.stdout:6/115: creat d8/f2d x:0 0 0 2024-05-25T06:11:18.676 INFO:tasks.workunit.client.1.smithi203.stdout:8/116: mkdir d2/d5/d2d 0 2024-05-25T06:11:18.677 INFO:tasks.workunit.client.1.smithi203.stdout:5/115: creat dc/d11/f29 x:0 0 0 2024-05-25T06:11:18.677 INFO:tasks.workunit.client.1.smithi203.stdout:8/117: readlink d2/d5/d10/d11/d1f/l25 0 2024-05-25T06:11:18.678 INFO:tasks.workunit.client.1.smithi203.stdout:6/116: creat d8/d11/d15/f2e x:0 0 0 2024-05-25T06:11:18.681 INFO:tasks.workunit.client.1.smithi203.stdout:5/116: rename dc/l16 to dc/d11/d17/d1a/l2a 0 2024-05-25T06:11:18.681 INFO:tasks.workunit.client.1.smithi203.stdout:8/118: rename d2/d5/d10/d11 to d2/d5/d10/d11/de/d2e 22 2024-05-25T06:11:18.683 INFO:tasks.workunit.client.1.smithi203.stdout:6/117: symlink d8/d2a/l2f 0 2024-05-25T06:11:18.684 INFO:tasks.workunit.client.1.smithi203.stdout:5/117: symlink dc/d11/d18/l2b 0 2024-05-25T06:11:18.691 INFO:tasks.workunit.client.1.smithi203.stdout:6/118: mkdir d8/d29/d30 0 2024-05-25T06:11:18.693 INFO:tasks.workunit.client.1.smithi203.stdout:5/118: dread dc/fe [0,4194304] 0 2024-05-25T06:11:18.693 INFO:tasks.workunit.client.1.smithi203.stdout:6/119: creat d8/d11/d15/f31 x:0 0 0 2024-05-25T06:11:18.695 INFO:tasks.workunit.client.1.smithi203.stdout:6/120: truncate d8/d11/d15/f31 60372 0 2024-05-25T06:11:18.697 INFO:tasks.workunit.client.1.smithi203.stdout:5/119: rename c5 to dc/d11/c2c 0 2024-05-25T06:11:18.698 INFO:tasks.workunit.client.1.smithi203.stdout:8/119: link d2/l21 d2/d5/l2f 0 2024-05-25T06:11:18.699 INFO:tasks.workunit.client.1.smithi203.stdout:8/120: stat d2/d5/d2d 0 2024-05-25T06:11:18.699 INFO:tasks.workunit.client.1.smithi203.stdout:8/121: stat d2/d5/f29 0 2024-05-25T06:11:18.700 INFO:tasks.workunit.client.1.smithi203.stdout:8/122: write d2/f2a [16947,76046] 0 2024-05-25T06:11:18.700 INFO:tasks.workunit.client.1.smithi203.stdout:5/120: mkdir dc/d11/d17/d1a/d2d 0 2024-05-25T06:11:18.702 INFO:tasks.workunit.client.1.smithi203.stdout:8/123: write d2/d5/d10/d11/de/f13 [987816,18743] 0 2024-05-25T06:11:18.707 INFO:tasks.workunit.client.1.smithi203.stdout:8/124: mknod d2/d5/d10/d11/d1f/c30 0 2024-05-25T06:11:18.709 INFO:tasks.workunit.client.1.smithi203.stdout:8/125: write d2/d5/d18/f1b [4899764,101491] 0 2024-05-25T06:11:18.711 INFO:tasks.workunit.client.1.smithi203.stdout:5/121: dwrite dc/d11/d17/d1a/d22/f24 [0,4194304] 0 2024-05-25T06:11:18.713 INFO:tasks.workunit.client.1.smithi203.stdout:6/121: rmdir d8/d11/d15 39 2024-05-25T06:11:18.714 INFO:tasks.workunit.client.1.smithi203.stdout:5/122: write dc/d11/f1c [12594,88340] 0 2024-05-25T06:11:18.714 INFO:tasks.workunit.client.1.smithi203.stdout:6/122: stat d8/d29/d30 0 2024-05-25T06:11:18.716 INFO:tasks.workunit.client.1.smithi203.stdout:8/126: fsync d2/d5/f8 0 2024-05-25T06:11:18.718 INFO:tasks.workunit.client.1.smithi203.stdout:5/123: fdatasync f8 0 2024-05-25T06:11:18.718 INFO:tasks.workunit.client.1.smithi203.stdout:5/124: chown l4 72453901 1 2024-05-25T06:11:18.722 INFO:tasks.workunit.client.1.smithi203.stdout:6/123: link d8/d11/d1b/f26 d8/d11/d1b/d24/f32 0 2024-05-25T06:11:18.722 INFO:tasks.workunit.client.1.smithi203.stdout:8/127: dread d2/f14 [0,4194304] 0 2024-05-25T06:11:18.723 INFO:tasks.workunit.client.1.smithi203.stdout:5/125: creat dc/d11/d17/d1a/f2e x:0 0 0 2024-05-25T06:11:18.724 INFO:tasks.workunit.client.1.smithi203.stdout:6/124: unlink d8/f2d 0 2024-05-25T06:11:18.726 INFO:tasks.workunit.client.1.smithi203.stdout:8/128: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:18.727 INFO:tasks.workunit.client.1.smithi203.stdout:5/126: creat dc/d11/d17/f2f x:0 0 0 2024-05-25T06:11:18.729 INFO:tasks.workunit.client.1.smithi203.stdout:6/125: symlink d8/d11/d15/l33 0 2024-05-25T06:11:18.729 INFO:tasks.workunit.client.1.smithi203.stdout:6/126: readlink l5 0 2024-05-25T06:11:18.730 INFO:tasks.workunit.client.1.smithi203.stdout:5/127: unlink dc/f19 0 2024-05-25T06:11:18.733 INFO:tasks.workunit.client.1.smithi203.stdout:5/128: dread dc/d11/f1c [0,4194304] 0 2024-05-25T06:11:18.735 INFO:tasks.workunit.client.1.smithi203.stdout:6/127: mknod d8/d29/d30/c34 0 2024-05-25T06:11:18.736 INFO:tasks.workunit.client.1.smithi203.stdout:6/128: chown d8/d29/d30/c34 1583464759 1 2024-05-25T06:11:18.738 INFO:tasks.workunit.client.1.smithi203.stdout:6/129: truncate d8/d11/d15/f31 1047591 0 2024-05-25T06:11:18.739 INFO:tasks.workunit.client.1.smithi203.stdout:5/129: getdents dc/d11/d17/d1a 0 2024-05-25T06:11:18.739 INFO:tasks.workunit.client.1.smithi203.stdout:6/130: truncate d8/fb 1229312 0 2024-05-25T06:11:18.743 INFO:tasks.workunit.client.1.smithi203.stdout:8/129: truncate d2/d5/d10/d11/de/f1a 589627 0 2024-05-25T06:11:18.744 INFO:tasks.workunit.client.1.smithi203.stdout:8/130: fsync d2/f2a 0 2024-05-25T06:11:18.745 INFO:tasks.workunit.client.1.smithi203.stdout:6/131: link d8/d11/d15/f2e d8/d11/d15/f35 0 2024-05-25T06:11:18.746 INFO:tasks.workunit.client.1.smithi203.stdout:6/132: dread - d8/d11/d15/f2e zero size 2024-05-25T06:11:18.748 INFO:tasks.workunit.client.1.smithi203.stdout:8/131: mkdir d2/d5/d18/d31 0 2024-05-25T06:11:18.750 INFO:tasks.workunit.client.1.smithi203.stdout:5/130: rename dc/c1b to dc/c30 0 2024-05-25T06:11:18.754 INFO:tasks.workunit.client.1.smithi203.stdout:5/131: creat dc/d11/d17/f31 x:0 0 0 2024-05-25T06:11:18.756 INFO:tasks.workunit.client.1.smithi203.stdout:8/132: symlink d2/d5/l32 0 2024-05-25T06:11:18.769 INFO:tasks.workunit.client.1.smithi203.stdout:6/133: dwrite d8/d11/d1b/d24/f32 [0,4194304] 0 2024-05-25T06:11:18.770 INFO:tasks.workunit.client.1.smithi203.stdout:8/133: dwrite d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:18.772 INFO:tasks.workunit.client.1.smithi203.stdout:5/132: dwrite fb [0,4194304] 0 2024-05-25T06:11:18.773 INFO:tasks.workunit.client.1.smithi203.stdout:6/134: dread - d8/d11/f19 zero size 2024-05-25T06:11:18.779 INFO:tasks.workunit.client.1.smithi203.stdout:8/134: write d2/d5/d10/d11/de/ff [4132545,24988] 0 2024-05-25T06:11:18.780 INFO:tasks.workunit.client.1.smithi203.stdout:5/133: dread fb [0,4194304] 0 2024-05-25T06:11:18.782 INFO:tasks.workunit.client.1.smithi203.stdout:6/135: rename d8/l13 to d8/d11/d1b/l36 0 2024-05-25T06:11:18.784 INFO:tasks.workunit.client.1.smithi203.stdout:8/135: symlink d2/d5/d10/d11/l33 0 2024-05-25T06:11:18.786 INFO:tasks.workunit.client.1.smithi203.stdout:8/136: unlink d2/f3 0 2024-05-25T06:11:18.786 INFO:tasks.workunit.client.1.smithi203.stdout:5/134: dread dc/d11/f13 [0,4194304] 0 2024-05-25T06:11:18.789 INFO:tasks.workunit.client.1.smithi203.stdout:5/135: chown dc/d11/d17/d1a/d22/f24 141 1 2024-05-25T06:11:18.792 INFO:tasks.workunit.client.1.smithi203.stdout:6/136: dwrite d8/d11/d15/f31 [0,4194304] 0 2024-05-25T06:11:18.794 INFO:tasks.workunit.client.1.smithi203.stdout:5/136: creat dc/d11/d17/d1a/f32 x:0 0 0 2024-05-25T06:11:18.795 INFO:tasks.workunit.client.1.smithi203.stdout:8/137: dread d2/d5/d18/f1b [0,4194304] 0 2024-05-25T06:11:18.796 INFO:tasks.workunit.client.1.smithi203.stdout:6/137: creat d8/d11/d15/f37 x:0 0 0 2024-05-25T06:11:18.798 INFO:tasks.workunit.client.1.smithi203.stdout:5/137: rename l4 to dc/d11/l33 0 2024-05-25T06:11:18.800 INFO:tasks.workunit.client.1.smithi203.stdout:5/138: truncate dc/d11/d17/d1a/f32 526884 0 2024-05-25T06:11:18.801 INFO:tasks.workunit.client.1.smithi203.stdout:6/138: mkdir d8/d29/d30/d38 0 2024-05-25T06:11:18.801 INFO:tasks.workunit.client.1.smithi203.stdout:6/139: fsync f1 0 2024-05-25T06:11:18.803 INFO:tasks.workunit.client.1.smithi203.stdout:8/138: creat d2/d5/d18/d31/f34 x:0 0 0 2024-05-25T06:11:18.807 INFO:tasks.workunit.client.1.smithi203.stdout:5/139: rename dc/d11/c2c to dc/d11/d17/d1a/d22/c34 0 2024-05-25T06:11:18.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/139: creat d2/d5/d10/d11/d1f/f35 x:0 0 0 2024-05-25T06:11:18.811 INFO:tasks.workunit.client.1.smithi203.stdout:5/140: rename dc/d11/d17/d1a/d22 to dc/d11/d17/d1a/d22/d35 22 2024-05-25T06:11:18.813 INFO:tasks.workunit.client.1.smithi203.stdout:6/140: dread f0 [0,4194304] 0 2024-05-25T06:11:18.815 INFO:tasks.workunit.client.1.smithi203.stdout:6/141: rmdir d8/d11 39 2024-05-25T06:11:18.817 INFO:tasks.workunit.client.1.smithi203.stdout:6/142: dread - d8/f9 zero size 2024-05-25T06:11:18.818 INFO:tasks.workunit.client.1.smithi203.stdout:8/140: write d2/d5/d10/d11/de/f1d [3939593,2099] 0 2024-05-25T06:11:18.821 INFO:tasks.workunit.client.1.smithi203.stdout:5/141: dwrite dc/d11/d17/d1a/f2e [0,4194304] 0 2024-05-25T06:11:18.821 INFO:tasks.workunit.client.1.smithi203.stdout:8/141: chown d2/l21 1 1 2024-05-25T06:11:18.824 INFO:tasks.workunit.client.1.smithi203.stdout:5/142: mknod dc/d11/d17/d1a/d22/c36 0 2024-05-25T06:11:18.827 INFO:tasks.workunit.client.1.smithi203.stdout:8/142: rename d2/d5/d10/d11/l33 to d2/d5/d18/d31/l36 0 2024-05-25T06:11:18.828 INFO:tasks.workunit.client.1.smithi203.stdout:5/143: link dc/d11/d17/f31 dc/d11/d18/f37 0 2024-05-25T06:11:18.829 INFO:tasks.workunit.client.1.smithi203.stdout:5/144: read f6 [2930810,29786] 0 2024-05-25T06:11:18.831 INFO:tasks.workunit.client.1.smithi203.stdout:5/145: chown dc/d11/d17/f2f 9486 1 2024-05-25T06:11:18.837 INFO:tasks.workunit.client.1.smithi203.stdout:8/143: dwrite d2/d5/d10/d11/de/f13 [0,4194304] 0 2024-05-25T06:11:18.837 INFO:tasks.workunit.client.1.smithi203.stdout:5/146: rmdir dc/d11 39 2024-05-25T06:11:18.840 INFO:tasks.workunit.client.1.smithi203.stdout:5/147: chown dc/d11/d17/d1a/f27 429802 1 2024-05-25T06:11:18.846 INFO:tasks.workunit.client.1.smithi203.stdout:8/144: dwrite d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:18.847 INFO:tasks.workunit.client.1.smithi203.stdout:5/148: dread f6 [0,4194304] 0 2024-05-25T06:11:18.854 INFO:tasks.workunit.client.1.smithi203.stdout:5/149: truncate dc/d11/f12 1393701 0 2024-05-25T06:11:18.867 INFO:tasks.workunit.client.1.smithi203.stdout:5/150: truncate dc/f21 1646853 0 2024-05-25T06:11:18.871 INFO:tasks.workunit.client.1.smithi203.stdout:5/151: dread f1 [0,4194304] 0 2024-05-25T06:11:18.873 INFO:tasks.workunit.client.1.smithi203.stdout:5/152: creat dc/d11/d17/f38 x:0 0 0 2024-05-25T06:11:18.877 INFO:tasks.workunit.client.1.smithi203.stdout:5/153: rename dc/d11/f13 to dc/d11/d17/d1a/d22/f39 0 2024-05-25T06:11:18.879 INFO:tasks.workunit.client.1.smithi203.stdout:5/154: symlink dc/d11/l3a 0 2024-05-25T06:11:18.879 INFO:tasks.workunit.client.1.smithi203.stdout:5/155: chown dc/d11/d17/f31 159047638 1 2024-05-25T06:11:18.881 INFO:tasks.workunit.client.1.smithi203.stdout:5/156: mknod dc/d11/d17/d1a/d22/c3b 0 2024-05-25T06:11:18.882 INFO:tasks.workunit.client.1.smithi203.stdout:5/157: unlink dc/l28 0 2024-05-25T06:11:18.883 INFO:tasks.workunit.client.1.smithi203.stdout:5/158: readlink dc/l1f 0 2024-05-25T06:11:18.884 INFO:tasks.workunit.client.1.smithi203.stdout:5/159: write dc/d11/f29 [429984,41702] 0 2024-05-25T06:11:18.887 INFO:tasks.workunit.client.1.smithi203.stdout:5/160: mkdir dc/d3c 0 2024-05-25T06:11:18.887 INFO:tasks.workunit.client.1.smithi203.stdout:5/161: chown c7 125 1 2024-05-25T06:11:18.888 INFO:tasks.workunit.client.1.smithi203.stdout:5/162: mknod dc/d11/d18/c3d 0 2024-05-25T06:11:18.890 INFO:tasks.workunit.client.1.smithi203.stdout:5/163: creat dc/d11/f3e x:0 0 0 2024-05-25T06:11:18.891 INFO:tasks.workunit.client.1.smithi203.stdout:5/164: write dc/d11/d17/f2f [265009,104141] 0 2024-05-25T06:11:18.893 INFO:tasks.workunit.client.1.smithi203.stdout:5/165: dread dc/d11/d17/d1a/d22/f39 [0,4194304] 0 2024-05-25T06:11:18.894 INFO:tasks.workunit.client.1.smithi203.stdout:5/166: dread dc/d11/d17/d1a/d22/f39 [0,4194304] 0 2024-05-25T06:11:18.896 INFO:tasks.workunit.client.1.smithi203.stdout:5/167: mkdir dc/d3c/d3f 0 2024-05-25T06:11:18.901 INFO:tasks.workunit.client.1.smithi203.stdout:5/168: fsync dc/d11/d17/f38 0 2024-05-25T06:11:18.902 INFO:tasks.workunit.client.1.smithi203.stdout:5/169: fsync dc/d11/d17/d1a/d22/f15 0 2024-05-25T06:11:18.905 INFO:tasks.workunit.client.1.smithi203.stdout:5/170: mknod dc/d11/d17/d1a/c40 0 2024-05-25T06:11:18.906 INFO:tasks.workunit.client.1.smithi203.stdout:5/171: chown dc/d11/d17/d1a/d22/l20 224 1 2024-05-25T06:11:18.907 INFO:tasks.workunit.client.1.smithi203.stdout:5/172: mknod dc/d11/d17/d1a/c41 0 2024-05-25T06:11:18.909 INFO:tasks.workunit.client.1.smithi203.stdout:5/173: read dc/d11/f29 [392843,21073] 0 2024-05-25T06:11:18.909 INFO:tasks.workunit.client.1.smithi203.stdout:5/174: chown dc/d11/d17/f31 51764685 1 2024-05-25T06:11:18.910 INFO:tasks.workunit.client.1.smithi203.stdout:5/175: read - dc/d11/d17/f31 zero size 2024-05-25T06:11:18.912 INFO:tasks.workunit.client.1.smithi203.stdout:5/176: truncate dc/d11/d17/d1a/d22/f15 717640 0 2024-05-25T06:11:18.913 INFO:tasks.workunit.client.1.smithi203.stdout:5/177: read dc/d11/d17/d1a/f2e [3292701,31834] 0 2024-05-25T06:11:18.917 INFO:tasks.workunit.client.1.smithi203.stdout:5/178: fdatasync dc/f21 0 2024-05-25T06:11:18.924 INFO:tasks.workunit.client.1.smithi203.stdout:5/179: dwrite dc/d11/d17/d1a/f2e [0,4194304] 0 2024-05-25T06:11:18.925 INFO:tasks.workunit.client.1.smithi203.stdout:5/180: chown f1 12 1 2024-05-25T06:11:18.936 INFO:tasks.workunit.client.1.smithi203.stdout:5/181: dwrite dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:18.945 INFO:tasks.workunit.client.1.smithi203.stdout:5/182: dwrite f1 [0,4194304] 0 2024-05-25T06:11:18.948 INFO:tasks.workunit.client.1.smithi203.stdout:5/183: mkdir dc/d11/d42 0 2024-05-25T06:11:18.951 INFO:tasks.workunit.client.1.smithi203.stdout:5/184: write fb [2944135,24782] 0 2024-05-25T06:11:18.953 INFO:tasks.workunit.client.1.smithi203.stdout:5/185: truncate dc/d11/d17/f26 359994 0 2024-05-25T06:11:18.957 INFO:tasks.workunit.client.1.smithi203.stdout:5/186: dread dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:18.960 INFO:tasks.workunit.client.1.smithi203.stdout:5/187: unlink dc/d11/d17/d1a/d22/f15 0 2024-05-25T06:11:18.962 INFO:tasks.workunit.client.1.smithi203.stdout:5/188: mknod dc/d11/d17/c43 0 2024-05-25T06:11:18.964 INFO:tasks.workunit.client.1.smithi203.stdout:5/189: read dc/f21 [1608028,88123] 0 2024-05-25T06:11:18.970 INFO:tasks.workunit.client.1.smithi203.stdout:5/190: dwrite fb [0,4194304] 0 2024-05-25T06:11:18.972 INFO:tasks.workunit.client.1.smithi203.stdout:5/191: mkdir dc/d11/d17/d44 0 2024-05-25T06:11:18.974 INFO:tasks.workunit.client.1.smithi203.stdout:5/192: mkdir dc/d11/d17/d45 0 2024-05-25T06:11:18.982 INFO:tasks.workunit.client.1.smithi203.stdout:5/193: write dc/d11/d18/f37 [866971,46048] 0 2024-05-25T06:11:18.989 INFO:tasks.workunit.client.1.smithi203.stdout:5/194: dwrite dc/d11/d17/f2f [0,4194304] 0 2024-05-25T06:11:18.991 INFO:tasks.workunit.client.1.smithi203.stdout:5/195: fdatasync dc/f21 0 2024-05-25T06:11:18.994 INFO:tasks.workunit.client.1.smithi203.stdout:5/196: getdents dc/d3c 0 2024-05-25T06:11:19.320 INFO:tasks.workunit.client.1.smithi203.stdout:3/53: sync 2024-05-25T06:11:19.320 INFO:tasks.workunit.client.1.smithi203.stdout:4/44: sync 2024-05-25T06:11:19.326 INFO:tasks.workunit.client.1.smithi203.stdout:3/54: dwrite f0 [0,4194304] 0 2024-05-25T06:11:19.327 INFO:tasks.workunit.client.1.smithi203.stdout:4/45: dwrite d0/f4 [0,4194304] 0 2024-05-25T06:11:19.330 INFO:tasks.workunit.client.1.smithi203.stdout:4/46: creat d0/d8/ff x:0 0 0 2024-05-25T06:11:19.331 INFO:tasks.workunit.client.1.smithi203.stdout:3/55: creat d10/f13 x:0 0 0 2024-05-25T06:11:19.336 INFO:tasks.workunit.client.1.smithi203.stdout:3/56: symlink d10/l14 0 2024-05-25T06:11:19.336 INFO:tasks.workunit.client.1.smithi203.stdout:4/47: dread d0/f5 [0,4194304] 0 2024-05-25T06:11:19.337 INFO:tasks.workunit.client.1.smithi203.stdout:4/48: dread - d0/d8/ff zero size 2024-05-25T06:11:19.340 INFO:tasks.workunit.client.1.smithi203.stdout:4/49: rename d0/d8/dd to d0/d8/d10 0 2024-05-25T06:11:19.341 INFO:tasks.workunit.client.1.smithi203.stdout:4/50: mkdir d0/d9/d11 0 2024-05-25T06:11:19.342 INFO:tasks.workunit.client.1.smithi203.stdout:4/51: dread - d0/d8/ff zero size 2024-05-25T06:11:19.349 INFO:tasks.workunit.client.1.smithi203.stdout:4/52: dwrite d0/d8/ff [0,4194304] 0 2024-05-25T06:11:19.352 INFO:tasks.workunit.client.1.smithi203.stdout:3/57: rmdir d10 39 2024-05-25T06:11:19.354 INFO:tasks.workunit.client.1.smithi203.stdout:4/53: dread d0/f4 [0,4194304] 0 2024-05-25T06:11:19.356 INFO:tasks.workunit.client.1.smithi203.stdout:4/54: read d0/d8/ff [3710651,36103] 0 2024-05-25T06:11:19.362 INFO:tasks.workunit.client.1.smithi203.stdout:4/55: dwrite d0/f3 [0,4194304] 0 2024-05-25T06:11:19.364 INFO:tasks.workunit.client.1.smithi203.stdout:4/56: rmdir d0 39 2024-05-25T06:11:19.373 INFO:tasks.workunit.client.1.smithi203.stdout:4/57: dwrite d0/f4 [0,4194304] 0 2024-05-25T06:11:19.373 INFO:tasks.workunit.client.1.smithi203.stdout:4/58: chown d0/d8/ca 924 1 2024-05-25T06:11:19.376 INFO:tasks.workunit.client.1.smithi203.stdout:4/59: creat d0/d9/f12 x:0 0 0 2024-05-25T06:11:19.379 INFO:tasks.workunit.client.1.smithi203.stdout:4/60: getdents d0/d9/d11 0 2024-05-25T06:11:19.380 INFO:tasks.workunit.client.1.smithi203.stdout:4/61: truncate d0/d9/f12 743517 0 2024-05-25T06:11:19.388 INFO:tasks.workunit.client.1.smithi203.stdout:4/62: fsync d0/d9/f12 0 2024-05-25T06:11:19.390 INFO:tasks.workunit.client.1.smithi203.stdout:4/63: rmdir d0/d9 39 2024-05-25T06:11:19.392 INFO:tasks.workunit.client.1.smithi203.stdout:4/64: write d0/f4 [3177590,90057] 0 2024-05-25T06:11:19.397 INFO:tasks.workunit.client.1.smithi203.stdout:4/65: truncate d0/f3 838841 0 2024-05-25T06:11:19.398 INFO:tasks.workunit.client.1.smithi203.stdout:4/66: stat d0/d9 0 2024-05-25T06:11:19.404 INFO:tasks.workunit.client.1.smithi203.stdout:4/67: truncate d0/d8/ff 906109 0 2024-05-25T06:11:19.418 INFO:tasks.workunit.client.1.smithi203.stdout:4/68: dwrite d0/f3 [0,4194304] 0 2024-05-25T06:11:19.420 INFO:tasks.workunit.client.1.smithi203.stdout:4/69: symlink d0/d8/d10/l13 0 2024-05-25T06:11:19.421 INFO:tasks.workunit.client.1.smithi203.stdout:4/70: read d0/d9/f12 [582132,97498] 0 2024-05-25T06:11:19.422 INFO:tasks.workunit.client.1.smithi203.stdout:4/71: rename d0/d8/d10 to d0/d8/d10/d14 22 2024-05-25T06:11:19.423 INFO:tasks.workunit.client.1.smithi203.stdout:4/72: read d0/f3 [4122322,115352] 0 2024-05-25T06:11:19.426 INFO:tasks.workunit.client.1.smithi203.stdout:4/73: creat d0/d8/d10/f15 x:0 0 0 2024-05-25T06:11:19.433 INFO:tasks.workunit.client.1.smithi203.stdout:4/74: fsync d0/d8/d10/f15 0 2024-05-25T06:11:19.435 INFO:tasks.workunit.client.1.smithi203.stdout:4/75: creat d0/f16 x:0 0 0 2024-05-25T06:11:19.438 INFO:tasks.workunit.client.1.smithi203.stdout:4/76: unlink d0/f4 0 2024-05-25T06:11:19.438 INFO:tasks.workunit.client.1.smithi203.stdout:4/77: write d0/f16 [81973,96828] 0 2024-05-25T06:11:19.439 INFO:tasks.workunit.client.1.smithi203.stdout:4/78: stat d0/d8 0 2024-05-25T06:11:19.440 INFO:tasks.workunit.client.1.smithi203.stdout:4/79: fsync d0/f5 0 2024-05-25T06:11:19.442 INFO:tasks.workunit.client.1.smithi203.stdout:4/80: dread d0/d8/ff [0,4194304] 0 2024-05-25T06:11:19.443 INFO:tasks.workunit.client.1.smithi203.stdout:4/81: chown d0/d9/f12 2006 1 2024-05-25T06:11:19.445 INFO:tasks.workunit.client.1.smithi203.stdout:4/82: creat d0/d9/f17 x:0 0 0 2024-05-25T06:11:19.509 INFO:tasks.workunit.client.1.smithi203.stdout:0/71: sync 2024-05-25T06:11:19.509 INFO:tasks.workunit.client.1.smithi203.stdout:9/69: sync 2024-05-25T06:11:19.511 INFO:tasks.workunit.client.1.smithi203.stdout:9/70: write d0/fc [4503044,83901] 0 2024-05-25T06:11:19.516 INFO:tasks.workunit.client.1.smithi203.stdout:0/72: dwrite dd/f17 [0,4194304] 0 2024-05-25T06:11:19.518 INFO:tasks.workunit.client.1.smithi203.stdout:9/71: dread d0/df/f15 [0,4194304] 0 2024-05-25T06:11:19.520 INFO:tasks.workunit.client.1.smithi203.stdout:9/72: mkdir d0/d17 0 2024-05-25T06:11:19.520 INFO:tasks.workunit.client.1.smithi203.stdout:0/73: dread f6 [0,4194304] 0 2024-05-25T06:11:19.521 INFO:tasks.workunit.client.1.smithi203.stdout:9/73: chown d0/fc 27547117 1 2024-05-25T06:11:19.522 INFO:tasks.workunit.client.1.smithi203.stdout:0/74: creat dd/d10/f18 x:0 0 0 2024-05-25T06:11:19.524 INFO:tasks.workunit.client.1.smithi203.stdout:9/74: rename d0/fc to d0/f18 0 2024-05-25T06:11:19.525 INFO:tasks.workunit.client.1.smithi203.stdout:0/75: symlink dd/l19 0 2024-05-25T06:11:19.526 INFO:tasks.workunit.client.1.smithi203.stdout:9/75: rename d0/df/f11 to d0/f19 0 2024-05-25T06:11:19.530 INFO:tasks.workunit.client.1.smithi203.stdout:9/76: dread d0/f18 [0,4194304] 0 2024-05-25T06:11:19.531 INFO:tasks.workunit.client.1.smithi203.stdout:0/76: getdents dd 0 2024-05-25T06:11:19.537 INFO:tasks.workunit.client.1.smithi203.stdout:0/77: dread dd/f17 [0,4194304] 0 2024-05-25T06:11:19.538 INFO:tasks.workunit.client.1.smithi203.stdout:0/78: chown l7 557670987 1 2024-05-25T06:11:19.538 INFO:tasks.workunit.client.1.smithi203.stdout:9/77: dwrite d0/f16 [4194304,4194304] 0 2024-05-25T06:11:19.539 INFO:tasks.workunit.client.1.smithi203.stdout:9/78: readlink d0/l3 0 2024-05-25T06:11:19.540 INFO:tasks.workunit.client.1.smithi203.stdout:0/79: creat dd/d10/f1a x:0 0 0 2024-05-25T06:11:19.541 INFO:tasks.workunit.client.1.smithi203.stdout:9/79: creat d0/df/f1a x:0 0 0 2024-05-25T06:11:19.541 INFO:tasks.workunit.client.1.smithi203.stdout:9/80: chown d0/f18 30301 1 2024-05-25T06:11:19.542 INFO:tasks.workunit.client.1.smithi203.stdout:0/80: mknod dd/d10/c1b 0 2024-05-25T06:11:19.542 INFO:tasks.workunit.client.1.smithi203.stdout:9/81: read d0/f19 [2742094,13851] 0 2024-05-25T06:11:19.543 INFO:tasks.workunit.client.1.smithi203.stdout:0/81: fdatasync dd/f11 0 2024-05-25T06:11:19.545 INFO:tasks.workunit.client.1.smithi203.stdout:0/82: creat dd/f1c x:0 0 0 2024-05-25T06:11:19.546 INFO:tasks.workunit.client.1.smithi203.stdout:9/82: getdents d0/d17 0 2024-05-25T06:11:19.549 INFO:tasks.workunit.client.1.smithi203.stdout:0/83: rename dd/d10/c16 to dd/c1d 0 2024-05-25T06:11:19.549 INFO:tasks.workunit.client.1.smithi203.stdout:0/84: readlink dd/lf 0 2024-05-25T06:11:19.551 INFO:tasks.workunit.client.1.smithi203.stdout:0/85: symlink dd/l1e 0 2024-05-25T06:11:19.553 INFO:tasks.workunit.client.1.smithi203.stdout:0/86: creat dd/f1f x:0 0 0 2024-05-25T06:11:19.554 INFO:tasks.workunit.client.1.smithi203.stdout:9/83: dwrite d0/f19 [4194304,4194304] 0 2024-05-25T06:11:19.558 INFO:tasks.workunit.client.1.smithi203.stdout:9/84: rename d0/fe to d0/d17/f1b 0 2024-05-25T06:11:19.560 INFO:tasks.workunit.client.1.smithi203.stdout:9/85: symlink d0/df/l1c 0 2024-05-25T06:11:19.562 INFO:tasks.workunit.client.1.smithi203.stdout:9/86: creat d0/df/f1d x:0 0 0 2024-05-25T06:11:19.563 INFO:tasks.workunit.client.1.smithi203.stdout:0/87: write f9 [3991410,110302] 0 2024-05-25T06:11:19.565 INFO:tasks.workunit.client.1.smithi203.stdout:0/88: creat dd/f20 x:0 0 0 2024-05-25T06:11:19.565 INFO:tasks.workunit.client.1.smithi203.stdout:0/89: write f9 [3676147,1478] 0 2024-05-25T06:11:19.566 INFO:tasks.workunit.client.1.smithi203.stdout:0/90: chown f5 2055738 1 2024-05-25T06:11:19.568 INFO:tasks.workunit.client.1.smithi203.stdout:0/91: mknod dd/c21 0 2024-05-25T06:11:19.568 INFO:tasks.workunit.client.1.smithi203.stdout:0/92: dread - dd/d10/f1a zero size 2024-05-25T06:11:19.570 INFO:tasks.workunit.client.1.smithi203.stdout:0/93: symlink dd/d10/l22 0 2024-05-25T06:11:19.572 INFO:tasks.workunit.client.1.smithi203.stdout:0/94: creat dd/f23 x:0 0 0 2024-05-25T06:11:19.574 INFO:tasks.workunit.client.1.smithi203.stdout:0/95: rename dd/f1c to dd/f24 0 2024-05-25T06:11:19.575 INFO:tasks.workunit.client.1.smithi203.stdout:0/96: creat dd/f25 x:0 0 0 2024-05-25T06:11:19.576 INFO:tasks.workunit.client.1.smithi203.stdout:0/97: read f9 [1753969,61425] 0 2024-05-25T06:11:19.578 INFO:tasks.workunit.client.1.smithi203.stdout:0/98: creat dd/d10/f26 x:0 0 0 2024-05-25T06:11:19.579 INFO:tasks.workunit.client.1.smithi203.stdout:0/99: symlink dd/l27 0 2024-05-25T06:11:19.581 INFO:tasks.workunit.client.1.smithi203.stdout:0/100: creat dd/d10/f28 x:0 0 0 2024-05-25T06:11:19.583 INFO:tasks.workunit.client.1.smithi203.stdout:0/101: link dd/f1f dd/d10/f29 0 2024-05-25T06:11:19.584 INFO:tasks.workunit.client.1.smithi203.stdout:0/102: read - dd/d10/f28 zero size 2024-05-25T06:11:19.586 INFO:tasks.workunit.client.1.smithi203.stdout:0/103: unlink dd/f20 0 2024-05-25T06:11:19.586 INFO:tasks.workunit.client.1.smithi203.stdout:0/104: stat dd/f23 0 2024-05-25T06:11:19.588 INFO:tasks.workunit.client.1.smithi203.stdout:0/105: unlink dd/l1e 0 2024-05-25T06:11:19.588 INFO:tasks.workunit.client.1.smithi203.stdout:0/106: rename dd to dd/d10/d2a 22 2024-05-25T06:11:19.589 INFO:tasks.workunit.client.1.smithi203.stdout:0/107: chown dd/d10/f29 821187 1 2024-05-25T06:11:19.589 INFO:tasks.workunit.client.1.smithi203.stdout:0/108: dread - dd/d10/f1a zero size 2024-05-25T06:11:19.657 INFO:tasks.workunit.client.1.smithi203.stdout:3/58: sync 2024-05-25T06:11:19.657 INFO:tasks.workunit.client.1.smithi203.stdout:7/76: sync 2024-05-25T06:11:19.658 INFO:tasks.workunit.client.1.smithi203.stdout:2/53: sync 2024-05-25T06:11:19.658 INFO:tasks.workunit.client.1.smithi203.stdout:6/143: sync 2024-05-25T06:11:19.658 INFO:tasks.workunit.client.1.smithi203.stdout:8/145: sync 2024-05-25T06:11:19.658 INFO:tasks.workunit.client.1.smithi203.stdout:5/197: sync 2024-05-25T06:11:19.658 INFO:tasks.workunit.client.1.smithi203.stdout:1/64: sync 2024-05-25T06:11:19.660 INFO:tasks.workunit.client.1.smithi203.stdout:1/65: mkdir d14 0 2024-05-25T06:11:19.662 INFO:tasks.workunit.client.1.smithi203.stdout:3/59: unlink d10/l12 0 2024-05-25T06:11:19.665 INFO:tasks.workunit.client.1.smithi203.stdout:7/77: rename d1/d9/l15 to d1/l1b 0 2024-05-25T06:11:19.665 INFO:tasks.workunit.client.1.smithi203.stdout:7/78: chown d1/l4 0 1 2024-05-25T06:11:19.666 INFO:tasks.workunit.client.1.smithi203.stdout:5/198: creat dc/d11/d17/d45/f46 x:0 0 0 2024-05-25T06:11:19.666 INFO:tasks.workunit.client.1.smithi203.stdout:1/66: mknod d14/c15 0 2024-05-25T06:11:19.668 INFO:tasks.workunit.client.1.smithi203.stdout:2/54: getdents d2 0 2024-05-25T06:11:19.668 INFO:tasks.workunit.client.1.smithi203.stdout:1/67: dread f10 [0,4194304] 0 2024-05-25T06:11:19.669 INFO:tasks.workunit.client.1.smithi203.stdout:1/68: read f10 [28576,71570] 0 2024-05-25T06:11:19.669 INFO:tasks.workunit.client.1.smithi203.stdout:7/79: unlink d1/d9/c10 0 2024-05-25T06:11:19.670 INFO:tasks.workunit.client.1.smithi203.stdout:1/69: write f12 [192689,83244] 0 2024-05-25T06:11:19.671 INFO:tasks.workunit.client.1.smithi203.stdout:8/146: rename d2/d5/d18/d31/l36 to d2/d5/d10/d11/l37 0 2024-05-25T06:11:19.671 INFO:tasks.workunit.client.1.smithi203.stdout:1/70: write fc [2388878,96592] 0 2024-05-25T06:11:19.672 INFO:tasks.workunit.client.1.smithi203.stdout:2/55: mknod d2/d6/da/c13 0 2024-05-25T06:11:19.675 INFO:tasks.workunit.client.1.smithi203.stdout:5/199: rename l9 to dc/d11/d17/d1a/l47 0 2024-05-25T06:11:19.675 INFO:tasks.workunit.client.1.smithi203.stdout:5/200: chown dc 1999 1 2024-05-25T06:11:19.677 INFO:tasks.workunit.client.1.smithi203.stdout:1/71: mknod d14/c16 0 2024-05-25T06:11:19.679 INFO:tasks.workunit.client.1.smithi203.stdout:2/56: unlink d2/f3 0 2024-05-25T06:11:19.682 INFO:tasks.workunit.client.1.smithi203.stdout:1/72: fsync fa 0 2024-05-25T06:11:19.685 INFO:tasks.workunit.client.1.smithi203.stdout:8/147: mkdir d2/d5/d28/d38 0 2024-05-25T06:11:19.688 INFO:tasks.workunit.client.1.smithi203.stdout:8/148: creat d2/d5/d18/d31/f39 x:0 0 0 2024-05-25T06:11:19.688 INFO:tasks.workunit.client.1.smithi203.stdout:5/201: link l0 dc/d11/d17/d1a/d2d/l48 0 2024-05-25T06:11:19.692 INFO:tasks.workunit.client.1.smithi203.stdout:5/202: unlink dc/d11/d17/f26 0 2024-05-25T06:11:19.695 INFO:tasks.workunit.client.1.smithi203.stdout:1/73: dwrite f10 [0,4194304] 0 2024-05-25T06:11:19.696 INFO:tasks.workunit.client.1.smithi203.stdout:8/149: dread d2/d5/d18/f1b [0,4194304] 0 2024-05-25T06:11:19.698 INFO:tasks.workunit.client.1.smithi203.stdout:1/74: creat d14/f17 x:0 0 0 2024-05-25T06:11:19.698 INFO:tasks.workunit.client.1.smithi203.stdout:1/75: write f10 [94361,80972] 0 2024-05-25T06:11:19.701 INFO:tasks.workunit.client.1.smithi203.stdout:8/150: creat d2/d5/d28/f3a x:0 0 0 2024-05-25T06:11:19.708 INFO:tasks.workunit.client.1.smithi203.stdout:8/151: rename d2/d5/d10/d11/l37 to d2/d5/d18/l3b 0 2024-05-25T06:11:19.708 INFO:tasks.workunit.client.1.smithi203.stdout:7/80: truncate d1/f2 1090977 0 2024-05-25T06:11:19.714 INFO:tasks.workunit.client.1.smithi203.stdout:2/57: mkdir d2/d6/d14 0 2024-05-25T06:11:19.715 INFO:tasks.workunit.client.1.smithi203.stdout:8/152: creat d2/d5/d28/d38/f3c x:0 0 0 2024-05-25T06:11:19.716 INFO:tasks.workunit.client.1.smithi203.stdout:7/81: link d1/d9/fd d1/f1c 0 2024-05-25T06:11:19.718 INFO:tasks.workunit.client.1.smithi203.stdout:5/203: write dc/d11/f12 [1994454,130911] 0 2024-05-25T06:11:19.725 INFO:tasks.workunit.client.1.smithi203.stdout:7/82: dread d1/d11/f17 [0,4194304] 0 2024-05-25T06:11:19.726 INFO:tasks.workunit.client.1.smithi203.stdout:2/58: symlink d2/d4/d10/l15 0 2024-05-25T06:11:19.728 INFO:tasks.workunit.client.1.smithi203.stdout:8/153: dwrite d2/d5/d18/f1b [4194304,4194304] 0 2024-05-25T06:11:19.728 INFO:tasks.workunit.client.1.smithi203.stdout:7/83: write d1/d9/fe [2017228,22262] 0 2024-05-25T06:11:19.729 INFO:tasks.workunit.client.1.smithi203.stdout:2/59: write d2/d6/fb [1793832,121699] 0 2024-05-25T06:11:19.731 INFO:tasks.workunit.client.1.smithi203.stdout:7/84: creat d1/f1d x:0 0 0 2024-05-25T06:11:19.733 INFO:tasks.workunit.client.1.smithi203.stdout:2/60: symlink d2/d4/l16 0 2024-05-25T06:11:19.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/154: dread d2/d5/d18/f1b [0,4194304] 0 2024-05-25T06:11:19.736 INFO:tasks.workunit.client.1.smithi203.stdout:8/155: chown d2/d5/d18/d31/f34 183551659 1 2024-05-25T06:11:19.736 INFO:tasks.workunit.client.1.smithi203.stdout:7/85: truncate d1/f1c 3130528 0 2024-05-25T06:11:19.737 INFO:tasks.workunit.client.1.smithi203.stdout:2/61: mknod d2/d6/d14/c17 0 2024-05-25T06:11:19.737 INFO:tasks.workunit.client.1.smithi203.stdout:8/156: chown d2/d5/d10/d11/de/f13 517693 1 2024-05-25T06:11:19.738 INFO:tasks.workunit.client.1.smithi203.stdout:7/86: mknod d1/d9/c1e 0 2024-05-25T06:11:19.738 INFO:tasks.workunit.client.1.smithi203.stdout:8/157: stat d2/d5/l6 0 2024-05-25T06:11:19.739 INFO:tasks.workunit.client.1.smithi203.stdout:2/62: symlink d2/l18 0 2024-05-25T06:11:19.741 INFO:tasks.workunit.client.1.smithi203.stdout:8/158: rmdir d2/d5/d10 39 2024-05-25T06:11:19.742 INFO:tasks.workunit.client.1.smithi203.stdout:8/159: fsync d2/d5/d18/d31/f39 0 2024-05-25T06:11:19.743 INFO:tasks.workunit.client.1.smithi203.stdout:7/87: dread d1/d9/fb [0,4194304] 0 2024-05-25T06:11:19.744 INFO:tasks.workunit.client.1.smithi203.stdout:8/160: rmdir d2/d5/d10 39 2024-05-25T06:11:19.746 INFO:tasks.workunit.client.1.smithi203.stdout:7/88: creat d1/d11/f1f x:0 0 0 2024-05-25T06:11:19.751 INFO:tasks.workunit.client.1.smithi203.stdout:7/89: dread d1/d9/fe [0,4194304] 0 2024-05-25T06:11:19.753 INFO:tasks.workunit.client.1.smithi203.stdout:7/90: mkdir d1/d11/d20 0 2024-05-25T06:11:19.753 INFO:tasks.workunit.client.1.smithi203.stdout:8/161: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:19.754 INFO:tasks.workunit.client.1.smithi203.stdout:2/63: dread d2/d6/fb [0,4194304] 0 2024-05-25T06:11:19.754 INFO:tasks.workunit.client.1.smithi203.stdout:8/162: write d2/d5/d28/f3a [398090,105509] 0 2024-05-25T06:11:19.755 INFO:tasks.workunit.client.1.smithi203.stdout:7/91: creat d1/d11/f21 x:0 0 0 2024-05-25T06:11:19.756 INFO:tasks.workunit.client.1.smithi203.stdout:2/64: truncate d2/d4/f11 627183 0 2024-05-25T06:11:19.756 INFO:tasks.workunit.client.1.smithi203.stdout:7/92: chown d1/d9/l16 0 1 2024-05-25T06:11:19.757 INFO:tasks.workunit.client.1.smithi203.stdout:8/163: read - d2/d5/d10/d11/d1f/f35 zero size 2024-05-25T06:11:19.759 INFO:tasks.workunit.client.1.smithi203.stdout:7/93: unlink d1/d11/f1f 0 2024-05-25T06:11:19.761 INFO:tasks.workunit.client.1.smithi203.stdout:8/164: creat d2/d5/d18/d31/f3d x:0 0 0 2024-05-25T06:11:19.764 INFO:tasks.workunit.client.1.smithi203.stdout:2/65: link d2/d6/da/lf d2/d6/da/l19 0 2024-05-25T06:11:19.765 INFO:tasks.workunit.client.1.smithi203.stdout:7/94: dread d1/f2 [0,4194304] 0 2024-05-25T06:11:19.766 INFO:tasks.workunit.client.1.smithi203.stdout:8/165: creat d2/d5/d2d/f3e x:0 0 0 2024-05-25T06:11:19.768 INFO:tasks.workunit.client.1.smithi203.stdout:2/66: mkdir d2/d1a 0 2024-05-25T06:11:19.772 INFO:tasks.workunit.client.1.smithi203.stdout:7/95: link d1/d9/c1e d1/d11/d20/c22 0 2024-05-25T06:11:19.773 INFO:tasks.workunit.client.1.smithi203.stdout:8/166: creat d2/d5/f3f x:0 0 0 2024-05-25T06:11:19.781 INFO:tasks.workunit.client.1.smithi203.stdout:7/96: rmdir d1/d9 39 2024-05-25T06:11:19.784 INFO:tasks.workunit.client.1.smithi203.stdout:7/97: creat d1/d11/d20/f23 x:0 0 0 2024-05-25T06:11:19.787 INFO:tasks.workunit.client.1.smithi203.stdout:7/98: dread - d1/d9/f18 zero size 2024-05-25T06:11:19.788 INFO:tasks.workunit.client.1.smithi203.stdout:2/67: dwrite d2/fe [0,4194304] 0 2024-05-25T06:11:19.792 INFO:tasks.workunit.client.1.smithi203.stdout:8/167: dwrite d2/d5/d10/d11/de/f1a [0,4194304] 0 2024-05-25T06:11:19.794 INFO:tasks.workunit.client.1.smithi203.stdout:2/68: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:19.795 INFO:tasks.workunit.client.1.smithi203.stdout:2/69: fdatasync d2/d4/f11 0 2024-05-25T06:11:19.796 INFO:tasks.workunit.client.1.smithi203.stdout:7/99: creat d1/f24 x:0 0 0 2024-05-25T06:11:19.796 INFO:tasks.workunit.client.1.smithi203.stdout:2/70: stat d2 0 2024-05-25T06:11:19.798 INFO:tasks.workunit.client.1.smithi203.stdout:7/100: rmdir d1/d9 39 2024-05-25T06:11:19.798 INFO:tasks.workunit.client.1.smithi203.stdout:7/101: write d1/f24 [1044112,63191] 0 2024-05-25T06:11:19.800 INFO:tasks.workunit.client.1.smithi203.stdout:2/71: write d2/d6/fb [2381356,35333] 0 2024-05-25T06:11:19.805 INFO:tasks.workunit.client.1.smithi203.stdout:2/72: link d2/d6/fb d2/d6/d14/f1b 0 2024-05-25T06:11:19.807 INFO:tasks.workunit.client.1.smithi203.stdout:7/102: link d1/d9/fc d1/d11/d20/f25 0 2024-05-25T06:11:19.809 INFO:tasks.workunit.client.1.smithi203.stdout:7/103: creat d1/d11/d20/f26 x:0 0 0 2024-05-25T06:11:19.812 INFO:tasks.workunit.client.1.smithi203.stdout:7/104: creat d1/d11/d20/f27 x:0 0 0 2024-05-25T06:11:19.815 INFO:tasks.workunit.client.1.smithi203.stdout:7/105: fsync d1/d9/fd 0 2024-05-25T06:11:19.815 INFO:tasks.workunit.client.1.smithi203.stdout:2/73: dwrite d2/d6/fb [0,4194304] 0 2024-05-25T06:11:19.818 INFO:tasks.workunit.client.1.smithi203.stdout:7/106: creat d1/d11/d20/f28 x:0 0 0 2024-05-25T06:11:19.819 INFO:tasks.workunit.client.1.smithi203.stdout:7/107: chown d1/c6 10070 1 2024-05-25T06:11:19.829 INFO:tasks.workunit.client.1.smithi203.stdout:2/74: dwrite d2/f8 [0,4194304] 0 2024-05-25T06:11:19.830 INFO:tasks.workunit.client.1.smithi203.stdout:2/75: write d2/fe [2609163,47572] 0 2024-05-25T06:11:19.834 INFO:tasks.workunit.client.1.smithi203.stdout:2/76: creat d2/d6/d14/f1c x:0 0 0 2024-05-25T06:11:19.843 INFO:tasks.workunit.client.1.smithi203.stdout:2/77: fsync d2/f8 0 2024-05-25T06:11:19.843 INFO:tasks.workunit.client.1.smithi203.stdout:2/78: chown d2/d6/da/c13 1522 1 2024-05-25T06:11:19.845 INFO:tasks.workunit.client.1.smithi203.stdout:2/79: dread d2/d4/f11 [0,4194304] 0 2024-05-25T06:11:19.846 INFO:tasks.workunit.client.1.smithi203.stdout:2/80: readlink d2/d4/l16 0 2024-05-25T06:11:19.847 INFO:tasks.workunit.client.1.smithi203.stdout:2/81: write d2/d6/d14/f1b [2574523,92923] 0 2024-05-25T06:11:19.851 INFO:tasks.workunit.client.1.smithi203.stdout:2/82: dread d2/d6/fb [0,4194304] 0 2024-05-25T06:11:19.853 INFO:tasks.workunit.client.1.smithi203.stdout:2/83: readlink d2/d6/da/l19 0 2024-05-25T06:11:19.853 INFO:tasks.workunit.client.1.smithi203.stdout:2/84: stat d2/d6/da/l19 0 2024-05-25T06:11:19.858 INFO:tasks.workunit.client.1.smithi203.stdout:2/85: truncate d2/d6/fb 3799581 0 2024-05-25T06:11:19.860 INFO:tasks.workunit.client.1.smithi203.stdout:2/86: creat d2/d6/da/f1d x:0 0 0 2024-05-25T06:11:19.867 INFO:tasks.workunit.client.1.smithi203.stdout:2/87: dread d2/d6/fb [0,4194304] 0 2024-05-25T06:11:19.872 INFO:tasks.workunit.client.1.smithi203.stdout:2/88: truncate d2/fe 3350610 0 2024-05-25T06:11:19.872 INFO:tasks.workunit.client.1.smithi203.stdout:2/89: stat d2/d6/da 0 2024-05-25T06:11:19.873 INFO:tasks.workunit.client.1.smithi203.stdout:2/90: rename d2 to d2/d1a/d1e 22 2024-05-25T06:11:19.873 INFO:tasks.workunit.client.1.smithi203.stdout:2/91: write d2/d4/f11 [301100,42360] 0 2024-05-25T06:11:20.028 INFO:tasks.workunit.client.1.smithi203.stdout:0/109: sync 2024-05-25T06:11:20.028 INFO:tasks.workunit.client.1.smithi203.stdout:4/83: sync 2024-05-25T06:11:20.029 INFO:tasks.workunit.client.1.smithi203.stdout:9/87: sync 2024-05-25T06:11:20.029 INFO:tasks.workunit.client.1.smithi203.stdout:4/84: chown d0/d8/d10/l13 845 1 2024-05-25T06:11:20.030 INFO:tasks.workunit.client.1.smithi203.stdout:0/110: mkdir dd/d2b 0 2024-05-25T06:11:20.031 INFO:tasks.workunit.client.1.smithi203.stdout:9/88: rmdir d0/df 39 2024-05-25T06:11:20.037 INFO:tasks.workunit.client.1.smithi203.stdout:4/85: rename d0/d8/ca to d0/c18 0 2024-05-25T06:11:20.039 INFO:tasks.workunit.client.1.smithi203.stdout:0/111: rmdir dd/d2b 0 2024-05-25T06:11:20.040 INFO:tasks.workunit.client.1.smithi203.stdout:9/89: creat d0/f1e x:0 0 0 2024-05-25T06:11:20.042 INFO:tasks.workunit.client.1.smithi203.stdout:4/86: mkdir d0/d8/d10/d19 0 2024-05-25T06:11:20.048 INFO:tasks.workunit.client.1.smithi203.stdout:0/112: rmdir dd 39 2024-05-25T06:11:20.048 INFO:tasks.workunit.client.1.smithi203.stdout:4/87: dwrite d0/f5 [0,4194304] 0 2024-05-25T06:11:20.049 INFO:tasks.workunit.client.1.smithi203.stdout:4/88: chown d0/d8/d10/f15 157866255 1 2024-05-25T06:11:20.050 INFO:tasks.workunit.client.1.smithi203.stdout:9/90: truncate d0/f19 1902736 0 2024-05-25T06:11:20.050 INFO:tasks.workunit.client.1.smithi203.stdout:4/89: fdatasync d0/d9/f17 0 2024-05-25T06:11:20.051 INFO:tasks.workunit.client.1.smithi203.stdout:4/90: stat d0/d8/d10/f15 0 2024-05-25T06:11:20.051 INFO:tasks.workunit.client.1.smithi203.stdout:4/91: chown d0/d8/ff 8156766 1 2024-05-25T06:11:20.053 INFO:tasks.workunit.client.1.smithi203.stdout:0/113: dread f6 [0,4194304] 0 2024-05-25T06:11:20.053 INFO:tasks.workunit.client.1.smithi203.stdout:9/91: mknod d0/df/c1f 0 2024-05-25T06:11:20.054 INFO:tasks.workunit.client.1.smithi203.stdout:4/92: symlink d0/d8/d10/l1a 0 2024-05-25T06:11:20.057 INFO:tasks.workunit.client.1.smithi203.stdout:9/92: mkdir d0/d17/d20 0 2024-05-25T06:11:20.057 INFO:tasks.workunit.client.1.smithi203.stdout:4/93: mknod d0/d8/c1b 0 2024-05-25T06:11:20.058 INFO:tasks.workunit.client.1.smithi203.stdout:0/114: symlink dd/d10/l2c 0 2024-05-25T06:11:20.062 INFO:tasks.workunit.client.1.smithi203.stdout:9/93: link d0/f4 d0/f21 0 2024-05-25T06:11:20.062 INFO:tasks.workunit.client.1.smithi203.stdout:4/94: creat d0/f1c x:0 0 0 2024-05-25T06:11:20.065 INFO:tasks.workunit.client.1.smithi203.stdout:9/94: truncate d0/df/f14 974356 0 2024-05-25T06:11:20.066 INFO:tasks.workunit.client.1.smithi203.stdout:4/95: truncate d0/d9/f17 151036 0 2024-05-25T06:11:20.067 INFO:tasks.workunit.client.1.smithi203.stdout:0/115: dwrite dd/d10/f1a [0,4194304] 0 2024-05-25T06:11:20.069 INFO:tasks.workunit.client.1.smithi203.stdout:4/96: rmdir d0/d8 39 2024-05-25T06:11:20.075 INFO:tasks.workunit.client.1.smithi203.stdout:0/116: link dd/f17 dd/d10/f2d 0 2024-05-25T06:11:20.078 INFO:tasks.workunit.client.1.smithi203.stdout:0/117: symlink dd/d10/l2e 0 2024-05-25T06:11:20.079 INFO:tasks.workunit.client.1.smithi203.stdout:4/97: symlink d0/d8/d10/d19/l1d 0 2024-05-25T06:11:20.079 INFO:tasks.workunit.client.1.smithi203.stdout:0/118: fdatasync dd/d10/f26 0 2024-05-25T06:11:20.080 INFO:tasks.workunit.client.1.smithi203.stdout:4/98: chown d0/d8/c1b 9341 1 2024-05-25T06:11:20.081 INFO:tasks.workunit.client.1.smithi203.stdout:4/99: write d0/d8/d10/f15 [787869,60463] 0 2024-05-25T06:11:20.084 INFO:tasks.workunit.client.1.smithi203.stdout:4/100: creat d0/d8/d10/d19/f1e x:0 0 0 2024-05-25T06:11:20.091 INFO:tasks.workunit.client.1.smithi203.stdout:4/101: getdents d0/d8/d10/d19 0 2024-05-25T06:11:20.093 INFO:tasks.workunit.client.1.smithi203.stdout:4/102: mkdir d0/d8/d1f 0 2024-05-25T06:11:20.095 INFO:tasks.workunit.client.1.smithi203.stdout:4/103: creat d0/d8/d10/d19/f20 x:0 0 0 2024-05-25T06:11:20.095 INFO:tasks.workunit.client.1.smithi203.stdout:4/104: read - d0/f1c zero size 2024-05-25T06:11:20.097 INFO:tasks.workunit.client.1.smithi203.stdout:4/105: write d0/d8/d10/f15 [995960,85914] 0 2024-05-25T06:11:20.100 INFO:tasks.workunit.client.1.smithi203.stdout:4/106: rename d0/c7 to d0/d8/d1f/c21 0 2024-05-25T06:11:20.101 INFO:tasks.workunit.client.1.smithi203.stdout:4/107: fdatasync d0/d8/d10/d19/f1e 0 2024-05-25T06:11:20.104 INFO:tasks.workunit.client.1.smithi203.stdout:4/108: dread d0/f3 [0,4194304] 0 2024-05-25T06:11:20.109 INFO:tasks.workunit.client.1.smithi203.stdout:4/109: getdents d0/d8/d1f 0 2024-05-25T06:11:20.111 INFO:tasks.workunit.client.1.smithi203.stdout:4/110: rmdir d0/d9 39 2024-05-25T06:11:20.114 INFO:tasks.workunit.client.1.smithi203.stdout:4/111: creat d0/d9/d11/f22 x:0 0 0 2024-05-25T06:11:20.115 INFO:tasks.workunit.client.1.smithi203.stdout:4/112: stat d0/d8/d1f 0 2024-05-25T06:11:20.116 INFO:tasks.workunit.client.1.smithi203.stdout:4/113: write d0/d8/d10/f15 [549915,83796] 0 2024-05-25T06:11:20.122 INFO:tasks.workunit.client.1.smithi203.stdout:4/114: dwrite d0/d8/d10/d19/f1e [0,4194304] 0 2024-05-25T06:11:20.126 INFO:tasks.workunit.client.1.smithi203.stdout:4/115: link d0/d9/d11/f22 d0/d8/f23 0 2024-05-25T06:11:20.130 INFO:tasks.workunit.client.1.smithi203.stdout:4/116: truncate d0/d9/f12 202177 0 2024-05-25T06:11:20.132 INFO:tasks.workunit.client.1.smithi203.stdout:4/117: unlink d0/d8/ff 0 2024-05-25T06:11:20.134 INFO:tasks.workunit.client.1.smithi203.stdout:4/118: mkdir d0/d9/d24 0 2024-05-25T06:11:20.135 INFO:tasks.workunit.client.1.smithi203.stdout:4/119: write d0/f1c [199334,100163] 0 2024-05-25T06:11:20.141 INFO:tasks.workunit.client.1.smithi203.stdout:4/120: dread d0/d9/f12 [0,4194304] 0 2024-05-25T06:11:20.142 INFO:tasks.workunit.client.1.smithi203.stdout:4/121: read - d0/d8/d10/d19/f20 zero size 2024-05-25T06:11:20.143 INFO:tasks.workunit.client.1.smithi203.stdout:4/122: chown d0/d9/d11/f22 64343151 1 2024-05-25T06:11:20.144 INFO:tasks.workunit.client.1.smithi203.stdout:4/123: symlink d0/d9/d24/l25 0 2024-05-25T06:11:20.151 INFO:tasks.workunit.client.1.smithi203.stdout:4/124: dwrite d0/d8/d10/f15 [0,4194304] 0 2024-05-25T06:11:20.202 INFO:tasks.workunit.client.1.smithi203.stdout:6/144: sync 2024-05-25T06:11:20.204 INFO:tasks.workunit.client.1.smithi203.stdout:6/145: readlink d8/d11/l21 0 2024-05-25T06:11:20.205 INFO:tasks.workunit.client.1.smithi203.stdout:6/146: chown d8/d11/d1b/c25 597 1 2024-05-25T06:11:20.208 INFO:tasks.workunit.client.1.smithi203.stdout:6/147: rename d8/d11/d15/f27 to d8/d29/d30/d38/f39 0 2024-05-25T06:11:20.215 INFO:tasks.workunit.client.1.smithi203.stdout:6/148: dwrite d8/fd [0,4194304] 0 2024-05-25T06:11:20.217 INFO:tasks.workunit.client.1.smithi203.stdout:6/149: symlink d8/d2a/l3a 0 2024-05-25T06:11:20.220 INFO:tasks.workunit.client.1.smithi203.stdout:6/150: rmdir d8/d11/d15 39 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 290 op/s 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi160 to 1557M 2024-05-25T06:11:20.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:20.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:19 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:20.221 INFO:tasks.workunit.client.1.smithi203.stdout:6/151: chown d8/d11/d1b/l36 1 1 2024-05-25T06:11:20.231 INFO:tasks.workunit.client.1.smithi203.stdout:6/152: dwrite f0 [0,4194304] 0 2024-05-25T06:11:20.239 INFO:tasks.workunit.client.1.smithi203.stdout:6/153: dwrite d8/f9 [0,4194304] 0 2024-05-25T06:11:20.240 INFO:tasks.workunit.client.1.smithi203.stdout:6/154: stat d8/d11/d1b 0 2024-05-25T06:11:20.243 INFO:tasks.workunit.client.1.smithi203.stdout:6/155: mknod d8/d11/d1b/d24/c3b 0 2024-05-25T06:11:20.344 INFO:tasks.workunit.client.1.smithi203.stdout:3/60: sync 2024-05-25T06:11:20.345 INFO:tasks.workunit.client.1.smithi203.stdout:3/61: symlink d10/l15 0 2024-05-25T06:11:20.346 INFO:tasks.workunit.client.1.smithi203.stdout:3/62: write f5 [2139895,15219] 0 2024-05-25T06:11:20.348 INFO:tasks.workunit.client.1.smithi203.stdout:3/63: mkdir d10/d16 0 2024-05-25T06:11:20.350 INFO:tasks.workunit.client.1.smithi203.stdout:3/64: creat d10/d16/f17 x:0 0 0 2024-05-25T06:11:20.355 INFO:tasks.workunit.client.1.smithi203.stdout:3/65: rmdir d10 39 2024-05-25T06:11:20.355 INFO:tasks.workunit.client.1.smithi203.stdout:3/66: readlink l9 0 2024-05-25T06:11:20.357 INFO:tasks.workunit.client.1.smithi203.stdout:3/67: symlink d10/l18 0 2024-05-25T06:11:20.358 INFO:tasks.workunit.client.1.smithi203.stdout:3/68: creat d10/d16/f19 x:0 0 0 2024-05-25T06:11:20.360 INFO:tasks.workunit.client.1.smithi203.stdout:3/69: creat d10/f1a x:0 0 0 2024-05-25T06:11:20.364 INFO:tasks.workunit.client.1.smithi203.stdout:3/70: rmdir d10/d16 39 2024-05-25T06:11:20.366 INFO:tasks.workunit.client.1.smithi203.stdout:3/71: rmdir d10/d16 39 2024-05-25T06:11:20.368 INFO:tasks.workunit.client.1.smithi203.stdout:3/72: mknod d10/c1b 0 2024-05-25T06:11:20.369 INFO:tasks.workunit.client.1.smithi203.stdout:3/73: truncate f0 4761855 0 2024-05-25T06:11:20.371 INFO:tasks.workunit.client.1.smithi203.stdout:3/74: symlink d10/d16/l1c 0 2024-05-25T06:11:20.371 INFO:tasks.workunit.client.1.smithi203.stdout:3/75: write f8 [1102027,27852] 0 2024-05-25T06:11:20.376 INFO:tasks.workunit.client.1.smithi203.stdout:3/76: rename d10/c11 to d10/c1d 0 2024-05-25T06:11:20.376 INFO:tasks.workunit.client.1.smithi203.stdout:3/77: chown la 49 1 2024-05-25T06:11:20.382 INFO:tasks.workunit.client.1.smithi203.stdout:3/78: truncate f0 3964133 0 2024-05-25T06:11:20.383 INFO:tasks.workunit.client.1.smithi203.stdout:3/79: read - d10/d16/f19 zero size 2024-05-25T06:11:20.386 INFO:tasks.workunit.client.1.smithi203.stdout:3/80: dread f1 [0,4194304] 0 2024-05-25T06:11:20.388 INFO:tasks.workunit.client.1.smithi203.stdout:3/81: mknod d10/d16/c1e 0 2024-05-25T06:11:20.391 INFO:tasks.workunit.client.1.smithi203.stdout:3/82: rename lc to d10/l1f 0 2024-05-25T06:11:20.394 INFO:tasks.workunit.client.1.smithi203.stdout:3/83: creat d10/f20 x:0 0 0 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 290 op/s 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi160 to 1557M 2024-05-25T06:11:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:20.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:19 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:20.400 INFO:tasks.workunit.client.1.smithi203.stdout:3/84: dwrite f1 [0,4194304] 0 2024-05-25T06:11:20.400 INFO:tasks.workunit.client.1.smithi203.stdout:3/85: chown l9 1133 1 2024-05-25T06:11:20.403 INFO:tasks.workunit.client.1.smithi203.stdout:3/86: creat d10/f21 x:0 0 0 2024-05-25T06:11:20.403 INFO:tasks.workunit.client.1.smithi203.stdout:3/87: readlink d10/l14 0 2024-05-25T06:11:20.410 INFO:tasks.workunit.client.1.smithi203.stdout:3/88: dwrite f5 [0,4194304] 0 2024-05-25T06:11:20.414 INFO:tasks.workunit.client.1.smithi203.stdout:3/89: mkdir d10/d22 0 2024-05-25T06:11:20.417 INFO:tasks.workunit.client.1.smithi203.stdout:3/90: dread f8 [4194304,4194304] 0 2024-05-25T06:11:20.419 INFO:tasks.workunit.client.1.smithi203.stdout:3/91: dread - d10/d16/f17 zero size 2024-05-25T06:11:20.419 INFO:tasks.workunit.client.1.smithi203.stdout:3/92: fdatasync d10/f1a 0 2024-05-25T06:11:20.419 INFO:tasks.workunit.client.1.smithi203.stdout:3/93: read - d10/f13 zero size 2024-05-25T06:11:20.423 INFO:tasks.workunit.client.1.smithi203.stdout:3/94: dread f5 [4194304,4194304] 0 2024-05-25T06:11:20.424 INFO:tasks.workunit.client.1.smithi203.stdout:3/95: truncate d10/f21 831141 0 2024-05-25T06:11:20.424 INFO:tasks.workunit.client.1.smithi203.stdout:3/96: chown d10/f13 1119266 1 2024-05-25T06:11:20.425 INFO:tasks.workunit.client.1.smithi203.stdout:3/97: truncate d10/f20 639243 0 2024-05-25T06:11:20.427 INFO:tasks.workunit.client.1.smithi203.stdout:3/98: mkdir d10/d16/d23 0 2024-05-25T06:11:20.429 INFO:tasks.workunit.client.1.smithi203.stdout:3/99: write f5 [998704,4366] 0 2024-05-25T06:11:20.431 INFO:tasks.workunit.client.1.smithi203.stdout:3/100: symlink d10/d16/d23/l24 0 2024-05-25T06:11:20.479 INFO:tasks.workunit.client.1.smithi203.stdout:5/204: sync 2024-05-25T06:11:20.479 INFO:tasks.workunit.client.1.smithi203.stdout:1/76: sync 2024-05-25T06:11:20.479 INFO:tasks.workunit.client.1.smithi203.stdout:1/77: fdatasync fc 0 2024-05-25T06:11:20.481 INFO:tasks.workunit.client.1.smithi203.stdout:3/101: creat d10/d16/d23/f25 x:0 0 0 2024-05-25T06:11:20.481 INFO:tasks.workunit.client.1.smithi203.stdout:5/205: truncate f8 1497426 0 2024-05-25T06:11:20.482 INFO:tasks.workunit.client.1.smithi203.stdout:3/102: truncate d10/f21 1131191 0 2024-05-25T06:11:20.484 INFO:tasks.workunit.client.1.smithi203.stdout:1/78: rename fb to d14/f18 0 2024-05-25T06:11:20.485 INFO:tasks.workunit.client.1.smithi203.stdout:5/206: rename dc/d11/d18 to dc/d11/d18/d49 22 2024-05-25T06:11:20.485 INFO:tasks.workunit.client.1.smithi203.stdout:3/103: rmdir d10/d16 39 2024-05-25T06:11:20.490 INFO:tasks.workunit.client.1.smithi203.stdout:5/207: rename dc/d11/d17/d1a/d2d/l48 to dc/d11/d17/d44/l4a 0 2024-05-25T06:11:20.491 INFO:tasks.workunit.client.1.smithi203.stdout:1/79: link c11 d14/c19 0 2024-05-25T06:11:20.492 INFO:tasks.workunit.client.1.smithi203.stdout:1/80: creat d14/f1a x:0 0 0 2024-05-25T06:11:20.493 INFO:tasks.workunit.client.1.smithi203.stdout:1/81: readlink - no filename 2024-05-25T06:11:20.493 INFO:tasks.workunit.client.1.smithi203.stdout:1/82: chown c7 370 1 2024-05-25T06:11:20.493 INFO:tasks.workunit.client.1.smithi203.stdout:5/208: symlink dc/d11/d17/d1a/d22/l4b 0 2024-05-25T06:11:20.493 INFO:tasks.workunit.client.1.smithi203.stdout:3/104: link la d10/l26 0 2024-05-25T06:11:20.494 INFO:tasks.workunit.client.1.smithi203.stdout:5/209: chown dc/d11/d17/d1a/l2a 4041 1 2024-05-25T06:11:20.495 INFO:tasks.workunit.client.1.smithi203.stdout:1/83: mkdir d14/d1b 0 2024-05-25T06:11:20.496 INFO:tasks.workunit.client.1.smithi203.stdout:1/84: truncate fa 5076517 0 2024-05-25T06:11:20.498 INFO:tasks.workunit.client.1.smithi203.stdout:1/85: mknod d14/c1c 0 2024-05-25T06:11:20.499 INFO:tasks.workunit.client.1.smithi203.stdout:3/105: rename d10/d16/l1c to d10/l27 0 2024-05-25T06:11:20.503 INFO:tasks.workunit.client.1.smithi203.stdout:3/106: creat d10/d16/f28 x:0 0 0 2024-05-25T06:11:20.505 INFO:tasks.workunit.client.1.smithi203.stdout:3/107: creat d10/d16/d23/f29 x:0 0 0 2024-05-25T06:11:20.507 INFO:tasks.workunit.client.1.smithi203.stdout:3/108: mkdir d10/d22/d2a 0 2024-05-25T06:11:20.517 INFO:tasks.workunit.client.1.smithi203.stdout:5/210: truncate dc/d11/f29 4015349 0 2024-05-25T06:11:20.519 INFO:tasks.workunit.client.1.smithi203.stdout:5/211: dread dc/f21 [0,4194304] 0 2024-05-25T06:11:20.521 INFO:tasks.workunit.client.1.smithi203.stdout:5/212: symlink dc/d11/d17/d1a/l4c 0 2024-05-25T06:11:20.528 INFO:tasks.workunit.client.1.smithi203.stdout:3/109: dwrite f0 [0,4194304] 0 2024-05-25T06:11:20.528 INFO:tasks.workunit.client.1.smithi203.stdout:5/213: dwrite dc/d11/f14 [0,4194304] 0 2024-05-25T06:11:20.532 INFO:tasks.workunit.client.1.smithi203.stdout:3/110: link d10/f13 d10/d16/d23/f2b 0 2024-05-25T06:11:20.533 INFO:tasks.workunit.client.1.smithi203.stdout:3/111: write d10/d16/f17 [505286,40673] 0 2024-05-25T06:11:20.534 INFO:tasks.workunit.client.1.smithi203.stdout:3/112: dread - d10/f13 zero size 2024-05-25T06:11:20.540 INFO:tasks.workunit.client.1.smithi203.stdout:3/113: unlink d10/d16/f28 0 2024-05-25T06:11:20.543 INFO:tasks.workunit.client.1.smithi203.stdout:3/114: creat d10/d16/f2c x:0 0 0 2024-05-25T06:11:20.548 INFO:tasks.workunit.client.1.smithi203.stdout:5/214: dwrite dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:20.552 INFO:tasks.workunit.client.1.smithi203.stdout:5/215: getdents dc/d11/d17/d45 0 2024-05-25T06:11:20.554 INFO:tasks.workunit.client.1.smithi203.stdout:5/216: fdatasync dc/d11/f1c 0 2024-05-25T06:11:20.556 INFO:tasks.workunit.client.1.smithi203.stdout:3/115: dwrite d10/d16/d23/f2b [0,4194304] 0 2024-05-25T06:11:20.558 INFO:tasks.workunit.client.1.smithi203.stdout:5/217: read dc/d11/f12 [1713524,15450] 0 2024-05-25T06:11:20.558 INFO:tasks.workunit.client.1.smithi203.stdout:3/116: write d10/f1a [328361,38548] 0 2024-05-25T06:11:20.560 INFO:tasks.workunit.client.1.smithi203.stdout:5/218: write dc/d11/f14 [4870429,25493] 0 2024-05-25T06:11:20.566 INFO:tasks.workunit.client.1.smithi203.stdout:5/219: dread f8 [0,4194304] 0 2024-05-25T06:11:20.567 INFO:tasks.workunit.client.1.smithi203.stdout:5/220: chown dc/d11/d18/f37 839803 1 2024-05-25T06:11:20.568 INFO:tasks.workunit.client.1.smithi203.stdout:5/221: fsync dc/d11/f14 0 2024-05-25T06:11:20.570 INFO:tasks.workunit.client.1.smithi203.stdout:5/222: mkdir dc/d11/d17/d1a/d2d/d4d 0 2024-05-25T06:11:20.572 INFO:tasks.workunit.client.1.smithi203.stdout:5/223: dread dc/f21 [0,4194304] 0 2024-05-25T06:11:20.574 INFO:tasks.workunit.client.1.smithi203.stdout:5/224: write dc/f10 [938621,99630] 0 2024-05-25T06:11:20.574 INFO:tasks.workunit.client.1.smithi203.stdout:3/117: truncate d10/d16/f17 81394 0 2024-05-25T06:11:20.577 INFO:tasks.workunit.client.1.smithi203.stdout:5/225: creat dc/d3c/d3f/f4e x:0 0 0 2024-05-25T06:11:20.578 INFO:tasks.workunit.client.1.smithi203.stdout:5/226: fdatasync fb 0 2024-05-25T06:11:20.581 INFO:tasks.workunit.client.1.smithi203.stdout:5/227: symlink dc/d11/d17/d1a/d2d/d4d/l4f 0 2024-05-25T06:11:20.582 INFO:tasks.workunit.client.1.smithi203.stdout:3/118: read d10/d16/f17 [70020,56910] 0 2024-05-25T06:11:20.585 INFO:tasks.workunit.client.1.smithi203.stdout:5/228: truncate dc/d11/f14 334042 0 2024-05-25T06:11:20.586 INFO:tasks.workunit.client.1.smithi203.stdout:5/229: write dc/d11/d17/d1a/f32 [264791,125550] 0 2024-05-25T06:11:20.586 INFO:tasks.workunit.client.1.smithi203.stdout:3/119: rename d10/d16/d23/l24 to d10/d22/d2a/l2d 0 2024-05-25T06:11:20.589 INFO:tasks.workunit.client.1.smithi203.stdout:3/120: symlink d10/d22/l2e 0 2024-05-25T06:11:20.589 INFO:tasks.workunit.client.1.smithi203.stdout:5/230: creat dc/d11/f50 x:0 0 0 2024-05-25T06:11:20.592 INFO:tasks.workunit.client.1.smithi203.stdout:5/231: creat dc/d11/d17/d44/f51 x:0 0 0 2024-05-25T06:11:20.594 INFO:tasks.workunit.client.1.smithi203.stdout:3/121: rename d10/l26 to d10/d16/l2f 0 2024-05-25T06:11:20.595 INFO:tasks.workunit.client.1.smithi203.stdout:5/232: creat dc/d11/d17/d45/f52 x:0 0 0 2024-05-25T06:11:20.596 INFO:tasks.workunit.client.1.smithi203.stdout:3/122: truncate d10/d16/f17 441665 0 2024-05-25T06:11:20.596 INFO:tasks.workunit.client.1.smithi203.stdout:3/123: chown d10/d22/l2e 6982 1 2024-05-25T06:11:20.597 INFO:tasks.workunit.client.1.smithi203.stdout:3/124: write f5 [5487200,98042] 0 2024-05-25T06:11:20.597 INFO:tasks.workunit.client.1.smithi203.stdout:5/233: mkdir dc/d11/d17/d1a/d2d/d53 0 2024-05-25T06:11:20.599 INFO:tasks.workunit.client.1.smithi203.stdout:5/234: creat dc/d11/d17/f54 x:0 0 0 2024-05-25T06:11:20.600 INFO:tasks.workunit.client.1.smithi203.stdout:3/125: rename d10/l14 to d10/d16/d23/l30 0 2024-05-25T06:11:20.601 INFO:tasks.workunit.client.1.smithi203.stdout:5/235: creat dc/d11/d17/d1a/d2d/d53/f55 x:0 0 0 2024-05-25T06:11:20.603 INFO:tasks.workunit.client.1.smithi203.stdout:3/126: dread d10/f21 [0,4194304] 0 2024-05-25T06:11:20.604 INFO:tasks.workunit.client.1.smithi203.stdout:5/236: truncate fb 3143151 0 2024-05-25T06:11:20.605 INFO:tasks.workunit.client.1.smithi203.stdout:5/237: chown dc/d11/d17/d45/f52 5 1 2024-05-25T06:11:20.606 INFO:tasks.workunit.client.1.smithi203.stdout:3/127: creat d10/f31 x:0 0 0 2024-05-25T06:11:20.610 INFO:tasks.workunit.client.1.smithi203.stdout:5/238: getdents dc/d11/d42 0 2024-05-25T06:11:20.610 INFO:tasks.workunit.client.1.smithi203.stdout:7/108: sync 2024-05-25T06:11:20.610 INFO:tasks.workunit.client.1.smithi203.stdout:8/168: sync 2024-05-25T06:11:20.610 INFO:tasks.workunit.client.1.smithi203.stdout:2/92: sync 2024-05-25T06:11:20.611 INFO:tasks.workunit.client.1.smithi203.stdout:3/128: dread f5 [0,4194304] 0 2024-05-25T06:11:20.612 INFO:tasks.workunit.client.1.smithi203.stdout:2/93: creat d2/d6/d14/f1f x:0 0 0 2024-05-25T06:11:20.614 INFO:tasks.workunit.client.1.smithi203.stdout:7/109: creat d1/d9/f29 x:0 0 0 2024-05-25T06:11:20.614 INFO:tasks.workunit.client.1.smithi203.stdout:2/94: chown d2/d6/fb 6 1 2024-05-25T06:11:20.616 INFO:tasks.workunit.client.1.smithi203.stdout:2/95: chown d2/d6/da 3229585 1 2024-05-25T06:11:20.617 INFO:tasks.workunit.client.1.smithi203.stdout:3/129: creat d10/f32 x:0 0 0 2024-05-25T06:11:20.617 INFO:tasks.workunit.client.1.smithi203.stdout:8/169: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:20.619 INFO:tasks.workunit.client.1.smithi203.stdout:5/239: dwrite dc/d11/d17/d1a/f27 [0,4194304] 0 2024-05-25T06:11:20.621 INFO:tasks.workunit.client.1.smithi203.stdout:3/130: creat d10/d22/f33 x:0 0 0 2024-05-25T06:11:20.623 INFO:tasks.workunit.client.1.smithi203.stdout:3/131: fdatasync d10/f20 0 2024-05-25T06:11:20.625 INFO:tasks.workunit.client.1.smithi203.stdout:7/110: link d1/c12 d1/c2a 0 2024-05-25T06:11:20.626 INFO:tasks.workunit.client.1.smithi203.stdout:5/240: dread - dc/d11/d17/d1a/d2d/d53/f55 zero size 2024-05-25T06:11:20.629 INFO:tasks.workunit.client.1.smithi203.stdout:3/132: unlink d10/d16/d23/f25 0 2024-05-25T06:11:20.630 INFO:tasks.workunit.client.1.smithi203.stdout:2/96: dwrite d2/d6/d14/f1c [0,4194304] 0 2024-05-25T06:11:20.630 INFO:tasks.workunit.client.1.smithi203.stdout:5/241: chown dc/d11/f14 6196838 1 2024-05-25T06:11:20.631 INFO:tasks.workunit.client.1.smithi203.stdout:5/242: dread - dc/d11/f3e zero size 2024-05-25T06:11:20.634 INFO:tasks.workunit.client.1.smithi203.stdout:2/97: mknod d2/d1a/c20 0 2024-05-25T06:11:20.635 INFO:tasks.workunit.client.1.smithi203.stdout:5/243: read dc/d11/f12 [1554196,8615] 0 2024-05-25T06:11:20.638 INFO:tasks.workunit.client.1.smithi203.stdout:2/98: creat d2/d4/d10/f21 x:0 0 0 2024-05-25T06:11:20.638 INFO:tasks.workunit.client.1.smithi203.stdout:5/244: creat dc/d11/d17/f56 x:0 0 0 2024-05-25T06:11:20.639 INFO:tasks.workunit.client.1.smithi203.stdout:2/99: truncate d2/d4/f11 1443561 0 2024-05-25T06:11:20.641 INFO:tasks.workunit.client.1.smithi203.stdout:5/245: creat dc/d3c/d3f/f57 x:0 0 0 2024-05-25T06:11:20.641 INFO:tasks.workunit.client.1.smithi203.stdout:2/100: creat d2/d6/d14/f22 x:0 0 0 2024-05-25T06:11:20.642 INFO:tasks.workunit.client.1.smithi203.stdout:8/170: truncate d2/f14 2544395 0 2024-05-25T06:11:20.642 INFO:tasks.workunit.client.1.smithi203.stdout:2/101: chown d2/d4/l16 1081 1 2024-05-25T06:11:20.645 INFO:tasks.workunit.client.1.smithi203.stdout:3/133: rmdir d10 39 2024-05-25T06:11:20.649 INFO:tasks.workunit.client.1.smithi203.stdout:3/134: write d10/d16/d23/f29 [997185,52881] 0 2024-05-25T06:11:20.650 INFO:tasks.workunit.client.1.smithi203.stdout:2/102: truncate d2/f8 1206505 0 2024-05-25T06:11:20.653 INFO:tasks.workunit.client.1.smithi203.stdout:8/171: rename d2/d5/l2f to d2/d5/l40 0 2024-05-25T06:11:20.653 INFO:tasks.workunit.client.1.smithi203.stdout:7/111: dwrite d1/f3 [0,4194304] 0 2024-05-25T06:11:20.654 INFO:tasks.workunit.client.1.smithi203.stdout:0/119: sync 2024-05-25T06:11:20.654 INFO:tasks.workunit.client.1.smithi203.stdout:6/156: sync 2024-05-25T06:11:20.654 INFO:tasks.workunit.client.1.smithi203.stdout:4/125: sync 2024-05-25T06:11:20.655 INFO:tasks.workunit.client.1.smithi203.stdout:9/95: sync 2024-05-25T06:11:20.655 INFO:tasks.workunit.client.1.smithi203.stdout:1/86: sync 2024-05-25T06:11:20.655 INFO:tasks.workunit.client.1.smithi203.stdout:3/135: creat d10/d22/d2a/f34 x:0 0 0 2024-05-25T06:11:20.657 INFO:tasks.workunit.client.1.smithi203.stdout:7/112: readlink d1/d11/l13 0 2024-05-25T06:11:20.658 INFO:tasks.workunit.client.1.smithi203.stdout:2/103: unlink d2/d6/l9 0 2024-05-25T06:11:20.660 INFO:tasks.workunit.client.1.smithi203.stdout:3/136: dread - d10/f31 zero size 2024-05-25T06:11:20.666 INFO:tasks.workunit.client.1.smithi203.stdout:2/104: mkdir d2/d1a/d23 0 2024-05-25T06:11:20.670 INFO:tasks.workunit.client.1.smithi203.stdout:0/120: rename dd/d10/f1a to dd/d10/f2f 0 2024-05-25T06:11:20.671 INFO:tasks.workunit.client.1.smithi203.stdout:5/246: dwrite dc/d11/f14 [0,4194304] 0 2024-05-25T06:11:20.672 INFO:tasks.workunit.client.1.smithi203.stdout:6/157: truncate f2 803149 0 2024-05-25T06:11:20.674 INFO:tasks.workunit.client.1.smithi203.stdout:3/137: creat d10/d22/f35 x:0 0 0 2024-05-25T06:11:20.674 INFO:tasks.workunit.client.1.smithi203.stdout:0/121: read - dd/d10/f26 zero size 2024-05-25T06:11:20.675 INFO:tasks.workunit.client.1.smithi203.stdout:0/122: chown l7 352 1 2024-05-25T06:11:20.675 INFO:tasks.workunit.client.1.smithi203.stdout:2/105: creat d2/d6/da/f24 x:0 0 0 2024-05-25T06:11:20.676 INFO:tasks.workunit.client.1.smithi203.stdout:7/113: rename d1/d11/d20/f26 to d1/d11/f2b 0 2024-05-25T06:11:20.680 INFO:tasks.workunit.client.1.smithi203.stdout:9/96: getdents d0/df 0 2024-05-25T06:11:20.680 INFO:tasks.workunit.client.1.smithi203.stdout:5/247: rmdir dc/d11/d18 39 2024-05-25T06:11:20.680 INFO:tasks.workunit.client.1.smithi203.stdout:6/158: chown d8/d11/f23 2008 1 2024-05-25T06:11:20.682 INFO:tasks.workunit.client.1.smithi203.stdout:1/87: dwrite f10 [0,4194304] 0 2024-05-25T06:11:20.684 INFO:tasks.workunit.client.1.smithi203.stdout:3/138: write f5 [2192502,15637] 0 2024-05-25T06:11:20.686 INFO:tasks.workunit.client.1.smithi203.stdout:0/123: mknod dd/c30 0 2024-05-25T06:11:20.686 INFO:tasks.workunit.client.1.smithi203.stdout:3/139: fdatasync d10/d16/d23/f2b 0 2024-05-25T06:11:20.688 INFO:tasks.workunit.client.1.smithi203.stdout:5/248: symlink dc/d3c/d3f/l58 0 2024-05-25T06:11:20.689 INFO:tasks.workunit.client.1.smithi203.stdout:8/172: dread d2/d5/d10/d11/de/f1a [0,4194304] 0 2024-05-25T06:11:20.691 INFO:tasks.workunit.client.1.smithi203.stdout:1/88: chown d14/f1a 272151202 1 2024-05-25T06:11:20.693 INFO:tasks.workunit.client.1.smithi203.stdout:9/97: symlink d0/l22 0 2024-05-25T06:11:20.694 INFO:tasks.workunit.client.1.smithi203.stdout:1/89: fsync f12 0 2024-05-25T06:11:20.695 INFO:tasks.workunit.client.1.smithi203.stdout:2/106: link d2/d6/d14/f1c d2/d1a/d23/f25 0 2024-05-25T06:11:20.695 INFO:tasks.workunit.client.1.smithi203.stdout:4/126: truncate d0/d8/d10/d19/f1e 3941593 0 2024-05-25T06:11:20.699 INFO:tasks.workunit.client.1.smithi203.stdout:6/159: symlink d8/d11/d15/l3c 0 2024-05-25T06:11:20.701 INFO:tasks.workunit.client.1.smithi203.stdout:5/249: symlink dc/d11/d17/d1a/d2d/d53/l59 0 2024-05-25T06:11:20.701 INFO:tasks.workunit.client.1.smithi203.stdout:7/114: dwrite d1/d11/f17 [0,4194304] 0 2024-05-25T06:11:20.702 INFO:tasks.workunit.client.1.smithi203.stdout:3/140: symlink d10/l36 0 2024-05-25T06:11:20.703 INFO:tasks.workunit.client.1.smithi203.stdout:8/173: rename d2/d5/d18/l22 to d2/l41 0 2024-05-25T06:11:20.706 INFO:tasks.workunit.client.1.smithi203.stdout:5/250: stat dc/d11/c1e 0 2024-05-25T06:11:20.706 INFO:tasks.workunit.client.1.smithi203.stdout:1/90: mkdir d14/d1b/d1d 0 2024-05-25T06:11:20.707 INFO:tasks.workunit.client.1.smithi203.stdout:7/115: rmdir d1/d9 39 2024-05-25T06:11:20.708 INFO:tasks.workunit.client.1.smithi203.stdout:3/141: write d10/f32 [611469,42975] 0 2024-05-25T06:11:20.708 INFO:tasks.workunit.client.1.smithi203.stdout:0/124: dwrite dd/f23 [0,4194304] 0 2024-05-25T06:11:20.716 INFO:tasks.workunit.client.1.smithi203.stdout:1/91: write d14/f17 [118495,2901] 0 2024-05-25T06:11:20.717 INFO:tasks.workunit.client.1.smithi203.stdout:7/116: write d1/d11/f2b [592034,3030] 0 2024-05-25T06:11:20.718 INFO:tasks.workunit.client.1.smithi203.stdout:4/127: rename d0/d8/d10/d19 to d0/d8/d1f/d26 0 2024-05-25T06:11:20.718 INFO:tasks.workunit.client.1.smithi203.stdout:7/117: stat d1/f1c 0 2024-05-25T06:11:20.719 INFO:tasks.workunit.client.1.smithi203.stdout:3/142: symlink d10/d16/d23/l37 0 2024-05-25T06:11:20.722 INFO:tasks.workunit.client.1.smithi203.stdout:1/92: truncate d14/f17 446496 0 2024-05-25T06:11:20.722 INFO:tasks.workunit.client.1.smithi203.stdout:1/93: fsync f12 0 2024-05-25T06:11:20.723 INFO:tasks.workunit.client.1.smithi203.stdout:4/128: chown d0/f16 987408 1 2024-05-25T06:11:20.724 INFO:tasks.workunit.client.1.smithi203.stdout:5/251: mknod dc/d11/d17/d44/c5a 0 2024-05-25T06:11:20.725 INFO:tasks.workunit.client.1.smithi203.stdout:8/174: mkdir d2/d5/d42 0 2024-05-25T06:11:20.725 INFO:tasks.workunit.client.1.smithi203.stdout:9/98: dwrite d0/d17/f1b [0,4194304] 0 2024-05-25T06:11:20.730 INFO:tasks.workunit.client.1.smithi203.stdout:9/99: stat d0/fb 0 2024-05-25T06:11:20.734 INFO:tasks.workunit.client.1.smithi203.stdout:2/107: write d2/fe [168381,91650] 0 2024-05-25T06:11:20.734 INFO:tasks.workunit.client.1.smithi203.stdout:5/252: write f1 [1728465,54851] 0 2024-05-25T06:11:20.734 INFO:tasks.workunit.client.1.smithi203.stdout:4/129: truncate d0/d9/d11/f22 835722 0 2024-05-25T06:11:20.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/175: read d2/d5/d10/d11/de/f1a [3025695,65895] 0 2024-05-25T06:11:20.735 INFO:tasks.workunit.client.1.smithi203.stdout:3/143: getdents d10/d16/d23 0 2024-05-25T06:11:20.736 INFO:tasks.workunit.client.1.smithi203.stdout:9/100: mkdir d0/d17/d23 0 2024-05-25T06:11:20.736 INFO:tasks.workunit.client.1.smithi203.stdout:0/125: dread dd/f17 [0,4194304] 0 2024-05-25T06:11:20.736 INFO:tasks.workunit.client.1.smithi203.stdout:1/94: symlink d14/d1b/d1d/l1e 0 2024-05-25T06:11:20.738 INFO:tasks.workunit.client.1.smithi203.stdout:3/144: mkdir d10/d16/d23/d38 0 2024-05-25T06:11:20.740 INFO:tasks.workunit.client.1.smithi203.stdout:2/108: creat d2/d6/f26 x:0 0 0 2024-05-25T06:11:20.743 INFO:tasks.workunit.client.1.smithi203.stdout:0/126: rename dd/f17 to dd/d10/f31 0 2024-05-25T06:11:20.746 INFO:tasks.workunit.client.1.smithi203.stdout:9/101: creat d0/f24 x:0 0 0 2024-05-25T06:11:20.747 INFO:tasks.workunit.client.1.smithi203.stdout:4/130: fsync d0/d8/f23 0 2024-05-25T06:11:20.749 INFO:tasks.workunit.client.1.smithi203.stdout:8/176: dwrite d2/d5/d2d/f3e [0,4194304] 0 2024-05-25T06:11:20.753 INFO:tasks.workunit.client.1.smithi203.stdout:2/109: truncate d2/d1a/d23/f25 2603276 0 2024-05-25T06:11:20.754 INFO:tasks.workunit.client.1.smithi203.stdout:9/102: fdatasync d0/df/f14 0 2024-05-25T06:11:20.758 INFO:tasks.workunit.client.1.smithi203.stdout:9/103: stat d0 0 2024-05-25T06:11:20.758 INFO:tasks.workunit.client.1.smithi203.stdout:1/95: dwrite fc [0,4194304] 0 2024-05-25T06:11:20.758 INFO:tasks.workunit.client.1.smithi203.stdout:4/131: write d0/d9/f17 [1094926,27596] 0 2024-05-25T06:11:20.762 INFO:tasks.workunit.client.1.smithi203.stdout:0/127: getdents dd/d10 0 2024-05-25T06:11:20.763 INFO:tasks.workunit.client.1.smithi203.stdout:1/96: rmdir d14/d1b 39 2024-05-25T06:11:20.768 INFO:tasks.workunit.client.1.smithi203.stdout:4/132: rmdir d0/d9/d11 39 2024-05-25T06:11:20.768 INFO:tasks.workunit.client.1.smithi203.stdout:2/110: dread d2/d4/f11 [0,4194304] 0 2024-05-25T06:11:20.772 INFO:tasks.workunit.client.1.smithi203.stdout:3/145: dwrite d10/d22/d2a/f34 [0,4194304] 0 2024-05-25T06:11:20.777 INFO:tasks.workunit.client.1.smithi203.stdout:2/111: creat d2/d1a/f27 x:0 0 0 2024-05-25T06:11:20.777 INFO:tasks.workunit.client.1.smithi203.stdout:9/104: getdents d0/d17/d23 0 2024-05-25T06:11:20.781 INFO:tasks.workunit.client.1.smithi203.stdout:0/128: write dd/d10/f2f [3458209,82096] 0 2024-05-25T06:11:20.782 INFO:tasks.workunit.client.1.smithi203.stdout:9/105: chown d0/l2 3987313 1 2024-05-25T06:11:20.782 INFO:tasks.workunit.client.1.smithi203.stdout:8/177: dwrite d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:20.783 INFO:tasks.workunit.client.1.smithi203.stdout:1/97: dwrite f10 [4194304,4194304] 0 2024-05-25T06:11:20.785 INFO:tasks.workunit.client.1.smithi203.stdout:3/146: symlink d10/d16/d23/d38/l39 0 2024-05-25T06:11:20.786 INFO:tasks.workunit.client.1.smithi203.stdout:1/98: chown fa 1029 1 2024-05-25T06:11:20.786 INFO:tasks.workunit.client.1.smithi203.stdout:2/112: truncate d2/d6/da/f1d 8147 0 2024-05-25T06:11:20.787 INFO:tasks.workunit.client.1.smithi203.stdout:8/178: write d2/d5/d10/d11/de/f13 [2474338,41474] 0 2024-05-25T06:11:20.789 INFO:tasks.workunit.client.1.smithi203.stdout:2/113: read - d2/d6/d14/f22 zero size 2024-05-25T06:11:20.790 INFO:tasks.workunit.client.1.smithi203.stdout:0/129: mkdir dd/d32 0 2024-05-25T06:11:20.791 INFO:tasks.workunit.client.1.smithi203.stdout:4/133: dread d0/f5 [0,4194304] 0 2024-05-25T06:11:20.792 INFO:tasks.workunit.client.1.smithi203.stdout:8/179: rmdir d2/d5/d2d 39 2024-05-25T06:11:20.794 INFO:tasks.workunit.client.1.smithi203.stdout:4/134: write d0/d8/f23 [774058,31696] 0 2024-05-25T06:11:20.797 INFO:tasks.workunit.client.1.smithi203.stdout:1/99: mknod d14/c1f 0 2024-05-25T06:11:20.799 INFO:tasks.workunit.client.1.smithi203.stdout:3/147: dread d10/d16/f17 [0,4194304] 0 2024-05-25T06:11:20.800 INFO:tasks.workunit.client.1.smithi203.stdout:4/135: dread d0/f16 [0,4194304] 0 2024-05-25T06:11:20.801 INFO:tasks.workunit.client.1.smithi203.stdout:8/180: chown d2/l41 2097955017 1 2024-05-25T06:11:20.803 INFO:tasks.workunit.client.1.smithi203.stdout:2/114: dwrite d2/d6/da/f24 [0,4194304] 0 2024-05-25T06:11:20.806 INFO:tasks.workunit.client.1.smithi203.stdout:9/106: getdents d0 0 2024-05-25T06:11:20.806 INFO:tasks.workunit.client.1.smithi203.stdout:9/107: readlink d0/l22 0 2024-05-25T06:11:20.808 INFO:tasks.workunit.client.1.smithi203.stdout:0/130: link dd/f15 dd/d10/f33 0 2024-05-25T06:11:20.811 INFO:tasks.workunit.client.1.smithi203.stdout:2/115: creat d2/d6/d14/f28 x:0 0 0 2024-05-25T06:11:20.812 INFO:tasks.workunit.client.1.smithi203.stdout:0/131: dread dd/d10/f2f [0,4194304] 0 2024-05-25T06:11:20.814 INFO:tasks.workunit.client.1.smithi203.stdout:8/181: rename d2/d5/d10/d11/de/c2c to d2/d5/d18/d31/c43 0 2024-05-25T06:11:20.819 INFO:tasks.workunit.client.1.smithi203.stdout:0/132: mkdir dd/d10/d34 0 2024-05-25T06:11:20.819 INFO:tasks.workunit.client.1.smithi203.stdout:9/108: mknod d0/d17/d20/c25 0 2024-05-25T06:11:20.821 INFO:tasks.workunit.client.1.smithi203.stdout:3/148: dwrite d10/f21 [0,4194304] 0 2024-05-25T06:11:20.822 INFO:tasks.workunit.client.1.smithi203.stdout:3/149: write d10/f1a [109187,31997] 0 2024-05-25T06:11:20.824 INFO:tasks.workunit.client.1.smithi203.stdout:2/116: fsync d2/d6/d14/f1c 0 2024-05-25T06:11:20.833 INFO:tasks.workunit.client.1.smithi203.stdout:0/133: mknod dd/d32/c35 0 2024-05-25T06:11:20.835 INFO:tasks.workunit.client.1.smithi203.stdout:2/117: dwrite d2/d6/f26 [0,4194304] 0 2024-05-25T06:11:20.836 INFO:tasks.workunit.client.1.smithi203.stdout:8/182: creat d2/d5/f44 x:0 0 0 2024-05-25T06:11:20.836 INFO:tasks.workunit.client.1.smithi203.stdout:9/109: symlink d0/d17/d23/l26 0 2024-05-25T06:11:20.836 INFO:tasks.workunit.client.1.smithi203.stdout:4/136: dwrite d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:20.838 INFO:tasks.workunit.client.1.smithi203.stdout:2/118: dread - d2/d6/d14/f22 zero size 2024-05-25T06:11:20.838 INFO:tasks.workunit.client.1.smithi203.stdout:9/110: fdatasync d0/fb 0 2024-05-25T06:11:20.842 INFO:tasks.workunit.client.1.smithi203.stdout:3/150: unlink d10/l27 0 2024-05-25T06:11:20.842 INFO:tasks.workunit.client.1.smithi203.stdout:2/119: fsync d2/d6/d14/f22 0 2024-05-25T06:11:20.842 INFO:tasks.workunit.client.1.smithi203.stdout:3/151: write d10/f20 [1423394,12662] 0 2024-05-25T06:11:20.845 INFO:tasks.workunit.client.1.smithi203.stdout:2/120: chown d2/d6/d14/f1c 40062 1 2024-05-25T06:11:20.845 INFO:tasks.workunit.client.1.smithi203.stdout:9/111: truncate d0/f1e 303130 0 2024-05-25T06:11:20.845 INFO:tasks.workunit.client.1.smithi203.stdout:0/134: mknod dd/c36 0 2024-05-25T06:11:20.847 INFO:tasks.workunit.client.1.smithi203.stdout:4/137: write d0/f3 [1980899,10156] 0 2024-05-25T06:11:20.848 INFO:tasks.workunit.client.1.smithi203.stdout:3/152: chown d10/d22/d2a/l2d 11425 1 2024-05-25T06:11:20.849 INFO:tasks.workunit.client.1.smithi203.stdout:9/112: creat d0/df/f27 x:0 0 0 2024-05-25T06:11:20.851 INFO:tasks.workunit.client.1.smithi203.stdout:2/121: creat d2/d6/da/f29 x:0 0 0 2024-05-25T06:11:20.852 INFO:tasks.workunit.client.1.smithi203.stdout:8/183: dwrite d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:20.854 INFO:tasks.workunit.client.1.smithi203.stdout:0/135: mkdir dd/d32/d37 0 2024-05-25T06:11:20.855 INFO:tasks.workunit.client.1.smithi203.stdout:4/138: mkdir d0/d9/d24/d27 0 2024-05-25T06:11:20.857 INFO:tasks.workunit.client.1.smithi203.stdout:2/122: mknod d2/d6/c2a 0 2024-05-25T06:11:20.857 INFO:tasks.workunit.client.1.smithi203.stdout:2/123: read - d2/d6/d14/f22 zero size 2024-05-25T06:11:20.859 INFO:tasks.workunit.client.1.smithi203.stdout:3/153: mkdir d10/d16/d23/d3a 0 2024-05-25T06:11:20.860 INFO:tasks.workunit.client.1.smithi203.stdout:0/136: creat dd/d32/f38 x:0 0 0 2024-05-25T06:11:20.862 INFO:tasks.workunit.client.1.smithi203.stdout:2/124: symlink d2/d4/l2b 0 2024-05-25T06:11:20.863 INFO:tasks.workunit.client.1.smithi203.stdout:2/125: chown d2/d4/l16 8761 1 2024-05-25T06:11:20.864 INFO:tasks.workunit.client.1.smithi203.stdout:9/113: link d0/l1 d0/d17/d23/l28 0 2024-05-25T06:11:20.865 INFO:tasks.workunit.client.1.smithi203.stdout:2/126: mknod d2/d1a/c2c 0 2024-05-25T06:11:20.867 INFO:tasks.workunit.client.1.smithi203.stdout:2/127: chown d2/d4/d10 79 1 2024-05-25T06:11:20.868 INFO:tasks.workunit.client.1.smithi203.stdout:4/139: getdents d0/d9/d24 0 2024-05-25T06:11:20.869 INFO:tasks.workunit.client.1.smithi203.stdout:2/128: readlink d2/d4/l7 0 2024-05-25T06:11:20.873 INFO:tasks.workunit.client.1.smithi203.stdout:4/140: dread d0/f16 [0,4194304] 0 2024-05-25T06:11:20.873 INFO:tasks.workunit.client.1.smithi203.stdout:2/129: creat d2/d6/f2d x:0 0 0 2024-05-25T06:11:20.875 INFO:tasks.workunit.client.1.smithi203.stdout:3/154: dwrite d10/d16/d23/f2b [0,4194304] 0 2024-05-25T06:11:20.875 INFO:tasks.workunit.client.1.smithi203.stdout:9/114: dwrite d0/d17/f1b [0,4194304] 0 2024-05-25T06:11:20.877 INFO:tasks.workunit.client.1.smithi203.stdout:2/130: chown d2/d6/c2a 712885 1 2024-05-25T06:11:20.881 INFO:tasks.workunit.client.1.smithi203.stdout:4/141: creat d0/d9/d24/f28 x:0 0 0 2024-05-25T06:11:20.882 INFO:tasks.workunit.client.1.smithi203.stdout:0/137: dwrite dd/d10/f29 [0,4194304] 0 2024-05-25T06:11:20.883 INFO:tasks.workunit.client.1.smithi203.stdout:0/138: readlink la 0 2024-05-25T06:11:20.883 INFO:tasks.workunit.client.1.smithi203.stdout:4/142: readlink d0/d8/d10/l1a 0 2024-05-25T06:11:20.884 INFO:tasks.workunit.client.1.smithi203.stdout:9/115: mkdir d0/d17/d20/d29 0 2024-05-25T06:11:20.884 INFO:tasks.workunit.client.1.smithi203.stdout:2/131: write d2/d6/da/f29 [740408,127848] 0 2024-05-25T06:11:20.890 INFO:tasks.workunit.client.1.smithi203.stdout:9/116: symlink d0/d17/d20/l2a 0 2024-05-25T06:11:20.891 INFO:tasks.workunit.client.1.smithi203.stdout:9/117: chown d0/f13 447 1 2024-05-25T06:11:20.892 INFO:tasks.workunit.client.1.smithi203.stdout:2/132: rename d2/d6/da/f24 to d2/d4/f2e 0 2024-05-25T06:11:20.895 INFO:tasks.workunit.client.1.smithi203.stdout:2/133: write d2/d1a/f27 [864510,80330] 0 2024-05-25T06:11:20.897 INFO:tasks.workunit.client.1.smithi203.stdout:3/155: dwrite d10/f1a [0,4194304] 0 2024-05-25T06:11:20.902 INFO:tasks.workunit.client.1.smithi203.stdout:9/118: dwrite d0/f18 [4194304,4194304] 0 2024-05-25T06:11:20.902 INFO:tasks.workunit.client.1.smithi203.stdout:2/134: mkdir d2/d6/da/d2f 0 2024-05-25T06:11:20.905 INFO:tasks.workunit.client.1.smithi203.stdout:9/119: write d0/df/f14 [1250024,92653] 0 2024-05-25T06:11:20.907 INFO:tasks.workunit.client.1.smithi203.stdout:2/135: mkdir d2/d6/d30 0 2024-05-25T06:11:20.908 INFO:tasks.workunit.client.1.smithi203.stdout:4/143: write d0/d9/f12 [429749,114926] 0 2024-05-25T06:11:20.913 INFO:tasks.workunit.client.1.smithi203.stdout:2/136: read d2/d6/da/f1d [6376,7929] 0 2024-05-25T06:11:20.916 INFO:tasks.workunit.client.1.smithi203.stdout:4/144: write d0/f3 [2974107,71642] 0 2024-05-25T06:11:20.917 INFO:tasks.workunit.client.1.smithi203.stdout:3/156: dread d10/d16/d23/f29 [0,4194304] 0 2024-05-25T06:11:20.919 INFO:tasks.workunit.client.1.smithi203.stdout:2/137: write d2/d6/f2d [303990,58150] 0 2024-05-25T06:11:20.920 INFO:tasks.workunit.client.1.smithi203.stdout:0/139: dwrite dd/f15 [0,4194304] 0 2024-05-25T06:11:20.921 INFO:tasks.workunit.client.1.smithi203.stdout:3/157: rmdir d10/d16/d23 39 2024-05-25T06:11:20.922 INFO:tasks.workunit.client.1.smithi203.stdout:0/140: read dd/d10/f31 [2820853,121810] 0 2024-05-25T06:11:20.924 INFO:tasks.workunit.client.1.smithi203.stdout:0/141: chown dd/d10/f28 1 1 2024-05-25T06:11:20.924 INFO:tasks.workunit.client.1.smithi203.stdout:2/138: dread d2/d6/da/f1d [0,4194304] 0 2024-05-25T06:11:20.924 INFO:tasks.workunit.client.1.smithi203.stdout:3/158: creat d10/d22/d2a/f3b x:0 0 0 2024-05-25T06:11:20.927 INFO:tasks.workunit.client.1.smithi203.stdout:9/120: truncate d0/d17/f1b 1053135 0 2024-05-25T06:11:20.927 INFO:tasks.workunit.client.1.smithi203.stdout:0/142: dread - dd/d10/f28 zero size 2024-05-25T06:11:20.930 INFO:tasks.workunit.client.1.smithi203.stdout:9/121: write d0/df/f15 [1061396,27809] 0 2024-05-25T06:11:20.931 INFO:tasks.workunit.client.1.smithi203.stdout:2/139: rename d2/d6/fb to d2/d6/da/d2f/f31 0 2024-05-25T06:11:20.932 INFO:tasks.workunit.client.1.smithi203.stdout:3/159: mknod d10/d16/c3c 0 2024-05-25T06:11:20.933 INFO:tasks.workunit.client.1.smithi203.stdout:4/145: truncate d0/f5 3622748 0 2024-05-25T06:11:20.934 INFO:tasks.workunit.client.1.smithi203.stdout:0/143: symlink dd/d10/l39 0 2024-05-25T06:11:20.935 INFO:tasks.workunit.client.1.smithi203.stdout:0/144: fsync dd/d10/f26 0 2024-05-25T06:11:20.935 INFO:tasks.workunit.client.1.smithi203.stdout:9/122: mkdir d0/df/d2b 0 2024-05-25T06:11:20.938 INFO:tasks.workunit.client.1.smithi203.stdout:2/140: rename d2/d6/d14 to d2/d4/d10/d32 0 2024-05-25T06:11:20.938 INFO:tasks.workunit.client.1.smithi203.stdout:3/160: chown d10/d16/d23/d3a 749739631 1 2024-05-25T06:11:20.939 INFO:tasks.workunit.client.1.smithi203.stdout:3/161: dread - d10/d22/f35 zero size 2024-05-25T06:11:20.940 INFO:tasks.workunit.client.1.smithi203.stdout:4/146: creat d0/d8/d1f/f29 x:0 0 0 2024-05-25T06:11:20.941 INFO:tasks.workunit.client.1.smithi203.stdout:9/123: unlink d0/f18 0 2024-05-25T06:11:20.942 INFO:tasks.workunit.client.1.smithi203.stdout:4/147: dread d0/f1c [0,4194304] 0 2024-05-25T06:11:20.942 INFO:tasks.workunit.client.1.smithi203.stdout:2/141: rename d2/d6/d30 to d2/d1a/d33 0 2024-05-25T06:11:20.950 INFO:tasks.workunit.client.1.smithi203.stdout:4/148: dwrite d0/d8/d1f/f29 [0,4194304] 0 2024-05-25T06:11:20.952 INFO:tasks.workunit.client.1.smithi203.stdout:2/142: creat d2/d6/da/d2f/f34 x:0 0 0 2024-05-25T06:11:20.953 INFO:tasks.workunit.client.1.smithi203.stdout:2/143: chown d2/d4 38516 1 2024-05-25T06:11:20.958 INFO:tasks.workunit.client.1.smithi203.stdout:4/149: creat d0/d8/d1f/d26/f2a x:0 0 0 2024-05-25T06:11:20.958 INFO:tasks.workunit.client.1.smithi203.stdout:2/144: dread d2/d1a/f27 [0,4194304] 0 2024-05-25T06:11:20.960 INFO:tasks.workunit.client.1.smithi203.stdout:0/145: getdents dd/d10 0 2024-05-25T06:11:20.962 INFO:tasks.workunit.client.1.smithi203.stdout:3/162: write d10/d16/f17 [1290545,54988] 0 2024-05-25T06:11:20.963 INFO:tasks.workunit.client.1.smithi203.stdout:4/150: creat d0/d8/d1f/f2b x:0 0 0 2024-05-25T06:11:20.963 INFO:tasks.workunit.client.1.smithi203.stdout:2/145: mknod d2/d1a/d33/c35 0 2024-05-25T06:11:20.964 INFO:tasks.workunit.client.1.smithi203.stdout:3/163: truncate d10/d16/f2c 396863 0 2024-05-25T06:11:20.964 INFO:tasks.workunit.client.1.smithi203.stdout:4/151: write d0/d8/d1f/d26/f1e [8491893,78894] 0 2024-05-25T06:11:20.965 INFO:tasks.workunit.client.1.smithi203.stdout:0/146: mkdir dd/d3a 0 2024-05-25T06:11:20.967 INFO:tasks.workunit.client.1.smithi203.stdout:3/164: write d10/d22/d2a/f34 [2769918,25756] 0 2024-05-25T06:11:20.972 INFO:tasks.workunit.client.1.smithi203.stdout:3/165: chown d10/d16/f2c 4082752 1 2024-05-25T06:11:20.975 INFO:tasks.workunit.client.1.smithi203.stdout:2/146: dwrite d2/d6/da/d2f/f34 [0,4194304] 0 2024-05-25T06:11:20.975 INFO:tasks.workunit.client.1.smithi203.stdout:0/147: truncate dd/d10/f2f 1379473 0 2024-05-25T06:11:20.979 INFO:tasks.workunit.client.1.smithi203.stdout:3/166: symlink d10/d22/d2a/l3d 0 2024-05-25T06:11:20.981 INFO:tasks.workunit.client.1.smithi203.stdout:4/152: dwrite d0/d8/d1f/d26/f1e [0,4194304] 0 2024-05-25T06:11:20.983 INFO:tasks.workunit.client.1.smithi203.stdout:2/147: rename d2/d1a/d23 to d2/d4/d10/d32/d36 0 2024-05-25T06:11:20.984 INFO:tasks.workunit.client.1.smithi203.stdout:3/167: write d10/d22/f35 [579064,106748] 0 2024-05-25T06:11:20.987 INFO:tasks.workunit.client.1.smithi203.stdout:9/124: dwrite d0/f16 [4194304,4194304] 0 2024-05-25T06:11:20.991 INFO:tasks.workunit.client.1.smithi203.stdout:3/168: chown f0 9532953 1 2024-05-25T06:11:20.994 INFO:tasks.workunit.client.1.smithi203.stdout:9/125: creat d0/d17/d20/f2c x:0 0 0 2024-05-25T06:11:20.995 INFO:tasks.workunit.client.1.smithi203.stdout:3/169: write d10/d16/f19 [249356,126971] 0 2024-05-25T06:11:20.997 INFO:tasks.workunit.client.1.smithi203.stdout:4/153: dwrite d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:21.000 INFO:tasks.workunit.client.1.smithi203.stdout:2/148: dwrite d2/d6/f2d [0,4194304] 0 2024-05-25T06:11:21.001 INFO:tasks.workunit.client.1.smithi203.stdout:4/154: truncate d0/d9/f12 737560 0 2024-05-25T06:11:21.003 INFO:tasks.workunit.client.1.smithi203.stdout:9/126: mkdir d0/df/d2d 0 2024-05-25T06:11:21.004 INFO:tasks.workunit.client.1.smithi203.stdout:2/149: symlink d2/d4/d10/l37 0 2024-05-25T06:11:21.006 INFO:tasks.workunit.client.1.smithi203.stdout:4/155: symlink d0/d9/l2c 0 2024-05-25T06:11:21.009 INFO:tasks.workunit.client.1.smithi203.stdout:9/127: creat d0/d17/d23/f2e x:0 0 0 2024-05-25T06:11:21.011 INFO:tasks.workunit.client.1.smithi203.stdout:9/128: write d0/df/f15 [4294826,108682] 0 2024-05-25T06:11:21.011 INFO:tasks.workunit.client.1.smithi203.stdout:3/170: fdatasync d10/d16/f19 0 2024-05-25T06:11:21.012 INFO:tasks.workunit.client.1.smithi203.stdout:4/156: symlink d0/l2d 0 2024-05-25T06:11:21.015 INFO:tasks.workunit.client.1.smithi203.stdout:2/150: rename d2/d6/da to d2/d38 0 2024-05-25T06:11:21.016 INFO:tasks.workunit.client.1.smithi203.stdout:4/157: dread d0/f1c [0,4194304] 0 2024-05-25T06:11:21.018 INFO:tasks.workunit.client.1.smithi203.stdout:3/171: getdents d10 0 2024-05-25T06:11:21.019 INFO:tasks.workunit.client.1.smithi203.stdout:2/151: link d2/l18 d2/d4/d10/d32/d36/l39 0 2024-05-25T06:11:21.020 INFO:tasks.workunit.client.1.smithi203.stdout:2/152: read - d2/d4/d10/f21 zero size 2024-05-25T06:11:21.021 INFO:tasks.workunit.client.1.smithi203.stdout:3/172: creat d10/d16/d23/d38/f3e x:0 0 0 2024-05-25T06:11:21.022 INFO:tasks.workunit.client.1.smithi203.stdout:2/153: write d2/d6/f2d [1216568,62434] 0 2024-05-25T06:11:21.023 INFO:tasks.workunit.client.1.smithi203.stdout:3/173: write d10/d16/d23/f2b [2084767,55699] 0 2024-05-25T06:11:21.023 INFO:tasks.workunit.client.1.smithi203.stdout:2/154: stat d2/d4/d10/d32/f22 0 2024-05-25T06:11:21.027 INFO:tasks.workunit.client.1.smithi203.stdout:2/155: unlink d2/d4/d10/l37 0 2024-05-25T06:11:21.030 INFO:tasks.workunit.client.1.smithi203.stdout:3/174: dwrite d10/f1a [0,4194304] 0 2024-05-25T06:11:21.033 INFO:tasks.workunit.client.1.smithi203.stdout:3/175: mknod d10/d16/d23/d38/c3f 0 2024-05-25T06:11:21.035 INFO:tasks.workunit.client.1.smithi203.stdout:3/176: write d10/d22/d2a/f3b [459102,7615] 0 2024-05-25T06:11:21.036 INFO:tasks.workunit.client.1.smithi203.stdout:3/177: write d10/d16/d23/f2b [1654556,28497] 0 2024-05-25T06:11:21.048 INFO:tasks.workunit.client.1.smithi203.stdout:3/178: dwrite d10/d16/d23/f29 [0,4194304] 0 2024-05-25T06:11:21.049 INFO:tasks.workunit.client.1.smithi203.stdout:4/158: truncate d0/d8/d1f/f29 3369700 0 2024-05-25T06:11:21.050 INFO:tasks.workunit.client.1.smithi203.stdout:2/156: write d2/d4/d10/d32/f1c [87730,45021] 0 2024-05-25T06:11:21.053 INFO:tasks.workunit.client.1.smithi203.stdout:2/157: write d2/d6/f26 [4945052,40859] 0 2024-05-25T06:11:21.056 INFO:tasks.workunit.client.1.smithi203.stdout:4/159: dread d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:21.057 INFO:tasks.workunit.client.1.smithi203.stdout:3/179: rename d10/f13 to d10/d16/f40 0 2024-05-25T06:11:21.058 INFO:tasks.workunit.client.1.smithi203.stdout:4/160: truncate d0/d9/f12 1183564 0 2024-05-25T06:11:21.058 INFO:tasks.workunit.client.1.smithi203.stdout:4/161: chown d0/d9/d24/l25 3 1 2024-05-25T06:11:21.062 INFO:tasks.workunit.client.1.smithi203.stdout:4/162: unlink d0/d8/d10/l13 0 2024-05-25T06:11:21.065 INFO:tasks.workunit.client.1.smithi203.stdout:2/158: dwrite d2/d38/d2f/f31 [0,4194304] 0 2024-05-25T06:11:21.067 INFO:tasks.workunit.client.1.smithi203.stdout:2/159: write d2/d4/d10/d32/f1f [815512,76507] 0 2024-05-25T06:11:21.070 INFO:tasks.workunit.client.1.smithi203.stdout:2/160: rmdir d2 39 2024-05-25T06:11:21.073 INFO:tasks.workunit.client.1.smithi203.stdout:4/163: write d0/f16 [365155,19260] 0 2024-05-25T06:11:21.074 INFO:tasks.workunit.client.1.smithi203.stdout:3/180: symlink d10/d22/d2a/l41 0 2024-05-25T06:11:21.075 INFO:tasks.workunit.client.1.smithi203.stdout:2/161: mknod d2/d4/d10/d32/c3a 0 2024-05-25T06:11:21.077 INFO:tasks.workunit.client.1.smithi203.stdout:3/181: creat d10/d22/f42 x:0 0 0 2024-05-25T06:11:21.077 INFO:tasks.workunit.client.1.smithi203.stdout:2/162: symlink d2/d4/l3b 0 2024-05-25T06:11:21.079 INFO:tasks.workunit.client.1.smithi203.stdout:3/182: chown la 1750 1 2024-05-25T06:11:21.081 INFO:tasks.workunit.client.1.smithi203.stdout:3/183: write d10/d22/f35 [494677,14924] 0 2024-05-25T06:11:21.081 INFO:tasks.workunit.client.1.smithi203.stdout:4/164: getdents d0/d8/d1f/d26 0 2024-05-25T06:11:21.082 INFO:tasks.workunit.client.1.smithi203.stdout:4/165: fdatasync d0/d9/f12 0 2024-05-25T06:11:21.082 INFO:tasks.workunit.client.1.smithi203.stdout:3/184: stat d10/d22/d2a/l2d 0 2024-05-25T06:11:21.082 INFO:tasks.workunit.client.1.smithi203.stdout:2/163: dread d2/fe [0,4194304] 0 2024-05-25T06:11:21.083 INFO:tasks.workunit.client.1.smithi203.stdout:4/166: readlink d0/d9/l2c 0 2024-05-25T06:11:21.083 INFO:tasks.workunit.client.1.smithi203.stdout:3/185: chown d10/d22/d2a/f34 0 1 2024-05-25T06:11:21.084 INFO:tasks.workunit.client.1.smithi203.stdout:2/164: mknod d2/d4/c3c 0 2024-05-25T06:11:21.085 INFO:tasks.workunit.client.1.smithi203.stdout:2/165: fdatasync d2/d38/d2f/f31 0 2024-05-25T06:11:21.088 INFO:tasks.workunit.client.1.smithi203.stdout:4/167: unlink d0/d8/d1f/f29 0 2024-05-25T06:11:21.089 INFO:tasks.workunit.client.1.smithi203.stdout:2/166: fsync d2/d38/f1d 0 2024-05-25T06:11:21.090 INFO:tasks.workunit.client.1.smithi203.stdout:4/168: write d0/d8/f23 [343999,91116] 0 2024-05-25T06:11:21.093 INFO:tasks.workunit.client.1.smithi203.stdout:3/186: dwrite d10/f1a [0,4194304] 0 2024-05-25T06:11:21.095 INFO:tasks.workunit.client.1.smithi203.stdout:3/187: write f0 [3677517,127529] 0 2024-05-25T06:11:21.095 INFO:tasks.workunit.client.1.smithi203.stdout:4/169: creat d0/d8/d1f/d26/f2e x:0 0 0 2024-05-25T06:11:21.100 INFO:tasks.workunit.client.1.smithi203.stdout:2/167: dwrite d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:21.102 INFO:tasks.workunit.client.1.smithi203.stdout:2/168: chown d2/d1a/d33 247201 1 2024-05-25T06:11:21.102 INFO:tasks.workunit.client.1.smithi203.stdout:3/188: unlink d10/d16/f2c 0 2024-05-25T06:11:21.107 INFO:tasks.workunit.client.1.smithi203.stdout:3/189: symlink d10/d16/l43 0 2024-05-25T06:11:21.108 INFO:tasks.workunit.client.1.smithi203.stdout:4/170: dwrite d0/d8/d1f/f2b [0,4194304] 0 2024-05-25T06:11:21.114 INFO:tasks.workunit.client.1.smithi203.stdout:2/169: dwrite d2/d4/d10/d32/f1c [0,4194304] 0 2024-05-25T06:11:21.116 INFO:tasks.workunit.client.1.smithi203.stdout:4/171: creat d0/d9/d11/f2f x:0 0 0 2024-05-25T06:11:21.119 INFO:tasks.workunit.client.1.smithi203.stdout:2/170: creat d2/d4/d10/d32/f3d x:0 0 0 2024-05-25T06:11:21.122 INFO:tasks.workunit.client.1.smithi203.stdout:3/190: getdents d10/d16/d23/d38 0 2024-05-25T06:11:21.123 INFO:tasks.workunit.client.1.smithi203.stdout:2/171: mknod d2/d1a/d33/c3e 0 2024-05-25T06:11:21.124 INFO:tasks.workunit.client.1.smithi203.stdout:3/191: chown d10/f32 2 1 2024-05-25T06:11:21.125 INFO:tasks.workunit.client.1.smithi203.stdout:3/192: readlink la 0 2024-05-25T06:11:21.126 INFO:tasks.workunit.client.1.smithi203.stdout:2/172: mknod d2/d1a/d33/c3f 0 2024-05-25T06:11:21.127 INFO:tasks.workunit.client.1.smithi203.stdout:3/193: dread d10/d16/f17 [0,4194304] 0 2024-05-25T06:11:21.129 INFO:tasks.workunit.client.1.smithi203.stdout:2/173: symlink d2/d4/d10/l40 0 2024-05-25T06:11:21.129 INFO:tasks.workunit.client.1.smithi203.stdout:4/172: link d0/f5 d0/d9/d24/d27/f30 0 2024-05-25T06:11:21.131 INFO:tasks.workunit.client.1.smithi203.stdout:2/174: symlink d2/d6/l41 0 2024-05-25T06:11:21.254 INFO:tasks.workunit.client.1.smithi203.stdout:3/194: creat d10/d16/f44 x:0 0 0 2024-05-25T06:11:21.257 INFO:tasks.workunit.client.1.smithi203.stdout:2/175: dread d2/d1a/f27 [0,4194304] 0 2024-05-25T06:11:21.258 INFO:tasks.workunit.client.1.smithi203.stdout:4/173: dwrite d0/f16 [0,4194304] 0 2024-05-25T06:11:21.260 INFO:tasks.workunit.client.1.smithi203.stdout:2/176: mkdir d2/d42 0 2024-05-25T06:11:21.263 INFO:tasks.workunit.client.1.smithi203.stdout:3/195: dwrite f5 [0,4194304] 0 2024-05-25T06:11:21.264 INFO:tasks.workunit.client.1.smithi203.stdout:2/177: dread d2/d4/f11 [0,4194304] 0 2024-05-25T06:11:21.264 INFO:tasks.workunit.client.1.smithi203.stdout:4/174: rename d0/c18 to d0/d8/c31 0 2024-05-25T06:11:21.265 INFO:tasks.workunit.client.1.smithi203.stdout:4/175: chown d0/d9/l2c 6999738 1 2024-05-25T06:11:21.268 INFO:tasks.workunit.client.1.smithi203.stdout:2/178: dread d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:21.269 INFO:tasks.workunit.client.1.smithi203.stdout:3/196: mkdir d10/d16/d23/d45 0 2024-05-25T06:11:21.270 INFO:tasks.workunit.client.1.smithi203.stdout:2/179: dread d2/d38/f1d [0,4194304] 0 2024-05-25T06:11:21.273 INFO:tasks.workunit.client.1.smithi203.stdout:4/176: rmdir d0/d8/d1f 39 2024-05-25T06:11:21.274 INFO:tasks.workunit.client.1.smithi203.stdout:3/197: chown d10/f20 1643460 1 2024-05-25T06:11:21.278 INFO:tasks.workunit.client.1.smithi203.stdout:2/180: dwrite d2/d4/d10/d32/f3d [0,4194304] 0 2024-05-25T06:11:21.278 INFO:tasks.workunit.client.1.smithi203.stdout:3/198: chown d10/d16/d23/l30 630 1 2024-05-25T06:11:21.281 INFO:tasks.workunit.client.1.smithi203.stdout:4/177: getdents d0/d9/d11 0 2024-05-25T06:11:21.283 INFO:tasks.workunit.client.1.smithi203.stdout:2/181: mkdir d2/d38/d43 0 2024-05-25T06:11:21.284 INFO:tasks.workunit.client.1.smithi203.stdout:3/199: unlink d10/d16/l2f 0 2024-05-25T06:11:21.284 INFO:tasks.workunit.client.1.smithi203.stdout:3/200: chown f1 1 1 2024-05-25T06:11:21.287 INFO:tasks.workunit.client.1.smithi203.stdout:3/201: mkdir d10/d16/d23/d46 0 2024-05-25T06:11:21.290 INFO:tasks.workunit.client.1.smithi203.stdout:2/182: dwrite d2/d6/f2d [4194304,4194304] 0 2024-05-25T06:11:21.292 INFO:tasks.workunit.client.1.smithi203.stdout:3/202: dread d10/d16/d23/f2b [0,4194304] 0 2024-05-25T06:11:21.293 INFO:tasks.workunit.client.1.smithi203.stdout:2/183: unlink d2/d6/l41 0 2024-05-25T06:11:21.296 INFO:tasks.workunit.client.1.smithi203.stdout:2/184: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:21.302 INFO:tasks.workunit.client.1.smithi203.stdout:2/185: dwrite d2/d38/d2f/f31 [0,4194304] 0 2024-05-25T06:11:21.304 INFO:tasks.workunit.client.1.smithi203.stdout:2/186: creat d2/d4/d10/d32/f44 x:0 0 0 2024-05-25T06:11:21.313 INFO:tasks.workunit.client.1.smithi203.stdout:2/187: dwrite d2/d4/d10/d32/f28 [0,4194304] 0 2024-05-25T06:11:21.315 INFO:tasks.workunit.client.1.smithi203.stdout:2/188: write d2/d4/d10/f21 [397793,41879] 0 2024-05-25T06:11:21.317 INFO:tasks.workunit.client.1.smithi203.stdout:2/189: mknod d2/d1a/d33/c45 0 2024-05-25T06:11:21.326 INFO:tasks.workunit.client.1.smithi203.stdout:2/190: dwrite d2/d6/f26 [0,4194304] 0 2024-05-25T06:11:21.327 INFO:tasks.workunit.client.1.smithi203.stdout:2/191: truncate d2/d4/d10/d32/f1f 1591586 0 2024-05-25T06:11:21.330 INFO:tasks.workunit.client.1.smithi203.stdout:2/192: creat d2/d6/f46 x:0 0 0 2024-05-25T06:11:21.331 INFO:tasks.workunit.client.1.smithi203.stdout:6/160: sync 2024-05-25T06:11:21.332 INFO:tasks.workunit.client.1.smithi203.stdout:2/193: write d2/fe [954791,130372] 0 2024-05-25T06:11:21.334 INFO:tasks.workunit.client.1.smithi203.stdout:6/161: mkdir d8/d3d 0 2024-05-25T06:11:21.339 INFO:tasks.workunit.client.1.smithi203.stdout:2/194: dwrite d2/d38/d2f/f34 [0,4194304] 0 2024-05-25T06:11:21.342 INFO:tasks.workunit.client.1.smithi203.stdout:2/195: symlink d2/d38/d2f/l47 0 2024-05-25T06:11:21.443 INFO:tasks.workunit.client.1.smithi203.stdout:2/196: symlink d2/d38/l48 0 2024-05-25T06:11:21.445 INFO:tasks.workunit.client.1.smithi203.stdout:2/197: mknod d2/d4/d10/d32/d36/c49 0 2024-05-25T06:11:21.447 INFO:tasks.workunit.client.1.smithi203.stdout:2/198: symlink d2/d38/l4a 0 2024-05-25T06:11:21.449 INFO:tasks.workunit.client.1.smithi203.stdout:2/199: mknod d2/d4/d10/c4b 0 2024-05-25T06:11:21.449 INFO:tasks.workunit.client.1.smithi203.stdout:6/162: dread d8/d11/d1b/d24/f32 [0,4194304] 0 2024-05-25T06:11:21.451 INFO:tasks.workunit.client.1.smithi203.stdout:2/200: symlink d2/d4/d10/d32/d36/l4c 0 2024-05-25T06:11:21.452 INFO:tasks.workunit.client.1.smithi203.stdout:6/163: mkdir d8/d29/d3e 0 2024-05-25T06:11:21.454 INFO:tasks.workunit.client.1.smithi203.stdout:2/201: creat d2/d38/d43/f4d x:0 0 0 2024-05-25T06:11:21.455 INFO:tasks.workunit.client.1.smithi203.stdout:2/202: readlink d2/d38/d2f/l47 0 2024-05-25T06:11:21.456 INFO:tasks.workunit.client.1.smithi203.stdout:2/203: chown d2/d4/d10/d32/f22 3 1 2024-05-25T06:11:21.458 INFO:tasks.workunit.client.1.smithi203.stdout:2/204: mkdir d2/d42/d4e 0 2024-05-25T06:11:21.458 INFO:tasks.workunit.client.1.smithi203.stdout:6/164: dwrite f0 [4194304,4194304] 0 2024-05-25T06:11:21.460 INFO:tasks.workunit.client.1.smithi203.stdout:6/165: chown d8/d11/d15/d28 179743 1 2024-05-25T06:11:21.462 INFO:tasks.workunit.client.1.smithi203.stdout:2/205: creat d2/d38/d2f/f4f x:0 0 0 2024-05-25T06:11:21.466 INFO:tasks.workunit.client.1.smithi203.stdout:6/166: dwrite d8/d11/d15/f2b [0,4194304] 0 2024-05-25T06:11:21.466 INFO:tasks.workunit.client.1.smithi203.stdout:6/167: readlink d8/d11/l21 0 2024-05-25T06:11:21.467 INFO:tasks.workunit.client.1.smithi203.stdout:6/168: chown d8/ff 7515 1 2024-05-25T06:11:21.469 INFO:tasks.workunit.client.1.smithi203.stdout:5/253: sync 2024-05-25T06:11:21.469 INFO:tasks.workunit.client.1.smithi203.stdout:7/118: sync 2024-05-25T06:11:21.472 INFO:tasks.workunit.client.1.smithi203.stdout:2/206: mkdir d2/d4/d10/d50 0 2024-05-25T06:11:21.480 INFO:tasks.workunit.client.1.smithi203.stdout:5/254: creat dc/d11/d17/d1a/d2d/d53/f5b x:0 0 0 2024-05-25T06:11:21.484 INFO:tasks.workunit.client.1.smithi203.stdout:7/119: mkdir d1/d9/d2c 0 2024-05-25T06:11:21.485 INFO:tasks.workunit.client.1.smithi203.stdout:5/255: creat dc/d11/d17/d45/f5c x:0 0 0 2024-05-25T06:11:21.485 INFO:tasks.workunit.client.1.smithi203.stdout:2/207: creat d2/d4/d10/d50/f51 x:0 0 0 2024-05-25T06:11:21.487 INFO:tasks.workunit.client.1.smithi203.stdout:7/120: creat d1/d9/d2c/f2d x:0 0 0 2024-05-25T06:11:21.489 INFO:tasks.workunit.client.1.smithi203.stdout:7/121: truncate d1/d11/f21 399601 0 2024-05-25T06:11:21.491 INFO:tasks.workunit.client.1.smithi203.stdout:7/122: creat d1/d9/f2e x:0 0 0 2024-05-25T06:11:21.491 INFO:tasks.workunit.client.1.smithi203.stdout:2/208: dread d2/d38/d2f/f31 [0,4194304] 0 2024-05-25T06:11:21.493 INFO:tasks.workunit.client.1.smithi203.stdout:7/123: symlink d1/d11/l2f 0 2024-05-25T06:11:21.494 INFO:tasks.workunit.client.1.smithi203.stdout:2/209: chown d2/d4/d10/d32/d36/l39 227 1 2024-05-25T06:11:21.495 INFO:tasks.workunit.client.1.smithi203.stdout:7/124: creat d1/d11/f30 x:0 0 0 2024-05-25T06:11:21.495 INFO:tasks.workunit.client.1.smithi203.stdout:2/210: fsync d2/d4/d10/d32/f22 0 2024-05-25T06:11:21.496 INFO:tasks.workunit.client.1.smithi203.stdout:6/169: write f2 [1626956,99508] 0 2024-05-25T06:11:21.498 INFO:tasks.workunit.client.1.smithi203.stdout:2/211: creat d2/d4/d10/d32/f52 x:0 0 0 2024-05-25T06:11:21.502 INFO:tasks.workunit.client.1.smithi203.stdout:7/125: link d1/f24 d1/f31 0 2024-05-25T06:11:21.503 INFO:tasks.workunit.client.1.smithi203.stdout:5/256: truncate dc/d11/f14 1381406 0 2024-05-25T06:11:21.504 INFO:tasks.workunit.client.1.smithi203.stdout:2/212: mknod d2/c53 0 2024-05-25T06:11:21.507 INFO:tasks.workunit.client.1.smithi203.stdout:7/126: dread d1/d9/f14 [0,4194304] 0 2024-05-25T06:11:21.508 INFO:tasks.workunit.client.1.smithi203.stdout:5/257: rename dc/d11/d17/f2f to dc/d11/d17/f5d 0 2024-05-25T06:11:21.508 INFO:tasks.workunit.client.1.smithi203.stdout:7/127: chown d1/f24 165546 1 2024-05-25T06:11:21.509 INFO:tasks.workunit.client.1.smithi203.stdout:5/258: write dc/d11/d17/f56 [427257,90578] 0 2024-05-25T06:11:21.510 INFO:tasks.workunit.client.1.smithi203.stdout:2/213: symlink d2/d4/d10/l54 0 2024-05-25T06:11:21.511 INFO:tasks.workunit.client.1.smithi203.stdout:6/170: write d8/d11/d15/f2e [345554,83054] 0 2024-05-25T06:11:21.516 INFO:tasks.workunit.client.1.smithi203.stdout:2/214: symlink d2/l55 0 2024-05-25T06:11:21.520 INFO:tasks.workunit.client.1.smithi203.stdout:5/259: rename dc/d11/d17/d44/c5a to dc/d11/d18/c5e 0 2024-05-25T06:11:21.520 INFO:tasks.workunit.client.1.smithi203.stdout:7/128: creat d1/f32 x:0 0 0 2024-05-25T06:11:21.521 INFO:tasks.workunit.client.1.smithi203.stdout:5/260: write dc/d11/d17/f31 [476776,24641] 0 2024-05-25T06:11:21.522 INFO:tasks.workunit.client.1.smithi203.stdout:2/215: write d2/d38/d2f/f31 [1014541,40317] 0 2024-05-25T06:11:21.523 INFO:tasks.workunit.client.1.smithi203.stdout:5/261: write dc/d11/d18/f37 [393611,70840] 0 2024-05-25T06:11:21.524 INFO:tasks.workunit.client.1.smithi203.stdout:6/171: getdents d8/d29 0 2024-05-25T06:11:21.528 INFO:tasks.workunit.client.1.smithi203.stdout:7/129: mkdir d1/d33 0 2024-05-25T06:11:21.529 INFO:tasks.workunit.client.1.smithi203.stdout:7/130: dread - d1/f1d zero size 2024-05-25T06:11:21.529 INFO:tasks.workunit.client.1.smithi203.stdout:5/262: chown dc/d11/d17/d1a/l47 6 1 2024-05-25T06:11:21.530 INFO:tasks.workunit.client.1.smithi203.stdout:2/216: symlink d2/l56 0 2024-05-25T06:11:21.531 INFO:tasks.workunit.client.1.smithi203.stdout:6/172: dwrite f0 [4194304,4194304] 0 2024-05-25T06:11:21.532 INFO:tasks.workunit.client.1.smithi203.stdout:5/263: write dc/d11/d17/f56 [15728,65177] 0 2024-05-25T06:11:21.534 INFO:tasks.workunit.client.1.smithi203.stdout:2/217: mknod d2/d38/d43/c57 0 2024-05-25T06:11:21.535 INFO:tasks.workunit.client.1.smithi203.stdout:6/173: mknod d8/d29/c3f 0 2024-05-25T06:11:21.535 INFO:tasks.workunit.client.1.smithi203.stdout:2/218: fdatasync d2/d4/d10/d32/f44 0 2024-05-25T06:11:21.536 INFO:tasks.workunit.client.1.smithi203.stdout:5/264: creat dc/d11/d17/d45/f5f x:0 0 0 2024-05-25T06:11:21.536 INFO:tasks.workunit.client.1.smithi203.stdout:6/174: write d8/f2c [15747,12257] 0 2024-05-25T06:11:21.536 INFO:tasks.workunit.client.1.smithi203.stdout:7/131: creat d1/f34 x:0 0 0 2024-05-25T06:11:21.540 INFO:tasks.workunit.client.1.smithi203.stdout:2/219: mkdir d2/d4/d10/d58 0 2024-05-25T06:11:21.541 INFO:tasks.workunit.client.1.smithi203.stdout:5/265: mknod dc/d11/d17/d44/c60 0 2024-05-25T06:11:21.542 INFO:tasks.workunit.client.1.smithi203.stdout:7/132: mknod d1/d9/d2c/c35 0 2024-05-25T06:11:21.544 INFO:tasks.workunit.client.1.smithi203.stdout:5/266: fsync f8 0 2024-05-25T06:11:21.546 INFO:tasks.workunit.client.1.smithi203.stdout:7/133: getdents d1/d33 0 2024-05-25T06:11:21.551 INFO:tasks.workunit.client.1.smithi203.stdout:6/175: dwrite d8/d11/f19 [0,4194304] 0 2024-05-25T06:11:21.551 INFO:tasks.workunit.client.1.smithi203.stdout:5/267: truncate dc/d11/d17/f38 404430 0 2024-05-25T06:11:21.551 INFO:tasks.workunit.client.1.smithi203.stdout:7/134: mkdir d1/d9/d2c/d36 0 2024-05-25T06:11:21.556 INFO:tasks.workunit.client.1.smithi203.stdout:7/135: fsync d1/f3 0 2024-05-25T06:11:21.558 INFO:tasks.workunit.client.1.smithi203.stdout:2/220: dwrite d2/d38/d43/f4d [0,4194304] 0 2024-05-25T06:11:21.560 INFO:tasks.workunit.client.1.smithi203.stdout:6/176: dwrite d8/d11/d15/f31 [0,4194304] 0 2024-05-25T06:11:21.560 INFO:tasks.workunit.client.1.smithi203.stdout:7/136: truncate d1/d11/d20/f23 291257 0 2024-05-25T06:11:21.561 INFO:tasks.workunit.client.1.smithi203.stdout:6/177: chown d8/d11/f19 80230530 1 2024-05-25T06:11:21.563 INFO:tasks.workunit.client.1.smithi203.stdout:5/268: dread dc/fe [0,4194304] 0 2024-05-25T06:11:21.564 INFO:tasks.workunit.client.1.smithi203.stdout:6/178: write d8/d11/d15/d28/f1d [981364,52708] 0 2024-05-25T06:11:21.564 INFO:tasks.workunit.client.1.smithi203.stdout:7/137: creat d1/d33/f37 x:0 0 0 2024-05-25T06:11:21.566 INFO:tasks.workunit.client.1.smithi203.stdout:6/179: readlink d8/d11/d1b/l36 0 2024-05-25T06:11:21.569 INFO:tasks.workunit.client.1.smithi203.stdout:5/269: creat dc/d11/d17/d1a/f61 x:0 0 0 2024-05-25T06:11:21.570 INFO:tasks.workunit.client.1.smithi203.stdout:5/270: chown dc/d11/f3e 0 1 2024-05-25T06:11:21.570 INFO:tasks.workunit.client.1.smithi203.stdout:6/180: mknod d8/d3d/c40 0 2024-05-25T06:11:21.573 INFO:tasks.workunit.client.1.smithi203.stdout:5/271: creat dc/d11/d17/d45/f62 x:0 0 0 2024-05-25T06:11:21.579 INFO:tasks.workunit.client.1.smithi203.stdout:6/181: dwrite d8/fd [4194304,4194304] 0 2024-05-25T06:11:21.582 INFO:tasks.workunit.client.1.smithi203.stdout:5/272: rename dc/d11/d17/d1a/d22/c34 to dc/d11/d17/d1a/d2d/d4d/c63 0 2024-05-25T06:11:21.585 INFO:tasks.workunit.client.1.smithi203.stdout:6/182: read d8/ff [1775019,7222] 0 2024-05-25T06:11:21.586 INFO:tasks.workunit.client.1.smithi203.stdout:6/183: fsync f2 0 2024-05-25T06:11:21.590 INFO:tasks.workunit.client.1.smithi203.stdout:5/273: dwrite dc/d3c/d3f/f4e [0,4194304] 0 2024-05-25T06:11:21.590 INFO:tasks.workunit.client.1.smithi203.stdout:6/184: mknod d8/d11/d1b/c41 0 2024-05-25T06:11:21.593 INFO:tasks.workunit.client.1.smithi203.stdout:7/138: dread f0 [0,4194304] 0 2024-05-25T06:11:21.594 INFO:tasks.workunit.client.1.smithi203.stdout:6/185: creat d8/d11/d15/f42 x:0 0 0 2024-05-25T06:11:21.596 INFO:tasks.workunit.client.1.smithi203.stdout:7/139: rmdir d1/d33 39 2024-05-25T06:11:21.596 INFO:tasks.workunit.client.1.smithi203.stdout:6/186: read d8/f9 [3299784,1186] 0 2024-05-25T06:11:21.597 INFO:tasks.workunit.client.1.smithi203.stdout:5/274: chown dc/d11/d18/c5e 1042601 1 2024-05-25T06:11:21.598 INFO:tasks.workunit.client.1.smithi203.stdout:6/187: chown d8/d29/d30/d38 246 1 2024-05-25T06:11:21.599 INFO:tasks.workunit.client.1.smithi203.stdout:7/140: unlink d1/d11/f17 0 2024-05-25T06:11:21.599 INFO:tasks.workunit.client.1.smithi203.stdout:0/148: sync 2024-05-25T06:11:21.600 INFO:tasks.workunit.client.1.smithi203.stdout:5/275: fdatasync dc/d11/f29 0 2024-05-25T06:11:21.601 INFO:tasks.workunit.client.1.smithi203.stdout:6/188: fdatasync d8/d11/d15/f2b 0 2024-05-25T06:11:21.602 INFO:tasks.workunit.client.1.smithi203.stdout:7/141: write d1/d33/f37 [866235,74296] 0 2024-05-25T06:11:21.602 INFO:tasks.workunit.client.1.smithi203.stdout:0/149: unlink dd/d10/f18 0 2024-05-25T06:11:21.604 INFO:tasks.workunit.client.1.smithi203.stdout:5/276: read dc/d11/f1c [13739,48192] 0 2024-05-25T06:11:21.605 INFO:tasks.workunit.client.1.smithi203.stdout:0/150: getdents dd/d10/d34 0 2024-05-25T06:11:21.605 INFO:tasks.workunit.client.1.smithi203.stdout:0/151: readlink dd/d10/l2c 0 2024-05-25T06:11:21.607 INFO:tasks.workunit.client.1.smithi203.stdout:7/142: rename d1/d11/d20/f25 to d1/f38 0 2024-05-25T06:11:21.610 INFO:tasks.workunit.client.1.smithi203.stdout:6/189: rename d8/d11/d1b/d24 to d8/d29/d3e/d43 0 2024-05-25T06:11:21.611 INFO:tasks.workunit.client.1.smithi203.stdout:0/152: dread dd/d10/f2d [0,4194304] 0 2024-05-25T06:11:21.612 INFO:tasks.workunit.client.1.smithi203.stdout:0/153: write dd/f25 [873792,63001] 0 2024-05-25T06:11:21.613 INFO:tasks.workunit.client.1.smithi203.stdout:0/154: dread - dd/fe zero size 2024-05-25T06:11:21.619 INFO:tasks.workunit.client.1.smithi203.stdout:0/155: creat dd/d3a/f3b x:0 0 0 2024-05-25T06:11:21.619 INFO:tasks.workunit.client.1.smithi203.stdout:5/277: write dc/f21 [1002642,107325] 0 2024-05-25T06:11:21.622 INFO:tasks.workunit.client.1.smithi203.stdout:0/156: fdatasync dd/d10/f2d 0 2024-05-25T06:11:21.623 INFO:tasks.workunit.client.1.smithi203.stdout:5/278: mkdir dc/d11/d17/d1a/d2d/d4d/d64 0 2024-05-25T06:11:21.625 INFO:tasks.workunit.client.1.smithi203.stdout:7/143: dread d1/d9/fb [0,4194304] 0 2024-05-25T06:11:21.625 INFO:tasks.workunit.client.1.smithi203.stdout:0/157: rename dd/f25 to dd/d10/d34/f3c 0 2024-05-25T06:11:21.626 INFO:tasks.workunit.client.1.smithi203.stdout:7/144: stat d1/d11/d20 0 2024-05-25T06:11:21.627 INFO:tasks.workunit.client.1.smithi203.stdout:0/158: mkdir dd/d3d 0 2024-05-25T06:11:21.633 INFO:tasks.workunit.client.1.smithi203.stdout:5/279: dwrite dc/d11/d17/d1a/d22/f39 [0,4194304] 0 2024-05-25T06:11:21.636 INFO:tasks.workunit.client.1.smithi203.stdout:7/145: write d1/d9/fc [1730364,48332] 0 2024-05-25T06:11:21.639 INFO:tasks.workunit.client.1.smithi203.stdout:7/146: symlink d1/l39 0 2024-05-25T06:11:21.643 INFO:tasks.workunit.client.1.smithi203.stdout:7/147: mkdir d1/d9/d2c/d3a 0 2024-05-25T06:11:21.643 INFO:tasks.workunit.client.1.smithi203.stdout:7/148: chown d1/f1d 1 1 2024-05-25T06:11:21.644 INFO:tasks.workunit.client.1.smithi203.stdout:7/149: fdatasync d1/d11/d20/f23 0 2024-05-25T06:11:21.646 INFO:tasks.workunit.client.1.smithi203.stdout:7/150: mkdir d1/d9/d2c/d36/d3b 0 2024-05-25T06:11:21.647 INFO:tasks.workunit.client.1.smithi203.stdout:7/151: fsync d1/f32 0 2024-05-25T06:11:21.648 INFO:tasks.workunit.client.1.smithi203.stdout:7/152: dread - d1/d9/d2c/f2d zero size 2024-05-25T06:11:21.649 INFO:tasks.workunit.client.1.smithi203.stdout:0/159: dread f5 [0,4194304] 0 2024-05-25T06:11:21.650 INFO:tasks.workunit.client.1.smithi203.stdout:7/153: creat d1/f3c x:0 0 0 2024-05-25T06:11:21.651 INFO:tasks.workunit.client.1.smithi203.stdout:0/160: mknod dd/d32/c3e 0 2024-05-25T06:11:21.652 INFO:tasks.workunit.client.1.smithi203.stdout:7/154: write d1/d11/d20/f23 [941715,16634] 0 2024-05-25T06:11:21.654 INFO:tasks.workunit.client.1.smithi203.stdout:7/155: dread - d1/d9/f29 zero size 2024-05-25T06:11:21.656 INFO:tasks.workunit.client.1.smithi203.stdout:7/156: mkdir d1/d11/d3d 0 2024-05-25T06:11:21.657 INFO:tasks.workunit.client.1.smithi203.stdout:5/280: dread dc/d11/d17/d1a/d22/f24 [0,4194304] 0 2024-05-25T06:11:21.659 INFO:tasks.workunit.client.1.smithi203.stdout:7/157: unlink d1/d9/d2c/f2d 0 2024-05-25T06:11:21.661 INFO:tasks.workunit.client.1.smithi203.stdout:7/158: creat d1/d9/d2c/d3a/f3e x:0 0 0 2024-05-25T06:11:21.664 INFO:tasks.workunit.client.1.smithi203.stdout:5/281: dwrite dc/d11/d17/d1a/f27 [0,4194304] 0 2024-05-25T06:11:21.666 INFO:tasks.workunit.client.1.smithi203.stdout:7/159: rename d1/c6 to d1/d9/c3f 0 2024-05-25T06:11:21.668 INFO:tasks.workunit.client.1.smithi203.stdout:7/160: chown d1/d11/d20/f28 33827077 1 2024-05-25T06:11:21.669 INFO:tasks.workunit.client.1.smithi203.stdout:5/282: rename dc/d11/d17/d1a/d2d/d53/f5b to dc/d11/d17/d1a/d2d/f65 0 2024-05-25T06:11:21.669 INFO:tasks.workunit.client.1.smithi203.stdout:7/161: chown d1/d11/d20/f28 85 1 2024-05-25T06:11:21.670 INFO:tasks.workunit.client.1.smithi203.stdout:5/283: write dc/d11/d17/d45/f5f [832612,77890] 0 2024-05-25T06:11:21.673 INFO:tasks.workunit.client.1.smithi203.stdout:5/284: mkdir dc/d11/d17/d1a/d2d/d4d/d64/d66 0 2024-05-25T06:11:21.679 INFO:tasks.workunit.client.1.smithi203.stdout:5/285: dwrite dc/d11/d17/d45/f5f [0,4194304] 0 2024-05-25T06:11:21.683 INFO:tasks.workunit.client.1.smithi203.stdout:5/286: symlink dc/d11/d17/d45/l67 0 2024-05-25T06:11:21.684 INFO:tasks.workunit.client.1.smithi203.stdout:7/162: write d1/d9/fd [1589756,78941] 0 2024-05-25T06:11:21.688 INFO:tasks.workunit.client.1.smithi203.stdout:7/163: rename d1/f1d to d1/d9/d2c/d36/f40 0 2024-05-25T06:11:21.689 INFO:tasks.workunit.client.1.smithi203.stdout:7/164: chown d1/d9/fc 767382 1 2024-05-25T06:11:21.692 INFO:tasks.workunit.client.1.smithi203.stdout:7/165: rmdir d1/d11 39 2024-05-25T06:11:21.692 INFO:tasks.workunit.client.1.smithi203.stdout:5/287: dwrite dc/d11/f12 [0,4194304] 0 2024-05-25T06:11:21.696 INFO:tasks.workunit.client.1.smithi203.stdout:5/288: symlink dc/d11/d17/d1a/d2d/d53/l68 0 2024-05-25T06:11:21.698 INFO:tasks.workunit.client.1.smithi203.stdout:7/166: creat d1/d11/f41 x:0 0 0 2024-05-25T06:11:21.699 INFO:tasks.workunit.client.1.smithi203.stdout:5/289: symlink dc/d11/d17/d45/l69 0 2024-05-25T06:11:21.699 INFO:tasks.workunit.client.1.smithi203.stdout:7/167: write d1/f32 [751070,111641] 0 2024-05-25T06:11:21.701 INFO:tasks.workunit.client.1.smithi203.stdout:5/290: truncate dc/d11/d17/d1a/d2d/d53/f55 543025 0 2024-05-25T06:11:21.708 INFO:tasks.workunit.client.1.smithi203.stdout:5/291: truncate dc/f10 692661 0 2024-05-25T06:11:21.711 INFO:tasks.workunit.client.1.smithi203.stdout:5/292: unlink dc/d11/d18/c23 0 2024-05-25T06:11:21.715 INFO:tasks.workunit.client.1.smithi203.stdout:5/293: getdents dc/d11/d17/d1a/d2d/d4d 0 2024-05-25T06:11:21.717 INFO:tasks.workunit.client.1.smithi203.stdout:5/294: write dc/d11/d17/f5d [743653,126455] 0 2024-05-25T06:11:21.717 INFO:tasks.workunit.client.1.smithi203.stdout:5/295: chown dc/d11/d17/f31 3494631 1 2024-05-25T06:11:21.724 INFO:tasks.workunit.client.1.smithi203.stdout:5/296: write f1 [248108,81947] 0 2024-05-25T06:11:21.728 INFO:tasks.workunit.client.1.smithi203.stdout:5/297: mkdir dc/d6a 0 2024-05-25T06:11:21.730 INFO:tasks.workunit.client.1.smithi203.stdout:5/298: mkdir dc/d11/d6b 0 2024-05-25T06:11:21.731 INFO:tasks.workunit.client.1.smithi203.stdout:5/299: write dc/d11/f14 [747682,51290] 0 2024-05-25T06:11:21.738 INFO:tasks.workunit.client.1.smithi203.stdout:5/300: dwrite dc/d3c/d3f/f4e [0,4194304] 0 2024-05-25T06:11:21.741 INFO:tasks.workunit.client.1.smithi203.stdout:5/301: symlink dc/d11/d6b/l6c 0 2024-05-25T06:11:21.743 INFO:tasks.workunit.client.1.smithi203.stdout:5/302: symlink dc/d11/d17/d1a/d2d/d4d/l6d 0 2024-05-25T06:11:21.747 INFO:tasks.workunit.client.1.smithi203.stdout:5/303: link dc/d11/d17/d1a/c41 dc/d3c/c6e 0 2024-05-25T06:11:21.748 INFO:tasks.workunit.client.1.smithi203.stdout:5/304: chown dc/d11/d17/d45/l67 7 1 2024-05-25T06:11:21.749 INFO:tasks.workunit.client.1.smithi203.stdout:5/305: write dc/d11/d17/d45/f5f [3691543,10138] 0 2024-05-25T06:11:21.750 INFO:tasks.workunit.client.1.smithi203.stdout:5/306: chown dc/d11/d18 908 1 2024-05-25T06:11:21.755 INFO:tasks.workunit.client.1.smithi203.stdout:5/307: stat dc/d11/d17/d1a/d2d/d53 0 2024-05-25T06:11:21.755 INFO:tasks.workunit.client.1.smithi203.stdout:5/308: fsync dc/d11/d17/f54 0 2024-05-25T06:11:21.756 INFO:tasks.workunit.client.1.smithi203.stdout:5/309: rmdir dc/d11/d17/d1a/d2d/d4d/d64/d66 0 2024-05-25T06:11:21.757 INFO:tasks.workunit.client.1.smithi203.stdout:5/310: symlink dc/d11/d17/d1a/d2d/d4d/d64/l6f 0 2024-05-25T06:11:21.759 INFO:tasks.workunit.client.1.smithi203.stdout:5/311: mknod dc/d11/d6b/c70 0 2024-05-25T06:11:21.761 INFO:tasks.workunit.client.1.smithi203.stdout:5/312: rmdir dc/d11/d17/d1a/d2d 39 2024-05-25T06:11:21.763 INFO:tasks.workunit.client.1.smithi203.stdout:5/313: symlink dc/d11/d18/l71 0 2024-05-25T06:11:21.765 INFO:tasks.workunit.client.1.smithi203.stdout:5/314: mkdir dc/d11/d17/d1a/d72 0 2024-05-25T06:11:21.767 INFO:tasks.workunit.client.1.smithi203.stdout:5/315: mkdir dc/d11/d17/d44/d73 0 2024-05-25T06:11:21.769 INFO:tasks.workunit.client.1.smithi203.stdout:5/316: creat dc/d11/d17/d1a/d72/f74 x:0 0 0 2024-05-25T06:11:21.772 INFO:tasks.workunit.client.1.smithi203.stdout:5/317: mkdir dc/d11/d17/d1a/d75 0 2024-05-25T06:11:21.774 INFO:tasks.workunit.client.1.smithi203.stdout:3/203: sync 2024-05-25T06:11:21.774 INFO:tasks.workunit.client.1.smithi203.stdout:4/178: sync 2024-05-25T06:11:21.774 INFO:tasks.workunit.client.1.smithi203.stdout:1/100: sync 2024-05-25T06:11:21.774 INFO:tasks.workunit.client.1.smithi203.stdout:9/129: sync 2024-05-25T06:11:21.774 INFO:tasks.workunit.client.1.smithi203.stdout:8/184: sync 2024-05-25T06:11:21.774 INFO:tasks.workunit.client.1.smithi203.stdout:5/318: mknod dc/c76 0 2024-05-25T06:11:21.776 INFO:tasks.workunit.client.1.smithi203.stdout:1/101: mkdir d14/d20 0 2024-05-25T06:11:21.777 INFO:tasks.workunit.client.1.smithi203.stdout:4/179: dread - d0/d8/d1f/d26/f2e zero size 2024-05-25T06:11:21.779 INFO:tasks.workunit.client.1.smithi203.stdout:5/319: creat dc/d11/d17/d1a/d75/f77 x:0 0 0 2024-05-25T06:11:21.780 INFO:tasks.workunit.client.1.smithi203.stdout:1/102: write f10 [5439434,13663] 0 2024-05-25T06:11:21.782 INFO:tasks.workunit.client.1.smithi203.stdout:4/180: symlink d0/d8/l32 0 2024-05-25T06:11:21.784 INFO:tasks.workunit.client.1.smithi203.stdout:5/320: read dc/d11/f29 [3404968,104877] 0 2024-05-25T06:11:21.786 INFO:tasks.workunit.client.1.smithi203.stdout:1/103: chown d14/d1b/d1d 47659517 1 2024-05-25T06:11:21.791 INFO:tasks.workunit.client.1.smithi203.stdout:5/321: fdatasync dc/d11/d17/d45/f46 0 2024-05-25T06:11:21.793 INFO:tasks.workunit.client.1.smithi203.stdout:9/130: dwrite d0/df/f1d [0,4194304] 0 2024-05-25T06:11:21.794 INFO:tasks.workunit.client.1.smithi203.stdout:3/204: dwrite d10/d22/d2a/f3b [0,4194304] 0 2024-05-25T06:11:21.795 INFO:tasks.workunit.client.1.smithi203.stdout:8/185: dwrite d2/d5/d18/d31/f3d [0,4194304] 0 2024-05-25T06:11:21.795 INFO:tasks.workunit.client.1.smithi203.stdout:1/104: symlink d14/l21 0 2024-05-25T06:11:21.798 INFO:tasks.workunit.client.1.smithi203.stdout:5/322: creat dc/d3c/f78 x:0 0 0 2024-05-25T06:11:21.799 INFO:tasks.workunit.client.1.smithi203.stdout:5/323: fsync dc/d11/f14 0 2024-05-25T06:11:21.800 INFO:tasks.workunit.client.1.smithi203.stdout:5/324: chown dc/d11/d17/d1a/f2e 672 1 2024-05-25T06:11:21.801 INFO:tasks.workunit.client.1.smithi203.stdout:9/131: rename d0/df/f27 to d0/df/d2b/f2f 0 2024-05-25T06:11:21.803 INFO:tasks.workunit.client.1.smithi203.stdout:4/181: truncate d0/d9/d24/d27/f30 2378531 0 2024-05-25T06:11:21.803 INFO:tasks.workunit.client.1.smithi203.stdout:1/105: dread f12 [0,4194304] 0 2024-05-25T06:11:21.806 INFO:tasks.workunit.client.1.smithi203.stdout:5/325: write dc/fe [293450,51631] 0 2024-05-25T06:11:21.807 INFO:tasks.workunit.client.1.smithi203.stdout:3/205: creat d10/d16/f47 x:0 0 0 2024-05-25T06:11:21.809 INFO:tasks.workunit.client.1.smithi203.stdout:4/182: mknod d0/d9/d11/c33 0 2024-05-25T06:11:21.811 INFO:tasks.workunit.client.1.smithi203.stdout:1/106: dwrite f10 [4194304,4194304] 0 2024-05-25T06:11:21.817 INFO:tasks.workunit.client.1.smithi203.stdout:4/183: rename d0/f1c to d0/d9/d11/f34 0 2024-05-25T06:11:21.817 INFO:tasks.workunit.client.1.smithi203.stdout:5/326: dwrite dc/d11/d17/d44/f51 [0,4194304] 0 2024-05-25T06:11:21.818 INFO:tasks.workunit.client.1.smithi203.stdout:1/107: mkdir d14/d1b/d22 0 2024-05-25T06:11:21.823 INFO:tasks.workunit.client.1.smithi203.stdout:9/132: link d0/l1 d0/df/d2d/l30 0 2024-05-25T06:11:21.824 INFO:tasks.workunit.client.1.smithi203.stdout:9/133: chown d0/d17/d23 119755722 1 2024-05-25T06:11:21.825 INFO:tasks.workunit.client.1.smithi203.stdout:9/134: chown d0/df/f14 7 1 2024-05-25T06:11:21.828 INFO:tasks.workunit.client.1.smithi203.stdout:5/327: creat dc/d11/d6b/f79 x:0 0 0 2024-05-25T06:11:21.828 INFO:tasks.workunit.client.1.smithi203.stdout:3/206: write d10/d16/f17 [361689,80830] 0 2024-05-25T06:11:21.898 INFO:tasks.workunit.client.1.smithi203.stdout:9/135: creat d0/d17/d20/f31 x:0 0 0 2024-05-25T06:11:21.899 INFO:tasks.workunit.client.1.smithi203.stdout:1/108: mkdir d14/d23 0 2024-05-25T06:11:21.899 INFO:tasks.workunit.client.1.smithi203.stdout:5/328: write f6 [866522,80928] 0 2024-05-25T06:11:21.901 INFO:tasks.workunit.client.1.smithi203.stdout:1/109: chown d14/l21 2 1 2024-05-25T06:11:21.904 INFO:tasks.workunit.client.1.smithi203.stdout:5/329: unlink dc/d11/d17/d1a/d72/f74 0 2024-05-25T06:11:21.906 INFO:tasks.workunit.client.1.smithi203.stdout:3/207: mkdir d10/d16/d23/d3a/d48 0 2024-05-25T06:11:21.912 INFO:tasks.workunit.client.1.smithi203.stdout:5/330: rmdir dc/d11/d17 39 2024-05-25T06:11:21.913 INFO:tasks.workunit.client.1.smithi203.stdout:4/184: read d0/f5 [33639,118513] 0 2024-05-25T06:11:21.913 INFO:tasks.workunit.client.1.smithi203.stdout:9/136: dwrite d0/d17/f1b [0,4194304] 0 2024-05-25T06:11:21.914 INFO:tasks.workunit.client.1.smithi203.stdout:9/137: dread - d0/df/f1a zero size 2024-05-25T06:11:21.914 INFO:tasks.workunit.client.1.smithi203.stdout:1/110: creat d14/f24 x:0 0 0 2024-05-25T06:11:21.916 INFO:tasks.workunit.client.1.smithi203.stdout:3/208: mknod d10/c49 0 2024-05-25T06:11:21.917 INFO:tasks.workunit.client.1.smithi203.stdout:1/111: write d14/f24 [695113,106773] 0 2024-05-25T06:11:21.917 INFO:tasks.workunit.client.1.smithi203.stdout:5/331: chown dc/d11/d17/d1a/f2e 84554 1 2024-05-25T06:11:21.918 INFO:tasks.workunit.client.1.smithi203.stdout:9/138: creat d0/d17/d20/d29/f32 x:0 0 0 2024-05-25T06:11:21.920 INFO:tasks.workunit.client.1.smithi203.stdout:4/185: unlink d0/d9/d11/f34 0 2024-05-25T06:11:21.922 INFO:tasks.workunit.client.1.smithi203.stdout:5/332: symlink dc/d11/d6b/l7a 0 2024-05-25T06:11:21.923 INFO:tasks.workunit.client.1.smithi203.stdout:9/139: creat d0/d17/f33 x:0 0 0 2024-05-25T06:11:21.924 INFO:tasks.workunit.client.1.smithi203.stdout:9/140: chown d0/d17/d20 0 1 2024-05-25T06:11:21.925 INFO:tasks.workunit.client.1.smithi203.stdout:5/333: rmdir dc/d3c 39 2024-05-25T06:11:21.926 INFO:tasks.workunit.client.1.smithi203.stdout:9/141: readlink d0/d17/d20/l2a 0 2024-05-25T06:11:21.928 INFO:tasks.workunit.client.1.smithi203.stdout:1/112: creat d14/f25 x:0 0 0 2024-05-25T06:11:21.928 INFO:tasks.workunit.client.1.smithi203.stdout:3/209: getdents d10 0 2024-05-25T06:11:21.930 INFO:tasks.workunit.client.1.smithi203.stdout:4/186: dwrite d0/d8/d1f/d26/f2e [0,4194304] 0 2024-05-25T06:11:21.938 INFO:tasks.workunit.client.1.smithi203.stdout:4/187: symlink d0/d8/l35 0 2024-05-25T06:11:21.940 INFO:tasks.workunit.client.1.smithi203.stdout:3/210: dread f5 [0,4194304] 0 2024-05-25T06:11:21.940 INFO:tasks.workunit.client.1.smithi203.stdout:5/334: rename fb to dc/d11/d17/d1a/d2d/d4d/f7b 0 2024-05-25T06:11:21.942 INFO:tasks.workunit.client.1.smithi203.stdout:9/142: dwrite d0/f24 [0,4194304] 0 2024-05-25T06:11:21.944 INFO:tasks.workunit.client.1.smithi203.stdout:1/113: dwrite d14/f25 [0,4194304] 0 2024-05-25T06:11:21.946 INFO:tasks.workunit.client.1.smithi203.stdout:3/211: write f0 [4280776,56112] 0 2024-05-25T06:11:21.946 INFO:tasks.workunit.client.1.smithi203.stdout:4/188: mknod d0/d9/c36 0 2024-05-25T06:11:21.946 INFO:tasks.workunit.client.1.smithi203.stdout:1/114: readlink d14/l21 0 2024-05-25T06:11:21.948 INFO:tasks.workunit.client.1.smithi203.stdout:5/335: write dc/d11/d18/f37 [980661,4631] 0 2024-05-25T06:11:21.954 INFO:tasks.workunit.client.1.smithi203.stdout:1/115: mknod d14/d23/c26 0 2024-05-25T06:11:21.959 INFO:tasks.workunit.client.1.smithi203.stdout:3/212: mkdir d10/d16/d23/d4a 0 2024-05-25T06:11:21.959 INFO:tasks.workunit.client.1.smithi203.stdout:3/213: write d10/d16/f44 [839783,95344] 0 2024-05-25T06:11:21.965 INFO:tasks.workunit.client.1.smithi203.stdout:5/336: dwrite dc/d11/d17/d45/f52 [0,4194304] 0 2024-05-25T06:11:21.965 INFO:tasks.workunit.client.1.smithi203.stdout:3/214: dread d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:21.966 INFO:tasks.workunit.client.1.smithi203.stdout:1/116: dwrite d14/f24 [0,4194304] 0 2024-05-25T06:11:21.973 INFO:tasks.workunit.client.1.smithi203.stdout:1/117: symlink d14/l27 0 2024-05-25T06:11:21.974 INFO:tasks.workunit.client.1.smithi203.stdout:9/143: write d0/f19 [1078213,51692] 0 2024-05-25T06:11:21.975 INFO:tasks.workunit.client.1.smithi203.stdout:9/144: chown d0/df 51 1 2024-05-25T06:11:21.976 INFO:tasks.workunit.client.1.smithi203.stdout:5/337: creat dc/f7c x:0 0 0 2024-05-25T06:11:21.976 INFO:tasks.workunit.client.1.smithi203.stdout:3/215: dread d10/f21 [0,4194304] 0 2024-05-25T06:11:21.978 INFO:tasks.workunit.client.1.smithi203.stdout:5/338: chown dc/d11/d42 2627031 1 2024-05-25T06:11:21.979 INFO:tasks.workunit.client.1.smithi203.stdout:9/145: creat d0/df/d2b/f34 x:0 0 0 2024-05-25T06:11:21.981 INFO:tasks.workunit.client.1.smithi203.stdout:1/118: dread d14/f18 [0,4194304] 0 2024-05-25T06:11:21.981 INFO:tasks.workunit.client.1.smithi203.stdout:1/119: chown d14/d1b 3898 1 2024-05-25T06:11:21.982 INFO:tasks.workunit.client.1.smithi203.stdout:3/216: rename d10/d16/f47 to d10/d16/d23/d46/f4b 0 2024-05-25T06:11:21.983 INFO:tasks.workunit.client.1.smithi203.stdout:3/217: chown d10/d16/d23/l30 26439215 1 2024-05-25T06:11:21.984 INFO:tasks.workunit.client.1.smithi203.stdout:1/120: mknod d14/d1b/c28 0 2024-05-25T06:11:21.985 INFO:tasks.workunit.client.1.smithi203.stdout:5/339: rename f8 to dc/d11/d17/d1a/d2d/f7d 0 2024-05-25T06:11:21.986 INFO:tasks.workunit.client.1.smithi203.stdout:5/340: fsync dc/d11/f29 0 2024-05-25T06:11:21.988 INFO:tasks.workunit.client.1.smithi203.stdout:1/121: mkdir d14/d1b/d29 0 2024-05-25T06:11:21.988 INFO:tasks.workunit.client.1.smithi203.stdout:9/146: link d0/d17/d20/l2a d0/df/l35 0 2024-05-25T06:11:21.989 INFO:tasks.workunit.client.1.smithi203.stdout:5/341: symlink dc/d11/d17/d1a/d2d/d4d/d64/l7e 0 2024-05-25T06:11:21.991 INFO:tasks.workunit.client.1.smithi203.stdout:1/122: creat d14/d1b/d1d/f2a x:0 0 0 2024-05-25T06:11:21.992 INFO:tasks.workunit.client.1.smithi203.stdout:1/123: write f10 [346456,71434] 0 2024-05-25T06:11:21.993 INFO:tasks.workunit.client.1.smithi203.stdout:1/124: read d14/f18 [356568,25144] 0 2024-05-25T06:11:21.993 INFO:tasks.workunit.client.1.smithi203.stdout:5/342: getdents dc/d11/d17/d44/d73 0 2024-05-25T06:11:21.995 INFO:tasks.workunit.client.1.smithi203.stdout:1/125: write d14/d1b/d1d/f2a [423430,18220] 0 2024-05-25T06:11:21.995 INFO:tasks.workunit.client.1.smithi203.stdout:2/221: sync 2024-05-25T06:11:21.996 INFO:tasks.workunit.client.1.smithi203.stdout:9/147: getdents d0/d17/d20 0 2024-05-25T06:11:21.998 INFO:tasks.workunit.client.1.smithi203.stdout:1/126: chown d14/d1b/d1d/l1e 0 1 2024-05-25T06:11:21.998 INFO:tasks.workunit.client.1.smithi203.stdout:1/127: readlink d14/l21 0 2024-05-25T06:11:21.999 INFO:tasks.workunit.client.1.smithi203.stdout:2/222: symlink d2/d1a/l59 0 2024-05-25T06:11:22.001 INFO:tasks.workunit.client.1.smithi203.stdout:1/128: write d14/f24 [2597003,78215] 0 2024-05-25T06:11:22.003 INFO:tasks.workunit.client.1.smithi203.stdout:2/223: symlink d2/d1a/l5a 0 2024-05-25T06:11:22.004 INFO:tasks.workunit.client.1.smithi203.stdout:5/343: dwrite dc/d11/d17/d1a/d22/f24 [0,4194304] 0 2024-05-25T06:11:22.006 INFO:tasks.workunit.client.1.smithi203.stdout:9/148: rmdir d0/df 39 2024-05-25T06:11:22.006 INFO:tasks.workunit.client.1.smithi203.stdout:1/129: creat d14/d1b/d29/f2b x:0 0 0 2024-05-25T06:11:22.009 INFO:tasks.workunit.client.1.smithi203.stdout:5/344: symlink dc/d11/d6b/l7f 0 2024-05-25T06:11:22.009 INFO:tasks.workunit.client.1.smithi203.stdout:2/224: creat d2/d4/d10/d32/d36/f5b x:0 0 0 2024-05-25T06:11:22.011 INFO:tasks.workunit.client.1.smithi203.stdout:3/218: dread f8 [4194304,4194304] 0 2024-05-25T06:11:22.012 INFO:tasks.workunit.client.1.smithi203.stdout:2/225: getdents d2/d42/d4e 0 2024-05-25T06:11:22.014 INFO:tasks.workunit.client.1.smithi203.stdout:9/149: creat d0/f36 x:0 0 0 2024-05-25T06:11:22.014 INFO:tasks.workunit.client.1.smithi203.stdout:3/219: creat d10/d22/f4c x:0 0 0 2024-05-25T06:11:22.015 INFO:tasks.workunit.client.1.smithi203.stdout:3/220: fdatasync d10/f32 0 2024-05-25T06:11:22.015 INFO:tasks.workunit.client.1.smithi203.stdout:2/226: mknod d2/d4/d10/d50/c5c 0 2024-05-25T06:11:22.018 INFO:tasks.workunit.client.1.smithi203.stdout:3/221: rename f5 to d10/d22/d2a/f4d 0 2024-05-25T06:11:22.023 INFO:tasks.workunit.client.1.smithi203.stdout:2/227: dwrite d2/d4/d10/d32/f44 [0,4194304] 0 2024-05-25T06:11:22.023 INFO:tasks.workunit.client.1.smithi203.stdout:3/222: creat d10/d16/d23/d45/f4e x:0 0 0 2024-05-25T06:11:22.026 INFO:tasks.workunit.client.1.smithi203.stdout:2/228: truncate d2/d4/d10/d32/f28 4451861 0 2024-05-25T06:11:22.026 INFO:tasks.workunit.client.1.smithi203.stdout:9/150: write d0/f19 [1567194,113669] 0 2024-05-25T06:11:22.026 INFO:tasks.workunit.client.1.smithi203.stdout:2/229: readlink d2/l56 0 2024-05-25T06:11:22.027 INFO:tasks.workunit.client.1.smithi203.stdout:3/223: creat d10/f4f x:0 0 0 2024-05-25T06:11:22.027 INFO:tasks.workunit.client.1.smithi203.stdout:2/230: chown d2/d4/d10/d50/f51 2486583 1 2024-05-25T06:11:22.028 INFO:tasks.workunit.client.1.smithi203.stdout:2/231: fdatasync d2/d38/d2f/f4f 0 2024-05-25T06:11:22.029 INFO:tasks.workunit.client.1.smithi203.stdout:2/232: write d2/d6/f2d [7148841,32186] 0 2024-05-25T06:11:22.032 INFO:tasks.workunit.client.1.smithi203.stdout:3/224: dread d10/d16/f17 [0,4194304] 0 2024-05-25T06:11:22.033 INFO:tasks.workunit.client.1.smithi203.stdout:9/151: mkdir d0/df/d37 0 2024-05-25T06:11:22.034 INFO:tasks.workunit.client.1.smithi203.stdout:2/233: mknod d2/d4/d10/d32/c5d 0 2024-05-25T06:11:22.039 INFO:tasks.workunit.client.1.smithi203.stdout:9/152: mknod d0/d17/d20/d29/c38 0 2024-05-25T06:11:22.044 INFO:tasks.workunit.client.1.smithi203.stdout:3/225: dwrite d10/d16/f19 [0,4194304] 0 2024-05-25T06:11:22.044 INFO:tasks.workunit.client.1.smithi203.stdout:9/153: dread d0/f1e [0,4194304] 0 2024-05-25T06:11:22.045 INFO:tasks.workunit.client.1.smithi203.stdout:2/234: dwrite d2/d6/f26 [4194304,4194304] 0 2024-05-25T06:11:22.046 INFO:tasks.workunit.client.1.smithi203.stdout:9/154: fdatasync d0/df/d2b/f34 0 2024-05-25T06:11:22.049 INFO:tasks.workunit.client.1.smithi203.stdout:9/155: write d0/f36 [740237,5903] 0 2024-05-25T06:11:22.051 INFO:tasks.workunit.client.1.smithi203.stdout:2/235: rmdir d2/d38/d43 39 2024-05-25T06:11:22.051 INFO:tasks.workunit.client.1.smithi203.stdout:3/226: mkdir d10/d16/d50 0 2024-05-25T06:11:22.053 INFO:tasks.workunit.client.1.smithi203.stdout:2/236: creat d2/d4/d10/f5e x:0 0 0 2024-05-25T06:11:22.059 INFO:tasks.workunit.client.1.smithi203.stdout:3/227: creat d10/f51 x:0 0 0 2024-05-25T06:11:22.060 INFO:tasks.workunit.client.1.smithi203.stdout:3/228: readlink d10/d16/l43 0 2024-05-25T06:11:22.062 INFO:tasks.workunit.client.1.smithi203.stdout:9/156: truncate d0/f24 4086855 0 2024-05-25T06:11:22.064 INFO:tasks.workunit.client.1.smithi203.stdout:3/229: creat d10/d16/d23/d3a/d48/f52 x:0 0 0 2024-05-25T06:11:22.064 INFO:tasks.workunit.client.1.smithi203.stdout:3/230: chown l9 0 1 2024-05-25T06:11:22.065 INFO:tasks.workunit.client.1.smithi203.stdout:9/157: creat d0/f39 x:0 0 0 2024-05-25T06:11:22.067 INFO:tasks.workunit.client.1.smithi203.stdout:3/231: creat d10/d16/d23/d3a/f53 x:0 0 0 2024-05-25T06:11:22.067 INFO:tasks.workunit.client.1.smithi203.stdout:3/232: readlink l3 0 2024-05-25T06:11:22.070 INFO:tasks.workunit.client.1.smithi203.stdout:9/158: creat d0/d17/d20/f3a x:0 0 0 2024-05-25T06:11:22.074 INFO:tasks.workunit.client.1.smithi203.stdout:9/159: getdents d0/d17 0 2024-05-25T06:11:22.075 INFO:tasks.workunit.client.1.smithi203.stdout:3/233: dwrite d10/d16/d23/d3a/f53 [0,4194304] 0 2024-05-25T06:11:22.081 INFO:tasks.workunit.client.1.smithi203.stdout:9/160: symlink d0/df/d37/l3b 0 2024-05-25T06:11:22.083 INFO:tasks.workunit.client.1.smithi203.stdout:9/161: chown d0/c10 158128 1 2024-05-25T06:11:22.084 INFO:tasks.workunit.client.1.smithi203.stdout:3/234: dread d10/f32 [0,4194304] 0 2024-05-25T06:11:22.084 INFO:tasks.workunit.client.1.smithi203.stdout:3/235: chown d10 34 1 2024-05-25T06:11:22.086 INFO:tasks.workunit.client.1.smithi203.stdout:9/162: mkdir d0/d17/d20/d3c 0 2024-05-25T06:11:22.086 INFO:tasks.workunit.client.1.smithi203.stdout:9/163: read - d0/d17/d20/d29/f32 zero size 2024-05-25T06:11:22.089 INFO:tasks.workunit.client.1.smithi203.stdout:9/164: readlink d0/df/d2d/l30 0 2024-05-25T06:11:22.091 INFO:tasks.workunit.client.1.smithi203.stdout:9/165: truncate d0/d17/f33 785643 0 2024-05-25T06:11:22.091 INFO:tasks.workunit.client.1.smithi203.stdout:3/236: dwrite d10/f1a [0,4194304] 0 2024-05-25T06:11:22.093 INFO:tasks.workunit.client.1.smithi203.stdout:3/237: truncate d10/f51 424563 0 2024-05-25T06:11:22.094 INFO:tasks.workunit.client.1.smithi203.stdout:9/166: write d0/f13 [3826505,49374] 0 2024-05-25T06:11:22.103 INFO:tasks.workunit.client.1.smithi203.stdout:3/238: dwrite d10/f20 [0,4194304] 0 2024-05-25T06:11:22.105 INFO:tasks.workunit.client.1.smithi203.stdout:9/167: dwrite d0/df/d2b/f2f [0,4194304] 0 2024-05-25T06:11:22.108 INFO:tasks.workunit.client.1.smithi203.stdout:9/168: truncate d0/df/f14 1428308 0 2024-05-25T06:11:22.109 INFO:tasks.workunit.client.1.smithi203.stdout:3/239: write d10/d22/d2a/f4d [1270639,81778] 0 2024-05-25T06:11:22.112 INFO:tasks.workunit.client.1.smithi203.stdout:9/169: symlink d0/d17/d20/d3c/l3d 0 2024-05-25T06:11:22.115 INFO:tasks.workunit.client.1.smithi203.stdout:9/170: symlink d0/d17/d20/d3c/l3e 0 2024-05-25T06:11:22.120 INFO:tasks.workunit.client.1.smithi203.stdout:9/171: fsync d0/f1e 0 2024-05-25T06:11:22.126 INFO:tasks.workunit.client.1.smithi203.stdout:9/172: dwrite d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:21 smithi203 ceph-mon[36030]: pgmap v11: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 290 op/s 2024-05-25T06:11:22.177 INFO:tasks.workunit.client.1.smithi203.stdout:6/190: sync 2024-05-25T06:11:22.179 INFO:tasks.workunit.client.1.smithi203.stdout:6/191: creat d8/d3d/f44 x:0 0 0 2024-05-25T06:11:22.180 INFO:tasks.workunit.client.1.smithi203.stdout:6/192: write d8/d11/d15/f42 [5303,92622] 0 2024-05-25T06:11:22.181 INFO:tasks.workunit.client.1.smithi203.stdout:6/193: write d8/d11/f23 [226155,99009] 0 2024-05-25T06:11:22.183 INFO:tasks.workunit.client.1.smithi203.stdout:6/194: truncate d8/d11/f23 795567 0 2024-05-25T06:11:22.184 INFO:tasks.workunit.client.1.smithi203.stdout:6/195: rmdir d8/d11 39 2024-05-25T06:11:22.186 INFO:tasks.workunit.client.1.smithi203.stdout:6/196: mknod d8/d29/d3e/c45 0 2024-05-25T06:11:22.186 INFO:tasks.workunit.client.1.smithi203.stdout:6/197: chown d8/d29/c3f 25 1 2024-05-25T06:11:22.188 INFO:tasks.workunit.client.1.smithi203.stdout:6/198: write d8/d11/d15/f18 [148028,6715] 0 2024-05-25T06:11:22.189 INFO:tasks.workunit.client.1.smithi203.stdout:6/199: chown d8/d29/d3e/c45 501793 1 2024-05-25T06:11:22.194 INFO:tasks.workunit.client.1.smithi203.stdout:6/200: dwrite d8/d11/f23 [0,4194304] 0 2024-05-25T06:11:22.196 INFO:tasks.workunit.client.1.smithi203.stdout:6/201: write d8/d11/d15/f42 [355968,13247] 0 2024-05-25T06:11:22.200 INFO:tasks.workunit.client.1.smithi203.stdout:6/202: dread d8/d11/d15/f35 [0,4194304] 0 2024-05-25T06:11:22.217 INFO:tasks.workunit.client.1.smithi203.stdout:6/203: dread f0 [0,4194304] 0 2024-05-25T06:11:22.219 INFO:tasks.workunit.client.1.smithi203.stdout:6/204: mkdir d8/d29/d46 0 2024-05-25T06:11:22.223 INFO:tasks.workunit.client.1.smithi203.stdout:6/205: dread d8/d29/d3e/d43/f32 [0,4194304] 0 2024-05-25T06:11:22.225 INFO:tasks.workunit.client.1.smithi203.stdout:6/206: mkdir d8/d11/d47 0 2024-05-25T06:11:22.226 INFO:tasks.workunit.client.1.smithi203.stdout:6/207: read d8/fb [749300,99792] 0 2024-05-25T06:11:22.227 INFO:tasks.workunit.client.1.smithi203.stdout:6/208: read f0 [7333170,98445] 0 2024-05-25T06:11:22.228 INFO:tasks.workunit.client.1.smithi203.stdout:6/209: write d8/fd [7957810,120713] 0 2024-05-25T06:11:22.367 INFO:tasks.workunit.client.1.smithi203.stdout:7/168: sync 2024-05-25T06:11:22.367 INFO:tasks.workunit.client.1.smithi203.stdout:0/161: sync 2024-05-25T06:11:22.367 INFO:tasks.workunit.client.1.smithi203.stdout:0/162: symlink dd/d10/l3f 0 2024-05-25T06:11:22.369 INFO:tasks.workunit.client.1.smithi203.stdout:0/163: rmdir dd/d3a 39 2024-05-25T06:11:22.369 INFO:tasks.workunit.client.1.smithi203.stdout:0/164: readlink la 0 2024-05-25T06:11:22.376 INFO:tasks.workunit.client.1.smithi203.stdout:0/165: truncate dd/d10/f33 670944 0 2024-05-25T06:11:22.378 INFO:tasks.workunit.client.1.smithi203.stdout:0/166: creat dd/d32/f40 x:0 0 0 2024-05-25T06:11:22.379 INFO:tasks.workunit.client.1.smithi203.stdout:0/167: readlink dd/d10/l22 0 2024-05-25T06:11:22.380 INFO:tasks.workunit.client.1.smithi203.stdout:7/169: dwrite f0 [4194304,4194304] 0 2024-05-25T06:11:22.382 INFO:tasks.workunit.client.1.smithi203.stdout:7/170: read d1/d11/f21 [363614,40843] 0 2024-05-25T06:11:22.384 INFO:tasks.workunit.client.1.smithi203.stdout:0/168: unlink dd/d10/f2d 0 2024-05-25T06:11:22.385 INFO:tasks.workunit.client.1.smithi203.stdout:0/169: truncate dd/fe 226146 0 2024-05-25T06:11:22.387 INFO:tasks.workunit.client.1.smithi203.stdout:7/171: dread d1/d11/f2b [0,4194304] 0 2024-05-25T06:11:22.391 INFO:tasks.workunit.client.1.smithi203.stdout:7/172: link d1/d9/fd d1/d9/d2c/f42 0 2024-05-25T06:11:22.392 INFO:tasks.workunit.client.1.smithi203.stdout:0/170: dwrite dd/f11 [0,4194304] 0 2024-05-25T06:11:22.394 INFO:tasks.workunit.client.1.smithi203.stdout:7/173: truncate d1/d9/f18 112813 0 2024-05-25T06:11:22.396 INFO:tasks.workunit.client.1.smithi203.stdout:0/171: rename dd/f11 to dd/d10/f41 0 2024-05-25T06:11:22.400 INFO:tasks.workunit.client.1.smithi203.stdout:0/172: creat dd/d32/d37/f42 x:0 0 0 2024-05-25T06:11:22.407 INFO:tasks.workunit.client.1.smithi203.stdout:7/174: dread d1/f1c [0,4194304] 0 2024-05-25T06:11:22.407 INFO:tasks.workunit.client.1.smithi203.stdout:7/175: chown d1/d9/f18 9576514 1 2024-05-25T06:11:22.410 INFO:tasks.workunit.client.1.smithi203.stdout:7/176: mkdir d1/d9/d43 0 2024-05-25T06:11:22.412 INFO:tasks.workunit.client.1.smithi203.stdout:7/177: chown d1/d11/d20/c22 1359 1 2024-05-25T06:11:22.412 INFO:tasks.workunit.client.1.smithi203.stdout:0/173: chown dd/f15 10692392 1 2024-05-25T06:11:22.415 INFO:tasks.workunit.client.1.smithi203.stdout:7/178: dread d1/f32 [0,4194304] 0 2024-05-25T06:11:22.415 INFO:tasks.workunit.client.1.smithi203.stdout:0/174: symlink dd/l43 0 2024-05-25T06:11:22.416 INFO:tasks.workunit.client.1.smithi203.stdout:7/179: write d1/d9/fc [913477,124022] 0 2024-05-25T06:11:22.421 INFO:tasks.workunit.client.1.smithi203.stdout:7/180: dread d1/d9/d2c/f42 [0,4194304] 0 2024-05-25T06:11:22.421 INFO:tasks.workunit.client.1.smithi203.stdout:0/175: dwrite dd/f23 [4194304,4194304] 0 2024-05-25T06:11:22.422 INFO:tasks.workunit.client.1.smithi203.stdout:0/176: stat la 0 2024-05-25T06:11:22.426 INFO:tasks.workunit.client.1.smithi203.stdout:7/181: rmdir d1/d9 39 2024-05-25T06:11:22.426 INFO:tasks.workunit.client.1.smithi203.stdout:0/177: unlink dd/d10/l12 0 2024-05-25T06:11:22.429 INFO:tasks.workunit.client.1.smithi203.stdout:7/182: getdents d1/d11 0 2024-05-25T06:11:22.432 INFO:tasks.workunit.client.1.smithi203.stdout:7/183: chown d1/c12 1416559803 1 2024-05-25T06:11:22.435 INFO:tasks.workunit.client.1.smithi203.stdout:7/184: rename d1/d9/fd to d1/d9/d2c/d3a/f44 0 2024-05-25T06:11:22.436 INFO:tasks.workunit.client.1.smithi203.stdout:7/185: write d1/d9/fc [1612068,35621] 0 2024-05-25T06:11:22.438 INFO:tasks.workunit.client.1.smithi203.stdout:7/186: creat d1/f45 x:0 0 0 2024-05-25T06:11:22.439 INFO:tasks.workunit.client.1.smithi203.stdout:7/187: dread - d1/d11/f30 zero size 2024-05-25T06:11:22.439 INFO:tasks.workunit.client.1.smithi203.stdout:7/188: write d1/f45 [224947,53556] 0 2024-05-25T06:11:22.442 INFO:tasks.workunit.client.1.smithi203.stdout:7/189: fdatasync d1/d9/d2c/f42 0 2024-05-25T06:11:22.443 INFO:tasks.workunit.client.1.smithi203.stdout:7/190: chown d1/l39 758579 1 2024-05-25T06:11:22.444 INFO:tasks.workunit.client.1.smithi203.stdout:7/191: fdatasync d1/f31 0 2024-05-25T06:11:22.449 INFO:tasks.workunit.client.1.smithi203.stdout:7/192: dread d1/f24 [0,4194304] 0 2024-05-25T06:11:22.450 INFO:tasks.workunit.client.1.smithi203.stdout:7/193: creat d1/d33/f46 x:0 0 0 2024-05-25T06:11:22.451 INFO:tasks.workunit.client.1.smithi203.stdout:7/194: fsync d1/d11/d20/f27 0 2024-05-25T06:11:22.451 INFO:tasks.workunit.client.1.smithi203.stdout:7/195: dread - d1/d9/d2c/d3a/f3e zero size 2024-05-25T06:11:22.452 INFO:tasks.workunit.client.1.smithi203.stdout:7/196: stat d1/d9/f29 0 2024-05-25T06:11:22.454 INFO:tasks.workunit.client.1.smithi203.stdout:7/197: link d1/d9/f18 d1/d9/d2c/d36/d3b/f47 0 2024-05-25T06:11:22.455 INFO:tasks.workunit.client.1.smithi203.stdout:7/198: stat d1/d9/f18 0 2024-05-25T06:11:22.455 INFO:tasks.workunit.client.1.smithi203.stdout:7/199: fsync d1/d11/d20/f23 0 2024-05-25T06:11:22.456 INFO:tasks.workunit.client.1.smithi203.stdout:7/200: chown d1/d9/c3f 224 1 2024-05-25T06:11:22.458 INFO:tasks.workunit.client.1.smithi203.stdout:7/201: getdents d1/d11/d3d 0 2024-05-25T06:11:22.460 INFO:tasks.workunit.client.1.smithi203.stdout:7/202: creat d1/d9/f48 x:0 0 0 2024-05-25T06:11:22.461 INFO:tasks.workunit.client.1.smithi203.stdout:7/203: unlink d1/f45 0 2024-05-25T06:11:22.463 INFO:tasks.workunit.client.1.smithi203.stdout:7/204: creat d1/d9/d2c/d3a/f49 x:0 0 0 2024-05-25T06:11:22.465 INFO:tasks.workunit.client.1.smithi203.stdout:7/205: symlink d1/d9/d2c/d36/d3b/l4a 0 2024-05-25T06:11:22.465 INFO:tasks.workunit.client.1.smithi203.stdout:7/206: dread - d1/d11/d20/f27 zero size 2024-05-25T06:11:22.466 INFO:tasks.workunit.client.1.smithi203.stdout:7/207: creat d1/d9/d2c/d3a/f4b x:0 0 0 2024-05-25T06:11:22.469 INFO:tasks.workunit.client.1.smithi203.stdout:7/208: rmdir d1/d9/d43 0 2024-05-25T06:11:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:21 smithi160 ceph-mon[29041]: pgmap v11: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 290 op/s 2024-05-25T06:11:22.470 INFO:tasks.workunit.client.1.smithi203.stdout:7/209: dread - d1/d11/f41 zero size 2024-05-25T06:11:22.471 INFO:tasks.workunit.client.1.smithi203.stdout:7/210: mkdir d1/d11/d3d/d4c 0 2024-05-25T06:11:22.471 INFO:tasks.workunit.client.1.smithi203.stdout:7/211: chown d1/d11/f30 106053 1 2024-05-25T06:11:22.472 INFO:tasks.workunit.client.1.smithi203.stdout:7/212: chown d1/d9/d2c 35 1 2024-05-25T06:11:22.477 INFO:tasks.workunit.client.1.smithi203.stdout:7/213: dread d1/d9/fc [0,4194304] 0 2024-05-25T06:11:22.479 INFO:tasks.workunit.client.1.smithi203.stdout:7/214: creat d1/f4d x:0 0 0 2024-05-25T06:11:22.484 INFO:tasks.workunit.client.1.smithi203.stdout:7/215: dwrite d1/d11/d20/f28 [0,4194304] 0 2024-05-25T06:11:22.488 INFO:tasks.workunit.client.1.smithi203.stdout:7/216: dread d1/f1c [0,4194304] 0 2024-05-25T06:11:22.492 INFO:tasks.workunit.client.1.smithi203.stdout:7/217: rename d1/d33 to d1/d9/d2c/d3a/d4e 0 2024-05-25T06:11:22.493 INFO:tasks.workunit.client.1.smithi203.stdout:7/218: truncate d1/d9/d2c/d3a/f49 222997 0 2024-05-25T06:11:22.493 INFO:tasks.workunit.client.1.smithi203.stdout:7/219: readlink d1/l39 0 2024-05-25T06:11:22.493 INFO:tasks.workunit.client.1.smithi203.stdout:7/220: chown d1/fa 426 1 2024-05-25T06:11:22.495 INFO:tasks.workunit.client.1.smithi203.stdout:7/221: creat d1/d9/d2c/d3a/f4f x:0 0 0 2024-05-25T06:11:22.496 INFO:tasks.workunit.client.1.smithi203.stdout:7/222: mkdir d1/d9/d2c/d3a/d4e/d50 0 2024-05-25T06:11:22.497 INFO:tasks.workunit.client.1.smithi203.stdout:7/223: chown d1/d11/d3d 0 1 2024-05-25T06:11:22.497 INFO:tasks.workunit.client.1.smithi203.stdout:7/224: write d1/f3 [3794222,15227] 0 2024-05-25T06:11:22.498 INFO:tasks.workunit.client.1.smithi203.stdout:7/225: write d1/d9/f29 [828540,39223] 0 2024-05-25T06:11:22.500 INFO:tasks.workunit.client.1.smithi203.stdout:7/226: unlink d1/d11/d20/c22 0 2024-05-25T06:11:22.502 INFO:tasks.workunit.client.1.smithi203.stdout:7/227: symlink d1/l51 0 2024-05-25T06:11:22.508 INFO:tasks.workunit.client.1.smithi203.stdout:7/228: write d1/d9/d2c/d3a/f44 [3963861,41493] 0 2024-05-25T06:11:22.510 INFO:tasks.workunit.client.1.smithi203.stdout:7/229: symlink d1/l52 0 2024-05-25T06:11:22.513 INFO:tasks.workunit.client.1.smithi203.stdout:7/230: rename d1/l39 to d1/d11/d3d/d4c/l53 0 2024-05-25T06:11:22.514 INFO:tasks.workunit.client.1.smithi203.stdout:7/231: rmdir d1/d11/d20 39 2024-05-25T06:11:22.515 INFO:tasks.workunit.client.1.smithi203.stdout:7/232: read d1/d9/fb [2624643,68823] 0 2024-05-25T06:11:22.517 INFO:tasks.workunit.client.1.smithi203.stdout:7/233: mkdir d1/d9/d2c/d36/d54 0 2024-05-25T06:11:22.521 INFO:tasks.workunit.client.1.smithi203.stdout:7/234: write d1/f2 [1079146,107653] 0 2024-05-25T06:11:22.558 INFO:tasks.workunit.client.1.smithi203.stdout:8/186: sync 2024-05-25T06:11:22.570 INFO:tasks.workunit.client.1.smithi203.stdout:8/187: mkdir d2/d5/d10/d45 0 2024-05-25T06:11:22.572 INFO:tasks.workunit.client.1.smithi203.stdout:7/235: dwrite d1/d11/f2b [0,4194304] 0 2024-05-25T06:11:22.572 INFO:tasks.workunit.client.1.smithi203.stdout:8/188: write d2/d5/f29 [779623,42870] 0 2024-05-25T06:11:22.573 INFO:tasks.workunit.client.1.smithi203.stdout:8/189: fdatasync d2/d5/f44 0 2024-05-25T06:11:22.631 INFO:tasks.workunit.client.1.smithi203.stdout:8/190: symlink d2/d5/d28/l46 0 2024-05-25T06:11:22.632 INFO:tasks.workunit.client.1.smithi203.stdout:8/191: fdatasync d2/d5/d10/d11/de/f1d 0 2024-05-25T06:11:22.635 INFO:tasks.workunit.client.1.smithi203.stdout:7/236: dwrite d1/d9/d2c/d3a/f49 [0,4194304] 0 2024-05-25T06:11:22.635 INFO:tasks.workunit.client.1.smithi203.stdout:8/192: creat d2/d5/d28/d38/f47 x:0 0 0 2024-05-25T06:11:22.636 INFO:tasks.workunit.client.1.smithi203.stdout:7/237: chown d1/d11/f21 2 1 2024-05-25T06:11:22.639 INFO:tasks.workunit.client.1.smithi203.stdout:7/238: mknod d1/d9/d2c/d36/d54/c55 0 2024-05-25T06:11:22.639 INFO:tasks.workunit.client.1.smithi203.stdout:7/239: dread - d1/d11/f41 zero size 2024-05-25T06:11:22.640 INFO:tasks.workunit.client.1.smithi203.stdout:8/193: getdents d2/d5/d2d 0 2024-05-25T06:11:22.660 INFO:tasks.workunit.client.1.smithi203.stdout:7/240: dread d1/d9/fe [0,4194304] 0 2024-05-25T06:11:22.666 INFO:tasks.workunit.client.1.smithi203.stdout:7/241: dwrite d1/d11/d20/f27 [0,4194304] 0 2024-05-25T06:11:22.671 INFO:tasks.workunit.client.1.smithi203.stdout:7/242: link d1/d9/f2e d1/d9/d2c/f56 0 2024-05-25T06:11:22.673 INFO:tasks.workunit.client.1.smithi203.stdout:7/243: mknod d1/d9/d2c/d3a/d4e/c57 0 2024-05-25T06:11:22.674 INFO:tasks.workunit.client.1.smithi203.stdout:7/244: fsync d1/d9/d2c/d3a/f4f 0 2024-05-25T06:11:22.676 INFO:tasks.workunit.client.1.smithi203.stdout:7/245: symlink d1/d11/d3d/d4c/l58 0 2024-05-25T06:11:22.678 INFO:tasks.workunit.client.1.smithi203.stdout:7/246: mkdir d1/d9/d2c/d36/d3b/d59 0 2024-05-25T06:11:22.678 INFO:tasks.workunit.client.1.smithi203.stdout:7/247: write d1/f3c [232294,16235] 0 2024-05-25T06:11:22.680 INFO:tasks.workunit.client.1.smithi203.stdout:7/248: symlink d1/d9/d2c/d3a/d4e/d50/l5a 0 2024-05-25T06:11:22.680 INFO:tasks.workunit.client.1.smithi203.stdout:7/249: stat d1 0 2024-05-25T06:11:22.682 INFO:tasks.workunit.client.1.smithi203.stdout:7/250: creat d1/d9/d2c/d3a/d4e/f5b x:0 0 0 2024-05-25T06:11:22.684 INFO:tasks.workunit.client.1.smithi203.stdout:7/251: dread d1/f32 [0,4194304] 0 2024-05-25T06:11:22.692 INFO:tasks.workunit.client.1.smithi203.stdout:7/252: dwrite d1/d9/d2c/f56 [0,4194304] 0 2024-05-25T06:11:22.696 INFO:tasks.workunit.client.1.smithi203.stdout:7/253: getdents d1/d9/d2c/d36/d54 0 2024-05-25T06:11:22.700 INFO:tasks.workunit.client.1.smithi203.stdout:7/254: link d1/d9/d2c/d36/f40 d1/d9/d2c/d3a/f5c 0 2024-05-25T06:11:22.703 INFO:tasks.workunit.client.1.smithi203.stdout:7/255: rename d1/f34 to d1/d9/d2c/f5d 0 2024-05-25T06:11:22.704 INFO:tasks.workunit.client.1.smithi203.stdout:7/256: mknod d1/d9/d2c/d36/d3b/c5e 0 2024-05-25T06:11:22.706 INFO:tasks.workunit.client.1.smithi203.stdout:7/257: symlink d1/d11/l5f 0 2024-05-25T06:11:22.712 INFO:tasks.workunit.client.1.smithi203.stdout:7/258: dwrite d1/d9/d2c/d3a/d4e/f37 [0,4194304] 0 2024-05-25T06:11:22.739 INFO:tasks.workunit.client.1.smithi203.stdout:1/130: sync 2024-05-25T06:11:22.739 INFO:tasks.workunit.client.1.smithi203.stdout:5/345: sync 2024-05-25T06:11:22.739 INFO:tasks.workunit.client.1.smithi203.stdout:4/189: sync 2024-05-25T06:11:22.740 INFO:tasks.workunit.client.1.smithi203.stdout:5/346: write f1 [2257464,42293] 0 2024-05-25T06:11:22.740 INFO:tasks.workunit.client.1.smithi203.stdout:1/131: write f10 [9132462,104062] 0 2024-05-25T06:11:22.741 INFO:tasks.workunit.client.1.smithi203.stdout:4/190: chown d0/d8/lc 153102900 1 2024-05-25T06:11:22.742 INFO:tasks.workunit.client.1.smithi203.stdout:4/191: fsync d0/d8/d10/f15 0 2024-05-25T06:11:22.743 INFO:tasks.workunit.client.1.smithi203.stdout:4/192: dread - d0/d9/d24/f28 zero size 2024-05-25T06:11:22.745 INFO:tasks.workunit.client.1.smithi203.stdout:5/347: link dc/f21 dc/f80 0 2024-05-25T06:11:22.746 INFO:tasks.workunit.client.1.smithi203.stdout:4/193: mknod d0/d8/d1f/d26/c37 0 2024-05-25T06:11:22.748 INFO:tasks.workunit.client.1.smithi203.stdout:1/132: dwrite d14/d1b/d1d/f2a [0,4194304] 0 2024-05-25T06:11:22.751 INFO:tasks.workunit.client.1.smithi203.stdout:5/348: dread dc/d11/d18/f37 [0,4194304] 0 2024-05-25T06:11:22.754 INFO:tasks.workunit.client.1.smithi203.stdout:4/194: creat d0/f38 x:0 0 0 2024-05-25T06:11:22.757 INFO:tasks.workunit.client.1.smithi203.stdout:1/133: rmdir d14/d1b/d22 0 2024-05-25T06:11:22.760 INFO:tasks.workunit.client.1.smithi203.stdout:1/134: creat d14/d1b/d29/f2c x:0 0 0 2024-05-25T06:11:22.761 INFO:tasks.workunit.client.1.smithi203.stdout:5/349: dwrite dc/d11/d17/f31 [0,4194304] 0 2024-05-25T06:11:22.763 INFO:tasks.workunit.client.1.smithi203.stdout:5/350: readlink dc/d11/d17/d1a/d2d/d4d/d64/l6f 0 2024-05-25T06:11:22.767 INFO:tasks.workunit.client.1.smithi203.stdout:5/351: mkdir dc/d11/d17/d44/d73/d81 0 2024-05-25T06:11:22.768 INFO:tasks.workunit.client.1.smithi203.stdout:5/352: read - dc/f7c zero size 2024-05-25T06:11:22.769 INFO:tasks.workunit.client.1.smithi203.stdout:4/195: write d0/f16 [4474310,36631] 0 2024-05-25T06:11:22.776 INFO:tasks.workunit.client.1.smithi203.stdout:5/353: dwrite dc/d11/d17/d1a/d2d/d53/f55 [0,4194304] 0 2024-05-25T06:11:22.777 INFO:tasks.workunit.client.1.smithi203.stdout:5/354: fdatasync f1 0 2024-05-25T06:11:22.785 INFO:tasks.workunit.client.1.smithi203.stdout:5/355: getdents dc/d11/d17 0 2024-05-25T06:11:22.788 INFO:tasks.workunit.client.1.smithi203.stdout:1/135: dread f10 [0,4194304] 0 2024-05-25T06:11:22.790 INFO:tasks.workunit.client.1.smithi203.stdout:1/136: mknod d14/c2d 0 2024-05-25T06:11:22.792 INFO:tasks.workunit.client.1.smithi203.stdout:5/356: rename dc/d3c/c6e to dc/d11/d17/d44/d73/d81/c82 0 2024-05-25T06:11:22.794 INFO:tasks.workunit.client.1.smithi203.stdout:5/357: truncate dc/d11/d6b/f79 705035 0 2024-05-25T06:11:22.798 INFO:tasks.workunit.client.1.smithi203.stdout:4/196: dread d0/d8/d1f/d26/f1e [0,4194304] 0 2024-05-25T06:11:22.798 INFO:tasks.workunit.client.1.smithi203.stdout:1/137: dwrite d14/d1b/d29/f2c [0,4194304] 0 2024-05-25T06:11:22.798 INFO:tasks.workunit.client.1.smithi203.stdout:5/358: chown dc/d11/l33 420838 1 2024-05-25T06:11:22.800 INFO:tasks.workunit.client.1.smithi203.stdout:1/138: chown d14/f25 118101 1 2024-05-25T06:11:22.800 INFO:tasks.workunit.client.1.smithi203.stdout:4/197: write d0/d8/f23 [1305001,13273] 0 2024-05-25T06:11:22.802 INFO:tasks.workunit.client.1.smithi203.stdout:4/198: truncate d0/d8/d1f/d26/f20 755370 0 2024-05-25T06:11:22.803 INFO:tasks.workunit.client.1.smithi203.stdout:4/199: chown d0/d9/l2c 12510 1 2024-05-25T06:11:22.805 INFO:tasks.workunit.client.1.smithi203.stdout:5/359: rename dc/f7c to dc/d11/d17/d1a/d2d/d4d/f83 0 2024-05-25T06:11:22.805 INFO:tasks.workunit.client.1.smithi203.stdout:1/139: chown c11 0 1 2024-05-25T06:11:22.806 INFO:tasks.workunit.client.1.smithi203.stdout:5/360: dread - dc/d11/d17/d45/f62 zero size 2024-05-25T06:11:22.807 INFO:tasks.workunit.client.1.smithi203.stdout:5/361: fsync dc/d11/f50 0 2024-05-25T06:11:22.812 INFO:tasks.workunit.client.1.smithi203.stdout:5/362: mknod dc/d11/d17/d1a/d2d/d4d/d64/c84 0 2024-05-25T06:11:22.814 INFO:tasks.workunit.client.1.smithi203.stdout:1/140: dwrite d14/d1b/d1d/f2a [0,4194304] 0 2024-05-25T06:11:22.814 INFO:tasks.workunit.client.1.smithi203.stdout:4/200: dwrite d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:22.816 INFO:tasks.workunit.client.1.smithi203.stdout:1/141: write d14/f25 [3037633,49937] 0 2024-05-25T06:11:22.819 INFO:tasks.workunit.client.1.smithi203.stdout:5/363: symlink dc/d11/d17/l85 0 2024-05-25T06:11:22.820 INFO:tasks.workunit.client.1.smithi203.stdout:5/364: write dc/d11/d18/f37 [564352,124260] 0 2024-05-25T06:11:22.824 INFO:tasks.workunit.client.1.smithi203.stdout:1/142: unlink d14/d1b/d29/f2c 0 2024-05-25T06:11:22.825 INFO:tasks.workunit.client.1.smithi203.stdout:1/143: chown d14/d1b/d1d/f2a 3520 1 2024-05-25T06:11:22.830 INFO:tasks.workunit.client.1.smithi203.stdout:5/365: rename dc/d11/f50 to dc/d11/d17/d1a/d22/f86 0 2024-05-25T06:11:22.830 INFO:tasks.workunit.client.1.smithi203.stdout:4/201: write d0/f5 [2742863,109929] 0 2024-05-25T06:11:22.831 INFO:tasks.workunit.client.1.smithi203.stdout:1/144: symlink d14/d1b/d29/l2e 0 2024-05-25T06:11:22.831 INFO:tasks.workunit.client.1.smithi203.stdout:5/366: chown dc/d11/d17/l85 14962658 1 2024-05-25T06:11:22.832 INFO:tasks.workunit.client.1.smithi203.stdout:1/145: write d14/d1b/d29/f2b [730154,22947] 0 2024-05-25T06:11:22.834 INFO:tasks.workunit.client.1.smithi203.stdout:4/202: rmdir d0/d8 39 2024-05-25T06:11:22.839 INFO:tasks.workunit.client.1.smithi203.stdout:5/367: getdents dc/d11/d17/d44 0 2024-05-25T06:11:22.839 INFO:tasks.workunit.client.1.smithi203.stdout:3/240: sync 2024-05-25T06:11:22.840 INFO:tasks.workunit.client.1.smithi203.stdout:2/237: sync 2024-05-25T06:11:22.840 INFO:tasks.workunit.client.1.smithi203.stdout:9/173: sync 2024-05-25T06:11:22.840 INFO:tasks.workunit.client.1.smithi203.stdout:6/210: sync 2024-05-25T06:11:22.840 INFO:tasks.workunit.client.1.smithi203.stdout:4/203: getdents d0/d9/d24/d27 0 2024-05-25T06:11:22.841 INFO:tasks.workunit.client.1.smithi203.stdout:6/211: chown d8 14610 1 2024-05-25T06:11:22.843 INFO:tasks.workunit.client.1.smithi203.stdout:1/146: dwrite f10 [8388608,4194304] 0 2024-05-25T06:11:22.843 INFO:tasks.workunit.client.1.smithi203.stdout:9/174: dread - d0/d17/d20/f3a zero size 2024-05-25T06:11:22.843 INFO:tasks.workunit.client.1.smithi203.stdout:2/238: read d2/d6/f26 [5141514,30033] 0 2024-05-25T06:11:22.846 INFO:tasks.workunit.client.1.smithi203.stdout:5/368: write dc/d11/d17/d1a/d22/f24 [3885388,105433] 0 2024-05-25T06:11:22.851 INFO:tasks.workunit.client.1.smithi203.stdout:5/369: readlink dc/d11/d17/d1a/d2d/d4d/l6d 0 2024-05-25T06:11:22.851 INFO:tasks.workunit.client.1.smithi203.stdout:1/147: creat d14/d1b/d1d/f2f x:0 0 0 2024-05-25T06:11:22.853 INFO:tasks.workunit.client.1.smithi203.stdout:6/212: getdents d8/d11/d15 0 2024-05-25T06:11:22.854 INFO:tasks.workunit.client.1.smithi203.stdout:4/204: truncate d0/d8/d10/f15 1834449 0 2024-05-25T06:11:22.855 INFO:tasks.workunit.client.1.smithi203.stdout:1/148: dread f12 [0,4194304] 0 2024-05-25T06:11:22.857 INFO:tasks.workunit.client.1.smithi203.stdout:3/241: dwrite d10/d16/f40 [0,4194304] 0 2024-05-25T06:11:22.857 INFO:tasks.workunit.client.1.smithi203.stdout:6/213: write f2 [1269378,90905] 0 2024-05-25T06:11:22.857 INFO:tasks.workunit.client.1.smithi203.stdout:2/239: rename d2/d4/d10/d32/d36 to d2/d4/d10/d5f 0 2024-05-25T06:11:22.860 INFO:tasks.workunit.client.1.smithi203.stdout:4/205: mknod d0/d8/d1f/c39 0 2024-05-25T06:11:22.860 INFO:tasks.workunit.client.1.smithi203.stdout:6/214: stat d8/d11/d1b/c25 0 2024-05-25T06:11:22.860 INFO:tasks.workunit.client.1.smithi203.stdout:4/206: readlink d0/l2d 0 2024-05-25T06:11:22.865 INFO:tasks.workunit.client.1.smithi203.stdout:1/149: rmdir d14/d20 0 2024-05-25T06:11:22.870 INFO:tasks.workunit.client.1.smithi203.stdout:3/242: link l3 d10/d22/l54 0 2024-05-25T06:11:22.872 INFO:tasks.workunit.client.1.smithi203.stdout:5/370: dwrite dc/d11/d17/d45/f46 [0,4194304] 0 2024-05-25T06:11:22.873 INFO:tasks.workunit.client.1.smithi203.stdout:4/207: dread d0/d9/d24/d27/f30 [0,4194304] 0 2024-05-25T06:11:22.875 INFO:tasks.workunit.client.1.smithi203.stdout:6/215: getdents d8/d29/d30/d38 0 2024-05-25T06:11:22.876 INFO:tasks.workunit.client.1.smithi203.stdout:9/175: dread d0/f7 [0,4194304] 0 2024-05-25T06:11:22.878 INFO:tasks.workunit.client.1.smithi203.stdout:1/150: dwrite d14/d1b/d1d/f2a [0,4194304] 0 2024-05-25T06:11:22.879 INFO:tasks.workunit.client.1.smithi203.stdout:3/243: creat d10/d16/d23/d3a/f55 x:0 0 0 2024-05-25T06:11:22.879 INFO:tasks.workunit.client.1.smithi203.stdout:5/371: write dc/d11/d17/d1a/f2e [2955600,115801] 0 2024-05-25T06:11:22.881 INFO:tasks.workunit.client.1.smithi203.stdout:4/208: rmdir d0 39 2024-05-25T06:11:22.882 INFO:tasks.workunit.client.1.smithi203.stdout:1/151: write d14/f25 [4591533,14338] 0 2024-05-25T06:11:22.883 INFO:tasks.workunit.client.1.smithi203.stdout:0/178: sync 2024-05-25T06:11:22.883 INFO:tasks.workunit.client.1.smithi203.stdout:8/194: sync 2024-05-25T06:11:22.885 INFO:tasks.workunit.client.1.smithi203.stdout:6/216: mkdir d8/d11/d15/d28/d48 0 2024-05-25T06:11:22.892 INFO:tasks.workunit.client.1.smithi203.stdout:5/372: rename dc/d11/d18/c5e to dc/d11/d42/c87 0 2024-05-25T06:11:22.892 INFO:tasks.workunit.client.1.smithi203.stdout:9/176: dwrite d0/df/f1a [0,4194304] 0 2024-05-25T06:11:22.893 INFO:tasks.workunit.client.1.smithi203.stdout:1/152: creat d14/d1b/d1d/f30 x:0 0 0 2024-05-25T06:11:22.895 INFO:tasks.workunit.client.1.smithi203.stdout:8/195: truncate d2/d5/d28/d38/f3c 894156 0 2024-05-25T06:11:22.897 INFO:tasks.workunit.client.1.smithi203.stdout:9/177: mknod d0/df/c3f 0 2024-05-25T06:11:22.898 INFO:tasks.workunit.client.1.smithi203.stdout:2/240: dwrite d2/d38/f1d [0,4194304] 0 2024-05-25T06:11:22.899 INFO:tasks.workunit.client.1.smithi203.stdout:1/153: read - d14/d1b/d1d/f30 zero size 2024-05-25T06:11:22.901 INFO:tasks.workunit.client.1.smithi203.stdout:0/179: creat dd/f44 x:0 0 0 2024-05-25T06:11:22.901 INFO:tasks.workunit.client.1.smithi203.stdout:4/209: mkdir d0/d8/d3a 0 2024-05-25T06:11:22.901 INFO:tasks.workunit.client.1.smithi203.stdout:8/196: dread d2/f2a [0,4194304] 0 2024-05-25T06:11:22.902 INFO:tasks.workunit.client.1.smithi203.stdout:6/217: getdents d8/d29/d30 0 2024-05-25T06:11:22.903 INFO:tasks.workunit.client.1.smithi203.stdout:4/210: fdatasync d0/f16 0 2024-05-25T06:11:22.907 INFO:tasks.workunit.client.1.smithi203.stdout:9/178: unlink d0/d17/d23/f2e 0 2024-05-25T06:11:22.908 INFO:tasks.workunit.client.1.smithi203.stdout:1/154: mkdir d14/d1b/d1d/d31 0 2024-05-25T06:11:22.910 INFO:tasks.workunit.client.1.smithi203.stdout:1/155: dread d14/f18 [0,4194304] 0 2024-05-25T06:11:22.911 INFO:tasks.workunit.client.1.smithi203.stdout:0/180: dread dd/f15 [0,4194304] 0 2024-05-25T06:11:22.914 INFO:tasks.workunit.client.1.smithi203.stdout:2/241: mkdir d2/d38/d43/d60 0 2024-05-25T06:11:22.914 INFO:tasks.workunit.client.1.smithi203.stdout:3/244: truncate f8 5063409 0 2024-05-25T06:11:22.915 INFO:tasks.workunit.client.1.smithi203.stdout:9/179: dwrite d0/f39 [0,4194304] 0 2024-05-25T06:11:22.916 INFO:tasks.workunit.client.1.smithi203.stdout:2/242: chown d2/d4/d10/l54 0 1 2024-05-25T06:11:22.919 INFO:tasks.workunit.client.1.smithi203.stdout:8/197: unlink d2/d5/f3f 0 2024-05-25T06:11:22.924 INFO:tasks.workunit.client.1.smithi203.stdout:6/218: symlink d8/d11/d15/d28/d48/l49 0 2024-05-25T06:11:22.926 INFO:tasks.workunit.client.1.smithi203.stdout:8/198: write d2/f2a [874098,9615] 0 2024-05-25T06:11:22.928 INFO:tasks.workunit.client.1.smithi203.stdout:2/243: creat d2/d38/d2f/f61 x:0 0 0 2024-05-25T06:11:22.930 INFO:tasks.workunit.client.1.smithi203.stdout:6/219: rmdir d8/d11/d15 39 2024-05-25T06:11:22.933 INFO:tasks.workunit.client.1.smithi203.stdout:1/156: creat d14/f32 x:0 0 0 2024-05-25T06:11:22.935 INFO:tasks.workunit.client.1.smithi203.stdout:9/180: dwrite d0/d17/f33 [0,4194304] 0 2024-05-25T06:11:22.936 INFO:tasks.workunit.client.1.smithi203.stdout:8/199: mknod d2/d5/d28/c48 0 2024-05-25T06:11:22.936 INFO:tasks.workunit.client.1.smithi203.stdout:7/259: sync 2024-05-25T06:11:22.938 INFO:tasks.workunit.client.1.smithi203.stdout:2/244: write d2/d4/d10/f5e [405737,92082] 0 2024-05-25T06:11:22.940 INFO:tasks.workunit.client.1.smithi203.stdout:3/245: dwrite d10/f32 [0,4194304] 0 2024-05-25T06:11:22.945 INFO:tasks.workunit.client.1.smithi203.stdout:5/373: truncate dc/d11/d17/d45/f5f 3514442 0 2024-05-25T06:11:22.947 INFO:tasks.workunit.client.1.smithi203.stdout:7/260: write d1/f38 [1239297,103421] 0 2024-05-25T06:11:22.949 INFO:tasks.workunit.client.1.smithi203.stdout:4/211: dwrite d0/d9/d24/d27/f30 [0,4194304] 0 2024-05-25T06:11:22.949 INFO:tasks.workunit.client.1.smithi203.stdout:9/181: write d0/f19 [663664,6272] 0 2024-05-25T06:11:22.950 INFO:tasks.workunit.client.1.smithi203.stdout:6/220: truncate d8/d11/d15/f17 470723 0 2024-05-25T06:11:22.951 INFO:tasks.workunit.client.1.smithi203.stdout:1/157: symlink d14/l33 0 2024-05-25T06:11:22.952 INFO:tasks.workunit.client.1.smithi203.stdout:9/182: write d0/f13 [4004359,14791] 0 2024-05-25T06:11:22.954 INFO:tasks.workunit.client.1.smithi203.stdout:0/181: dread dd/d10/f29 [0,4194304] 0 2024-05-25T06:11:22.954 INFO:tasks.workunit.client.1.smithi203.stdout:5/374: symlink dc/d11/d18/l88 0 2024-05-25T06:11:22.955 INFO:tasks.workunit.client.1.smithi203.stdout:0/182: write dd/d10/f26 [529732,4260] 0 2024-05-25T06:11:22.956 INFO:tasks.workunit.client.1.smithi203.stdout:0/183: chown dd/d10/f29 924509 1 2024-05-25T06:11:22.957 INFO:tasks.workunit.client.1.smithi203.stdout:6/221: creat d8/d2a/f4a x:0 0 0 2024-05-25T06:11:22.957 INFO:tasks.workunit.client.1.smithi203.stdout:6/222: readlink d8/d11/d15/l33 0 2024-05-25T06:11:22.958 INFO:tasks.workunit.client.1.smithi203.stdout:3/246: rename d10/c1b to d10/d22/d2a/c56 0 2024-05-25T06:11:22.960 INFO:tasks.workunit.client.1.smithi203.stdout:1/158: mknod d14/d1b/c34 0 2024-05-25T06:11:22.961 INFO:tasks.workunit.client.1.smithi203.stdout:7/261: mkdir d1/d9/d60 0 2024-05-25T06:11:22.964 INFO:tasks.workunit.client.1.smithi203.stdout:7/262: read d1/f3c [119674,130605] 0 2024-05-25T06:11:22.966 INFO:tasks.workunit.client.1.smithi203.stdout:3/247: rename d10/d22/f4c to d10/d22/f57 0 2024-05-25T06:11:22.967 INFO:tasks.workunit.client.1.smithi203.stdout:6/223: dwrite d8/d11/f23 [0,4194304] 0 2024-05-25T06:11:22.967 INFO:tasks.workunit.client.1.smithi203.stdout:9/183: symlink d0/df/d2d/l40 0 2024-05-25T06:11:22.968 INFO:tasks.workunit.client.1.smithi203.stdout:3/248: write d10/f1a [4459001,110620] 0 2024-05-25T06:11:22.969 INFO:tasks.workunit.client.1.smithi203.stdout:8/200: truncate d2/d5/f8 3211407 0 2024-05-25T06:11:22.973 INFO:tasks.workunit.client.1.smithi203.stdout:0/184: mkdir dd/d45 0 2024-05-25T06:11:22.977 INFO:tasks.workunit.client.1.smithi203.stdout:8/201: dwrite d2/d5/f44 [0,4194304] 0 2024-05-25T06:11:22.978 INFO:tasks.workunit.client.1.smithi203.stdout:6/224: rmdir d8/d11 39 2024-05-25T06:11:22.979 INFO:tasks.workunit.client.1.smithi203.stdout:9/184: symlink d0/df/d2b/l41 0 2024-05-25T06:11:22.980 INFO:tasks.workunit.client.1.smithi203.stdout:8/202: stat d2/d5/d10/d11 0 2024-05-25T06:11:22.980 INFO:tasks.workunit.client.1.smithi203.stdout:9/185: stat d0/d17/d23/l28 0 2024-05-25T06:11:22.983 INFO:tasks.workunit.client.1.smithi203.stdout:2/245: rename d2/d1a/d33/c45 to d2/d38/c62 0 2024-05-25T06:11:22.985 INFO:tasks.workunit.client.1.smithi203.stdout:9/186: dread d0/f36 [0,4194304] 0 2024-05-25T06:11:22.989 INFO:tasks.workunit.client.1.smithi203.stdout:3/249: creat d10/d16/d23/d4a/f58 x:0 0 0 2024-05-25T06:11:22.989 INFO:tasks.workunit.client.1.smithi203.stdout:6/225: truncate d8/d11/d1b/f26 4311884 0 2024-05-25T06:11:22.989 INFO:tasks.workunit.client.1.smithi203.stdout:3/250: chown d10 881 1 2024-05-25T06:11:22.990 INFO:tasks.workunit.client.1.smithi203.stdout:5/375: link dc/d11/l33 dc/d11/d17/d1a/l89 0 2024-05-25T06:11:22.991 INFO:tasks.workunit.client.1.smithi203.stdout:6/226: write d8/d3d/f44 [287111,5930] 0 2024-05-25T06:11:22.991 INFO:tasks.workunit.client.1.smithi203.stdout:3/251: read d10/d16/f19 [1106399,45063] 0 2024-05-25T06:11:22.992 INFO:tasks.workunit.client.1.smithi203.stdout:8/203: creat d2/d5/d28/d38/f49 x:0 0 0 2024-05-25T06:11:22.995 INFO:tasks.workunit.client.1.smithi203.stdout:0/185: creat dd/d3a/f46 x:0 0 0 2024-05-25T06:11:22.995 INFO:tasks.workunit.client.1.smithi203.stdout:9/187: mknod d0/d17/d20/d29/c42 0 2024-05-25T06:11:22.997 INFO:tasks.workunit.client.1.smithi203.stdout:1/159: getdents d14/d1b/d29 0 2024-05-25T06:11:22.999 INFO:tasks.workunit.client.1.smithi203.stdout:3/252: unlink c2 0 2024-05-25T06:11:23.003 INFO:tasks.workunit.client.1.smithi203.stdout:5/376: dwrite dc/d11/d17/d1a/f27 [0,4194304] 0 2024-05-25T06:11:23.004 INFO:tasks.workunit.client.1.smithi203.stdout:0/186: creat dd/d10/d34/f47 x:0 0 0 2024-05-25T06:11:23.006 INFO:tasks.workunit.client.1.smithi203.stdout:1/160: unlink cd 0 2024-05-25T06:11:23.008 INFO:tasks.workunit.client.1.smithi203.stdout:2/246: dread d2/d4/d10/d32/f3d [0,4194304] 0 2024-05-25T06:11:23.009 INFO:tasks.workunit.client.1.smithi203.stdout:9/188: creat d0/d17/d20/d3c/f43 x:0 0 0 2024-05-25T06:11:23.011 INFO:tasks.workunit.client.1.smithi203.stdout:2/247: dread - d2/d4/d10/d50/f51 zero size 2024-05-25T06:11:23.011 INFO:tasks.workunit.client.1.smithi203.stdout:3/253: write d10/f31 [511578,95056] 0 2024-05-25T06:11:23.012 INFO:tasks.workunit.client.1.smithi203.stdout:9/189: dread - d0/d17/d20/f31 zero size 2024-05-25T06:11:23.013 INFO:tasks.workunit.client.1.smithi203.stdout:3/254: dread - d10/d22/f57 zero size 2024-05-25T06:11:23.014 INFO:tasks.workunit.client.1.smithi203.stdout:6/227: link d8/f9 d8/d29/f4b 0 2024-05-25T06:11:23.015 INFO:tasks.workunit.client.1.smithi203.stdout:5/377: chown dc/d11/d17/d44/d73/d81/c82 1 1 2024-05-25T06:11:23.015 INFO:tasks.workunit.client.1.smithi203.stdout:0/187: mkdir dd/d3d/d48 0 2024-05-25T06:11:23.015 INFO:tasks.workunit.client.1.smithi203.stdout:2/248: fsync d2/d38/d2f/f31 0 2024-05-25T06:11:23.017 INFO:tasks.workunit.client.1.smithi203.stdout:3/255: chown d10/d22/f57 422161586 1 2024-05-25T06:11:23.018 INFO:tasks.workunit.client.1.smithi203.stdout:0/188: stat dd/f15 0 2024-05-25T06:11:23.020 INFO:tasks.workunit.client.1.smithi203.stdout:5/378: chown dc/d11/d17/d1a/d2d/d53/f55 3223825 1 2024-05-25T06:11:23.020 INFO:tasks.workunit.client.1.smithi203.stdout:1/161: link d14/d1b/d1d/f2a d14/f35 0 2024-05-25T06:11:23.021 INFO:tasks.workunit.client.1.smithi203.stdout:2/249: symlink d2/d4/l63 0 2024-05-25T06:11:23.022 INFO:tasks.workunit.client.1.smithi203.stdout:0/189: mknod dd/d10/c49 0 2024-05-25T06:11:23.024 INFO:tasks.workunit.client.1.smithi203.stdout:1/162: dread - d14/d1b/d1d/f2f zero size 2024-05-25T06:11:23.025 INFO:tasks.workunit.client.1.smithi203.stdout:3/256: chown d10/d16/d23/l37 16485574 1 2024-05-25T06:11:23.026 INFO:tasks.workunit.client.1.smithi203.stdout:9/190: dread d0/f39 [0,4194304] 0 2024-05-25T06:11:23.028 INFO:tasks.workunit.client.1.smithi203.stdout:6/228: dwrite d8/fc [0,4194304] 0 2024-05-25T06:11:23.028 INFO:tasks.workunit.client.1.smithi203.stdout:0/190: write dd/d3a/f3b [564550,18591] 0 2024-05-25T06:11:23.031 INFO:tasks.workunit.client.1.smithi203.stdout:6/229: fdatasync d8/d3d/f44 0 2024-05-25T06:11:23.031 INFO:tasks.workunit.client.1.smithi203.stdout:3/257: fsync d10/d16/f17 0 2024-05-25T06:11:23.033 INFO:tasks.workunit.client.1.smithi203.stdout:9/191: creat d0/d17/d20/f44 x:0 0 0 2024-05-25T06:11:23.035 INFO:tasks.workunit.client.1.smithi203.stdout:1/163: mkdir d14/d1b/d1d/d31/d36 0 2024-05-25T06:11:23.035 INFO:tasks.workunit.client.1.smithi203.stdout:8/204: dwrite d2/f14 [0,4194304] 0 2024-05-25T06:11:23.036 INFO:tasks.workunit.client.1.smithi203.stdout:5/379: getdents dc/d11/d17/d44/d73 0 2024-05-25T06:11:23.037 INFO:tasks.workunit.client.1.smithi203.stdout:1/164: stat d14/f32 0 2024-05-25T06:11:23.038 INFO:tasks.workunit.client.1.smithi203.stdout:0/191: rmdir dd/d32/d37 39 2024-05-25T06:11:23.041 INFO:tasks.workunit.client.1.smithi203.stdout:5/380: stat dc/d11/d17/d1a/d22/l20 0 2024-05-25T06:11:23.042 INFO:tasks.workunit.client.1.smithi203.stdout:1/165: mkdir d14/d37 0 2024-05-25T06:11:23.044 INFO:tasks.workunit.client.1.smithi203.stdout:1/166: stat d14/d23/c26 0 2024-05-25T06:11:23.052 INFO:tasks.workunit.client.1.smithi203.stdout:5/381: rename dc/d11/d17/d1a/d2d to dc/d11/d17/d1a/d2d/d8a 22 2024-05-25T06:11:23.053 INFO:tasks.workunit.client.1.smithi203.stdout:6/230: dwrite d8/fd [4194304,4194304] 0 2024-05-25T06:11:23.054 INFO:tasks.workunit.client.1.smithi203.stdout:2/250: dwrite d2/d1a/f27 [0,4194304] 0 2024-05-25T06:11:23.055 INFO:tasks.workunit.client.1.smithi203.stdout:3/258: truncate d10/d16/d23/d3a/f53 850327 0 2024-05-25T06:11:23.060 INFO:tasks.workunit.client.1.smithi203.stdout:8/205: dwrite d2/d5/d10/d11/de/f16 [0,4194304] 0 2024-05-25T06:11:23.063 INFO:tasks.workunit.client.1.smithi203.stdout:0/192: dwrite f5 [0,4194304] 0 2024-05-25T06:11:23.067 INFO:tasks.workunit.client.1.smithi203.stdout:3/259: rename d10/d16/d23/d38/l39 to d10/d16/d23/d38/l59 0 2024-05-25T06:11:23.068 INFO:tasks.workunit.client.1.smithi203.stdout:6/231: creat d8/d29/d30/d38/f4c x:0 0 0 2024-05-25T06:11:23.068 INFO:tasks.workunit.client.1.smithi203.stdout:0/193: unlink dd/d10/l2e 0 2024-05-25T06:11:23.071 INFO:tasks.workunit.client.1.smithi203.stdout:2/251: link d2/d4/d10/d32/f52 d2/d4/d10/d58/f64 0 2024-05-25T06:11:23.074 INFO:tasks.workunit.client.1.smithi203.stdout:6/232: chown d8/d11/d15/f31 1597508 1 2024-05-25T06:11:23.078 INFO:tasks.workunit.client.1.smithi203.stdout:2/252: symlink d2/d4/d10/d32/l65 0 2024-05-25T06:11:23.082 INFO:tasks.workunit.client.1.smithi203.stdout:6/233: fsync d8/d11/f19 0 2024-05-25T06:11:23.083 INFO:tasks.workunit.client.1.smithi203.stdout:6/234: stat f2 0 2024-05-25T06:11:23.084 INFO:tasks.workunit.client.1.smithi203.stdout:8/206: dwrite d2/d5/d28/d38/f3c [0,4194304] 0 2024-05-25T06:11:23.086 INFO:tasks.workunit.client.1.smithi203.stdout:5/382: dwrite dc/d11/d17/d1a/f27 [4194304,4194304] 0 2024-05-25T06:11:23.087 INFO:tasks.workunit.client.1.smithi203.stdout:6/235: write f2 [236306,51054] 0 2024-05-25T06:11:23.090 INFO:tasks.workunit.client.1.smithi203.stdout:6/236: stat d8/d11/d15/f35 0 2024-05-25T06:11:23.090 INFO:tasks.workunit.client.1.smithi203.stdout:2/253: rename d2/d4/d10/l54 to d2/d6/l66 0 2024-05-25T06:11:23.091 INFO:tasks.workunit.client.1.smithi203.stdout:0/194: dwrite dd/d3a/f46 [0,4194304] 0 2024-05-25T06:11:23.096 INFO:tasks.workunit.client.1.smithi203.stdout:0/195: dread - dd/f44 zero size 2024-05-25T06:11:23.096 INFO:tasks.workunit.client.1.smithi203.stdout:0/196: chown dd/f23 17030308 1 2024-05-25T06:11:23.098 INFO:tasks.workunit.client.1.smithi203.stdout:3/260: dread d10/d16/d23/d3a/f53 [0,4194304] 0 2024-05-25T06:11:23.099 INFO:tasks.workunit.client.1.smithi203.stdout:8/207: write d2/d5/d10/d11/de/f1a [610909,101982] 0 2024-05-25T06:11:23.100 INFO:tasks.workunit.client.1.smithi203.stdout:8/208: write d2/d5/d2d/f3e [897050,23142] 0 2024-05-25T06:11:23.101 INFO:tasks.workunit.client.1.smithi203.stdout:8/209: readlink d2/l26 0 2024-05-25T06:11:23.103 INFO:tasks.workunit.client.1.smithi203.stdout:2/254: creat d2/d4/d10/d50/f67 x:0 0 0 2024-05-25T06:11:23.108 INFO:tasks.workunit.client.1.smithi203.stdout:5/383: fsync dc/d11/d17/d45/f5f 0 2024-05-25T06:11:23.109 INFO:tasks.workunit.client.1.smithi203.stdout:3/261: symlink d10/d16/d23/d3a/d48/l5a 0 2024-05-25T06:11:23.111 INFO:tasks.workunit.client.1.smithi203.stdout:0/197: dwrite dd/f1f [0,4194304] 0 2024-05-25T06:11:23.111 INFO:tasks.workunit.client.1.smithi203.stdout:8/210: mkdir d2/d5/d10/d4a 0 2024-05-25T06:11:23.117 INFO:tasks.workunit.client.1.smithi203.stdout:2/255: mkdir d2/d42/d4e/d68 0 2024-05-25T06:11:23.118 INFO:tasks.workunit.client.1.smithi203.stdout:8/211: truncate d2/d5/d18/d31/f3d 5160576 0 2024-05-25T06:11:23.120 INFO:tasks.workunit.client.1.smithi203.stdout:0/198: rename dd/d32/f40 to dd/d10/f4a 0 2024-05-25T06:11:23.120 INFO:tasks.workunit.client.1.smithi203.stdout:3/262: dwrite d10/f32 [0,4194304] 0 2024-05-25T06:11:23.123 INFO:tasks.workunit.client.1.smithi203.stdout:2/256: mkdir d2/d38/d2f/d69 0 2024-05-25T06:11:23.124 INFO:tasks.workunit.client.1.smithi203.stdout:0/199: creat dd/d3a/f4b x:0 0 0 2024-05-25T06:11:23.128 INFO:tasks.workunit.client.1.smithi203.stdout:8/212: truncate d2/d5/d10/d11/de/f1e 109997 0 2024-05-25T06:11:23.129 INFO:tasks.workunit.client.1.smithi203.stdout:6/237: dwrite d8/d29/f4b [0,4194304] 0 2024-05-25T06:11:23.129 INFO:tasks.workunit.client.1.smithi203.stdout:5/384: truncate dc/d11/f12 4170958 0 2024-05-25T06:11:23.131 INFO:tasks.workunit.client.1.smithi203.stdout:0/200: truncate dd/d10/f4a 617716 0 2024-05-25T06:11:23.133 INFO:tasks.workunit.client.1.smithi203.stdout:5/385: write dc/d11/d17/f38 [66722,126965] 0 2024-05-25T06:11:23.135 INFO:tasks.workunit.client.1.smithi203.stdout:0/201: write dd/d10/f26 [1087715,48370] 0 2024-05-25T06:11:23.135 INFO:tasks.workunit.client.1.smithi203.stdout:8/213: creat d2/d5/d2d/f4b x:0 0 0 2024-05-25T06:11:23.137 INFO:tasks.workunit.client.1.smithi203.stdout:0/202: chown dd/f23 3454 1 2024-05-25T06:11:23.138 INFO:tasks.workunit.client.1.smithi203.stdout:3/263: dwrite d10/d16/f40 [0,4194304] 0 2024-05-25T06:11:23.138 INFO:tasks.workunit.client.1.smithi203.stdout:6/238: creat d8/d11/d1b/f4d x:0 0 0 2024-05-25T06:11:23.139 INFO:tasks.workunit.client.1.smithi203.stdout:8/214: write d2/d5/d18/d31/f3d [949725,38913] 0 2024-05-25T06:11:23.141 INFO:tasks.workunit.client.1.smithi203.stdout:2/257: link d2/cd d2/d42/c6a 0 2024-05-25T06:11:23.142 INFO:tasks.workunit.client.1.smithi203.stdout:5/386: rmdir dc/d3c 39 2024-05-25T06:11:23.144 INFO:tasks.workunit.client.1.smithi203.stdout:6/239: rename d8/d11/d1b/c20 to d8/d29/d30/c4e 0 2024-05-25T06:11:23.145 INFO:tasks.workunit.client.1.smithi203.stdout:8/215: truncate d2/d5/d28/d38/f3c 5145275 0 2024-05-25T06:11:23.145 INFO:tasks.workunit.client.1.smithi203.stdout:3/264: truncate d10/f4f 98692 0 2024-05-25T06:11:23.149 INFO:tasks.workunit.client.1.smithi203.stdout:8/216: creat d2/d5/d28/f4c x:0 0 0 2024-05-25T06:11:23.150 INFO:tasks.workunit.client.1.smithi203.stdout:3/265: creat d10/d22/f5b x:0 0 0 2024-05-25T06:11:23.151 INFO:tasks.workunit.client.1.smithi203.stdout:6/240: truncate d8/d11/d1b/f26 4653261 0 2024-05-25T06:11:23.152 INFO:tasks.workunit.client.1.smithi203.stdout:0/203: dwrite dd/d10/f26 [0,4194304] 0 2024-05-25T06:11:23.153 INFO:tasks.workunit.client.1.smithi203.stdout:3/266: write d10/d22/d2a/f34 [4048016,35930] 0 2024-05-25T06:11:23.154 INFO:tasks.workunit.client.1.smithi203.stdout:2/258: mkdir d2/d38/d43/d60/d6b 0 2024-05-25T06:11:23.159 INFO:tasks.workunit.client.1.smithi203.stdout:0/204: mkdir dd/d10/d34/d4c 0 2024-05-25T06:11:23.162 INFO:tasks.workunit.client.1.smithi203.stdout:3/267: mknod d10/d16/d23/d38/c5c 0 2024-05-25T06:11:23.163 INFO:tasks.workunit.client.1.smithi203.stdout:5/387: getdents dc/d11/d6b 0 2024-05-25T06:11:23.164 INFO:tasks.workunit.client.1.smithi203.stdout:6/241: dwrite d8/ff [0,4194304] 0 2024-05-25T06:11:23.171 INFO:tasks.workunit.client.1.smithi203.stdout:3/268: rename d10/d22/d2a/f34 to d10/d16/d23/d3a/f5d 0 2024-05-25T06:11:23.172 INFO:tasks.workunit.client.1.smithi203.stdout:8/217: getdents d2/d5/d18 0 2024-05-25T06:11:23.175 INFO:tasks.workunit.client.1.smithi203.stdout:3/269: fdatasync d10/d16/d23/f29 0 2024-05-25T06:11:23.175 INFO:tasks.workunit.client.1.smithi203.stdout:2/259: creat d2/f6c x:0 0 0 2024-05-25T06:11:23.177 INFO:tasks.workunit.client.1.smithi203.stdout:6/242: creat d8/d29/d30/f4f x:0 0 0 2024-05-25T06:11:23.178 INFO:tasks.workunit.client.1.smithi203.stdout:3/270: readlink d10/d22/l2e 0 2024-05-25T06:11:23.179 INFO:tasks.workunit.client.1.smithi203.stdout:8/218: creat d2/d5/d10/f4d x:0 0 0 2024-05-25T06:11:23.180 INFO:tasks.workunit.client.1.smithi203.stdout:0/205: dwrite dd/d32/d37/f42 [0,4194304] 0 2024-05-25T06:11:23.182 INFO:tasks.workunit.client.1.smithi203.stdout:8/219: dread - d2/d5/d28/d38/f49 zero size 2024-05-25T06:11:23.184 INFO:tasks.workunit.client.1.smithi203.stdout:3/271: stat d10/d16/d23/l30 0 2024-05-25T06:11:23.185 INFO:tasks.workunit.client.1.smithi203.stdout:2/260: creat d2/d4/d10/d58/f6d x:0 0 0 2024-05-25T06:11:23.185 INFO:tasks.workunit.client.1.smithi203.stdout:6/243: truncate d8/d11/d15/f2b 5040467 0 2024-05-25T06:11:23.189 INFO:tasks.workunit.client.1.smithi203.stdout:3/272: chown d10/d16/f19 2 1 2024-05-25T06:11:23.189 INFO:tasks.workunit.client.1.smithi203.stdout:6/244: dread - d8/d11/d1b/f4d zero size 2024-05-25T06:11:23.191 INFO:tasks.workunit.client.1.smithi203.stdout:2/261: fsync d2/f8 0 2024-05-25T06:11:23.191 INFO:tasks.workunit.client.1.smithi203.stdout:0/206: mknod dd/d32/c4d 0 2024-05-25T06:11:23.192 INFO:tasks.workunit.client.1.smithi203.stdout:0/207: stat dd/d10/f29 0 2024-05-25T06:11:23.193 INFO:tasks.workunit.client.1.smithi203.stdout:2/262: write d2/d1a/f27 [1527182,102125] 0 2024-05-25T06:11:23.193 INFO:tasks.workunit.client.1.smithi203.stdout:3/273: unlink d10/f32 0 2024-05-25T06:11:23.194 INFO:tasks.workunit.client.1.smithi203.stdout:6/245: mknod d8/d2a/c50 0 2024-05-25T06:11:23.194 INFO:tasks.workunit.client.1.smithi203.stdout:6/246: chown d8 3162 1 2024-05-25T06:11:23.194 INFO:tasks.workunit.client.1.smithi203.stdout:0/208: truncate dd/d10/f28 377830 0 2024-05-25T06:11:23.196 INFO:tasks.workunit.client.1.smithi203.stdout:3/274: creat d10/d22/f5e x:0 0 0 2024-05-25T06:11:23.197 INFO:tasks.workunit.client.1.smithi203.stdout:0/209: creat dd/d32/d37/f4e x:0 0 0 2024-05-25T06:11:23.200 INFO:tasks.workunit.client.1.smithi203.stdout:5/388: dread dc/d11/d17/d1a/f2e [0,4194304] 0 2024-05-25T06:11:23.200 INFO:tasks.workunit.client.1.smithi203.stdout:2/263: link d2/d4/d10/d32/f1c d2/d38/f6e 0 2024-05-25T06:11:23.206 INFO:tasks.workunit.client.1.smithi203.stdout:5/389: mkdir dc/d3c/d8b 0 2024-05-25T06:11:23.211 INFO:tasks.workunit.client.1.smithi203.stdout:6/247: dwrite d8/d11/d15/f37 [0,4194304] 0 2024-05-25T06:11:23.214 INFO:tasks.workunit.client.1.smithi203.stdout:0/210: dwrite dd/f24 [0,4194304] 0 2024-05-25T06:11:23.214 INFO:tasks.workunit.client.1.smithi203.stdout:5/390: creat dc/d11/d17/d1a/d72/f8c x:0 0 0 2024-05-25T06:11:23.214 INFO:tasks.workunit.client.1.smithi203.stdout:8/220: dread d2/d5/d10/d11/de/f13 [0,4194304] 0 2024-05-25T06:11:23.216 INFO:tasks.workunit.client.1.smithi203.stdout:6/248: symlink d8/l51 0 2024-05-25T06:11:23.221 INFO:tasks.workunit.client.1.smithi203.stdout:0/211: fsync dd/d10/f2f 0 2024-05-25T06:11:23.222 INFO:tasks.workunit.client.1.smithi203.stdout:5/391: mknod dc/d6a/c8d 0 2024-05-25T06:11:23.224 INFO:tasks.workunit.client.1.smithi203.stdout:5/392: dread - dc/d11/d17/d1a/d75/f77 zero size 2024-05-25T06:11:23.229 INFO:tasks.workunit.client.1.smithi203.stdout:6/249: dwrite f0 [8388608,4194304] 0 2024-05-25T06:11:23.230 INFO:tasks.workunit.client.1.smithi203.stdout:3/275: dread d10/d22/d2a/f3b [0,4194304] 0 2024-05-25T06:11:23.233 INFO:tasks.workunit.client.1.smithi203.stdout:0/212: dread dd/d3a/f3b [0,4194304] 0 2024-05-25T06:11:23.234 INFO:tasks.workunit.client.1.smithi203.stdout:2/264: dread d2/d38/d2f/f34 [0,4194304] 0 2024-05-25T06:11:23.234 INFO:tasks.workunit.client.1.smithi203.stdout:3/276: chown d10/d22/d2a 2 1 2024-05-25T06:11:23.236 INFO:tasks.workunit.client.1.smithi203.stdout:0/213: mkdir dd/d3d/d4f 0 2024-05-25T06:11:23.236 INFO:tasks.workunit.client.1.smithi203.stdout:5/393: dread dc/d11/d18/f37 [0,4194304] 0 2024-05-25T06:11:23.236 INFO:tasks.workunit.client.1.smithi203.stdout:3/277: write d10/f20 [688349,87204] 0 2024-05-25T06:11:23.240 INFO:tasks.workunit.client.1.smithi203.stdout:6/250: rename d8/d29/d3e to d8/d11/d15/d28/d52 0 2024-05-25T06:11:23.240 INFO:tasks.workunit.client.1.smithi203.stdout:2/265: write d2/d4/d10/d32/f44 [1889166,102996] 0 2024-05-25T06:11:23.241 INFO:tasks.workunit.client.1.smithi203.stdout:5/394: mkdir dc/d11/d18/d8e 0 2024-05-25T06:11:23.243 INFO:tasks.workunit.client.1.smithi203.stdout:0/214: creat dd/d3d/d4f/f50 x:0 0 0 2024-05-25T06:11:23.243 INFO:tasks.workunit.client.1.smithi203.stdout:0/215: stat dd/f23 0 2024-05-25T06:11:23.244 INFO:tasks.workunit.client.1.smithi203.stdout:6/251: mknod d8/d3d/c53 0 2024-05-25T06:11:23.245 INFO:tasks.workunit.client.1.smithi203.stdout:6/252: write d8/f9 [2657300,24674] 0 2024-05-25T06:11:23.246 INFO:tasks.workunit.client.1.smithi203.stdout:2/266: creat d2/f6f x:0 0 0 2024-05-25T06:11:23.247 INFO:tasks.workunit.client.1.smithi203.stdout:0/216: read dd/d10/f33 [220374,708] 0 2024-05-25T06:11:23.247 INFO:tasks.workunit.client.1.smithi203.stdout:3/278: dread d10/d16/f17 [0,4194304] 0 2024-05-25T06:11:23.250 INFO:tasks.workunit.client.1.smithi203.stdout:0/217: rmdir dd/d3a 39 2024-05-25T06:11:23.255 INFO:tasks.workunit.client.1.smithi203.stdout:3/279: creat d10/d16/d23/f5f x:0 0 0 2024-05-25T06:11:23.256 INFO:tasks.workunit.client.1.smithi203.stdout:5/395: dread dc/f10 [0,4194304] 0 2024-05-25T06:11:23.256 INFO:tasks.workunit.client.1.smithi203.stdout:2/267: getdents d2/d38 0 2024-05-25T06:11:23.256 INFO:tasks.workunit.client.1.smithi203.stdout:6/253: dwrite d8/d11/d15/d28/d52/d43/f32 [0,4194304] 0 2024-05-25T06:11:23.258 INFO:tasks.workunit.client.1.smithi203.stdout:3/280: chown d10/d22/d2a/f3b 372923 1 2024-05-25T06:11:23.259 INFO:tasks.workunit.client.1.smithi203.stdout:5/396: dread - dc/d3c/f78 zero size 2024-05-25T06:11:23.259 INFO:tasks.workunit.client.1.smithi203.stdout:2/268: symlink d2/d6/l70 0 2024-05-25T06:11:23.260 INFO:tasks.workunit.client.1.smithi203.stdout:8/221: dread d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:23.262 INFO:tasks.workunit.client.1.smithi203.stdout:0/218: getdents dd 0 2024-05-25T06:11:23.264 INFO:tasks.workunit.client.1.smithi203.stdout:3/281: symlink d10/d16/l60 0 2024-05-25T06:11:23.265 INFO:tasks.workunit.client.1.smithi203.stdout:2/269: truncate d2/d4/d10/d50/f67 1027178 0 2024-05-25T06:11:23.265 INFO:tasks.workunit.client.1.smithi203.stdout:8/222: mkdir d2/d5/d28/d38/d4e 0 2024-05-25T06:11:23.267 INFO:tasks.workunit.client.1.smithi203.stdout:2/270: stat d2/d4/d10/d5f/c49 0 2024-05-25T06:11:23.269 INFO:tasks.workunit.client.1.smithi203.stdout:3/282: fsync d10/d16/d23/d4a/f58 0 2024-05-25T06:11:23.271 INFO:tasks.workunit.client.1.smithi203.stdout:3/283: readlink lf 0 2024-05-25T06:11:23.275 INFO:tasks.workunit.client.1.smithi203.stdout:5/397: dwrite dc/d11/d17/d1a/f61 [0,4194304] 0 2024-05-25T06:11:23.282 INFO:tasks.workunit.client.1.smithi203.stdout:2/271: rename d2/d38/d43/f4d to d2/d1a/d33/f71 0 2024-05-25T06:11:23.282 INFO:tasks.workunit.client.1.smithi203.stdout:6/254: dwrite d8/f12 [0,4194304] 0 2024-05-25T06:11:23.284 INFO:tasks.workunit.client.1.smithi203.stdout:5/398: dread - dc/d11/f3e zero size 2024-05-25T06:11:23.289 INFO:tasks.workunit.client.1.smithi203.stdout:6/255: mkdir d8/d54 0 2024-05-25T06:11:23.291 INFO:tasks.workunit.client.1.smithi203.stdout:0/219: dwrite dd/d10/f41 [0,4194304] 0 2024-05-25T06:11:23.293 INFO:tasks.workunit.client.1.smithi203.stdout:6/256: read d8/ff [2309078,60192] 0 2024-05-25T06:11:23.294 INFO:tasks.workunit.client.1.smithi203.stdout:2/272: unlink d2/d4/d10/d32/c3a 0 2024-05-25T06:11:23.296 INFO:tasks.workunit.client.1.smithi203.stdout:3/284: dwrite d10/f1a [0,4194304] 0 2024-05-25T06:11:23.298 INFO:tasks.workunit.client.1.smithi203.stdout:3/285: write d10/f31 [107451,86914] 0 2024-05-25T06:11:23.298 INFO:tasks.workunit.client.1.smithi203.stdout:8/223: dwrite d2/d5/d18/d31/f39 [0,4194304] 0 2024-05-25T06:11:23.299 INFO:tasks.workunit.client.1.smithi203.stdout:0/220: creat dd/d10/f51 x:0 0 0 2024-05-25T06:11:23.302 INFO:tasks.workunit.client.1.smithi203.stdout:2/273: fdatasync d2/d4/d10/d32/f3d 0 2024-05-25T06:11:23.307 INFO:tasks.workunit.client.1.smithi203.stdout:3/286: stat d10/d22/l54 0 2024-05-25T06:11:23.307 INFO:tasks.workunit.client.1.smithi203.stdout:8/224: write d2/d5/d10/d11/de/f1a [2566422,24589] 0 2024-05-25T06:11:23.311 INFO:tasks.workunit.client.1.smithi203.stdout:3/287: dread - d10/d16/d23/f5f zero size 2024-05-25T06:11:23.314 INFO:tasks.workunit.client.1.smithi203.stdout:8/225: chown d2/d5/d10/d11/l20 659 1 2024-05-25T06:11:23.315 INFO:tasks.workunit.client.1.smithi203.stdout:5/399: dwrite dc/d11/d17/f31 [0,4194304] 0 2024-05-25T06:11:23.318 INFO:tasks.workunit.client.1.smithi203.stdout:6/257: truncate f2 1574687 0 2024-05-25T06:11:23.319 INFO:tasks.workunit.client.1.smithi203.stdout:0/221: dwrite dd/d10/f29 [4194304,4194304] 0 2024-05-25T06:11:23.320 INFO:tasks.workunit.client.1.smithi203.stdout:6/258: chown d8/l51 13718 1 2024-05-25T06:11:23.334 INFO:tasks.workunit.client.1.smithi203.stdout:6/259: creat d8/d11/d15/d28/d48/f55 x:0 0 0 2024-05-25T06:11:23.336 INFO:tasks.workunit.client.1.smithi203.stdout:5/400: getdents dc/d11/d17 0 2024-05-25T06:11:23.336 INFO:tasks.workunit.client.1.smithi203.stdout:0/222: link dd/d10/f41 dd/d32/d37/f52 0 2024-05-25T06:11:23.337 INFO:tasks.workunit.client.1.smithi203.stdout:2/274: dwrite d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:23.340 INFO:tasks.workunit.client.1.smithi203.stdout:2/275: chown d2/d4/d10/d5f 5 1 2024-05-25T06:11:23.340 INFO:tasks.workunit.client.1.smithi203.stdout:5/401: write dc/d11/f3e [393612,23051] 0 2024-05-25T06:11:23.340 INFO:tasks.workunit.client.1.smithi203.stdout:3/288: dwrite d10/d16/d23/d3a/d48/f52 [0,4194304] 0 2024-05-25T06:11:23.343 INFO:tasks.workunit.client.1.smithi203.stdout:3/289: chown d10/d16/d23/l37 1 1 2024-05-25T06:11:23.343 INFO:tasks.workunit.client.1.smithi203.stdout:2/276: write d2/d4/f2e [4602399,91091] 0 2024-05-25T06:11:23.345 INFO:tasks.workunit.client.1.smithi203.stdout:5/402: dread - dc/d11/d17/d1a/d75/f77 zero size 2024-05-25T06:11:23.346 INFO:tasks.workunit.client.1.smithi203.stdout:6/260: rename d8/d11/d1b/l36 to d8/d2a/l56 0 2024-05-25T06:11:23.346 INFO:tasks.workunit.client.1.smithi203.stdout:0/223: symlink dd/d3d/d4f/l53 0 2024-05-25T06:11:23.348 INFO:tasks.workunit.client.1.smithi203.stdout:2/277: write d2/d38/f29 [1067164,120728] 0 2024-05-25T06:11:23.350 INFO:tasks.workunit.client.1.smithi203.stdout:0/224: truncate dd/d10/f28 616357 0 2024-05-25T06:11:23.351 INFO:tasks.workunit.client.1.smithi203.stdout:5/403: creat dc/d11/d17/d44/f8f x:0 0 0 2024-05-25T06:11:23.352 INFO:tasks.workunit.client.1.smithi203.stdout:3/290: rename d10/d16/d23/f29 to d10/d16/d23/d4a/f61 0 2024-05-25T06:11:23.353 INFO:tasks.workunit.client.1.smithi203.stdout:2/278: dread d2/d4/d10/d50/f67 [0,4194304] 0 2024-05-25T06:11:23.354 INFO:tasks.workunit.client.1.smithi203.stdout:3/291: chown d10/f31 959 1 2024-05-25T06:11:23.356 INFO:tasks.workunit.client.1.smithi203.stdout:3/292: truncate d10/d16/d23/d3a/f55 842726 0 2024-05-25T06:11:23.359 INFO:tasks.workunit.client.1.smithi203.stdout:0/225: creat dd/d32/f54 x:0 0 0 2024-05-25T06:11:23.360 INFO:tasks.workunit.client.1.smithi203.stdout:3/293: mkdir d10/d16/d50/d62 0 2024-05-25T06:11:23.362 INFO:tasks.workunit.client.1.smithi203.stdout:8/226: truncate d2/d5/d10/d11/de/f1a 1430264 0 2024-05-25T06:11:23.363 INFO:tasks.workunit.client.1.smithi203.stdout:0/226: dread dd/d10/f28 [0,4194304] 0 2024-05-25T06:11:23.363 INFO:tasks.workunit.client.1.smithi203.stdout:6/261: readlink d8/d2a/l56 0 2024-05-25T06:11:23.364 INFO:tasks.workunit.client.1.smithi203.stdout:0/227: write dd/d10/f51 [273360,55123] 0 2024-05-25T06:11:23.365 INFO:tasks.workunit.client.1.smithi203.stdout:6/262: read d8/f12 [362195,99254] 0 2024-05-25T06:11:23.365 INFO:tasks.workunit.client.1.smithi203.stdout:8/227: truncate d2/d5/d10/d11/de/f1e 486675 0 2024-05-25T06:11:23.372 INFO:tasks.workunit.client.1.smithi203.stdout:2/279: write d2/d4/d10/d58/f64 [1036214,99730] 0 2024-05-25T06:11:23.374 INFO:tasks.workunit.client.1.smithi203.stdout:6/263: link d8/d11/d15/f17 d8/d11/d15/d28/d48/f57 0 2024-05-25T06:11:23.377 INFO:tasks.workunit.client.1.smithi203.stdout:6/264: chown d8/d11 449675376 1 2024-05-25T06:11:23.378 INFO:tasks.workunit.client.1.smithi203.stdout:5/404: dwrite dc/d11/f1c [0,4194304] 0 2024-05-25T06:11:23.382 INFO:tasks.workunit.client.1.smithi203.stdout:0/228: dwrite dd/d3d/d4f/f50 [0,4194304] 0 2024-05-25T06:11:23.384 INFO:tasks.workunit.client.1.smithi203.stdout:6/265: symlink d8/d11/d15/d28/d48/l58 0 2024-05-25T06:11:23.388 INFO:tasks.workunit.client.1.smithi203.stdout:5/405: rename dc/d11/d17/d1a/d2d/d53/f55 to dc/d11/d17/d1a/d22/f90 0 2024-05-25T06:11:23.388 INFO:tasks.workunit.client.1.smithi203.stdout:2/280: dwrite d2/d38/d2f/f34 [4194304,4194304] 0 2024-05-25T06:11:23.390 INFO:tasks.workunit.client.1.smithi203.stdout:8/228: dread d2/d5/d10/d11/de/ff [0,4194304] 0 2024-05-25T06:11:23.391 INFO:tasks.workunit.client.1.smithi203.stdout:2/281: fsync d2/fe 0 2024-05-25T06:11:23.392 INFO:tasks.workunit.client.1.smithi203.stdout:6/266: creat d8/d11/d15/f59 x:0 0 0 2024-05-25T06:11:23.393 INFO:tasks.workunit.client.1.smithi203.stdout:5/406: write dc/d11/f1c [397263,3032] 0 2024-05-25T06:11:23.396 INFO:tasks.workunit.client.1.smithi203.stdout:6/267: fsync d8/d11/f23 0 2024-05-25T06:11:23.403 INFO:tasks.workunit.client.1.smithi203.stdout:6/268: mkdir d8/d5a 0 2024-05-25T06:11:23.406 INFO:tasks.workunit.client.1.smithi203.stdout:5/407: rename dc/d11/d17/d1a/l4c to dc/d11/d17/d1a/d2d/d4d/l91 0 2024-05-25T06:11:23.407 INFO:tasks.workunit.client.1.smithi203.stdout:8/229: dwrite d2/d5/d28/f3a [0,4194304] 0 2024-05-25T06:11:23.407 INFO:tasks.workunit.client.1.smithi203.stdout:6/269: write d8/d11/d1b/f4d [13067,48939] 0 2024-05-25T06:11:23.410 INFO:tasks.workunit.client.1.smithi203.stdout:2/282: dwrite d2/d4/d10/f5e [0,4194304] 0 2024-05-25T06:11:23.410 INFO:tasks.workunit.client.1.smithi203.stdout:6/270: fsync d8/d11/d15/f59 0 2024-05-25T06:11:23.413 INFO:tasks.workunit.client.1.smithi203.stdout:5/408: write dc/d11/f1c [4296209,12877] 0 2024-05-25T06:11:23.413 INFO:tasks.workunit.client.1.smithi203.stdout:8/230: dread - d2/d5/d28/d38/f49 zero size 2024-05-25T06:11:23.415 INFO:tasks.workunit.client.1.smithi203.stdout:2/283: fdatasync d2/d4/d10/d5f/f5b 0 2024-05-25T06:11:23.415 INFO:tasks.workunit.client.1.smithi203.stdout:6/271: chown d8/d11/d15/d28/d52/c45 10709099 1 2024-05-25T06:11:23.416 INFO:tasks.workunit.client.1.smithi203.stdout:6/272: readlink d8/d2a/l3a 0 2024-05-25T06:11:23.419 INFO:tasks.workunit.client.1.smithi203.stdout:5/409: getdents dc 0 2024-05-25T06:11:23.423 INFO:tasks.workunit.client.1.smithi203.stdout:6/273: creat d8/d54/f5b x:0 0 0 2024-05-25T06:11:23.426 INFO:tasks.workunit.client.1.smithi203.stdout:8/231: dwrite d2/d5/d10/d11/de/f16 [0,4194304] 0 2024-05-25T06:11:23.426 INFO:tasks.workunit.client.1.smithi203.stdout:2/284: getdents d2/d38/d43 0 2024-05-25T06:11:23.426 INFO:tasks.workunit.client.1.smithi203.stdout:6/274: chown d8/d11/d15/l33 3 1 2024-05-25T06:11:23.428 INFO:tasks.workunit.client.1.smithi203.stdout:5/410: dwrite dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:23.429 INFO:tasks.workunit.client.1.smithi203.stdout:8/232: write d2/d5/d2d/f3e [2977253,88295] 0 2024-05-25T06:11:23.430 INFO:tasks.workunit.client.1.smithi203.stdout:2/285: mknod d2/c72 0 2024-05-25T06:11:23.431 INFO:tasks.workunit.client.1.smithi203.stdout:5/411: chown dc/fe 38 1 2024-05-25T06:11:23.432 INFO:tasks.workunit.client.1.smithi203.stdout:6/275: truncate d8/d11/f1e 1638624 0 2024-05-25T06:11:23.435 INFO:tasks.workunit.client.1.smithi203.stdout:8/233: readlink d2/d5/l40 0 2024-05-25T06:11:23.438 INFO:tasks.workunit.client.1.smithi203.stdout:5/412: creat dc/d11/d17/d1a/f92 x:0 0 0 2024-05-25T06:11:23.440 INFO:tasks.workunit.client.1.smithi203.stdout:8/234: write d2/d5/d10/d11/de/f1e [1133080,95360] 0 2024-05-25T06:11:23.442 INFO:tasks.workunit.client.1.smithi203.stdout:2/286: dwrite d2/f6f [0,4194304] 0 2024-05-25T06:11:23.447 INFO:tasks.workunit.client.1.smithi203.stdout:6/276: write d8/d11/d15/f17 [703784,81709] 0 2024-05-25T06:11:23.449 INFO:tasks.workunit.client.1.smithi203.stdout:2/287: rename d2/d4/l63 to d2/d38/l73 0 2024-05-25T06:11:23.453 INFO:tasks.workunit.client.1.smithi203.stdout:6/277: truncate d8/f12 639855 0 2024-05-25T06:11:23.454 INFO:tasks.workunit.client.1.smithi203.stdout:5/413: dwrite dc/d11/f3e [0,4194304] 0 2024-05-25T06:11:23.456 INFO:tasks.workunit.client.1.smithi203.stdout:2/288: rename d2/d4/d10/d32/f1c to d2/d4/f74 0 2024-05-25T06:11:23.459 INFO:tasks.workunit.client.1.smithi203.stdout:5/414: truncate dc/d11/f12 2575854 0 2024-05-25T06:11:23.471 INFO:tasks.workunit.client.1.smithi203.stdout:5/415: write dc/f80 [1354553,84544] 0 2024-05-25T06:11:23.474 INFO:tasks.workunit.client.1.smithi203.stdout:5/416: creat dc/d3c/d3f/f93 x:0 0 0 2024-05-25T06:11:23.475 INFO:tasks.workunit.client.1.smithi203.stdout:5/417: fdatasync f6 0 2024-05-25T06:11:23.478 INFO:tasks.workunit.client.1.smithi203.stdout:8/235: dread d2/d5/d18/f1b [4194304,4194304] 0 2024-05-25T06:11:23.480 INFO:tasks.workunit.client.1.smithi203.stdout:5/418: dread dc/d11/d17/d1a/f61 [0,4194304] 0 2024-05-25T06:11:23.480 INFO:tasks.workunit.client.1.smithi203.stdout:8/236: mkdir d2/d5/d18/d4f 0 2024-05-25T06:11:23.482 INFO:tasks.workunit.client.1.smithi203.stdout:8/237: write d2/d5/d10/d11/d1f/f35 [119808,3977] 0 2024-05-25T06:11:23.484 INFO:tasks.workunit.client.1.smithi203.stdout:2/289: dread d2/d1a/d33/f71 [0,4194304] 0 2024-05-25T06:11:23.485 INFO:tasks.workunit.client.1.smithi203.stdout:2/290: readlink d2/l55 0 2024-05-25T06:11:23.486 INFO:tasks.workunit.client.1.smithi203.stdout:5/419: dread dc/d11/d17/d1a/d2d/f7d [0,4194304] 0 2024-05-25T06:11:23.487 INFO:tasks.workunit.client.1.smithi203.stdout:2/291: mknod d2/d6/c75 0 2024-05-25T06:11:23.488 INFO:tasks.workunit.client.1.smithi203.stdout:8/238: dread d2/d5/d18/f1b [4194304,4194304] 0 2024-05-25T06:11:23.488 INFO:tasks.workunit.client.1.smithi203.stdout:5/420: mkdir dc/d11/d42/d94 0 2024-05-25T06:11:23.489 INFO:tasks.workunit.client.1.smithi203.stdout:2/292: symlink d2/d1a/d33/l76 0 2024-05-25T06:11:23.491 INFO:tasks.workunit.client.1.smithi203.stdout:8/239: rmdir d2/d5/d28 39 2024-05-25T06:11:23.495 INFO:tasks.workunit.client.1.smithi203.stdout:8/240: write d2/d5/d18/d31/f3d [1842175,7021] 0 2024-05-25T06:11:23.496 INFO:tasks.workunit.client.1.smithi203.stdout:5/421: dwrite dc/d11/d17/d45/f52 [0,4194304] 0 2024-05-25T06:11:23.497 INFO:tasks.workunit.client.1.smithi203.stdout:2/293: dread d2/d4/d10/d32/f3d [0,4194304] 0 2024-05-25T06:11:23.499 INFO:tasks.workunit.client.1.smithi203.stdout:2/294: chown d2/d1a/c20 0 1 2024-05-25T06:11:23.501 INFO:tasks.workunit.client.1.smithi203.stdout:2/295: stat d2/d6/c75 0 2024-05-25T06:11:23.501 INFO:tasks.workunit.client.1.smithi203.stdout:5/422: mkdir dc/d11/d42/d94/d95 0 2024-05-25T06:11:23.503 INFO:tasks.workunit.client.1.smithi203.stdout:2/296: fsync d2/d4/d10/d32/f28 0 2024-05-25T06:11:23.503 INFO:tasks.workunit.client.1.smithi203.stdout:5/423: chown dc/d11/d42 101208373 1 2024-05-25T06:11:23.505 INFO:tasks.workunit.client.1.smithi203.stdout:2/297: write d2/d38/d2f/f61 [136845,104391] 0 2024-05-25T06:11:23.506 INFO:tasks.workunit.client.1.smithi203.stdout:5/424: write f6 [2853596,58497] 0 2024-05-25T06:11:23.506 INFO:tasks.workunit.client.1.smithi203.stdout:8/241: dwrite d2/d5/d18/d31/f3d [0,4194304] 0 2024-05-25T06:11:23.509 INFO:tasks.workunit.client.1.smithi203.stdout:8/242: fsync d2/d5/fc 0 2024-05-25T06:11:23.509 INFO:tasks.workunit.client.1.smithi203.stdout:5/425: rename dc to dc/d3c/d3f/d96 22 2024-05-25T06:11:23.512 INFO:tasks.workunit.client.1.smithi203.stdout:5/426: read - dc/d11/d17/d45/f62 zero size 2024-05-25T06:11:23.513 INFO:tasks.workunit.client.1.smithi203.stdout:8/243: creat d2/d5/d18/d31/f50 x:0 0 0 2024-05-25T06:11:23.514 INFO:tasks.workunit.client.1.smithi203.stdout:5/427: rmdir dc/d6a 39 2024-05-25T06:11:23.518 INFO:tasks.workunit.client.1.smithi203.stdout:2/298: dwrite d2/d4/d10/d50/f51 [0,4194304] 0 2024-05-25T06:11:23.521 INFO:tasks.workunit.client.1.smithi203.stdout:5/428: write dc/d11/f1c [3427408,92034] 0 2024-05-25T06:11:23.523 INFO:tasks.workunit.client.1.smithi203.stdout:2/299: write d2/d4/d10/d32/f28 [3454912,90538] 0 2024-05-25T06:11:23.524 INFO:tasks.workunit.client.1.smithi203.stdout:8/244: dwrite d2/d5/fc [0,4194304] 0 2024-05-25T06:11:23.526 INFO:tasks.workunit.client.1.smithi203.stdout:8/245: stat d2 0 2024-05-25T06:11:23.529 INFO:tasks.workunit.client.1.smithi203.stdout:5/429: read dc/d11/d17/d1a/d22/f90 [4001168,5271] 0 2024-05-25T06:11:23.529 INFO:tasks.workunit.client.1.smithi203.stdout:8/246: write d2/f2a [1119101,27268] 0 2024-05-25T06:11:23.535 INFO:tasks.workunit.client.1.smithi203.stdout:5/430: symlink dc/d6a/l97 0 2024-05-25T06:11:23.535 INFO:tasks.workunit.client.1.smithi203.stdout:2/300: dwrite d2/f6f [0,4194304] 0 2024-05-25T06:11:23.537 INFO:tasks.workunit.client.1.smithi203.stdout:8/247: creat d2/d5/d28/d38/d4e/f51 x:0 0 0 2024-05-25T06:11:23.539 INFO:tasks.workunit.client.1.smithi203.stdout:8/248: write d2/f2a [383041,127356] 0 2024-05-25T06:11:23.539 INFO:tasks.workunit.client.1.smithi203.stdout:2/301: unlink d2/d6/f2d 0 2024-05-25T06:11:23.543 INFO:tasks.workunit.client.1.smithi203.stdout:5/431: dwrite dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:23.547 INFO:tasks.workunit.client.1.smithi203.stdout:5/432: truncate dc/d11/d17/d1a/d72/f8c 437727 0 2024-05-25T06:11:23.547 INFO:tasks.workunit.client.1.smithi203.stdout:8/249: getdents d2/d5/d28 0 2024-05-25T06:11:23.550 INFO:tasks.workunit.client.1.smithi203.stdout:8/250: creat d2/d5/d10/f52 x:0 0 0 2024-05-25T06:11:23.553 INFO:tasks.workunit.client.1.smithi203.stdout:5/433: dread dc/d11/d17/d1a/f2e [0,4194304] 0 2024-05-25T06:11:23.553 INFO:tasks.workunit.client.1.smithi203.stdout:8/251: creat d2/d5/d10/f53 x:0 0 0 2024-05-25T06:11:23.556 INFO:tasks.workunit.client.1.smithi203.stdout:8/252: creat d2/d5/d2d/f54 x:0 0 0 2024-05-25T06:11:23.558 INFO:tasks.workunit.client.1.smithi203.stdout:2/302: dwrite d2/d4/f74 [0,4194304] 0 2024-05-25T06:11:23.559 INFO:tasks.workunit.client.1.smithi203.stdout:8/253: dread - d2/d5/d28/d38/d4e/f51 zero size 2024-05-25T06:11:23.560 INFO:tasks.workunit.client.1.smithi203.stdout:8/254: dread - d2/d5/d28/f4c zero size 2024-05-25T06:11:23.565 INFO:tasks.workunit.client.1.smithi203.stdout:8/255: read - d2/d5/d28/f4c zero size 2024-05-25T06:11:23.568 INFO:tasks.workunit.client.1.smithi203.stdout:8/256: mknod d2/d5/d18/d4f/c55 0 2024-05-25T06:11:23.569 INFO:tasks.workunit.client.1.smithi203.stdout:2/303: dwrite d2/d38/d2f/f34 [0,4194304] 0 2024-05-25T06:11:23.579 INFO:tasks.workunit.client.1.smithi203.stdout:5/434: dread f6 [0,4194304] 0 2024-05-25T06:11:23.584 INFO:tasks.workunit.client.1.smithi203.stdout:2/304: dwrite d2/f8 [0,4194304] 0 2024-05-25T06:11:23.589 INFO:tasks.workunit.client.1.smithi203.stdout:5/435: dwrite dc/d11/f3e [0,4194304] 0 2024-05-25T06:11:23.592 INFO:tasks.workunit.client.1.smithi203.stdout:2/305: rename d2/d4/d10/d32/c5d to d2/d38/d43/d60/c77 0 2024-05-25T06:11:23.595 INFO:tasks.workunit.client.1.smithi203.stdout:5/436: mkdir dc/d11/d17/d44/d73/d98 0 2024-05-25T06:11:23.595 INFO:tasks.workunit.client.1.smithi203.stdout:2/306: symlink d2/d38/d43/l78 0 2024-05-25T06:11:23.598 INFO:tasks.workunit.client.1.smithi203.stdout:5/437: mknod dc/c99 0 2024-05-25T06:11:23.600 INFO:tasks.workunit.client.1.smithi203.stdout:5/438: chown dc/d11/d17/d1a/d2d/d4d/l6d 51 1 2024-05-25T06:11:23.602 INFO:tasks.workunit.client.1.smithi203.stdout:5/439: fdatasync dc/d11/d17/d45/f46 0 2024-05-25T06:11:23.603 INFO:tasks.workunit.client.1.smithi203.stdout:2/307: dwrite d2/d4/d10/d32/f22 [0,4194304] 0 2024-05-25T06:11:23.605 INFO:tasks.workunit.client.1.smithi203.stdout:2/308: read - d2/d4/d10/d5f/f5b zero size 2024-05-25T06:11:23.607 INFO:tasks.workunit.client.1.smithi203.stdout:2/309: fdatasync d2/d4/d10/d32/f52 0 2024-05-25T06:11:23.607 INFO:tasks.workunit.client.1.smithi203.stdout:5/440: dread dc/d11/d17/d1a/d2d/f7d [0,4194304] 0 2024-05-25T06:11:23.610 INFO:tasks.workunit.client.1.smithi203.stdout:2/310: creat d2/d4/d10/f79 x:0 0 0 2024-05-25T06:11:23.619 INFO:tasks.workunit.client.1.smithi203.stdout:5/441: dwrite dc/f80 [0,4194304] 0 2024-05-25T06:11:23.620 INFO:tasks.workunit.client.1.smithi203.stdout:2/311: dwrite d2/d4/f74 [0,4194304] 0 2024-05-25T06:11:23.630 INFO:tasks.workunit.client.1.smithi203.stdout:5/442: dwrite dc/d11/d17/d45/f5c [0,4194304] 0 2024-05-25T06:11:23.635 INFO:tasks.workunit.client.1.smithi203.stdout:2/312: dwrite d2/d38/f1d [4194304,4194304] 0 2024-05-25T06:11:23.637 INFO:tasks.workunit.client.1.smithi203.stdout:5/443: truncate dc/d11/d17/f38 1045128 0 2024-05-25T06:11:23.641 INFO:tasks.workunit.client.1.smithi203.stdout:2/313: dread d2/f6f [0,4194304] 0 2024-05-25T06:11:23.642 INFO:tasks.workunit.client.1.smithi203.stdout:5/444: getdents dc/d11/d17/d1a/d75 0 2024-05-25T06:11:23.651 INFO:tasks.workunit.client.1.smithi203.stdout:2/314: dwrite d2/d1a/d33/f71 [4194304,4194304] 0 2024-05-25T06:11:23.651 INFO:tasks.workunit.client.1.smithi203.stdout:5/445: dread dc/d11/d17/d1a/f32 [0,4194304] 0 2024-05-25T06:11:23.655 INFO:tasks.workunit.client.1.smithi203.stdout:5/446: mknod dc/d11/d17/d1a/d2d/d53/c9a 0 2024-05-25T06:11:23.655 INFO:tasks.workunit.client.1.smithi203.stdout:2/315: getdents d2/d42/d4e/d68 0 2024-05-25T06:11:23.659 INFO:tasks.workunit.client.1.smithi203.stdout:5/447: link dc/d11/d17/d1a/d22/f39 dc/d11/d42/f9b 0 2024-05-25T06:11:23.662 INFO:tasks.workunit.client.1.smithi203.stdout:5/448: symlink dc/d3c/d8b/l9c 0 2024-05-25T06:11:23.663 INFO:tasks.workunit.client.1.smithi203.stdout:5/449: readlink dc/d11/d17/d1a/d2d/d4d/l6d 0 2024-05-25T06:11:23.665 INFO:tasks.workunit.client.1.smithi203.stdout:5/450: symlink dc/d11/d17/d1a/d72/l9d 0 2024-05-25T06:11:23.666 INFO:tasks.workunit.client.1.smithi203.stdout:5/451: readlink dc/d11/d18/l71 0 2024-05-25T06:11:23.667 INFO:tasks.workunit.client.1.smithi203.stdout:5/452: chown dc/d11/d17/d1a/d22/f90 2 1 2024-05-25T06:11:23.673 INFO:tasks.workunit.client.1.smithi203.stdout:5/453: dwrite dc/d3c/f78 [0,4194304] 0 2024-05-25T06:11:23.676 INFO:tasks.workunit.client.1.smithi203.stdout:5/454: creat dc/d11/d17/d1a/d2d/d4d/d64/f9e x:0 0 0 2024-05-25T06:11:23.678 INFO:tasks.workunit.client.1.smithi203.stdout:5/455: rmdir dc/d11/d17/d1a/d2d 39 2024-05-25T06:11:23.681 INFO:tasks.workunit.client.1.smithi203.stdout:5/456: creat dc/d11/d17/d44/d73/d98/f9f x:0 0 0 2024-05-25T06:11:23.684 INFO:tasks.workunit.client.1.smithi203.stdout:5/457: rename dc/d11/d42/d94/d95 to dc/d11/d17/d1a/d2d/d53/da0 0 2024-05-25T06:11:23.686 INFO:tasks.workunit.client.1.smithi203.stdout:5/458: creat dc/d11/d17/d1a/d2d/d4d/d64/fa1 x:0 0 0 2024-05-25T06:11:23.687 INFO:tasks.workunit.client.1.smithi203.stdout:5/459: write dc/d11/f29 [4922136,58767] 0 2024-05-25T06:11:23.691 INFO:tasks.workunit.client.1.smithi203.stdout:5/460: rename dc/d11/d6b to dc/d11/d17/d1a/d2d/d53/da2 0 2024-05-25T06:11:23.693 INFO:tasks.workunit.client.1.smithi203.stdout:5/461: rename dc/d11/d18/c3d to dc/d11/d17/d44/d73/d98/ca3 0 2024-05-25T06:11:23.694 INFO:tasks.workunit.client.1.smithi203.stdout:5/462: fdatasync dc/d11/d17/d45/f5c 0 2024-05-25T06:11:23.697 INFO:tasks.workunit.client.1.smithi203.stdout:5/463: dread f6 [0,4194304] 0 2024-05-25T06:11:23.703 INFO:tasks.workunit.client.1.smithi203.stdout:5/464: link dc/d11/d17/d1a/d2d/d4d/d64/l6f dc/d11/d17/d1a/d2d/d4d/d64/la4 0 2024-05-25T06:11:23.704 INFO:tasks.workunit.client.1.smithi203.stdout:5/465: write dc/d11/d17/d1a/d2d/d4d/d64/f9e [235688,763] 0 2024-05-25T06:11:23.717 INFO:tasks.workunit.client.1.smithi203.stdout:5/466: dread dc/d11/f12 [0,4194304] 0 2024-05-25T06:11:23.720 INFO:tasks.workunit.client.1.smithi203.stdout:5/467: mknod dc/d11/d17/d1a/d2d/d53/ca5 0 2024-05-25T06:11:23.720 INFO:tasks.workunit.client.1.smithi203.stdout:5/468: read dc/d11/f3e [855090,55120] 0 2024-05-25T06:11:23.728 INFO:tasks.workunit.client.1.smithi203.stdout:5/469: dwrite dc/d11/d17/d1a/d22/f86 [0,4194304] 0 2024-05-25T06:11:23.743 INFO:tasks.workunit.client.1.smithi203.stdout:5/470: dread dc/d3c/d3f/f4e [0,4194304] 0 2024-05-25T06:11:24.006 INFO:tasks.workunit.client.1.smithi203.stdout:4/212: sync 2024-05-25T06:11:24.010 INFO:tasks.workunit.client.1.smithi203.stdout:4/213: mkdir d0/d8/d1f/d26/d3b 0 2024-05-25T06:11:24.116 INFO:tasks.workunit.client.0.smithi160.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-25T06:11:24.124 INFO:tasks.workunit.client.0.smithi160.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-25T06:11:24.125 INFO:tasks.workunit.client.0.smithi160.stderr:+ make 2024-05-25T06:11:24.155 INFO:tasks.workunit.client.0.smithi160.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-25T06:11:24.198 INFO:tasks.workunit.client.1.smithi203.stdout:9/192: sync 2024-05-25T06:11:24.198 INFO:tasks.workunit.client.1.smithi203.stdout:7/263: sync 2024-05-25T06:11:24.199 INFO:tasks.workunit.client.1.smithi203.stdout:9/193: write d0/d17/d20/f31 [151662,24084] 0 2024-05-25T06:11:24.200 INFO:tasks.workunit.client.1.smithi203.stdout:7/264: creat d1/d9/d2c/d3a/d4e/f61 x:0 0 0 2024-05-25T06:11:24.201 INFO:tasks.workunit.client.1.smithi203.stdout:7/265: dread - d1/d9/d2c/f5d zero size 2024-05-25T06:11:24.202 INFO:tasks.workunit.client.1.smithi203.stdout:9/194: mkdir d0/df/d2d/d45 0 2024-05-25T06:11:24.202 INFO:tasks.workunit.client.1.smithi203.stdout:9/195: chown d0/df/d2d/l30 19 1 2024-05-25T06:11:24.203 INFO:tasks.workunit.client.1.smithi203.stdout:7/266: mkdir d1/d9/d2c/d62 0 2024-05-25T06:11:24.206 INFO:tasks.workunit.client.1.smithi203.stdout:7/267: rename d1/d9/d2c/d3a/f44 to d1/d11/d3d/d4c/f63 0 2024-05-25T06:11:24.207 INFO:tasks.workunit.client.1.smithi203.stdout:9/196: getdents d0/df/d37 0 2024-05-25T06:11:24.207 INFO:tasks.workunit.client.1.smithi203.stdout:7/268: symlink d1/d11/d20/l64 0 2024-05-25T06:11:24.210 INFO:tasks.workunit.client.1.smithi203.stdout:7/269: rename d1/f3c to d1/d9/d60/f65 0 2024-05-25T06:11:24.212 INFO:tasks.workunit.client.1.smithi203.stdout:7/270: mkdir d1/d9/d2c/d36/d66 0 2024-05-25T06:11:24.213 INFO:tasks.workunit.client.1.smithi203.stdout:9/197: dwrite d0/d17/d20/f44 [0,4194304] 0 2024-05-25T06:11:24.214 INFO:tasks.workunit.client.1.smithi203.stdout:9/198: truncate d0/d17/d20/f31 565712 0 2024-05-25T06:11:24.215 INFO:tasks.workunit.client.1.smithi203.stdout:9/199: chown d0/f39 160672327 1 2024-05-25T06:11:24.218 INFO:tasks.workunit.client.1.smithi203.stdout:7/271: fsync d1/f1c 0 2024-05-25T06:11:24.219 INFO:tasks.workunit.client.1.smithi203.stdout:7/272: truncate d1/d11/f30 644990 0 2024-05-25T06:11:24.223 INFO:tasks.workunit.client.1.smithi203.stdout:7/273: link d1/d11/l2f d1/d9/d2c/d3a/d4e/l67 0 2024-05-25T06:11:24.225 INFO:tasks.workunit.client.1.smithi203.stdout:7/274: symlink d1/d11/l68 0 2024-05-25T06:11:24.227 INFO:tasks.workunit.client.1.smithi203.stdout:7/275: mknod d1/d11/d3d/c69 0 2024-05-25T06:11:24.229 INFO:tasks.workunit.client.1.smithi203.stdout:7/276: creat d1/d11/d3d/f6a x:0 0 0 2024-05-25T06:11:24.234 INFO:tasks.workunit.client.1.smithi203.stdout:7/277: dwrite d1/f3 [4194304,4194304] 0 2024-05-25T06:11:24.235 INFO:tasks.workunit.client.1.smithi203.stdout:9/200: dread d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:24.236 INFO:tasks.workunit.client.1.smithi203.stdout:7/278: mknod d1/d11/d20/c6b 0 2024-05-25T06:11:24.239 INFO:tasks.workunit.client.1.smithi203.stdout:7/279: creat d1/d9/d2c/d3a/d4e/f6c x:0 0 0 2024-05-25T06:11:24.239 INFO:tasks.workunit.client.1.smithi203.stdout:9/201: dread d0/f1e [0,4194304] 0 2024-05-25T06:11:24.239 INFO:tasks.workunit.client.1.smithi203.stdout:7/280: read d1/f38 [1737214,27973] 0 2024-05-25T06:11:24.241 INFO:tasks.workunit.client.1.smithi203.stdout:9/202: truncate d0/df/d2b/f34 249249 0 2024-05-25T06:11:24.241 INFO:tasks.workunit.client.1.smithi203.stdout:7/281: symlink d1/d9/d2c/d3a/l6d 0 2024-05-25T06:11:24.242 INFO:tasks.workunit.client.1.smithi203.stdout:9/203: creat d0/d17/d20/f46 x:0 0 0 2024-05-25T06:11:24.243 INFO:tasks.workunit.client.1.smithi203.stdout:7/282: read d1/d9/fe [1502667,71893] 0 2024-05-25T06:11:24.244 INFO:tasks.workunit.client.1.smithi203.stdout:9/204: mkdir d0/df/d47 0 2024-05-25T06:11:24.245 INFO:tasks.workunit.client.1.smithi203.stdout:7/283: creat d1/d11/d3d/f6e x:0 0 0 2024-05-25T06:11:24.246 INFO:tasks.workunit.client.1.smithi203.stdout:7/284: mknod d1/d9/d2c/d62/c6f 0 2024-05-25T06:11:24.250 INFO:tasks.workunit.client.1.smithi203.stdout:7/285: link d1/l1b d1/d9/d2c/d62/l70 0 2024-05-25T06:11:24.256 INFO:tasks.workunit.client.1.smithi203.stdout:7/286: dwrite d1/d9/f48 [0,4194304] 0 2024-05-25T06:11:24.258 INFO:tasks.workunit.client.1.smithi203.stdout:7/287: write d1/d9/d2c/f56 [2913376,91383] 0 2024-05-25T06:11:24.259 INFO:tasks.workunit.client.1.smithi203.stdout:7/288: chown d1/f4d 0 1 2024-05-25T06:11:24.262 INFO:tasks.workunit.client.1.smithi203.stdout:9/205: dread d0/f13 [0,4194304] 0 2024-05-25T06:11:24.264 INFO:tasks.workunit.client.1.smithi203.stdout:9/206: rmdir d0/d17/d20/d29 39 2024-05-25T06:11:24.264 INFO:tasks.workunit.client.1.smithi203.stdout:9/207: stat d0/d17/d20/d3c/l3d 0 2024-05-25T06:11:24.265 INFO:tasks.workunit.client.1.smithi203.stdout:9/208: fdatasync d0/df/d2b/f2f 0 2024-05-25T06:11:24.270 INFO:tasks.workunit.client.1.smithi203.stdout:9/209: dwrite d0/f19 [0,4194304] 0 2024-05-25T06:11:24.275 INFO:tasks.workunit.client.1.smithi203.stdout:9/210: rmdir d0/df/d47 0 2024-05-25T06:11:24.279 INFO:tasks.workunit.client.1.smithi203.stdout:9/211: creat d0/d17/d20/d29/f48 x:0 0 0 2024-05-25T06:11:24.282 INFO:tasks.workunit.client.1.smithi203.stdout:9/212: fsync d0/f4 0 2024-05-25T06:11:24.287 INFO:tasks.workunit.client.1.smithi203.stdout:9/213: truncate d0/f39 354252 0 2024-05-25T06:11:24.301 INFO:tasks.workunit.client.1.smithi203.stdout:9/214: dwrite d0/f4 [0,4194304] 0 2024-05-25T06:11:24.306 INFO:tasks.workunit.client.1.smithi203.stdout:9/215: link d0/f36 d0/df/d2d/d45/f49 0 2024-05-25T06:11:24.319 INFO:tasks.workunit.client.1.smithi203.stdout:9/216: dwrite d0/df/d2d/d45/f49 [0,4194304] 0 2024-05-25T06:11:24.327 INFO:tasks.workunit.client.1.smithi203.stdout:9/217: write d0/f39 [240593,69972] 0 2024-05-25T06:11:24.330 INFO:tasks.workunit.client.1.smithi203.stdout:9/218: creat d0/df/f4a x:0 0 0 2024-05-25T06:11:24.343 INFO:tasks.workunit.client.1.smithi203.stdout:9/219: dread d0/f24 [0,4194304] 0 2024-05-25T06:11:24.344 INFO:tasks.workunit.client.1.smithi203.stdout:9/220: dread d0/df/d2b/f34 [0,4194304] 0 2024-05-25T06:11:24.345 INFO:tasks.workunit.client.1.smithi203.stdout:9/221: rmdir d0/df/d2d/d45 39 2024-05-25T06:11:24.348 INFO:tasks.workunit.client.1.smithi203.stdout:9/222: rename d0/df/d2b/f34 to d0/d17/f4b 0 2024-05-25T06:11:24.353 INFO:tasks.workunit.client.1.smithi203.stdout:9/223: dwrite d0/f4 [0,4194304] 0 2024-05-25T06:11:24.353 INFO:tasks.workunit.client.1.smithi203.stdout:9/224: chown d0/f24 4 1 2024-05-25T06:11:24.354 INFO:tasks.workunit.client.1.smithi203.stdout:9/225: write d0/df/f1a [458130,95994] 0 2024-05-25T06:11:24.357 INFO:tasks.workunit.client.1.smithi203.stdout:9/226: write d0/fb [3720405,45679] 0 2024-05-25T06:11:24.371 INFO:tasks.workunit.client.1.smithi203.stdout:9/227: dwrite d0/d17/f4b [0,4194304] 0 2024-05-25T06:11:24.372 INFO:tasks.workunit.client.1.smithi203.stdout:9/228: chown d0/d17/d20/d29/c42 263960337 1 2024-05-25T06:11:24.376 INFO:tasks.workunit.client.1.smithi203.stdout:9/229: rename d0/df/d37/l3b to d0/d17/d20/d29/l4c 0 2024-05-25T06:11:24.376 INFO:tasks.workunit.client.1.smithi203.stdout:1/167: sync 2024-05-25T06:11:24.376 INFO:tasks.workunit.client.1.smithi203.stdout:3/294: sync 2024-05-25T06:11:24.377 INFO:tasks.workunit.client.1.smithi203.stdout:9/230: stat d0/d17/f4b 0 2024-05-25T06:11:24.378 INFO:tasks.workunit.client.1.smithi203.stdout:3/295: truncate d10/d16/d23/d45/f4e 312596 0 2024-05-25T06:11:24.379 INFO:tasks.workunit.client.1.smithi203.stdout:9/231: mknod d0/d17/d20/d29/c4d 0 2024-05-25T06:11:24.380 INFO:tasks.workunit.client.1.smithi203.stdout:3/296: creat d10/d16/d50/d62/f63 x:0 0 0 2024-05-25T06:11:24.381 INFO:tasks.workunit.client.1.smithi203.stdout:9/232: symlink d0/d17/d20/d29/l4e 0 2024-05-25T06:11:24.387 INFO:tasks.workunit.client.1.smithi203.stdout:9/233: dwrite d0/d17/d20/d29/f32 [0,4194304] 0 2024-05-25T06:11:24.388 INFO:tasks.workunit.client.1.smithi203.stdout:9/234: write d0/d17/f1b [3954694,32256] 0 2024-05-25T06:11:24.395 INFO:tasks.workunit.client.1.smithi203.stdout:9/235: dwrite d0/d17/f4b [4194304,4194304] 0 2024-05-25T06:11:24.396 INFO:tasks.workunit.client.1.smithi203.stdout:3/297: write d10/d16/f19 [854425,14313] 0 2024-05-25T06:11:24.397 INFO:tasks.workunit.client.1.smithi203.stdout:9/236: stat d0/d17/d23/l26 0 2024-05-25T06:11:24.398 INFO:tasks.workunit.client.1.smithi203.stdout:3/298: truncate d10/d22/f5e 310505 0 2024-05-25T06:11:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:23 smithi203 ceph-mon[36030]: pgmap v12: 65 pgs: 65 active+clean; 466 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 3.4 MiB/s rd, 40 MiB/s wr, 417 op/s 2024-05-25T06:11:24.400 INFO:tasks.workunit.client.1.smithi203.stdout:3/299: readlink d10/l15 0 2024-05-25T06:11:24.403 INFO:tasks.workunit.client.1.smithi203.stdout:9/237: dwrite d0/d17/d20/f46 [0,4194304] 0 2024-05-25T06:11:24.405 INFO:tasks.workunit.client.1.smithi203.stdout:9/238: write d0/d17/f1b [2088725,73090] 0 2024-05-25T06:11:24.406 INFO:tasks.workunit.client.1.smithi203.stdout:3/300: creat d10/d16/d50/f64 x:0 0 0 2024-05-25T06:11:24.407 INFO:tasks.workunit.client.1.smithi203.stdout:3/301: dread - d10/d16/d23/d46/f4b zero size 2024-05-25T06:11:24.409 INFO:tasks.workunit.client.1.smithi203.stdout:9/239: mkdir d0/df/d37/d4f 0 2024-05-25T06:11:24.410 INFO:tasks.workunit.client.1.smithi203.stdout:3/302: write d10/d16/d23/d4a/f61 [3061155,126242] 0 2024-05-25T06:11:24.410 INFO:tasks.workunit.client.1.smithi203.stdout:9/240: dread d0/f1e [0,4194304] 0 2024-05-25T06:11:24.411 INFO:tasks.workunit.client.1.smithi203.stdout:3/303: write d10/f1a [5038335,82543] 0 2024-05-25T06:11:24.412 INFO:tasks.workunit.client.1.smithi203.stdout:3/304: dread - d10/d16/d50/d62/f63 zero size 2024-05-25T06:11:24.413 INFO:tasks.workunit.client.1.smithi203.stdout:3/305: dread - d10/d22/f57 zero size 2024-05-25T06:11:24.413 INFO:tasks.workunit.client.1.smithi203.stdout:9/241: symlink d0/d17/d20/d29/l50 0 2024-05-25T06:11:24.414 INFO:tasks.workunit.client.1.smithi203.stdout:9/242: chown d0/d17/d20/f3a 15 1 2024-05-25T06:11:24.414 INFO:tasks.workunit.client.1.smithi203.stdout:3/306: write d10/d16/f40 [2854039,45802] 0 2024-05-25T06:11:24.416 INFO:tasks.workunit.client.1.smithi203.stdout:9/243: chown d0/d17/d20/d3c/l3d 2219 1 2024-05-25T06:11:24.416 INFO:tasks.workunit.client.1.smithi203.stdout:9/244: chown d0/d17/d20/d3c 1090777 1 2024-05-25T06:11:24.417 INFO:tasks.workunit.client.1.smithi203.stdout:3/307: creat d10/d16/d23/d3a/f65 x:0 0 0 2024-05-25T06:11:24.417 INFO:tasks.workunit.client.1.smithi203.stdout:9/245: chown d0/d17/d20/d3c/f43 1357665 1 2024-05-25T06:11:24.418 INFO:tasks.workunit.client.1.smithi203.stdout:3/308: write d10/d16/f44 [723855,128709] 0 2024-05-25T06:11:24.418 INFO:tasks.workunit.client.1.smithi203.stdout:9/246: readlink d0/df/l35 0 2024-05-25T06:11:24.419 INFO:tasks.workunit.client.1.smithi203.stdout:9/247: chown d0/d17 273 1 2024-05-25T06:11:24.421 INFO:tasks.workunit.client.1.smithi203.stdout:3/309: getdents d10/d16/d23/d45 0 2024-05-25T06:11:24.422 INFO:tasks.workunit.client.1.smithi203.stdout:9/248: creat d0/f51 x:0 0 0 2024-05-25T06:11:24.423 INFO:tasks.workunit.client.1.smithi203.stdout:9/249: chown d0/d17/d20/d29 195213 1 2024-05-25T06:11:24.424 INFO:tasks.workunit.client.1.smithi203.stdout:9/250: write d0/d17/d20/f44 [1712739,109184] 0 2024-05-25T06:11:24.428 INFO:tasks.workunit.client.1.smithi203.stdout:3/310: dwrite d10/d16/d23/d38/f3e [0,4194304] 0 2024-05-25T06:11:24.433 INFO:tasks.workunit.client.1.smithi203.stdout:3/311: dread d10/f31 [0,4194304] 0 2024-05-25T06:11:24.437 INFO:tasks.workunit.client.1.smithi203.stdout:3/312: unlink d10/d22/d2a/l2d 0 2024-05-25T06:11:24.439 INFO:tasks.workunit.client.1.smithi203.stdout:3/313: mkdir d10/d22/d2a/d66 0 2024-05-25T06:11:24.440 INFO:tasks.workunit.client.1.smithi203.stdout:3/314: chown d10/d16/f44 1034 1 2024-05-25T06:11:24.445 INFO:tasks.workunit.client.1.smithi203.stdout:3/315: fsync d10/d16/d23/d38/f3e 0 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:7/289: sync 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:2/316: sync 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:0/229: sync 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:5/471: sync 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:6/278: sync 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:4/214: sync 2024-05-25T06:11:24.463 INFO:tasks.workunit.client.1.smithi203.stdout:8/257: sync 2024-05-25T06:11:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:23 smithi160 ceph-mon[29041]: pgmap v12: 65 pgs: 65 active+clean; 466 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 3.4 MiB/s rd, 40 MiB/s wr, 417 op/s 2024-05-25T06:11:24.474 INFO:tasks.workunit.client.1.smithi203.stdout:4/215: fsync d0/d9/f17 0 2024-05-25T06:11:24.474 INFO:tasks.workunit.client.1.smithi203.stdout:3/316: mkdir d10/d16/d23/d67 0 2024-05-25T06:11:24.475 INFO:tasks.workunit.client.1.smithi203.stdout:8/258: symlink d2/l56 0 2024-05-25T06:11:24.477 INFO:tasks.workunit.client.1.smithi203.stdout:4/216: write d0/d8/d1f/d26/f1e [9186915,22086] 0 2024-05-25T06:11:24.477 INFO:tasks.workunit.client.1.smithi203.stdout:6/279: symlink d8/d29/d30/d38/l5c 0 2024-05-25T06:11:24.480 INFO:tasks.workunit.client.1.smithi203.stdout:0/230: rename l7 to dd/d3d/d4f/l55 0 2024-05-25T06:11:24.485 INFO:tasks.workunit.client.1.smithi203.stdout:6/280: symlink d8/d11/d15/d28/d52/l5d 0 2024-05-25T06:11:24.490 INFO:tasks.workunit.client.1.smithi203.stdout:3/317: dwrite d10/d16/d50/f64 [0,4194304] 0 2024-05-25T06:11:24.501 INFO:tasks.workunit.client.1.smithi203.stdout:5/472: dwrite dc/d11/d17/d1a/f2e [4194304,4194304] 0 2024-05-25T06:11:24.501 INFO:tasks.workunit.client.1.smithi203.stdout:3/318: mknod d10/c68 0 2024-05-25T06:11:24.502 INFO:tasks.workunit.client.1.smithi203.stdout:4/217: dwrite d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:24.503 INFO:tasks.workunit.client.1.smithi203.stdout:0/231: dwrite dd/f24 [0,4194304] 0 2024-05-25T06:11:24.504 INFO:tasks.workunit.client.1.smithi203.stdout:5/473: chown c7 13568251 1 2024-05-25T06:11:24.507 INFO:tasks.workunit.client.1.smithi203.stdout:7/290: dread f0 [4194304,4194304] 0 2024-05-25T06:11:24.510 INFO:tasks.workunit.client.1.smithi203.stdout:2/317: dwrite d2/d6/f26 [0,4194304] 0 2024-05-25T06:11:24.513 INFO:tasks.workunit.client.1.smithi203.stdout:7/291: dread - d1/d9/d2c/d3a/f5c zero size 2024-05-25T06:11:24.518 INFO:tasks.workunit.client.1.smithi203.stdout:0/232: dwrite dd/f1f [4194304,4194304] 0 2024-05-25T06:11:24.522 INFO:tasks.workunit.client.1.smithi203.stdout:2/318: dwrite d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:24.524 INFO:tasks.workunit.client.1.smithi203.stdout:5/474: rename dc/d11/d17/d45/l67 to dc/d11/d17/d1a/d75/la6 0 2024-05-25T06:11:24.524 INFO:tasks.workunit.client.1.smithi203.stdout:2/319: readlink d2/d4/l3b 0 2024-05-25T06:11:24.527 INFO:tasks.workunit.client.1.smithi203.stdout:2/320: write d2/d4/d10/d58/f6d [652778,86437] 0 2024-05-25T06:11:24.529 INFO:tasks.workunit.client.1.smithi203.stdout:7/292: rmdir d1/d9/d2c/d36/d3b/d59 0 2024-05-25T06:11:24.530 INFO:tasks.workunit.client.1.smithi203.stdout:0/233: chown dd/d3d/d4f/l55 16473601 1 2024-05-25T06:11:24.531 INFO:tasks.workunit.client.1.smithi203.stdout:0/234: chown dd/fe 150 1 2024-05-25T06:11:24.533 INFO:tasks.workunit.client.1.smithi203.stdout:2/321: truncate d2/d4/d10/d5f/f5b 425405 0 2024-05-25T06:11:24.537 INFO:tasks.workunit.client.1.smithi203.stdout:5/475: dread dc/f10 [0,4194304] 0 2024-05-25T06:11:24.539 INFO:tasks.workunit.client.1.smithi203.stdout:4/218: dwrite d0/d8/d10/f15 [0,4194304] 0 2024-05-25T06:11:24.543 INFO:tasks.workunit.client.1.smithi203.stdout:6/281: dread d8/d29/d30/d38/f39 [0,4194304] 0 2024-05-25T06:11:24.549 INFO:tasks.workunit.client.1.smithi203.stdout:7/293: dwrite d1/d9/d2c/d3a/d4e/f6c [0,4194304] 0 2024-05-25T06:11:24.552 INFO:tasks.workunit.client.1.smithi203.stdout:4/219: mknod d0/d8/d1f/d26/d3b/c3c 0 2024-05-25T06:11:24.554 INFO:tasks.workunit.client.1.smithi203.stdout:3/319: dread d10/f21 [0,4194304] 0 2024-05-25T06:11:24.555 INFO:tasks.workunit.client.1.smithi203.stdout:0/235: dwrite dd/d3a/f4b [0,4194304] 0 2024-05-25T06:11:24.557 INFO:tasks.workunit.client.1.smithi203.stdout:1/168: sync 2024-05-25T06:11:24.558 INFO:tasks.workunit.client.1.smithi203.stdout:4/220: write d0/f38 [385059,62439] 0 2024-05-25T06:11:24.558 INFO:tasks.workunit.client.1.smithi203.stdout:7/294: mknod d1/d9/d2c/d3a/d4e/c71 0 2024-05-25T06:11:24.564 INFO:tasks.workunit.client.1.smithi203.stdout:2/322: dwrite d2/d4/d10/d32/f28 [4194304,4194304] 0 2024-05-25T06:11:24.567 INFO:tasks.workunit.client.1.smithi203.stdout:3/320: chown d10/d16/d23/d38/l59 492287658 1 2024-05-25T06:11:24.567 INFO:tasks.workunit.client.1.smithi203.stdout:5/476: dwrite dc/d11/d17/d1a/f2e [8388608,4194304] 0 2024-05-25T06:11:24.569 INFO:tasks.workunit.client.0.smithi160.stderr:++ readlink -f fsstress 2024-05-25T06:11:24.570 INFO:tasks.workunit.client.1.smithi203.stdout:0/236: write dd/d32/d37/f42 [1406874,113231] 0 2024-05-25T06:11:24.570 INFO:tasks.workunit.client.1.smithi203.stdout:1/169: mknod d14/d23/c38 0 2024-05-25T06:11:24.570 INFO:tasks.workunit.client.1.smithi203.stdout:7/295: symlink d1/d9/d2c/d3a/d4e/l72 0 2024-05-25T06:11:24.571 INFO:tasks.workunit.client.0.smithi160.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-25T06:11:24.571 INFO:tasks.workunit.client.0.smithi160.stderr:+ popd 2024-05-25T06:11:24.572 INFO:tasks.workunit.client.1.smithi203.stdout:4/221: symlink d0/d9/d24/d27/l3d 0 2024-05-25T06:11:24.573 INFO:tasks.workunit.client.0.smithi160.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-25T06:11:24.573 INFO:tasks.workunit.client.0.smithi160.stderr:+ popd 2024-05-25T06:11:24.574 INFO:tasks.workunit.client.0.smithi160.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-25T06:11:24.575 INFO:tasks.workunit.client.1.smithi203.stdout:6/282: dwrite f1 [0,4194304] 0 2024-05-25T06:11:24.575 INFO:tasks.workunit.client.0.smithi160.stderr:++ mktemp -d -p . 2024-05-25T06:11:24.576 INFO:tasks.workunit.client.1.smithi203.stdout:7/296: write d1/d11/d20/f23 [1084239,21435] 0 2024-05-25T06:11:24.576 INFO:tasks.workunit.client.1.smithi203.stdout:3/321: dread - d10/d16/d50/d62/f63 zero size 2024-05-25T06:11:24.579 INFO:tasks.workunit.client.1.smithi203.stdout:4/222: write d0/d8/d1f/d26/f20 [2317,120702] 0 2024-05-25T06:11:24.581 INFO:tasks.workunit.client.1.smithi203.stdout:6/283: read d8/d11/d15/d28/d48/f57 [68059,66658] 0 2024-05-25T06:11:24.583 INFO:tasks.workunit.client.1.smithi203.stdout:7/297: fsync d1/d9/d2c/d36/f40 0 2024-05-25T06:11:24.584 INFO:tasks.workunit.client.1.smithi203.stdout:4/223: write d0/f38 [1076031,36274] 0 2024-05-25T06:11:24.584 INFO:tasks.workunit.client.1.smithi203.stdout:7/298: chown d1/f4d 32863 1 2024-05-25T06:11:24.585 INFO:tasks.workunit.client.1.smithi203.stdout:7/299: dread - d1/d9/d2c/d3a/f4b zero size 2024-05-25T06:11:24.586 INFO:tasks.workunit.client.1.smithi203.stdout:1/170: creat d14/d1b/d1d/d31/d36/f39 x:0 0 0 2024-05-25T06:11:24.587 INFO:tasks.workunit.client.1.smithi203.stdout:5/477: rename dc/d3c/d8b/l9c to dc/d11/d17/d1a/d2d/la7 0 2024-05-25T06:11:24.590 INFO:tasks.workunit.client.1.smithi203.stdout:5/478: dread - dc/d11/d17/d44/d73/d98/f9f zero size 2024-05-25T06:11:24.590 INFO:tasks.workunit.client.1.smithi203.stdout:6/284: creat d8/d11/d15/d28/f5e x:0 0 0 2024-05-25T06:11:24.591 INFO:tasks.workunit.client.1.smithi203.stdout:3/322: mknod d10/d16/d23/c69 0 2024-05-25T06:11:24.592 INFO:tasks.workunit.client.1.smithi203.stdout:5/479: write dc/d11/d17/d45/f5c [3021643,126351] 0 2024-05-25T06:11:24.593 INFO:tasks.workunit.client.1.smithi203.stdout:6/285: symlink d8/d11/d15/d28/d48/l5f 0 2024-05-25T06:11:24.594 INFO:tasks.workunit.client.1.smithi203.stdout:7/300: symlink d1/l73 0 2024-05-25T06:11:24.595 INFO:tasks.workunit.client.1.smithi203.stdout:7/301: fdatasync d1/d11/d3d/f6e 0 2024-05-25T06:11:24.595 INFO:tasks.workunit.client.1.smithi203.stdout:3/323: mknod d10/d22/c6a 0 2024-05-25T06:11:24.596 INFO:tasks.workunit.client.1.smithi203.stdout:7/302: fdatasync d1/d9/f2e 0 2024-05-25T06:11:24.596 INFO:tasks.workunit.client.1.smithi203.stdout:1/171: mknod d14/c3a 0 2024-05-25T06:11:24.597 INFO:tasks.workunit.client.1.smithi203.stdout:5/480: creat dc/d11/d42/d94/fa8 x:0 0 0 2024-05-25T06:11:24.597 INFO:tasks.workunit.client.1.smithi203.stdout:6/286: rmdir d8/d11/d15/d28/d52 39 2024-05-25T06:11:24.598 INFO:tasks.workunit.client.1.smithi203.stdout:5/481: chown dc/d11/d17/d1a/d75 615 1 2024-05-25T06:11:24.599 INFO:tasks.workunit.client.1.smithi203.stdout:3/324: write d10/d22/d2a/f3b [3423573,95606] 0 2024-05-25T06:11:24.600 INFO:tasks.workunit.client.1.smithi203.stdout:7/303: symlink d1/d9/l74 0 2024-05-25T06:11:24.602 INFO:tasks.workunit.client.1.smithi203.stdout:5/482: creat dc/d11/d42/fa9 x:0 0 0 2024-05-25T06:11:24.604 INFO:tasks.workunit.client.1.smithi203.stdout:7/304: chown d1/d11/d20/f23 63300 1 2024-05-25T06:11:24.608 INFO:tasks.workunit.client.1.smithi203.stdout:1/172: dwrite d14/d1b/d1d/f2f [0,4194304] 0 2024-05-25T06:11:24.608 INFO:tasks.workunit.client.1.smithi203.stdout:3/325: link d10/d22/d2a/l41 d10/d22/d2a/d66/l6b 0 2024-05-25T06:11:24.608 INFO:tasks.workunit.client.1.smithi203.stdout:6/287: dwrite d8/f9 [0,4194304] 0 2024-05-25T06:11:24.609 INFO:tasks.workunit.client.1.smithi203.stdout:7/305: creat d1/d9/d2c/d3a/d4e/f75 x:0 0 0 2024-05-25T06:11:24.611 INFO:tasks.workunit.client.1.smithi203.stdout:1/173: mknod d14/d23/c3b 0 2024-05-25T06:11:24.611 INFO:tasks.workunit.client.1.smithi203.stdout:7/306: dread - d1/d9/d2c/d3a/f4f zero size 2024-05-25T06:11:24.612 INFO:tasks.workunit.client.1.smithi203.stdout:5/483: truncate dc/d11/d17/d1a/d22/f39 1358761 0 2024-05-25T06:11:24.613 INFO:tasks.workunit.client.1.smithi203.stdout:1/174: write d14/f25 [2351804,42786] 0 2024-05-25T06:11:24.614 INFO:tasks.workunit.client.1.smithi203.stdout:5/484: readlink dc/d11/d17/d1a/d22/l20 0 2024-05-25T06:11:24.615 INFO:tasks.workunit.client.0.smithi160.stderr:+ T=./tmp.bD4KoTPC4G 2024-05-25T06:11:24.615 INFO:tasks.workunit.client.0.smithi160.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.bD4KoTPC4G -l 1 -n 1000 -p 10 -v 2024-05-25T06:11:24.615 INFO:tasks.workunit.client.1.smithi203.stdout:7/307: chown d1/d9/c1e 15561 1 2024-05-25T06:11:24.616 INFO:tasks.workunit.client.1.smithi203.stdout:5/485: write dc/d11/d17/d1a/d22/f86 [1570941,115772] 0 2024-05-25T06:11:24.616 INFO:tasks.workunit.client.1.smithi203.stdout:7/308: chown d1/d9/d2c/d3a/d4e 216430 1 2024-05-25T06:11:24.617 INFO:tasks.workunit.client.1.smithi203.stdout:7/309: write d1/d9/f2e [828571,65504] 0 2024-05-25T06:11:24.622 INFO:tasks.workunit.client.1.smithi203.stdout:3/326: rename d10/d16/d50/d62 to d10/d16/d23/d6c 0 2024-05-25T06:11:24.623 INFO:tasks.workunit.client.0.smithi160.stdout:seed = 1717045044 2024-05-25T06:11:24.626 INFO:tasks.workunit.client.1.smithi203.stdout:6/288: rename d8/d11/d15/d28/d48 to d8/d11/d47/d60 0 2024-05-25T06:11:24.627 INFO:tasks.workunit.client.1.smithi203.stdout:3/327: mkdir d10/d16/d23/d4a/d6d 0 2024-05-25T06:11:24.628 INFO:tasks.workunit.client.0.smithi160.stdout:0/0: chown . 9711 1 2024-05-25T06:11:24.628 INFO:tasks.workunit.client.0.smithi160.stdout:0/1: chown . 3834403 1 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/2: rename - no filename 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/3: truncate - no filename 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/4: rmdir - no directory 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/5: write - no filename 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/6: chown . 96 1 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/7: link - no file 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/8: write - no filename 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/9: rename - no filename 2024-05-25T06:11:24.629 INFO:tasks.workunit.client.1.smithi203.stdout:1/175: dread d14/d1b/d29/f2b [0,4194304] 0 2024-05-25T06:11:24.630 INFO:tasks.workunit.client.0.smithi160.stdout:1/0: chown . 54717 1 2024-05-25T06:11:24.630 INFO:tasks.workunit.client.0.smithi160.stdout:1/1: fdatasync - no filename 2024-05-25T06:11:24.630 INFO:tasks.workunit.client.0.smithi160.stdout:1/2: dread - no filename 2024-05-25T06:11:24.631 INFO:tasks.workunit.client.1.smithi203.stdout:5/486: rename dc/d11/d18/d8e to dc/daa 0 2024-05-25T06:11:24.632 INFO:tasks.workunit.client.1.smithi203.stdout:7/310: dwrite d1/f32 [0,4194304] 0 2024-05-25T06:11:24.632 INFO:tasks.workunit.client.0.smithi160.stdout:1/3: symlink l0 0 2024-05-25T06:11:24.632 INFO:tasks.workunit.client.0.smithi160.stdout:1/4: chown l0 839 1 2024-05-25T06:11:24.632 INFO:tasks.workunit.client.0.smithi160.stdout:1/5: write - no filename 2024-05-25T06:11:24.633 INFO:tasks.workunit.client.0.smithi160.stdout:1/6: read - no filename 2024-05-25T06:11:24.633 INFO:tasks.workunit.client.1.smithi203.stdout:1/176: truncate d14/d1b/d1d/d31/d36/f39 937456 0 2024-05-25T06:11:24.635 INFO:tasks.workunit.client.1.smithi203.stdout:1/177: chown d14/d1b/d29/l2e 3254012 1 2024-05-25T06:11:24.635 INFO:tasks.workunit.client.0.smithi160.stdout:1/7: creat f1 x:0 0 0 2024-05-25T06:11:24.635 INFO:tasks.workunit.client.1.smithi203.stdout:1/178: stat d14 0 2024-05-25T06:11:24.636 INFO:tasks.workunit.client.1.smithi203.stdout:7/311: read d1/d11/d20/f27 [186338,2931] 0 2024-05-25T06:11:24.637 INFO:tasks.workunit.client.0.smithi160.stdout:4/0: dread - no filename 2024-05-25T06:11:24.637 INFO:tasks.workunit.client.0.smithi160.stdout:4/1: chown . 11567586 1 2024-05-25T06:11:24.637 INFO:tasks.workunit.client.0.smithi160.stdout:1/8: creat f2 x:0 0 0 2024-05-25T06:11:24.638 INFO:tasks.workunit.client.1.smithi203.stdout:1/179: mknod d14/d1b/d29/c3c 0 2024-05-25T06:11:24.638 INFO:tasks.workunit.client.1.smithi203.stdout:1/180: stat d14/f24 0 2024-05-25T06:11:24.638 INFO:tasks.workunit.client.0.smithi160.stdout:1/9: write f2 [551766,1355] 0 2024-05-25T06:11:24.639 INFO:tasks.workunit.client.0.smithi160.stdout:2/0: creat f0 x:0 0 0 2024-05-25T06:11:24.639 INFO:tasks.workunit.client.1.smithi203.stdout:3/328: creat d10/d16/d23/f6e x:0 0 0 2024-05-25T06:11:24.639 INFO:tasks.workunit.client.1.smithi203.stdout:5/487: creat dc/d11/fab x:0 0 0 2024-05-25T06:11:24.640 INFO:tasks.workunit.client.0.smithi160.stdout:5/0: rename - no filename 2024-05-25T06:11:24.640 INFO:tasks.workunit.client.0.smithi160.stdout:2/1: write f0 [480488,122516] 0 2024-05-25T06:11:24.641 INFO:tasks.workunit.client.1.smithi203.stdout:3/329: fsync d10/d16/d23/d6c/f63 0 2024-05-25T06:11:24.641 INFO:tasks.workunit.client.1.smithi203.stdout:1/181: dread - d14/d1b/d1d/f30 zero size 2024-05-25T06:11:24.641 INFO:tasks.workunit.client.1.smithi203.stdout:5/488: readlink dc/d6a/l97 0 2024-05-25T06:11:24.641 INFO:tasks.workunit.client.0.smithi160.stdout:1/10: creat f3 x:0 0 0 2024-05-25T06:11:24.642 INFO:tasks.workunit.client.0.smithi160.stdout:4/2: mknod c0 0 2024-05-25T06:11:24.642 INFO:tasks.workunit.client.0.smithi160.stdout:2/2: chown f0 26228575 1 2024-05-25T06:11:24.643 INFO:tasks.workunit.client.0.smithi160.stdout:4/3: creat f1 x:0 0 0 2024-05-25T06:11:24.644 INFO:tasks.workunit.client.1.smithi203.stdout:1/182: symlink d14/l3d 0 2024-05-25T06:11:24.644 INFO:tasks.workunit.client.0.smithi160.stdout:5/1: symlink l0 0 2024-05-25T06:11:24.644 INFO:tasks.workunit.client.0.smithi160.stdout:4/4: read - f1 zero size 2024-05-25T06:11:24.647 INFO:tasks.workunit.client.0.smithi160.stdout:6/0: symlink l0 0 2024-05-25T06:11:24.647 INFO:tasks.workunit.client.0.smithi160.stdout:6/1: read - no filename 2024-05-25T06:11:24.647 INFO:tasks.workunit.client.0.smithi160.stdout:6/2: dwrite - no filename 2024-05-25T06:11:24.648 INFO:tasks.workunit.client.1.smithi203.stdout:5/489: symlink dc/d11/d18/lac 0 2024-05-25T06:11:24.648 INFO:tasks.workunit.client.1.smithi203.stdout:7/312: getdents d1/d11/d3d/d4c 0 2024-05-25T06:11:24.649 INFO:tasks.workunit.client.1.smithi203.stdout:7/313: read - d1/fa zero size 2024-05-25T06:11:24.651 INFO:tasks.workunit.client.1.smithi203.stdout:3/330: creat d10/d16/d23/d67/f6f x:0 0 0 2024-05-25T06:11:24.652 INFO:tasks.workunit.client.0.smithi160.stdout:6/3: rename l0 to l1 0 2024-05-25T06:11:24.654 INFO:tasks.workunit.client.1.smithi203.stdout:1/183: dwrite d14/d1b/d1d/f2f [0,4194304] 0 2024-05-25T06:11:24.654 INFO:tasks.workunit.client.0.smithi160.stdout:7/0: mknod c0 0 2024-05-25T06:11:24.655 INFO:tasks.workunit.client.0.smithi160.stdout:7/1: read - no filename 2024-05-25T06:11:24.655 INFO:tasks.workunit.client.1.smithi203.stdout:3/331: read f8 [3876366,68740] 0 2024-05-25T06:11:24.659 INFO:tasks.workunit.client.0.smithi160.stdout:7/2: creat f1 x:0 0 0 2024-05-25T06:11:24.660 INFO:tasks.workunit.client.0.smithi160.stdout:8/0: mknod c0 0 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/0: getdents . 0 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/1: readlink - no filename 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/2: unlink - no file 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/3: link - no file 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/4: dread - no filename 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/5: dwrite - no filename 2024-05-25T06:11:24.661 INFO:tasks.workunit.client.0.smithi160.stdout:7/3: rename c0 to c2 0 2024-05-25T06:11:24.662 INFO:tasks.workunit.client.0.smithi160.stdout:7/4: rmdir - no directory 2024-05-25T06:11:24.662 INFO:tasks.workunit.client.1.smithi203.stdout:3/332: creat d10/d16/d50/f70 x:0 0 0 2024-05-25T06:11:24.662 INFO:tasks.workunit.client.1.smithi203.stdout:3/333: write d10/d16/d23/d45/f4e [287114,6573] 0 2024-05-25T06:11:24.663 INFO:tasks.workunit.client.0.smithi160.stdout:7/5: chown f1 183468 1 2024-05-25T06:11:24.663 INFO:tasks.workunit.client.0.smithi160.stdout:8/1: mknod c1 0 2024-05-25T06:11:24.663 INFO:tasks.workunit.client.0.smithi160.stdout:8/2: rmdir - no directory 2024-05-25T06:11:24.663 INFO:tasks.workunit.client.0.smithi160.stdout:8/3: dwrite - no filename 2024-05-25T06:11:24.663 INFO:tasks.workunit.client.0.smithi160.stdout:8/4: write - no filename 2024-05-25T06:11:24.663 INFO:tasks.workunit.client.1.smithi203.stdout:3/334: chown d10/d16/c1e 103 1 2024-05-25T06:11:24.664 INFO:tasks.workunit.client.1.smithi203.stdout:6/289: truncate d8/d11/d15/f37 3603492 0 2024-05-25T06:11:24.664 INFO:tasks.workunit.client.1.smithi203.stdout:3/335: read - d10/d22/f33 zero size 2024-05-25T06:11:24.664 INFO:tasks.workunit.client.1.smithi203.stdout:3/336: readlink l9 0 2024-05-25T06:11:24.664 INFO:tasks.workunit.client.0.smithi160.stdout:9/6: symlink l0 0 2024-05-25T06:11:24.665 INFO:tasks.workunit.client.0.smithi160.stdout:9/7: write - no filename 2024-05-25T06:11:24.665 INFO:tasks.workunit.client.0.smithi160.stdout:9/8: write - no filename 2024-05-25T06:11:24.665 INFO:tasks.workunit.client.0.smithi160.stdout:7/6: mknod c3 0 2024-05-25T06:11:24.665 INFO:tasks.workunit.client.1.smithi203.stdout:7/314: creat d1/d9/f76 x:0 0 0 2024-05-25T06:11:24.666 INFO:tasks.workunit.client.1.smithi203.stdout:3/337: fsync d10/d22/f42 0 2024-05-25T06:11:24.666 INFO:tasks.workunit.client.1.smithi203.stdout:7/315: write d1/d9/f76 [806009,110364] 0 2024-05-25T06:11:24.666 INFO:tasks.workunit.client.0.smithi160.stdout:9/9: rename l0 to l1 0 2024-05-25T06:11:24.667 INFO:tasks.workunit.client.0.smithi160.stdout:8/5: mknod c2 0 2024-05-25T06:11:24.667 INFO:tasks.workunit.client.0.smithi160.stdout:8/6: write - no filename 2024-05-25T06:11:24.669 INFO:tasks.workunit.client.1.smithi203.stdout:6/290: write d8/d11/d15/f2e [533283,47093] 0 2024-05-25T06:11:24.669 INFO:tasks.workunit.client.1.smithi203.stdout:7/316: symlink d1/d11/d3d/l77 0 2024-05-25T06:11:24.670 INFO:tasks.workunit.client.0.smithi160.stdout:8/7: creat f3 x:0 0 0 2024-05-25T06:11:24.670 INFO:tasks.workunit.client.0.smithi160.stdout:8/8: rmdir - no directory 2024-05-25T06:11:24.673 INFO:tasks.workunit.client.0.smithi160.stdout:8/9: symlink l4 0 2024-05-25T06:11:24.674 INFO:tasks.workunit.client.1.smithi203.stdout:6/291: mknod d8/d11/d15/d28/d52/d43/c61 0 2024-05-25T06:11:24.675 INFO:tasks.workunit.client.0.smithi160.stdout:8/10: mkdir d5 0 2024-05-25T06:11:24.675 INFO:tasks.workunit.client.1.smithi203.stdout:7/317: dread d1/f32 [0,4194304] 0 2024-05-25T06:11:24.677 INFO:tasks.workunit.client.0.smithi160.stdout:8/11: mknod d5/c6 0 2024-05-25T06:11:24.678 INFO:tasks.workunit.client.1.smithi203.stdout:5/490: truncate dc/d3c/f78 1101651 0 2024-05-25T06:11:24.678 INFO:tasks.workunit.client.0.smithi160.stdout:8/12: read - f3 zero size 2024-05-25T06:11:24.680 INFO:tasks.workunit.client.1.smithi203.stdout:5/491: chown dc/d11/d17/d45 27 1 2024-05-25T06:11:24.682 INFO:tasks.workunit.client.1.smithi203.stdout:1/184: truncate d14/d1b/d1d/d31/d36/f39 619188 0 2024-05-25T06:11:24.682 INFO:tasks.workunit.client.1.smithi203.stdout:7/318: creat d1/f78 x:0 0 0 2024-05-25T06:11:24.684 INFO:tasks.workunit.client.1.smithi203.stdout:6/292: dwrite d8/fd [4194304,4194304] 0 2024-05-25T06:11:24.689 INFO:tasks.workunit.client.1.smithi203.stdout:7/319: write d1/d9/d2c/f42 [4333315,80859] 0 2024-05-25T06:11:24.690 INFO:tasks.workunit.client.1.smithi203.stdout:3/338: dread d10/d16/d23/d3a/f5d [0,4194304] 0 2024-05-25T06:11:24.691 INFO:tasks.workunit.client.1.smithi203.stdout:1/185: rename d14/d1b/d1d/d31 to d14/d1b/d1d/d3e 0 2024-05-25T06:11:24.692 INFO:tasks.workunit.client.1.smithi203.stdout:6/293: mkdir d8/d11/d47/d62 0 2024-05-25T06:11:24.693 INFO:tasks.workunit.client.1.smithi203.stdout:7/320: fsync d1/d11/d3d/d4c/f63 0 2024-05-25T06:11:24.693 INFO:tasks.workunit.client.1.smithi203.stdout:3/339: write d10/d22/d2a/f4d [1888381,104190] 0 2024-05-25T06:11:24.694 INFO:tasks.workunit.client.1.smithi203.stdout:3/340: readlink d10/d16/d23/l37 0 2024-05-25T06:11:24.697 INFO:tasks.workunit.client.0.smithi160.stdout:1/11: fsync f1 0 2024-05-25T06:11:24.697 INFO:tasks.workunit.client.1.smithi203.stdout:1/186: mkdir d14/d1b/d1d/d3f 0 2024-05-25T06:11:24.699 INFO:tasks.workunit.client.0.smithi160.stdout:1/12: mknod c4 0 2024-05-25T06:11:24.699 INFO:tasks.workunit.client.1.smithi203.stdout:7/321: mkdir d1/d9/d2c/d79 0 2024-05-25T06:11:24.701 INFO:tasks.workunit.client.1.smithi203.stdout:6/294: fsync d8/f12 0 2024-05-25T06:11:24.702 INFO:tasks.workunit.client.1.smithi203.stdout:1/187: creat d14/d1b/d1d/f40 x:0 0 0 2024-05-25T06:11:24.702 INFO:tasks.workunit.client.1.smithi203.stdout:7/322: mkdir d1/d9/d2c/d62/d7a 0 2024-05-25T06:11:24.703 INFO:tasks.workunit.client.1.smithi203.stdout:1/188: write f10 [11423689,130355] 0 2024-05-25T06:11:24.704 INFO:tasks.workunit.client.0.smithi160.stdout:1/13: unlink f3 0 2024-05-25T06:11:24.705 INFO:tasks.workunit.client.0.smithi160.stdout:4/5: fsync f1 0 2024-05-25T06:11:24.705 INFO:tasks.workunit.client.1.smithi203.stdout:3/341: rename d10/d22/d2a/d66 to d10/d16/d71 0 2024-05-25T06:11:24.706 INFO:tasks.workunit.client.0.smithi160.stdout:4/6: dread - f1 zero size 2024-05-25T06:11:24.706 INFO:tasks.workunit.client.0.smithi160.stdout:5/2: getdents . 0 2024-05-25T06:11:24.706 INFO:tasks.workunit.client.0.smithi160.stdout:5/3: dwrite - no filename 2024-05-25T06:11:24.707 INFO:tasks.workunit.client.0.smithi160.stdout:5/4: fsync - no filename 2024-05-25T06:11:24.707 INFO:tasks.workunit.client.1.smithi203.stdout:6/295: unlink d8/l51 0 2024-05-25T06:11:24.707 INFO:tasks.workunit.client.1.smithi203.stdout:5/492: dread dc/d11/d17/d44/f51 [0,4194304] 0 2024-05-25T06:11:24.709 INFO:tasks.workunit.client.0.smithi160.stdout:1/14: dread f2 [0,4194304] 0 2024-05-25T06:11:24.709 INFO:tasks.workunit.client.1.smithi203.stdout:7/323: symlink d1/d9/d2c/d3a/d4e/d50/l7b 0 2024-05-25T06:11:24.710 INFO:tasks.workunit.client.0.smithi160.stdout:5/5: creat f1 x:0 0 0 2024-05-25T06:11:24.710 INFO:tasks.workunit.client.0.smithi160.stdout:1/15: chown f1 2506546 1 2024-05-25T06:11:24.712 INFO:tasks.workunit.client.1.smithi203.stdout:6/296: rename d8/d54 to d8/d2a/d63 0 2024-05-25T06:11:24.713 INFO:tasks.workunit.client.0.smithi160.stdout:6/4: chown l1 92 1 2024-05-25T06:11:24.713 INFO:tasks.workunit.client.0.smithi160.stdout:6/5: read - no filename 2024-05-25T06:11:24.713 INFO:tasks.workunit.client.0.smithi160.stdout:5/6: rename l0 to l2 0 2024-05-25T06:11:24.714 INFO:tasks.workunit.client.1.smithi203.stdout:5/493: mkdir dc/d11/d42/d94/dad 0 2024-05-25T06:11:24.714 INFO:tasks.workunit.client.0.smithi160.stdout:5/7: dread - f1 zero size 2024-05-25T06:11:24.715 INFO:tasks.workunit.client.0.smithi160.stdout:4/7: dwrite f1 [0,4194304] 0 2024-05-25T06:11:24.716 INFO:tasks.workunit.client.1.smithi203.stdout:3/342: rename d10/d16/d23/d38/c3f to d10/d16/d23/d46/c72 0 2024-05-25T06:11:24.716 INFO:tasks.workunit.client.0.smithi160.stdout:4/8: chown f1 5383183 1 2024-05-25T06:11:24.717 INFO:tasks.workunit.client.0.smithi160.stdout:6/6: creat f2 x:0 0 0 2024-05-25T06:11:24.718 INFO:tasks.workunit.client.1.smithi203.stdout:5/494: creat dc/d3c/d3f/fae x:0 0 0 2024-05-25T06:11:24.719 INFO:tasks.workunit.client.0.smithi160.stdout:1/16: link f2 f5 0 2024-05-25T06:11:24.720 INFO:tasks.workunit.client.1.smithi203.stdout:7/324: rename d1/d11/d3d/c69 to d1/d9/d2c/d79/c7c 0 2024-05-25T06:11:24.722 INFO:tasks.workunit.client.0.smithi160.stdout:7/7: getdents . 0 2024-05-25T06:11:24.723 INFO:tasks.workunit.client.1.smithi203.stdout:3/343: symlink d10/d22/l73 0 2024-05-25T06:11:24.723 INFO:tasks.workunit.client.0.smithi160.stdout:6/7: mknod c3 0 2024-05-25T06:11:24.726 INFO:tasks.workunit.client.1.smithi203.stdout:7/325: mknod d1/d9/d2c/d62/c7d 0 2024-05-25T06:11:24.727 INFO:tasks.workunit.client.0.smithi160.stdout:1/17: symlink l6 0 2024-05-25T06:11:24.727 INFO:tasks.workunit.client.1.smithi203.stdout:6/297: dwrite d8/d29/d30/d38/f39 [0,4194304] 0 2024-05-25T06:11:24.728 INFO:tasks.workunit.client.0.smithi160.stdout:8/13: fsync f3 0 2024-05-25T06:11:24.728 INFO:tasks.workunit.client.0.smithi160.stdout:1/18: dread - f1 zero size 2024-05-25T06:11:24.729 INFO:tasks.workunit.client.1.smithi203.stdout:3/344: symlink d10/d16/d23/d3a/d48/l74 0 2024-05-25T06:11:24.729 INFO:tasks.workunit.client.1.smithi203.stdout:6/298: fsync d8/d11/d15/f18 0 2024-05-25T06:11:24.729 INFO:tasks.workunit.client.0.smithi160.stdout:6/8: rename l1 to l4 0 2024-05-25T06:11:24.730 INFO:tasks.workunit.client.1.smithi203.stdout:3/345: fsync d10/d16/f44 0 2024-05-25T06:11:24.730 INFO:tasks.workunit.client.0.smithi160.stdout:8/14: truncate f3 678702 0 2024-05-25T06:11:24.730 INFO:tasks.workunit.client.0.smithi160.stdout:7/8: dwrite f1 [0,4194304] 0 2024-05-25T06:11:24.733 INFO:tasks.workunit.client.0.smithi160.stdout:9/10: chown l1 309011042 1 2024-05-25T06:11:24.735 INFO:tasks.workunit.client.0.smithi160.stdout:1/19: creat f7 x:0 0 0 2024-05-25T06:11:24.737 INFO:tasks.workunit.client.0.smithi160.stdout:6/9: mkdir d5 0 2024-05-25T06:11:24.739 INFO:tasks.workunit.client.0.smithi160.stdout:6/10: rename d5 to d5/d6 22 2024-05-25T06:11:24.739 INFO:tasks.workunit.client.1.smithi203.stdout:7/326: symlink d1/l7e 0 2024-05-25T06:11:24.741 INFO:tasks.workunit.client.0.smithi160.stdout:7/9: dread f1 [0,4194304] 0 2024-05-25T06:11:24.743 INFO:tasks.workunit.client.0.smithi160.stdout:8/15: dwrite f3 [0,4194304] 0 2024-05-25T06:11:24.744 INFO:tasks.workunit.client.0.smithi160.stdout:9/11: creat f2 x:0 0 0 2024-05-25T06:11:24.744 INFO:tasks.workunit.client.0.smithi160.stdout:4/9: write f1 [4470064,109265] 0 2024-05-25T06:11:24.745 INFO:tasks.workunit.client.1.smithi203.stdout:7/327: creat d1/d11/d3d/f7f x:0 0 0 2024-05-25T06:11:24.746 INFO:tasks.workunit.client.0.smithi160.stdout:1/20: write f2 [643615,18943] 0 2024-05-25T06:11:24.746 INFO:tasks.workunit.client.0.smithi160.stdout:8/16: write f3 [476867,19195] 0 2024-05-25T06:11:24.746 INFO:tasks.workunit.client.0.smithi160.stdout:9/12: chown f2 1979 1 2024-05-25T06:11:24.746 INFO:tasks.workunit.client.0.smithi160.stdout:9/13: rmdir - no directory 2024-05-25T06:11:24.750 INFO:tasks.workunit.client.0.smithi160.stdout:9/14: write f2 [22390,115304] 0 2024-05-25T06:11:24.751 INFO:tasks.workunit.client.0.smithi160.stdout:5/8: chown l2 387012 1 2024-05-25T06:11:24.751 INFO:tasks.workunit.client.1.smithi203.stdout:3/346: getdents d10/d16/d23/d3a 0 2024-05-25T06:11:24.752 INFO:tasks.workunit.client.0.smithi160.stdout:5/9: truncate f1 940422 0 2024-05-25T06:11:24.752 INFO:tasks.workunit.client.1.smithi203.stdout:1/189: write d14/d1b/d1d/d3e/d36/f39 [77911,65008] 0 2024-05-25T06:11:24.754 INFO:tasks.workunit.client.0.smithi160.stdout:7/10: mknod c4 0 2024-05-25T06:11:24.755 INFO:tasks.workunit.client.1.smithi203.stdout:3/347: creat d10/d22/d2a/f75 x:0 0 0 2024-05-25T06:11:24.755 INFO:tasks.workunit.client.1.smithi203.stdout:5/495: truncate dc/d11/d17/f31 3004889 0 2024-05-25T06:11:24.757 INFO:tasks.workunit.client.1.smithi203.stdout:1/190: mknod d14/d1b/c41 0 2024-05-25T06:11:24.758 INFO:tasks.workunit.client.1.smithi203.stdout:6/299: truncate d8/d11/d15/f18 131474 0 2024-05-25T06:11:24.759 INFO:tasks.workunit.client.1.smithi203.stdout:3/348: creat d10/d16/d23/d45/f76 x:0 0 0 2024-05-25T06:11:24.759 INFO:tasks.workunit.client.1.smithi203.stdout:7/328: dwrite d1/d9/f29 [0,4194304] 0 2024-05-25T06:11:24.760 INFO:tasks.workunit.client.0.smithi160.stdout:5/10: dwrite f1 [0,4194304] 0 2024-05-25T06:11:24.762 INFO:tasks.workunit.client.1.smithi203.stdout:6/300: read d8/d11/d15/f35 [354392,55356] 0 2024-05-25T06:11:24.762 INFO:tasks.workunit.client.0.smithi160.stdout:6/11: creat d5/f7 x:0 0 0 2024-05-25T06:11:24.764 INFO:tasks.workunit.client.1.smithi203.stdout:5/496: dread dc/d11/d17/d1a/d22/f86 [0,4194304] 0 2024-05-25T06:11:24.766 INFO:tasks.workunit.client.1.smithi203.stdout:3/349: dread - d10/d22/f57 zero size 2024-05-25T06:11:24.767 INFO:tasks.workunit.client.0.smithi160.stdout:1/21: write f2 [1008356,74650] 0 2024-05-25T06:11:24.769 INFO:tasks.workunit.client.0.smithi160.stdout:5/11: symlink l3 0 2024-05-25T06:11:24.770 INFO:tasks.workunit.client.1.smithi203.stdout:6/301: mknod d8/d11/d47/d60/c64 0 2024-05-25T06:11:24.771 INFO:tasks.workunit.client.0.smithi160.stdout:5/12: chown l2 1 1 2024-05-25T06:11:24.774 INFO:tasks.workunit.client.0.smithi160.stdout:4/10: getdents . 0 2024-05-25T06:11:24.774 INFO:tasks.workunit.client.0.smithi160.stdout:4/11: readlink - no filename 2024-05-25T06:11:24.774 INFO:tasks.workunit.client.0.smithi160.stdout:5/13: chown l2 0 1 2024-05-25T06:11:24.775 INFO:tasks.workunit.client.1.smithi203.stdout:5/497: mknod dc/d11/d17/d1a/caf 0 2024-05-25T06:11:24.777 INFO:tasks.workunit.client.1.smithi203.stdout:5/498: dread - dc/d11/d17/f54 zero size 2024-05-25T06:11:24.778 INFO:tasks.workunit.client.1.smithi203.stdout:7/329: link d1/d9/d2c/d3a/d4e/f75 d1/d11/f80 0 2024-05-25T06:11:24.779 INFO:tasks.workunit.client.0.smithi160.stdout:5/14: creat f4 x:0 0 0 2024-05-25T06:11:24.779 INFO:tasks.workunit.client.1.smithi203.stdout:1/191: getdents d14/d1b/d1d 0 2024-05-25T06:11:24.780 INFO:tasks.workunit.client.0.smithi160.stdout:4/12: rename c0 to c2 0 2024-05-25T06:11:24.780 INFO:tasks.workunit.client.1.smithi203.stdout:6/302: creat d8/f65 x:0 0 0 2024-05-25T06:11:24.780 INFO:tasks.workunit.client.1.smithi203.stdout:1/192: stat d14/c1c 0 2024-05-25T06:11:24.781 INFO:tasks.workunit.client.0.smithi160.stdout:8/17: dwrite f3 [4194304,4194304] 0 2024-05-25T06:11:24.781 INFO:tasks.workunit.client.0.smithi160.stdout:6/12: dwrite f2 [0,4194304] 0 2024-05-25T06:11:24.782 INFO:tasks.workunit.client.1.smithi203.stdout:3/350: dwrite d10/f21 [4194304,4194304] 0 2024-05-25T06:11:24.783 INFO:tasks.workunit.client.1.smithi203.stdout:7/330: read d1/f31 [315638,61688] 0 2024-05-25T06:11:24.783 INFO:tasks.workunit.client.0.smithi160.stdout:5/15: creat f5 x:0 0 0 2024-05-25T06:11:24.785 INFO:tasks.workunit.client.1.smithi203.stdout:3/351: readlink d10/d22/d2a/l3d 0 2024-05-25T06:11:24.786 INFO:tasks.workunit.client.0.smithi160.stdout:1/22: fsync f2 0 2024-05-25T06:11:24.788 INFO:tasks.workunit.client.0.smithi160.stdout:1/23: fsync f5 0 2024-05-25T06:11:24.789 INFO:tasks.workunit.client.1.smithi203.stdout:1/193: creat d14/d23/f42 x:0 0 0 2024-05-25T06:11:24.789 INFO:tasks.workunit.client.1.smithi203.stdout:3/352: write d10/d22/f5e [472618,76384] 0 2024-05-25T06:11:24.790 INFO:tasks.workunit.client.0.smithi160.stdout:8/18: mkdir d5/d7 0 2024-05-25T06:11:24.792 INFO:tasks.workunit.client.1.smithi203.stdout:7/331: creat d1/d9/d2c/d79/f81 x:0 0 0 2024-05-25T06:11:24.792 INFO:tasks.workunit.client.1.smithi203.stdout:6/303: stat d8/d29/d30/c4e 0 2024-05-25T06:11:24.792 INFO:tasks.workunit.client.1.smithi203.stdout:5/499: dwrite dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:24.793 INFO:tasks.workunit.client.0.smithi160.stdout:4/13: dread f1 [0,4194304] 0 2024-05-25T06:11:24.794 INFO:tasks.workunit.client.1.smithi203.stdout:3/353: chown d10/d16/l43 91361975 1 2024-05-25T06:11:24.794 INFO:tasks.workunit.client.0.smithi160.stdout:1/24: mknod c8 0 2024-05-25T06:11:24.795 INFO:tasks.workunit.client.1.smithi203.stdout:5/500: chown dc/d11/d17/d1a/caf 13206 1 2024-05-25T06:11:24.796 INFO:tasks.workunit.client.1.smithi203.stdout:3/354: chown l9 660438 1 2024-05-25T06:11:24.796 INFO:tasks.workunit.client.0.smithi160.stdout:7/11: truncate f1 2180669 0 2024-05-25T06:11:24.796 INFO:tasks.workunit.client.0.smithi160.stdout:7/12: rmdir - no directory 2024-05-25T06:11:24.798 INFO:tasks.workunit.client.0.smithi160.stdout:4/14: truncate f1 5617960 0 2024-05-25T06:11:24.798 INFO:tasks.workunit.client.0.smithi160.stdout:6/13: dwrite f2 [0,4194304] 0 2024-05-25T06:11:24.799 INFO:tasks.workunit.client.1.smithi203.stdout:1/194: symlink d14/d37/l43 0 2024-05-25T06:11:24.800 INFO:tasks.workunit.client.1.smithi203.stdout:3/355: dread - d10/d22/f5b zero size 2024-05-25T06:11:24.801 INFO:tasks.workunit.client.0.smithi160.stdout:8/19: getdents d5/d7 0 2024-05-25T06:11:24.801 INFO:tasks.workunit.client.1.smithi203.stdout:3/356: dread - d10/d16/d23/f6e zero size 2024-05-25T06:11:24.802 INFO:tasks.workunit.client.0.smithi160.stdout:5/16: dwrite f5 [0,4194304] 0 2024-05-25T06:11:24.802 INFO:tasks.workunit.client.0.smithi160.stdout:5/17: fdatasync f4 0 2024-05-25T06:11:24.803 INFO:tasks.workunit.client.0.smithi160.stdout:5/18: dread - f4 zero size 2024-05-25T06:11:24.805 INFO:tasks.workunit.client.0.smithi160.stdout:5/19: write f1 [4456553,880] 0 2024-05-25T06:11:24.805 INFO:tasks.workunit.client.1.smithi203.stdout:6/304: mknod d8/d3d/c66 0 2024-05-25T06:11:24.809 INFO:tasks.workunit.client.0.smithi160.stdout:6/14: creat d5/f8 x:0 0 0 2024-05-25T06:11:24.811 INFO:tasks.workunit.client.1.smithi203.stdout:1/195: symlink d14/d37/l44 0 2024-05-25T06:11:24.811 INFO:tasks.workunit.client.1.smithi203.stdout:1/196: fsync f12 0 2024-05-25T06:11:24.811 INFO:tasks.workunit.client.0.smithi160.stdout:8/20: getdents d5/d7 0 2024-05-25T06:11:24.812 INFO:tasks.workunit.client.1.smithi203.stdout:1/197: dread - d14/f32 zero size 2024-05-25T06:11:24.812 INFO:tasks.workunit.client.0.smithi160.stdout:1/25: read f5 [725500,74248] 0 2024-05-25T06:11:24.814 INFO:tasks.workunit.client.1.smithi203.stdout:1/198: dread - d14/d23/f42 zero size 2024-05-25T06:11:24.817 INFO:tasks.workunit.client.0.smithi160.stdout:6/15: rename d5/f8 to d5/f9 0 2024-05-25T06:11:24.818 INFO:tasks.workunit.client.1.smithi203.stdout:3/357: fsync d10/f31 0 2024-05-25T06:11:24.818 INFO:tasks.workunit.client.1.smithi203.stdout:9/251: sync 2024-05-25T06:11:24.818 INFO:tasks.workunit.client.1.smithi203.stdout:8/259: sync 2024-05-25T06:11:24.819 INFO:tasks.workunit.client.1.smithi203.stdout:5/501: mkdir dc/d11/d42/d94/dad/db0 0 2024-05-25T06:11:24.820 INFO:tasks.workunit.client.0.smithi160.stdout:8/21: mkdir d5/d7/d8 0 2024-05-25T06:11:24.821 INFO:tasks.workunit.client.1.smithi203.stdout:1/199: mknod d14/d1b/d1d/d3e/c45 0 2024-05-25T06:11:24.824 INFO:tasks.workunit.client.1.smithi203.stdout:8/260: mknod d2/d5/d10/c57 0 2024-05-25T06:11:24.826 INFO:tasks.workunit.client.0.smithi160.stdout:8/22: mkdir d5/d9 0 2024-05-25T06:11:24.826 INFO:tasks.workunit.client.0.smithi160.stdout:5/20: dwrite f4 [0,4194304] 0 2024-05-25T06:11:24.826 INFO:tasks.workunit.client.0.smithi160.stdout:4/15: unlink c2 0 2024-05-25T06:11:24.827 INFO:tasks.workunit.client.1.smithi203.stdout:9/252: creat d0/f52 x:0 0 0 2024-05-25T06:11:24.827 INFO:tasks.workunit.client.0.smithi160.stdout:4/16: stat f1 0 2024-05-25T06:11:24.828 INFO:tasks.workunit.client.0.smithi160.stdout:1/26: dwrite f2 [0,4194304] 0 2024-05-25T06:11:24.829 INFO:tasks.workunit.client.0.smithi160.stdout:6/16: dread f2 [0,4194304] 0 2024-05-25T06:11:24.829 INFO:tasks.workunit.client.0.smithi160.stdout:1/27: chown c4 360346159 1 2024-05-25T06:11:24.830 INFO:tasks.workunit.client.0.smithi160.stdout:5/21: creat f6 x:0 0 0 2024-05-25T06:11:24.831 INFO:tasks.workunit.client.1.smithi203.stdout:8/261: mknod d2/d5/d2d/c58 0 2024-05-25T06:11:24.832 INFO:tasks.workunit.client.0.smithi160.stdout:4/17: write f1 [2918762,46045] 0 2024-05-25T06:11:24.832 INFO:tasks.workunit.client.0.smithi160.stdout:4/18: stat f1 0 2024-05-25T06:11:24.834 INFO:tasks.workunit.client.1.smithi203.stdout:8/262: write d2/d5/d10/f53 [866418,41668] 0 2024-05-25T06:11:24.835 INFO:tasks.workunit.client.0.smithi160.stdout:5/22: link f6 f7 0 2024-05-25T06:11:24.837 INFO:tasks.workunit.client.1.smithi203.stdout:5/502: dwrite dc/d3c/d3f/f57 [0,4194304] 0 2024-05-25T06:11:24.841 INFO:tasks.workunit.client.0.smithi160.stdout:4/19: dwrite f1 [4194304,4194304] 0 2024-05-25T06:11:24.841 INFO:tasks.workunit.client.1.smithi203.stdout:1/200: dwrite d14/f32 [0,4194304] 0 2024-05-25T06:11:24.841 INFO:tasks.workunit.client.1.smithi203.stdout:9/253: creat d0/df/d2d/d45/f53 x:0 0 0 2024-05-25T06:11:24.843 INFO:tasks.workunit.client.0.smithi160.stdout:5/23: creat f8 x:0 0 0 2024-05-25T06:11:24.844 INFO:tasks.workunit.client.1.smithi203.stdout:2/323: sync 2024-05-25T06:11:24.844 INFO:tasks.workunit.client.1.smithi203.stdout:0/237: sync 2024-05-25T06:11:24.844 INFO:tasks.workunit.client.1.smithi203.stdout:4/224: sync 2024-05-25T06:11:24.844 INFO:tasks.workunit.client.1.smithi203.stdout:9/254: write d0/d17/d20/f46 [4376474,126304] 0 2024-05-25T06:11:24.845 INFO:tasks.workunit.client.0.smithi160.stdout:8/23: fdatasync f3 0 2024-05-25T06:11:24.846 INFO:tasks.workunit.client.1.smithi203.stdout:9/255: fdatasync d0/df/d2d/d45/f49 0 2024-05-25T06:11:24.846 INFO:tasks.workunit.client.0.smithi160.stdout:6/17: fsync f2 0 2024-05-25T06:11:24.852 INFO:tasks.workunit.client.0.smithi160.stdout:4/20: dwrite f1 [4194304,4194304] 0 2024-05-25T06:11:24.853 INFO:tasks.workunit.client.1.smithi203.stdout:9/256: write d0/f16 [8696695,63786] 0 2024-05-25T06:11:24.853 INFO:tasks.workunit.client.0.smithi160.stdout:6/18: creat d5/fa x:0 0 0 2024-05-25T06:11:24.853 INFO:tasks.workunit.client.1.smithi203.stdout:1/201: chown d14/f35 129343 1 2024-05-25T06:11:24.854 INFO:tasks.workunit.client.1.smithi203.stdout:1/202: chown d14/d1b/c34 302 1 2024-05-25T06:11:24.856 INFO:tasks.workunit.client.0.smithi160.stdout:8/24: creat d5/d7/fa x:0 0 0 2024-05-25T06:11:24.858 INFO:tasks.workunit.client.1.smithi203.stdout:7/332: truncate f0 6640245 0 2024-05-25T06:11:24.861 INFO:tasks.workunit.client.0.smithi160.stdout:4/21: creat f3 x:0 0 0 2024-05-25T06:11:24.863 INFO:tasks.workunit.client.0.smithi160.stdout:4/22: write f3 [578539,102026] 0 2024-05-25T06:11:24.865 INFO:tasks.workunit.client.1.smithi203.stdout:2/324: mkdir d2/d38/d2f/d69/d7a 0 2024-05-25T06:11:24.866 INFO:tasks.workunit.client.1.smithi203.stdout:3/358: write f1 [4276583,68510] 0 2024-05-25T06:11:24.866 INFO:tasks.workunit.client.1.smithi203.stdout:8/263: dwrite d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:24.866 INFO:tasks.workunit.client.1.smithi203.stdout:8/264: chown d2/d5 43 1 2024-05-25T06:11:24.867 INFO:tasks.workunit.client.0.smithi160.stdout:8/25: dwrite d5/d7/fa [0,4194304] 0 2024-05-25T06:11:24.867 INFO:tasks.workunit.client.0.smithi160.stdout:8/26: readlink l4 0 2024-05-25T06:11:24.868 INFO:tasks.workunit.client.1.smithi203.stdout:9/257: readlink d0/l22 0 2024-05-25T06:11:24.869 INFO:tasks.workunit.client.1.smithi203.stdout:1/203: write d14/d1b/d29/f2b [1519258,88257] 0 2024-05-25T06:11:24.871 INFO:tasks.workunit.client.1.smithi203.stdout:5/503: chown dc/c30 3197665 1 2024-05-25T06:11:24.872 INFO:tasks.workunit.client.1.smithi203.stdout:9/258: write d0/f4 [159412,84238] 0 2024-05-25T06:11:24.872 INFO:tasks.workunit.client.1.smithi203.stdout:8/265: mkdir d2/d5/d10/d11/d59 0 2024-05-25T06:11:24.874 INFO:tasks.workunit.client.1.smithi203.stdout:3/359: symlink d10/d16/d23/d6c/l77 0 2024-05-25T06:11:24.874 INFO:tasks.workunit.client.0.smithi160.stdout:4/23: mkdir d4 0 2024-05-25T06:11:24.875 INFO:tasks.workunit.client.1.smithi203.stdout:6/305: dread d8/d11/d15/f31 [0,4194304] 0 2024-05-25T06:11:24.875 INFO:tasks.workunit.client.0.smithi160.stdout:4/24: stat d4 0 2024-05-25T06:11:24.876 INFO:tasks.workunit.client.1.smithi203.stdout:6/306: dread - d8/d11/d15/d28/f5e zero size 2024-05-25T06:11:24.878 INFO:tasks.workunit.client.1.smithi203.stdout:3/360: truncate d10/d22/f57 89896 0 2024-05-25T06:11:24.879 INFO:tasks.workunit.client.0.smithi160.stdout:8/27: creat d5/fb x:0 0 0 2024-05-25T06:11:24.879 INFO:tasks.workunit.client.0.smithi160.stdout:6/19: rename d5/f9 to d5/fb 0 2024-05-25T06:11:24.881 INFO:tasks.workunit.client.1.smithi203.stdout:2/325: creat d2/d4/d10/d5f/f7b x:0 0 0 2024-05-25T06:11:24.882 INFO:tasks.workunit.client.1.smithi203.stdout:5/504: mkdir dc/d11/d42/d94/dad/db1 0 2024-05-25T06:11:24.883 INFO:tasks.workunit.client.1.smithi203.stdout:2/326: fsync d2/d4/d10/d5f/f25 0 2024-05-25T06:11:24.883 INFO:tasks.workunit.client.1.smithi203.stdout:8/266: mknod d2/d5/d28/c5a 0 2024-05-25T06:11:24.884 INFO:tasks.workunit.client.0.smithi160.stdout:8/28: creat d5/d9/fc x:0 0 0 2024-05-25T06:11:24.886 INFO:tasks.workunit.client.1.smithi203.stdout:9/259: dread d0/d17/d20/f31 [0,4194304] 0 2024-05-25T06:11:24.886 INFO:tasks.workunit.client.0.smithi160.stdout:8/29: read f3 [6913477,88297] 0 2024-05-25T06:11:24.887 INFO:tasks.workunit.client.1.smithi203.stdout:0/238: write dd/d10/f33 [528073,124148] 0 2024-05-25T06:11:24.888 INFO:tasks.workunit.client.0.smithi160.stdout:4/25: dwrite f3 [0,4194304] 0 2024-05-25T06:11:24.888 INFO:tasks.workunit.client.1.smithi203.stdout:8/267: fsync d2/d5/d10/d11/de/f1e 0 2024-05-25T06:11:24.889 INFO:tasks.workunit.client.1.smithi203.stdout:2/327: chown d2/d6/l70 510906191 1 2024-05-25T06:11:24.889 INFO:tasks.workunit.client.0.smithi160.stdout:5/24: getdents . 0 2024-05-25T06:11:24.890 INFO:tasks.workunit.client.0.smithi160.stdout:6/20: dread f2 [0,4194304] 0 2024-05-25T06:11:24.891 INFO:tasks.workunit.client.0.smithi160.stdout:4/26: write f1 [7607566,106773] 0 2024-05-25T06:11:24.893 INFO:tasks.workunit.client.1.smithi203.stdout:8/268: readlink d2/d5/l6 0 2024-05-25T06:11:24.893 INFO:tasks.workunit.client.0.smithi160.stdout:5/25: creat f9 x:0 0 0 2024-05-25T06:11:24.893 INFO:tasks.workunit.client.0.smithi160.stdout:4/27: chown f3 4712798 1 2024-05-25T06:11:24.893 INFO:tasks.workunit.client.0.smithi160.stdout:4/28: readlink - no filename 2024-05-25T06:11:24.894 INFO:tasks.workunit.client.1.smithi203.stdout:0/239: write dd/d3a/f3b [1472854,86346] 0 2024-05-25T06:11:24.895 INFO:tasks.workunit.client.0.smithi160.stdout:6/21: mkdir d5/dc 0 2024-05-25T06:11:24.895 INFO:tasks.workunit.client.1.smithi203.stdout:5/505: symlink dc/d11/d18/lb2 0 2024-05-25T06:11:24.896 INFO:tasks.workunit.client.1.smithi203.stdout:1/204: link c13 d14/d1b/d1d/c46 0 2024-05-25T06:11:24.898 INFO:tasks.workunit.client.0.smithi160.stdout:5/26: creat fa x:0 0 0 2024-05-25T06:11:24.898 INFO:tasks.workunit.client.1.smithi203.stdout:2/328: unlink d2/d4/d10/d32/f44 0 2024-05-25T06:11:24.898 INFO:tasks.workunit.client.1.smithi203.stdout:0/240: chown dd/d10/l3f 23 1 2024-05-25T06:11:24.898 INFO:tasks.workunit.client.1.smithi203.stdout:9/260: dread d0/f13 [0,4194304] 0 2024-05-25T06:11:24.899 INFO:tasks.workunit.client.1.smithi203.stdout:1/205: chown d14/d1b/d29/c3c 6616175 1 2024-05-25T06:11:24.899 INFO:tasks.workunit.client.1.smithi203.stdout:5/506: stat dc/d11/d17/d1a/f32 0 2024-05-25T06:11:24.902 INFO:tasks.workunit.client.1.smithi203.stdout:0/241: creat dd/d3d/d4f/f56 x:0 0 0 2024-05-25T06:11:24.903 INFO:tasks.workunit.client.1.smithi203.stdout:4/225: dread d0/d9/d24/d27/f30 [0,4194304] 0 2024-05-25T06:11:24.903 INFO:tasks.workunit.client.1.smithi203.stdout:2/329: mknod d2/d38/d43/d60/c7c 0 2024-05-25T06:11:24.904 INFO:tasks.workunit.client.0.smithi160.stdout:8/30: dwrite d5/d7/fa [0,4194304] 0 2024-05-25T06:11:24.906 INFO:tasks.workunit.client.0.smithi160.stdout:8/31: stat d5/d7 0 2024-05-25T06:11:24.906 INFO:tasks.workunit.client.0.smithi160.stdout:4/29: dread f1 [0,4194304] 0 2024-05-25T06:11:24.906 INFO:tasks.workunit.client.0.smithi160.stdout:8/32: stat c2 0 2024-05-25T06:11:24.909 INFO:tasks.workunit.client.1.smithi203.stdout:9/261: write d0/d17/d20/f2c [2644074,56272] 0 2024-05-25T06:11:24.909 INFO:tasks.workunit.client.0.smithi160.stdout:8/33: dread - d5/d9/fc zero size 2024-05-25T06:11:24.909 INFO:tasks.workunit.client.0.smithi160.stdout:6/22: dwrite d5/fa [0,4194304] 0 2024-05-25T06:11:24.910 INFO:tasks.workunit.client.1.smithi203.stdout:0/242: mknod dd/d3d/d4f/c57 0 2024-05-25T06:11:24.911 INFO:tasks.workunit.client.0.smithi160.stdout:8/34: chown d5/d9/fc 112104 1 2024-05-25T06:11:24.911 INFO:tasks.workunit.client.0.smithi160.stdout:4/30: unlink f3 0 2024-05-25T06:11:24.915 INFO:tasks.workunit.client.0.smithi160.stdout:4/31: write f1 [2634023,98786] 0 2024-05-25T06:11:24.915 INFO:tasks.workunit.client.0.smithi160.stdout:4/32: readlink - no filename 2024-05-25T06:11:24.918 INFO:tasks.workunit.client.1.smithi203.stdout:2/330: rename d2/d4/d10/d32/f22 to d2/d38/f7d 0 2024-05-25T06:11:24.918 INFO:tasks.workunit.client.0.smithi160.stdout:8/35: dread f3 [4194304,4194304] 0 2024-05-25T06:11:24.919 INFO:tasks.workunit.client.0.smithi160.stdout:4/33: unlink f1 0 2024-05-25T06:11:24.919 INFO:tasks.workunit.client.0.smithi160.stdout:4/34: dread - no filename 2024-05-25T06:11:24.921 INFO:tasks.workunit.client.0.smithi160.stdout:4/35: rename d4 to d4/d5 22 2024-05-25T06:11:24.921 INFO:tasks.workunit.client.0.smithi160.stdout:4/36: truncate - no filename 2024-05-25T06:11:24.923 INFO:tasks.workunit.client.1.smithi203.stdout:9/262: rmdir d0/d17/d20/d3c 39 2024-05-25T06:11:24.923 INFO:tasks.workunit.client.0.smithi160.stdout:5/27: dwrite f7 [0,4194304] 0 2024-05-25T06:11:24.925 INFO:tasks.workunit.client.0.smithi160.stdout:8/36: dread f3 [0,4194304] 0 2024-05-25T06:11:24.927 INFO:tasks.workunit.client.0.smithi160.stdout:4/37: mkdir d4/d6 0 2024-05-25T06:11:24.928 INFO:tasks.workunit.client.0.smithi160.stdout:4/38: stat d4 0 2024-05-25T06:11:24.928 INFO:tasks.workunit.client.0.smithi160.stdout:8/37: write d5/fb [678734,20216] 0 2024-05-25T06:11:24.929 INFO:tasks.workunit.client.0.smithi160.stdout:8/38: stat d5/fb 0 2024-05-25T06:11:24.929 INFO:tasks.workunit.client.1.smithi203.stdout:6/307: dread d8/d11/d15/f37 [0,4194304] 0 2024-05-25T06:11:24.931 INFO:tasks.workunit.client.0.smithi160.stdout:8/39: symlink d5/ld 0 2024-05-25T06:11:24.932 INFO:tasks.workunit.client.1.smithi203.stdout:3/361: truncate d10/d16/d23/d3a/d48/f52 985368 0 2024-05-25T06:11:24.933 INFO:tasks.workunit.client.1.smithi203.stdout:4/226: dread d0/d8/f23 [0,4194304] 0 2024-05-25T06:11:24.933 INFO:tasks.workunit.client.0.smithi160.stdout:8/40: chown c2 170106 1 2024-05-25T06:11:24.933 INFO:tasks.workunit.client.1.smithi203.stdout:2/331: dwrite d2/d4/d10/f79 [0,4194304] 0 2024-05-25T06:11:24.935 INFO:tasks.workunit.client.0.smithi160.stdout:4/39: creat d4/d6/f7 x:0 0 0 2024-05-25T06:11:24.935 INFO:tasks.workunit.client.1.smithi203.stdout:7/333: dread d1/d11/d20/f27 [0,4194304] 0 2024-05-25T06:11:24.936 INFO:tasks.workunit.client.1.smithi203.stdout:9/263: symlink d0/df/l54 0 2024-05-25T06:11:24.936 INFO:tasks.workunit.client.0.smithi160.stdout:8/41: creat d5/fe x:0 0 0 2024-05-25T06:11:24.937 INFO:tasks.workunit.client.1.smithi203.stdout:4/227: dread - d0/d9/d11/f2f zero size 2024-05-25T06:11:24.940 INFO:tasks.workunit.client.1.smithi203.stdout:7/334: chown d1/d11/d20/c6b 118 1 2024-05-25T06:11:24.942 INFO:tasks.workunit.client.0.smithi160.stdout:4/40: dwrite d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:24.943 INFO:tasks.workunit.client.1.smithi203.stdout:6/308: creat d8/d2a/f67 x:0 0 0 2024-05-25T06:11:24.944 INFO:tasks.workunit.client.1.smithi203.stdout:3/362: creat d10/d16/d23/d67/f78 x:0 0 0 2024-05-25T06:11:24.944 INFO:tasks.workunit.client.1.smithi203.stdout:8/269: dread d2/d5/d10/d11/de/f13 [0,4194304] 0 2024-05-25T06:11:24.945 INFO:tasks.workunit.client.1.smithi203.stdout:0/243: write dd/d10/f41 [2875684,94757] 0 2024-05-25T06:11:24.945 INFO:tasks.workunit.client.1.smithi203.stdout:1/206: dread d14/d1b/d1d/f2a [0,4194304] 0 2024-05-25T06:11:24.946 INFO:tasks.workunit.client.0.smithi160.stdout:4/41: mknod d4/d6/c8 0 2024-05-25T06:11:24.947 INFO:tasks.workunit.client.1.smithi203.stdout:4/228: symlink d0/d8/d1f/d26/d3b/l3e 0 2024-05-25T06:11:24.948 INFO:tasks.workunit.client.1.smithi203.stdout:6/309: unlink d8/d11/f1e 0 2024-05-25T06:11:24.950 INFO:tasks.workunit.client.0.smithi160.stdout:4/42: dread d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:24.954 INFO:tasks.workunit.client.0.smithi160.stdout:4/43: dread d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:24.955 INFO:tasks.workunit.client.0.smithi160.stdout:4/44: rename d4 to d4/d6/d9 22 2024-05-25T06:11:24.957 INFO:tasks.workunit.client.1.smithi203.stdout:3/363: rename d10/d22/f5e to d10/d16/d23/d46/f79 0 2024-05-25T06:11:24.958 INFO:tasks.workunit.client.1.smithi203.stdout:5/507: dwrite dc/d11/f12 [0,4194304] 0 2024-05-25T06:11:24.961 INFO:tasks.workunit.client.0.smithi160.stdout:8/42: getdents d5 0 2024-05-25T06:11:24.961 INFO:tasks.workunit.client.1.smithi203.stdout:7/335: link d1/d11/d3d/d4c/l58 d1/d9/d2c/d3a/l82 0 2024-05-25T06:11:24.964 INFO:tasks.workunit.client.0.smithi160.stdout:4/45: dwrite d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:24.964 INFO:tasks.workunit.client.1.smithi203.stdout:8/270: dwrite d2/d5/fc [0,4194304] 0 2024-05-25T06:11:24.966 INFO:tasks.workunit.client.0.smithi160.stdout:4/46: write d4/d6/f7 [2087102,106224] 0 2024-05-25T06:11:24.968 INFO:tasks.workunit.client.0.smithi160.stdout:8/43: dread f3 [0,4194304] 0 2024-05-25T06:11:24.968 INFO:tasks.workunit.client.0.smithi160.stdout:8/44: chown c2 6470675 1 2024-05-25T06:11:24.969 INFO:tasks.workunit.client.0.smithi160.stdout:4/47: creat d4/d6/fa x:0 0 0 2024-05-25T06:11:24.971 INFO:tasks.workunit.client.1.smithi203.stdout:2/332: write d2/d4/d10/d50/f67 [994272,16832] 0 2024-05-25T06:11:24.971 INFO:tasks.workunit.client.0.smithi160.stdout:8/45: symlink d5/lf 0 2024-05-25T06:11:24.972 INFO:tasks.workunit.client.1.smithi203.stdout:0/244: rename f6 to dd/d10/d34/f58 0 2024-05-25T06:11:24.972 INFO:tasks.workunit.client.1.smithi203.stdout:5/508: chown dc/d11/d17/d1a/d2d/d4d/d64/fa1 31211 1 2024-05-25T06:11:24.974 INFO:tasks.workunit.client.1.smithi203.stdout:4/229: dwrite d0/d8/d1f/d26/f1e [0,4194304] 0 2024-05-25T06:11:24.974 INFO:tasks.workunit.client.0.smithi160.stdout:8/46: symlink d5/d7/l10 0 2024-05-25T06:11:24.978 INFO:tasks.workunit.client.0.smithi160.stdout:8/47: write d5/d7/fa [1462766,72230] 0 2024-05-25T06:11:24.979 INFO:tasks.workunit.client.1.smithi203.stdout:1/207: dwrite d14/d23/f42 [0,4194304] 0 2024-05-25T06:11:24.980 INFO:tasks.workunit.client.1.smithi203.stdout:8/271: rmdir d2/d5/d28/d38 39 2024-05-25T06:11:24.980 INFO:tasks.workunit.client.1.smithi203.stdout:2/333: mknod d2/d38/d2f/c7e 0 2024-05-25T06:11:24.980 INFO:tasks.workunit.client.0.smithi160.stdout:4/48: dwrite d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:24.981 INFO:tasks.workunit.client.1.smithi203.stdout:1/208: chown d14/d37/l43 160731 1 2024-05-25T06:11:24.982 INFO:tasks.workunit.client.1.smithi203.stdout:8/272: write d2/f2a [1434525,42030] 0 2024-05-25T06:11:24.983 INFO:tasks.workunit.client.1.smithi203.stdout:1/209: chown d14/f17 3619490 1 2024-05-25T06:11:24.983 INFO:tasks.workunit.client.1.smithi203.stdout:8/273: dread - d2/d5/d2d/f4b zero size 2024-05-25T06:11:24.983 INFO:tasks.workunit.client.1.smithi203.stdout:5/509: creat dc/d11/d17/d44/d73/d98/fb3 x:0 0 0 2024-05-25T06:11:24.985 INFO:tasks.workunit.client.1.smithi203.stdout:4/230: rename d0/d8/d10/f15 to d0/d9/d11/f3f 0 2024-05-25T06:11:24.986 INFO:tasks.workunit.client.0.smithi160.stdout:4/49: mknod d4/d6/cb 0 2024-05-25T06:11:24.988 INFO:tasks.workunit.client.1.smithi203.stdout:3/364: rmdir d10/d16/d23/d4a/d6d 0 2024-05-25T06:11:24.989 INFO:tasks.workunit.client.1.smithi203.stdout:6/310: getdents d8/d2a/d63 0 2024-05-25T06:11:24.989 INFO:tasks.workunit.client.1.smithi203.stdout:6/311: stat d8/f2c 0 2024-05-25T06:11:24.989 INFO:tasks.workunit.client.0.smithi160.stdout:4/50: write d4/d6/fa [903034,96515] 0 2024-05-25T06:11:24.990 INFO:tasks.workunit.client.0.smithi160.stdout:8/48: dwrite d5/fe [0,4194304] 0 2024-05-25T06:11:24.991 INFO:tasks.workunit.client.0.smithi160.stdout:8/49: chown l4 1417 1 2024-05-25T06:11:24.991 INFO:tasks.workunit.client.1.smithi203.stdout:3/365: fdatasync d10/d22/d2a/f4d 0 2024-05-25T06:11:24.994 INFO:tasks.workunit.client.1.smithi203.stdout:8/274: rmdir d2/d5/d10 39 2024-05-25T06:11:24.994 INFO:tasks.workunit.client.1.smithi203.stdout:4/231: dread d0/d9/d11/f3f [0,4194304] 0 2024-05-25T06:11:24.998 INFO:tasks.workunit.client.0.smithi160.stdout:8/50: dread d5/d7/fa [0,4194304] 0 2024-05-25T06:11:24.999 INFO:tasks.workunit.client.0.smithi160.stdout:8/51: truncate d5/d9/fc 180281 0 2024-05-25T06:11:25.000 INFO:tasks.workunit.client.0.smithi160.stdout:4/51: link d4/d6/fa d4/d6/fc 0 2024-05-25T06:11:25.001 INFO:tasks.workunit.client.0.smithi160.stdout:8/52: write d5/fb [605021,113869] 0 2024-05-25T06:11:25.003 INFO:tasks.workunit.client.0.smithi160.stdout:4/52: rmdir d4 39 2024-05-25T06:11:25.004 INFO:tasks.workunit.client.1.smithi203.stdout:1/210: symlink d14/d37/l47 0 2024-05-25T06:11:25.004 INFO:tasks.workunit.client.1.smithi203.stdout:6/312: symlink d8/d29/l68 0 2024-05-25T06:11:25.005 INFO:tasks.workunit.client.1.smithi203.stdout:3/366: dread d10/d16/f44 [0,4194304] 0 2024-05-25T06:11:25.007 INFO:tasks.workunit.client.0.smithi160.stdout:8/53: rename c1 to d5/d7/c11 0 2024-05-25T06:11:25.008 INFO:tasks.workunit.client.1.smithi203.stdout:4/232: creat d0/d9/d24/f40 x:0 0 0 2024-05-25T06:11:25.008 INFO:tasks.workunit.client.0.smithi160.stdout:4/53: symlink d4/ld 0 2024-05-25T06:11:25.009 INFO:tasks.workunit.client.0.smithi160.stdout:4/54: readlink d4/ld 0 2024-05-25T06:11:25.009 INFO:tasks.workunit.client.0.smithi160.stdout:8/54: mknod d5/d7/c12 0 2024-05-25T06:11:25.011 INFO:tasks.workunit.client.0.smithi160.stdout:4/55: write d4/d6/fa [656753,10173] 0 2024-05-25T06:11:25.011 INFO:tasks.workunit.client.1.smithi203.stdout:3/367: write d10/d16/d23/f6e [70936,28503] 0 2024-05-25T06:11:25.013 INFO:tasks.workunit.client.1.smithi203.stdout:4/233: write d0/f16 [571332,91864] 0 2024-05-25T06:11:25.013 INFO:tasks.workunit.client.0.smithi160.stdout:4/56: creat d4/d6/fe x:0 0 0 2024-05-25T06:11:25.017 INFO:tasks.workunit.client.1.smithi203.stdout:4/234: stat d0/f3 0 2024-05-25T06:11:25.017 INFO:tasks.workunit.client.0.smithi160.stdout:4/57: mknod d4/cf 0 2024-05-25T06:11:25.018 INFO:tasks.workunit.client.0.smithi160.stdout:8/55: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:25.019 INFO:tasks.workunit.client.1.smithi203.stdout:1/211: dwrite d14/d1b/d1d/f30 [0,4194304] 0 2024-05-25T06:11:25.023 INFO:tasks.workunit.client.1.smithi203.stdout:6/313: dwrite d8/d11/d47/d60/f55 [0,4194304] 0 2024-05-25T06:11:25.024 INFO:tasks.workunit.client.1.smithi203.stdout:4/235: write d0/d8/d1f/f2b [5075983,86980] 0 2024-05-25T06:11:25.025 INFO:tasks.workunit.client.1.smithi203.stdout:1/212: chown d14/d37/l44 156690 1 2024-05-25T06:11:25.025 INFO:tasks.workunit.client.0.smithi160.stdout:4/58: creat d4/d6/f10 x:0 0 0 2024-05-25T06:11:25.026 INFO:tasks.workunit.client.1.smithi203.stdout:8/275: dwrite d2/d5/d10/d11/de/f16 [0,4194304] 0 2024-05-25T06:11:25.026 INFO:tasks.workunit.client.1.smithi203.stdout:6/314: readlink d8/d11/d15/l3c 0 2024-05-25T06:11:25.027 INFO:tasks.workunit.client.1.smithi203.stdout:8/276: read d2/d5/fc [683774,74896] 0 2024-05-25T06:11:25.030 INFO:tasks.workunit.client.1.smithi203.stdout:6/315: write d8/d29/d30/f4f [690596,88577] 0 2024-05-25T06:11:25.034 INFO:tasks.workunit.client.1.smithi203.stdout:1/213: rmdir d14/d1b/d29 39 2024-05-25T06:11:25.035 INFO:tasks.workunit.client.0.smithi160.stdout:4/59: dwrite d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:25.035 INFO:tasks.workunit.client.1.smithi203.stdout:1/214: stat d14/d37 0 2024-05-25T06:11:25.035 INFO:tasks.workunit.client.1.smithi203.stdout:8/277: mknod d2/d5/d10/d11/de/c5b 0 2024-05-25T06:11:25.038 INFO:tasks.workunit.client.1.smithi203.stdout:6/316: fdatasync d8/d11/d15/f2e 0 2024-05-25T06:11:25.039 INFO:tasks.workunit.client.0.smithi160.stdout:4/60: unlink d4/d6/cb 0 2024-05-25T06:11:25.040 INFO:tasks.workunit.client.1.smithi203.stdout:2/334: truncate d2/d4/d10/d58/f6d 4716 0 2024-05-25T06:11:25.040 INFO:tasks.workunit.client.1.smithi203.stdout:4/236: dread d0/f38 [0,4194304] 0 2024-05-25T06:11:25.041 INFO:tasks.workunit.client.1.smithi203.stdout:1/215: read f10 [4464863,106651] 0 2024-05-25T06:11:25.041 INFO:tasks.workunit.client.0.smithi160.stdout:4/61: write d4/d6/fa [525328,120535] 0 2024-05-25T06:11:25.042 INFO:tasks.workunit.client.1.smithi203.stdout:8/278: truncate d2/d5/d10/f53 1822940 0 2024-05-25T06:11:25.043 INFO:tasks.workunit.client.0.smithi160.stdout:4/62: truncate d4/d6/f10 700214 0 2024-05-25T06:11:25.043 INFO:tasks.workunit.client.1.smithi203.stdout:4/237: fsync d0/f16 0 2024-05-25T06:11:25.044 INFO:tasks.workunit.client.1.smithi203.stdout:2/335: chown d2/d4/d10/d32/f1b 79619939 1 2024-05-25T06:11:25.044 INFO:tasks.workunit.client.1.smithi203.stdout:1/216: stat d14/d23/c38 0 2024-05-25T06:11:25.044 INFO:tasks.workunit.client.1.smithi203.stdout:5/510: truncate dc/d11/f1c 973316 0 2024-05-25T06:11:25.046 INFO:tasks.workunit.client.1.smithi203.stdout:2/336: stat d2/d6/f46 0 2024-05-25T06:11:25.050 INFO:tasks.workunit.client.1.smithi203.stdout:8/279: creat d2/d5/d42/f5c x:0 0 0 2024-05-25T06:11:25.051 INFO:tasks.workunit.client.0.smithi160.stdout:4/63: dwrite d4/d6/fa [0,4194304] 0 2024-05-25T06:11:25.052 INFO:tasks.workunit.client.1.smithi203.stdout:1/217: symlink d14/d1b/d1d/l48 0 2024-05-25T06:11:25.052 INFO:tasks.workunit.client.0.smithi160.stdout:8/56: truncate d5/d7/fa 185281 0 2024-05-25T06:11:25.054 INFO:tasks.workunit.client.0.smithi160.stdout:4/64: creat d4/f11 x:0 0 0 2024-05-25T06:11:25.055 INFO:tasks.workunit.client.1.smithi203.stdout:2/337: creat d2/d1a/f7f x:0 0 0 2024-05-25T06:11:25.055 INFO:tasks.workunit.client.0.smithi160.stdout:4/65: write d4/d6/fe [14233,84157] 0 2024-05-25T06:11:25.056 INFO:tasks.workunit.client.1.smithi203.stdout:5/511: creat dc/d11/d17/d1a/d22/fb4 x:0 0 0 2024-05-25T06:11:25.056 INFO:tasks.workunit.client.1.smithi203.stdout:6/317: dwrite d8/d11/d15/d28/d52/d43/f32 [0,4194304] 0 2024-05-25T06:11:25.057 INFO:tasks.workunit.client.1.smithi203.stdout:1/218: truncate d14/f18 824631 0 2024-05-25T06:11:25.057 INFO:tasks.workunit.client.0.smithi160.stdout:4/66: write d4/d6/f7 [564027,126001] 0 2024-05-25T06:11:25.057 INFO:tasks.workunit.client.0.smithi160.stdout:8/57: dread f3 [4194304,4194304] 0 2024-05-25T06:11:25.058 INFO:tasks.workunit.client.0.smithi160.stdout:4/67: write d4/d6/fa [129693,106619] 0 2024-05-25T06:11:25.063 INFO:tasks.workunit.client.1.smithi203.stdout:3/368: dread d10/d22/d2a/f3b [0,4194304] 0 2024-05-25T06:11:25.064 INFO:tasks.workunit.client.0.smithi160.stdout:4/68: mknod d4/c12 0 2024-05-25T06:11:25.068 INFO:tasks.workunit.client.1.smithi203.stdout:2/338: write d2/d4/d10/d32/f1f [776506,97834] 0 2024-05-25T06:11:25.069 INFO:tasks.workunit.client.1.smithi203.stdout:1/219: read d14/d1b/d1d/f2f [1782001,61608] 0 2024-05-25T06:11:25.069 INFO:tasks.workunit.client.1.smithi203.stdout:8/280: dwrite d2/d5/d18/d31/f39 [0,4194304] 0 2024-05-25T06:11:25.076 INFO:tasks.workunit.client.1.smithi203.stdout:6/318: dwrite d8/d29/d30/d38/f39 [0,4194304] 0 2024-05-25T06:11:25.077 INFO:tasks.workunit.client.1.smithi203.stdout:3/369: symlink d10/d22/d2a/l7a 0 2024-05-25T06:11:25.079 INFO:tasks.workunit.client.1.smithi203.stdout:1/220: symlink d14/d37/l49 0 2024-05-25T06:11:25.081 INFO:tasks.workunit.client.1.smithi203.stdout:3/370: read d10/d16/f40 [97650,96071] 0 2024-05-25T06:11:25.082 INFO:tasks.workunit.client.1.smithi203.stdout:2/339: mkdir d2/d1a/d33/d80 0 2024-05-25T06:11:25.084 INFO:tasks.workunit.client.1.smithi203.stdout:4/238: dread d0/d8/d1f/f2b [0,4194304] 0 2024-05-25T06:11:25.084 INFO:tasks.workunit.client.0.smithi160.stdout:4/69: write d4/d6/fa [5097152,28592] 0 2024-05-25T06:11:25.085 INFO:tasks.workunit.client.1.smithi203.stdout:2/340: write d2/d4/d10/d32/f3d [2365127,90757] 0 2024-05-25T06:11:25.086 INFO:tasks.workunit.client.0.smithi160.stdout:8/58: truncate d5/fe 1974345 0 2024-05-25T06:11:25.088 INFO:tasks.workunit.client.0.smithi160.stdout:4/70: creat d4/d6/f13 x:0 0 0 2024-05-25T06:11:25.088 INFO:tasks.workunit.client.1.smithi203.stdout:8/281: rename d2/d5/d42 to d2/d5/d10/d45/d5d 0 2024-05-25T06:11:25.089 INFO:tasks.workunit.client.1.smithi203.stdout:3/371: readlink d10/d16/d71/l6b 0 2024-05-25T06:11:25.090 INFO:tasks.workunit.client.0.smithi160.stdout:4/71: symlink d4/l14 0 2024-05-25T06:11:25.091 INFO:tasks.workunit.client.1.smithi203.stdout:8/282: chown d2/d5/d10/d45/d5d 24288 1 2024-05-25T06:11:25.092 INFO:tasks.workunit.client.0.smithi160.stdout:4/72: mkdir d4/d6/d15 0 2024-05-25T06:11:25.092 INFO:tasks.workunit.client.1.smithi203.stdout:6/319: link d8/d29/d30/d38/f4c d8/d11/d1b/f69 0 2024-05-25T06:11:25.095 INFO:tasks.workunit.client.1.smithi203.stdout:1/221: creat d14/d1b/d1d/d3f/f4a x:0 0 0 2024-05-25T06:11:25.095 INFO:tasks.workunit.client.0.smithi160.stdout:4/73: rename d4/ld to d4/d6/l16 0 2024-05-25T06:11:25.095 INFO:tasks.workunit.client.1.smithi203.stdout:3/372: mknod d10/d16/d23/d45/c7b 0 2024-05-25T06:11:25.097 INFO:tasks.workunit.client.1.smithi203.stdout:4/239: mkdir d0/d9/d41 0 2024-05-25T06:11:25.098 INFO:tasks.workunit.client.0.smithi160.stdout:4/74: creat d4/d6/d15/f17 x:0 0 0 2024-05-25T06:11:25.098 INFO:tasks.workunit.client.0.smithi160.stdout:4/75: stat d4/f11 0 2024-05-25T06:11:25.099 INFO:tasks.workunit.client.1.smithi203.stdout:8/283: chown d2/d5/d28/d38/d4e/f51 506655 1 2024-05-25T06:11:25.099 INFO:tasks.workunit.client.1.smithi203.stdout:6/320: unlink d8/d11/d15/f31 0 2024-05-25T06:11:25.100 INFO:tasks.workunit.client.1.smithi203.stdout:5/512: write dc/d11/f1c [1344609,8524] 0 2024-05-25T06:11:25.104 INFO:tasks.workunit.client.1.smithi203.stdout:3/373: dwrite d10/d16/d23/d3a/f65 [0,4194304] 0 2024-05-25T06:11:25.107 INFO:tasks.workunit.client.0.smithi160.stdout:4/76: dwrite d4/d6/f10 [0,4194304] 0 2024-05-25T06:11:25.110 INFO:tasks.workunit.client.1.smithi203.stdout:3/374: dread d10/f21 [4194304,4194304] 0 2024-05-25T06:11:25.111 INFO:tasks.workunit.client.0.smithi160.stdout:4/77: symlink d4/d6/d15/l18 0 2024-05-25T06:11:25.111 INFO:tasks.workunit.client.0.smithi160.stdout:8/59: dwrite d5/d7/fa [0,4194304] 0 2024-05-25T06:11:25.112 INFO:tasks.workunit.client.1.smithi203.stdout:4/240: mkdir d0/d9/d42 0 2024-05-25T06:11:25.113 INFO:tasks.workunit.client.0.smithi160.stdout:8/60: fsync d5/fb 0 2024-05-25T06:11:25.113 INFO:tasks.workunit.client.0.smithi160.stdout:8/61: fdatasync d5/d9/fc 0 2024-05-25T06:11:25.114 INFO:tasks.workunit.client.0.smithi160.stdout:8/62: chown d5/d7/d8 1830 1 2024-05-25T06:11:25.114 INFO:tasks.workunit.client.1.smithi203.stdout:2/341: getdents d2/d1a/d33 0 2024-05-25T06:11:25.116 INFO:tasks.workunit.client.1.smithi203.stdout:6/321: getdents d8/d29/d46 0 2024-05-25T06:11:25.117 INFO:tasks.workunit.client.0.smithi160.stdout:8/63: write f3 [1170776,11281] 0 2024-05-25T06:11:25.117 INFO:tasks.workunit.client.1.smithi203.stdout:1/222: creat d14/f4b x:0 0 0 2024-05-25T06:11:25.121 INFO:tasks.workunit.client.1.smithi203.stdout:8/284: link d2/d5/d2d/f54 d2/d5/d18/d31/f5e 0 2024-05-25T06:11:25.121 INFO:tasks.workunit.client.0.smithi160.stdout:4/78: dwrite d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:25.124 INFO:tasks.workunit.client.1.smithi203.stdout:3/375: dwrite d10/f31 [0,4194304] 0 2024-05-25T06:11:25.125 INFO:tasks.workunit.client.1.smithi203.stdout:4/241: fdatasync d0/d9/d11/f3f 0 2024-05-25T06:11:25.125 INFO:tasks.workunit.client.1.smithi203.stdout:3/376: fdatasync f0 0 2024-05-25T06:11:25.127 INFO:tasks.workunit.client.1.smithi203.stdout:6/322: creat d8/d3d/f6a x:0 0 0 2024-05-25T06:11:25.127 INFO:tasks.workunit.client.0.smithi160.stdout:4/79: link d4/c12 d4/d6/c19 0 2024-05-25T06:11:25.128 INFO:tasks.workunit.client.1.smithi203.stdout:5/513: creat dc/d11/fb5 x:0 0 0 2024-05-25T06:11:25.130 INFO:tasks.workunit.client.1.smithi203.stdout:6/323: readlink d8/d11/d47/d60/l49 0 2024-05-25T06:11:25.130 INFO:tasks.workunit.client.1.smithi203.stdout:3/377: write d10/d16/d23/d45/f76 [776215,106343] 0 2024-05-25T06:11:25.130 INFO:tasks.workunit.client.1.smithi203.stdout:2/342: creat d2/d4/d10/d5f/f81 x:0 0 0 2024-05-25T06:11:25.131 INFO:tasks.workunit.client.0.smithi160.stdout:4/80: unlink d4/d6/fc 0 2024-05-25T06:11:25.134 INFO:tasks.workunit.client.1.smithi203.stdout:6/324: write d8/d11/d1b/f26 [2753354,24563] 0 2024-05-25T06:11:25.134 INFO:tasks.workunit.client.1.smithi203.stdout:5/514: write dc/d11/f1c [802931,65188] 0 2024-05-25T06:11:25.135 INFO:tasks.workunit.client.0.smithi160.stdout:4/81: dread d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:25.136 INFO:tasks.workunit.client.1.smithi203.stdout:6/325: truncate d8/f2c 61455 0 2024-05-25T06:11:25.138 INFO:tasks.workunit.client.1.smithi203.stdout:3/378: mknod d10/d16/d23/d38/c7c 0 2024-05-25T06:11:25.140 INFO:tasks.workunit.client.0.smithi160.stdout:4/82: dread d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:25.141 INFO:tasks.workunit.client.0.smithi160.stdout:8/64: fdatasync f3 0 2024-05-25T06:11:25.142 INFO:tasks.workunit.client.1.smithi203.stdout:1/223: getdents d14/d1b/d1d/d3e/d36 0 2024-05-25T06:11:25.142 INFO:tasks.workunit.client.0.smithi160.stdout:4/83: truncate d4/d6/d15/f17 749404 0 2024-05-25T06:11:25.143 INFO:tasks.workunit.client.1.smithi203.stdout:3/379: mkdir d10/d16/d50/d7d 0 2024-05-25T06:11:25.144 INFO:tasks.workunit.client.1.smithi203.stdout:3/380: chown d10/d16/d23/d67 0 1 2024-05-25T06:11:25.144 INFO:tasks.workunit.client.0.smithi160.stdout:8/65: dread d5/d9/fc [0,4194304] 0 2024-05-25T06:11:25.144 INFO:tasks.workunit.client.0.smithi160.stdout:4/84: mkdir d4/d1a 0 2024-05-25T06:11:25.145 INFO:tasks.workunit.client.1.smithi203.stdout:1/224: mkdir d14/d23/d4c 0 2024-05-25T06:11:25.147 INFO:tasks.workunit.client.1.smithi203.stdout:3/381: mknod d10/d16/d23/d4a/c7e 0 2024-05-25T06:11:25.148 INFO:tasks.workunit.client.1.smithi203.stdout:5/515: getdents dc/d11/d17/d1a 0 2024-05-25T06:11:25.150 INFO:tasks.workunit.client.1.smithi203.stdout:1/225: rename d14/l33 to d14/d1b/d1d/d3e/l4d 0 2024-05-25T06:11:25.151 INFO:tasks.workunit.client.0.smithi160.stdout:4/85: dread d4/d6/f10 [0,4194304] 0 2024-05-25T06:11:25.152 INFO:tasks.workunit.client.1.smithi203.stdout:1/226: truncate d14/d1b/d1d/d3e/d36/f39 1543014 0 2024-05-25T06:11:25.152 INFO:tasks.workunit.client.1.smithi203.stdout:5/516: creat dc/d11/d17/d1a/d2d/d53/da0/fb6 x:0 0 0 2024-05-25T06:11:25.152 INFO:tasks.workunit.client.0.smithi160.stdout:8/66: dwrite d5/d7/fa [4194304,4194304] 0 2024-05-25T06:11:25.152 INFO:tasks.workunit.client.0.smithi160.stdout:4/86: dread - d4/d6/f13 zero size 2024-05-25T06:11:25.153 INFO:tasks.workunit.client.1.smithi203.stdout:5/517: truncate dc/d11/d17/f54 147122 0 2024-05-25T06:11:25.154 INFO:tasks.workunit.client.0.smithi160.stdout:8/67: truncate f3 8662147 0 2024-05-25T06:11:25.157 INFO:tasks.workunit.client.0.smithi160.stdout:8/68: creat d5/d9/f13 x:0 0 0 2024-05-25T06:11:25.157 INFO:tasks.workunit.client.1.smithi203.stdout:1/227: rename d14/d37/l44 to d14/d1b/d1d/d3e/d36/l4e 0 2024-05-25T06:11:25.160 INFO:tasks.workunit.client.1.smithi203.stdout:1/228: truncate d14/f1a 209426 0 2024-05-25T06:11:25.161 INFO:tasks.workunit.client.0.smithi160.stdout:8/69: mkdir d5/d9/d14 0 2024-05-25T06:11:25.163 INFO:tasks.workunit.client.1.smithi203.stdout:5/518: dwrite dc/d11/d17/d1a/d2d/d53/da0/fb6 [0,4194304] 0 2024-05-25T06:11:25.164 INFO:tasks.workunit.client.0.smithi160.stdout:4/87: dwrite d4/d6/f13 [0,4194304] 0 2024-05-25T06:11:25.165 INFO:tasks.workunit.client.1.smithi203.stdout:8/285: truncate d2/d5/f44 1577554 0 2024-05-25T06:11:25.166 INFO:tasks.workunit.client.1.smithi203.stdout:8/286: write d2/d5/d18/d31/f3d [5380607,40520] 0 2024-05-25T06:11:25.167 INFO:tasks.workunit.client.1.smithi203.stdout:6/326: write d8/d11/d1b/f69 [174524,113240] 0 2024-05-25T06:11:25.169 INFO:tasks.workunit.client.0.smithi160.stdout:8/70: getdents d5/d9/d14 0 2024-05-25T06:11:25.170 INFO:tasks.workunit.client.0.smithi160.stdout:4/88: mknod d4/d1a/c1b 0 2024-05-25T06:11:25.171 INFO:tasks.workunit.client.1.smithi203.stdout:5/519: creat dc/d11/d42/d94/fb7 x:0 0 0 2024-05-25T06:11:25.175 INFO:tasks.workunit.client.0.smithi160.stdout:8/71: mkdir d5/d7/d8/d15 0 2024-05-25T06:11:25.176 INFO:tasks.workunit.client.1.smithi203.stdout:8/287: readlink d2/d5/d10/l24 0 2024-05-25T06:11:25.177 INFO:tasks.workunit.client.1.smithi203.stdout:2/343: dwrite d2/d4/d10/d58/f6d [0,4194304] 0 2024-05-25T06:11:25.177 INFO:tasks.workunit.client.1.smithi203.stdout:8/288: readlink d2/d5/d10/d11/d1f/l25 0 2024-05-25T06:11:25.179 INFO:tasks.workunit.client.1.smithi203.stdout:5/520: symlink dc/d11/d17/d1a/lb8 0 2024-05-25T06:11:25.182 INFO:tasks.workunit.client.1.smithi203.stdout:3/382: truncate d10/d16/d23/d45/f76 574404 0 2024-05-25T06:11:25.182 INFO:tasks.workunit.client.0.smithi160.stdout:8/72: rename d5/d7/fa to d5/d7/d8/f16 0 2024-05-25T06:11:25.183 INFO:tasks.workunit.client.0.smithi160.stdout:4/89: dwrite d4/d6/d15/f17 [0,4194304] 0 2024-05-25T06:11:25.185 INFO:tasks.workunit.client.1.smithi203.stdout:3/383: read d10/d16/d50/f64 [3232078,76628] 0 2024-05-25T06:11:25.186 INFO:tasks.workunit.client.1.smithi203.stdout:1/229: chown d14/d1b/d1d/d3e/d36/l4e 0 1 2024-05-25T06:11:25.189 INFO:tasks.workunit.client.0.smithi160.stdout:8/73: rename d5/d7/c11 to d5/d9/d14/c17 0 2024-05-25T06:11:25.190 INFO:tasks.workunit.client.0.smithi160.stdout:4/90: fsync d4/d6/fa 0 2024-05-25T06:11:25.190 INFO:tasks.workunit.client.1.smithi203.stdout:6/327: link d8/d3d/c40 d8/d11/d15/d28/d52/d43/c6b 0 2024-05-25T06:11:25.191 INFO:tasks.workunit.client.1.smithi203.stdout:8/289: dwrite d2/d5/d10/d11/de/f13 [0,4194304] 0 2024-05-25T06:11:25.193 INFO:tasks.workunit.client.1.smithi203.stdout:5/521: mknod dc/d11/d42/d94/dad/db1/cb9 0 2024-05-25T06:11:25.194 INFO:tasks.workunit.client.0.smithi160.stdout:8/74: rename d5/lf to d5/d9/d14/l18 0 2024-05-25T06:11:25.194 INFO:tasks.workunit.client.1.smithi203.stdout:3/384: rename d10/d22/l54 to d10/d16/d23/d3a/l7f 0 2024-05-25T06:11:25.195 INFO:tasks.workunit.client.0.smithi160.stdout:8/75: write d5/d9/f13 [722842,5158] 0 2024-05-25T06:11:25.196 INFO:tasks.workunit.client.0.smithi160.stdout:4/91: rmdir d4/d1a 39 2024-05-25T06:11:25.197 INFO:tasks.workunit.client.0.smithi160.stdout:8/76: chown d5/d7/d8/f16 807354334 1 2024-05-25T06:11:25.198 INFO:tasks.workunit.client.1.smithi203.stdout:1/230: creat d14/d23/d4c/f4f x:0 0 0 2024-05-25T06:11:25.199 INFO:tasks.workunit.client.1.smithi203.stdout:8/290: dread d2/d5/d18/d31/f39 [0,4194304] 0 2024-05-25T06:11:25.200 INFO:tasks.workunit.client.1.smithi203.stdout:6/328: mkdir d8/d2a/d6c 0 2024-05-25T06:11:25.201 INFO:tasks.workunit.client.0.smithi160.stdout:8/77: creat d5/d7/f19 x:0 0 0 2024-05-25T06:11:25.202 INFO:tasks.workunit.client.0.smithi160.stdout:8/78: chown d5/ld 3 1 2024-05-25T06:11:25.202 INFO:tasks.workunit.client.0.smithi160.stdout:4/92: mkdir d4/d1c 0 2024-05-25T06:11:25.205 INFO:tasks.workunit.client.1.smithi203.stdout:1/231: write d14/d1b/d1d/f2a [1170030,23629] 0 2024-05-25T06:11:25.205 INFO:tasks.workunit.client.0.smithi160.stdout:8/79: dread f3 [4194304,4194304] 0 2024-05-25T06:11:25.207 INFO:tasks.workunit.client.0.smithi160.stdout:4/93: symlink d4/d1c/l1d 0 2024-05-25T06:11:25.210 INFO:tasks.workunit.client.0.smithi160.stdout:4/94: rename d4/d6/f10 to d4/d6/d15/f1e 0 2024-05-25T06:11:25.212 INFO:tasks.workunit.client.0.smithi160.stdout:4/95: read d4/d6/d15/f1e [3917001,71323] 0 2024-05-25T06:11:25.213 INFO:tasks.workunit.client.1.smithi203.stdout:1/232: getdents d14/d37 0 2024-05-25T06:11:25.215 INFO:tasks.workunit.client.1.smithi203.stdout:6/329: dwrite d8/d11/d15/f17 [0,4194304] 0 2024-05-25T06:11:25.216 INFO:tasks.workunit.client.1.smithi203.stdout:6/330: readlink d8/d29/d30/d38/l5c 0 2024-05-25T06:11:25.217 INFO:tasks.workunit.client.1.smithi203.stdout:1/233: rename f12 to d14/d23/d4c/f50 0 2024-05-25T06:11:25.218 INFO:tasks.workunit.client.0.smithi160.stdout:4/96: dwrite d4/d6/d15/f17 [4194304,4194304] 0 2024-05-25T06:11:25.219 INFO:tasks.workunit.client.1.smithi203.stdout:6/331: truncate d8/d11/d15/f37 285011 0 2024-05-25T06:11:25.220 INFO:tasks.workunit.client.1.smithi203.stdout:1/234: mknod d14/d1b/d1d/d3e/c51 0 2024-05-25T06:11:25.221 INFO:tasks.workunit.client.1.smithi203.stdout:6/332: write d8/d11/d15/f17 [1900425,86008] 0 2024-05-25T06:11:25.222 INFO:tasks.workunit.client.0.smithi160.stdout:4/97: link d4/l14 d4/d6/l1f 0 2024-05-25T06:11:25.223 INFO:tasks.workunit.client.1.smithi203.stdout:6/333: write d8/d11/d15/f59 [662220,21898] 0 2024-05-25T06:11:25.224 INFO:tasks.workunit.client.1.smithi203.stdout:6/334: fdatasync d8/d3d/f44 0 2024-05-25T06:11:25.224 INFO:tasks.workunit.client.1.smithi203.stdout:1/235: rename d14/d1b/d1d/d3e/d36/l4e to d14/d1b/d1d/d3e/l52 0 2024-05-25T06:11:25.225 INFO:tasks.workunit.client.0.smithi160.stdout:4/98: unlink d4/d6/c19 0 2024-05-25T06:11:25.225 INFO:tasks.workunit.client.1.smithi203.stdout:6/335: symlink d8/d11/d47/d60/l6d 0 2024-05-25T06:11:25.229 INFO:tasks.workunit.client.1.smithi203.stdout:1/236: dread d14/d1b/d1d/f30 [0,4194304] 0 2024-05-25T06:11:25.233 INFO:tasks.workunit.client.0.smithi160.stdout:4/99: dwrite d4/f11 [0,4194304] 0 2024-05-25T06:11:25.236 INFO:tasks.workunit.client.1.smithi203.stdout:1/237: dwrite d14/f17 [0,4194304] 0 2024-05-25T06:11:25.236 INFO:tasks.workunit.client.1.smithi203.stdout:1/238: fsync fc 0 2024-05-25T06:11:25.239 INFO:tasks.workunit.client.0.smithi160.stdout:4/100: dwrite d4/d6/fe [0,4194304] 0 2024-05-25T06:11:25.248 INFO:tasks.workunit.client.1.smithi203.stdout:6/336: dread d8/d11/f19 [0,4194304] 0 2024-05-25T06:11:25.249 INFO:tasks.workunit.client.0.smithi160.stdout:4/101: link d4/d6/d15/l18 d4/d1a/l20 0 2024-05-25T06:11:25.249 INFO:tasks.workunit.client.1.smithi203.stdout:1/239: dwrite d14/d23/d4c/f4f [0,4194304] 0 2024-05-25T06:11:25.254 INFO:tasks.workunit.client.1.smithi203.stdout:6/337: rename d8/d11/d47/d62 to d8/d29/d6e 0 2024-05-25T06:11:25.257 INFO:tasks.workunit.client.1.smithi203.stdout:6/338: creat d8/d2a/d6c/f6f x:0 0 0 2024-05-25T06:11:25.258 INFO:tasks.workunit.client.1.smithi203.stdout:1/240: dwrite d14/d1b/d1d/f2a [0,4194304] 0 2024-05-25T06:11:25.264 INFO:tasks.workunit.client.1.smithi203.stdout:6/339: dwrite d8/d11/d1b/f26 [0,4194304] 0 2024-05-25T06:11:25.265 INFO:tasks.workunit.client.1.smithi203.stdout:1/241: dwrite d14/f25 [0,4194304] 0 2024-05-25T06:11:25.271 INFO:tasks.workunit.client.1.smithi203.stdout:1/242: mknod d14/c53 0 2024-05-25T06:11:25.272 INFO:tasks.workunit.client.1.smithi203.stdout:6/340: getdents d8/d3d 0 2024-05-25T06:11:25.276 INFO:tasks.workunit.client.1.smithi203.stdout:1/243: dread d14/d1b/d29/f2b [0,4194304] 0 2024-05-25T06:11:25.279 INFO:tasks.workunit.client.1.smithi203.stdout:6/341: dwrite d8/d29/d30/d38/f39 [0,4194304] 0 2024-05-25T06:11:25.280 INFO:tasks.workunit.client.1.smithi203.stdout:6/342: chown d8/d11/d47/d60/f55 16343672 1 2024-05-25T06:11:25.281 INFO:tasks.workunit.client.1.smithi203.stdout:1/244: creat d14/f54 x:0 0 0 2024-05-25T06:11:25.281 INFO:tasks.workunit.client.1.smithi203.stdout:1/245: readlink d14/l27 0 2024-05-25T06:11:25.287 INFO:tasks.workunit.client.1.smithi203.stdout:5/522: readlink dc/d11/d17/d1a/lb8 0 2024-05-25T06:11:25.290 INFO:tasks.workunit.client.1.smithi203.stdout:5/523: write dc/f21 [159942,24025] 0 2024-05-25T06:11:25.291 INFO:tasks.workunit.client.1.smithi203.stdout:5/524: truncate dc/d11/d17/d44/d73/d98/f9f 728653 0 2024-05-25T06:11:25.292 INFO:tasks.workunit.client.1.smithi203.stdout:5/525: fdatasync dc/d3c/d3f/f57 0 2024-05-25T06:11:25.292 INFO:tasks.workunit.client.1.smithi203.stdout:1/246: link d14/f32 d14/d1b/d1d/d3f/f55 0 2024-05-25T06:11:25.293 INFO:tasks.workunit.client.1.smithi203.stdout:1/247: read d14/d1b/d1d/f2a [491864,4402] 0 2024-05-25T06:11:25.296 INFO:tasks.workunit.client.1.smithi203.stdout:5/526: stat dc/d11/d17/d1a/f27 0 2024-05-25T06:11:25.297 INFO:tasks.workunit.client.1.smithi203.stdout:5/527: write dc/d11/d42/d94/fb7 [550332,111553] 0 2024-05-25T06:11:25.298 INFO:tasks.workunit.client.1.smithi203.stdout:1/248: mknod d14/d37/c56 0 2024-05-25T06:11:25.298 INFO:tasks.workunit.client.1.smithi203.stdout:1/249: readlink d14/l27 0 2024-05-25T06:11:25.299 INFO:tasks.workunit.client.1.smithi203.stdout:1/250: chown d14/f1a 3428 1 2024-05-25T06:11:25.301 INFO:tasks.workunit.client.1.smithi203.stdout:5/528: write dc/d11/d17/d1a/d22/fb4 [308467,88167] 0 2024-05-25T06:11:25.304 INFO:tasks.workunit.client.1.smithi203.stdout:1/251: dread - d14/d1b/d1d/f40 zero size 2024-05-25T06:11:25.305 INFO:tasks.workunit.client.1.smithi203.stdout:6/343: rmdir d8/d11/d15 39 2024-05-25T06:11:25.308 INFO:tasks.workunit.client.1.smithi203.stdout:5/529: creat dc/d11/d17/fba x:0 0 0 2024-05-25T06:11:25.308 INFO:tasks.workunit.client.1.smithi203.stdout:1/252: chown d14/d1b/d1d/d3e 29425022 1 2024-05-25T06:11:25.315 INFO:tasks.workunit.client.1.smithi203.stdout:5/530: creat dc/d11/d17/d44/fbb x:0 0 0 2024-05-25T06:11:25.315 INFO:tasks.workunit.client.0.smithi160.stdout:4/102: truncate d4/d6/f13 216991 0 2024-05-25T06:11:25.315 INFO:tasks.workunit.client.1.smithi203.stdout:8/291: dwrite d2/d5/d18/d31/f34 [0,4194304] 0 2024-05-25T06:11:25.317 INFO:tasks.workunit.client.0.smithi160.stdout:4/103: mkdir d4/d6/d21 0 2024-05-25T06:11:25.318 INFO:tasks.workunit.client.1.smithi203.stdout:3/385: dwrite d10/d16/d23/d3a/d48/f52 [0,4194304] 0 2024-05-25T06:11:25.318 INFO:tasks.workunit.client.0.smithi160.stdout:4/104: fdatasync d4/d6/fe 0 2024-05-25T06:11:25.324 INFO:tasks.workunit.client.0.smithi160.stdout:4/105: dwrite d4/d6/fe [0,4194304] 0 2024-05-25T06:11:25.325 INFO:tasks.workunit.client.1.smithi203.stdout:5/531: creat dc/daa/fbc x:0 0 0 2024-05-25T06:11:25.329 INFO:tasks.workunit.client.1.smithi203.stdout:3/386: rename d10/d16/d23/f5f to d10/d16/d23/d3a/f80 0 2024-05-25T06:11:25.330 INFO:tasks.workunit.client.1.smithi203.stdout:5/532: fsync dc/d11/d17/f56 0 2024-05-25T06:11:25.330 INFO:tasks.workunit.client.1.smithi203.stdout:6/344: dwrite d8/d29/d30/f4f [0,4194304] 0 2024-05-25T06:11:25.330 INFO:tasks.workunit.client.1.smithi203.stdout:1/253: dwrite d14/f32 [0,4194304] 0 2024-05-25T06:11:25.341 INFO:tasks.workunit.client.1.smithi203.stdout:3/387: creat d10/d22/d2a/f81 x:0 0 0 2024-05-25T06:11:25.347 INFO:tasks.workunit.client.0.smithi160.stdout:4/106: write d4/d6/fa [3188960,51812] 0 2024-05-25T06:11:25.349 INFO:tasks.workunit.client.1.smithi203.stdout:1/254: dwrite fa [0,4194304] 0 2024-05-25T06:11:25.349 INFO:tasks.workunit.client.0.smithi160.stdout:4/107: getdents d4/d6/d21 0 2024-05-25T06:11:25.351 INFO:tasks.workunit.client.1.smithi203.stdout:6/345: rename d8/d29/d30 to d8/d70 0 2024-05-25T06:11:25.353 INFO:tasks.workunit.client.1.smithi203.stdout:6/346: chown d8/d2a/l3a 945099 1 2024-05-25T06:11:25.354 INFO:tasks.workunit.client.1.smithi203.stdout:8/292: dwrite d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:25.356 INFO:tasks.workunit.client.1.smithi203.stdout:6/347: write d8/f2c [556186,54183] 0 2024-05-25T06:11:25.356 INFO:tasks.workunit.client.1.smithi203.stdout:5/533: dwrite dc/d11/d17/d1a/d72/f8c [0,4194304] 0 2024-05-25T06:11:25.357 INFO:tasks.workunit.client.1.smithi203.stdout:1/255: rename d14/d1b to d14/d57 0 2024-05-25T06:11:25.360 INFO:tasks.workunit.client.1.smithi203.stdout:5/534: write dc/f80 [4664156,45467] 0 2024-05-25T06:11:25.362 INFO:tasks.workunit.client.1.smithi203.stdout:8/293: read d2/d5/fc [1210302,73904] 0 2024-05-25T06:11:25.362 INFO:tasks.workunit.client.1.smithi203.stdout:5/535: chown l0 28947 1 2024-05-25T06:11:25.365 INFO:tasks.workunit.client.1.smithi203.stdout:1/256: dread d14/f18 [0,4194304] 0 2024-05-25T06:11:25.369 INFO:tasks.workunit.client.1.smithi203.stdout:5/536: rename dc/d11/d17/d1a/d2d/d53/l68 to dc/d11/d42/lbd 0 2024-05-25T06:11:25.370 INFO:tasks.workunit.client.1.smithi203.stdout:3/388: fsync d10/d16/d23/d3a/f80 0 2024-05-25T06:11:25.373 INFO:tasks.workunit.client.1.smithi203.stdout:8/294: link d2/d5/d10/f4d d2/d5/d28/f5f 0 2024-05-25T06:11:25.376 INFO:tasks.workunit.client.1.smithi203.stdout:8/295: rename d2/d5/d18/d31 to d2/d5/d18/d31/d60 22 2024-05-25T06:11:25.376 INFO:tasks.workunit.client.1.smithi203.stdout:1/257: creat d14/d57/f58 x:0 0 0 2024-05-25T06:11:25.377 INFO:tasks.workunit.client.1.smithi203.stdout:5/537: dwrite dc/d11/d17/d45/f52 [4194304,4194304] 0 2024-05-25T06:11:25.377 INFO:tasks.workunit.client.1.smithi203.stdout:8/296: stat d2/d5/d18/d31/f3d 0 2024-05-25T06:11:25.381 INFO:tasks.workunit.client.1.smithi203.stdout:8/297: write d2/d5/d28/d38/d4e/f51 [113622,33726] 0 2024-05-25T06:11:25.382 INFO:tasks.workunit.client.1.smithi203.stdout:5/538: creat dc/d11/d17/d44/fbe x:0 0 0 2024-05-25T06:11:25.382 INFO:tasks.workunit.client.1.smithi203.stdout:1/258: write d14/d57/d1d/f2a [2492688,100126] 0 2024-05-25T06:11:25.386 INFO:tasks.workunit.client.1.smithi203.stdout:1/259: creat d14/d57/d1d/d3e/d36/f59 x:0 0 0 2024-05-25T06:11:25.390 INFO:tasks.workunit.client.1.smithi203.stdout:6/348: write d8/d11/d15/f37 [752233,111004] 0 2024-05-25T06:11:25.390 INFO:tasks.workunit.client.1.smithi203.stdout:5/539: dwrite dc/d11/d17/d44/fbe [0,4194304] 0 2024-05-25T06:11:25.393 INFO:tasks.workunit.client.1.smithi203.stdout:1/260: rename d14/d57/c34 to d14/d57/d1d/d3f/c5a 0 2024-05-25T06:11:25.395 INFO:tasks.workunit.client.1.smithi203.stdout:5/540: mknod dc/d11/d17/d1a/d2d/d53/da2/cbf 0 2024-05-25T06:11:25.395 INFO:tasks.workunit.client.1.smithi203.stdout:1/261: fdatasync d14/d57/d1d/d3e/d36/f39 0 2024-05-25T06:11:25.397 INFO:tasks.workunit.client.1.smithi203.stdout:1/262: chown c11 32269698 1 2024-05-25T06:11:25.400 INFO:tasks.workunit.client.1.smithi203.stdout:1/263: chown d14/d57/d1d/f40 3104765 1 2024-05-25T06:11:25.401 INFO:tasks.workunit.client.1.smithi203.stdout:5/541: rename dc/d11/d42/d94/dad/db0 to dc/d11/d18/dc0 0 2024-05-25T06:11:25.401 INFO:tasks.workunit.client.1.smithi203.stdout:6/349: dwrite d8/d11/d47/d60/f55 [0,4194304] 0 2024-05-25T06:11:25.405 INFO:tasks.workunit.client.1.smithi203.stdout:3/389: dread d10/d16/d23/f2b [0,4194304] 0 2024-05-25T06:11:25.407 INFO:tasks.workunit.client.1.smithi203.stdout:3/390: stat le 0 2024-05-25T06:11:25.407 INFO:tasks.workunit.client.1.smithi203.stdout:6/350: mknod d8/d11/d15/d28/d52/c71 0 2024-05-25T06:11:25.409 INFO:tasks.workunit.client.1.smithi203.stdout:3/391: mknod d10/d16/d23/c82 0 2024-05-25T06:11:25.409 INFO:tasks.workunit.client.1.smithi203.stdout:8/298: dread d2/d5/d28/f3a [0,4194304] 0 2024-05-25T06:11:25.410 INFO:tasks.workunit.client.1.smithi203.stdout:3/392: readlink d10/d22/d2a/l41 0 2024-05-25T06:11:25.411 INFO:tasks.workunit.client.1.smithi203.stdout:1/264: dwrite d14/f24 [0,4194304] 0 2024-05-25T06:11:25.411 INFO:tasks.workunit.client.1.smithi203.stdout:3/393: chown d10/d16/d50/f64 229 1 2024-05-25T06:11:25.416 INFO:tasks.workunit.client.1.smithi203.stdout:5/542: write dc/d11/f3e [986286,56829] 0 2024-05-25T06:11:25.418 INFO:tasks.workunit.client.1.smithi203.stdout:5/543: read - dc/d3c/d3f/f93 zero size 2024-05-25T06:11:25.419 INFO:tasks.workunit.client.1.smithi203.stdout:1/265: dread d14/d57/d1d/f2a [0,4194304] 0 2024-05-25T06:11:25.420 INFO:tasks.workunit.client.1.smithi203.stdout:1/266: read d14/d23/f42 [2288860,86163] 0 2024-05-25T06:11:25.423 INFO:tasks.workunit.client.1.smithi203.stdout:1/267: rename d14/d57/d1d/d3f/f4a to d14/d57/d1d/d3f/f5b 0 2024-05-25T06:11:25.425 INFO:tasks.workunit.client.1.smithi203.stdout:5/544: link dc/d11/d17/d44/d73/d98/f9f dc/d11/d17/d1a/fc1 0 2024-05-25T06:11:25.428 INFO:tasks.workunit.client.1.smithi203.stdout:8/299: dwrite d2/d5/d28/f5f [0,4194304] 0 2024-05-25T06:11:25.431 INFO:tasks.workunit.client.1.smithi203.stdout:3/394: dwrite d10/d16/d23/d3a/f53 [0,4194304] 0 2024-05-25T06:11:25.441 INFO:tasks.workunit.client.1.smithi203.stdout:1/268: dwrite d14/d57/d29/f2b [0,4194304] 0 2024-05-25T06:11:25.442 INFO:tasks.workunit.client.1.smithi203.stdout:3/395: dread d10/d16/d23/d45/f76 [0,4194304] 0 2024-05-25T06:11:25.443 INFO:tasks.workunit.client.1.smithi203.stdout:1/269: stat d14/f1a 0 2024-05-25T06:11:25.443 INFO:tasks.workunit.client.1.smithi203.stdout:8/300: dwrite d2/f14 [4194304,4194304] 0 2024-05-25T06:11:25.445 INFO:tasks.workunit.client.1.smithi203.stdout:3/396: symlink d10/d22/l83 0 2024-05-25T06:11:25.447 INFO:tasks.workunit.client.1.smithi203.stdout:3/397: creat d10/d16/d50/f84 x:0 0 0 2024-05-25T06:11:25.449 INFO:tasks.workunit.client.1.smithi203.stdout:1/270: link d14/d57/c41 d14/d57/d29/c5c 0 2024-05-25T06:11:25.451 INFO:tasks.workunit.client.1.smithi203.stdout:3/398: creat d10/d16/d23/d3a/d48/f85 x:0 0 0 2024-05-25T06:11:25.453 INFO:tasks.workunit.client.1.smithi203.stdout:3/399: fsync d10/d16/d23/d67/f78 0 2024-05-25T06:11:25.454 INFO:tasks.workunit.client.1.smithi203.stdout:8/301: dwrite d2/d5/d10/d11/de/ff [4194304,4194304] 0 2024-05-25T06:11:25.455 INFO:tasks.workunit.client.1.smithi203.stdout:1/271: creat d14/d23/d4c/f5d x:0 0 0 2024-05-25T06:11:25.458 INFO:tasks.workunit.client.1.smithi203.stdout:3/400: creat d10/d22/f86 x:0 0 0 2024-05-25T06:11:25.460 INFO:tasks.workunit.client.1.smithi203.stdout:1/272: rename d14/d57/d1d/d3f to d14/d37/d5e 0 2024-05-25T06:11:25.462 INFO:tasks.workunit.client.1.smithi203.stdout:3/401: stat d10/d16/d23/d38/c7c 0 2024-05-25T06:11:25.463 INFO:tasks.workunit.client.1.smithi203.stdout:3/402: fdatasync d10/d16/f19 0 2024-05-25T06:11:25.466 INFO:tasks.workunit.client.1.smithi203.stdout:8/302: dwrite d2/d5/d2d/f4b [0,4194304] 0 2024-05-25T06:11:25.467 INFO:tasks.workunit.client.1.smithi203.stdout:5/545: dread dc/d11/d17/f31 [0,4194304] 0 2024-05-25T06:11:25.468 INFO:tasks.workunit.client.1.smithi203.stdout:1/273: dread d14/f17 [0,4194304] 0 2024-05-25T06:11:25.470 INFO:tasks.workunit.client.1.smithi203.stdout:9/264: sync 2024-05-25T06:11:25.470 INFO:tasks.workunit.client.1.smithi203.stdout:5/546: creat dc/d11/d18/fc2 x:0 0 0 2024-05-25T06:11:25.473 INFO:tasks.workunit.client.1.smithi203.stdout:1/274: mkdir d14/d5f 0 2024-05-25T06:11:25.474 INFO:tasks.workunit.client.1.smithi203.stdout:9/265: write d0/df/f4a [198548,52540] 0 2024-05-25T06:11:25.476 INFO:tasks.workunit.client.1.smithi203.stdout:5/547: read dc/d11/d17/d45/f46 [1605566,47377] 0 2024-05-25T06:11:25.477 INFO:tasks.workunit.client.1.smithi203.stdout:9/266: dread - d0/f52 zero size 2024-05-25T06:11:25.477 INFO:tasks.workunit.client.1.smithi203.stdout:3/403: dwrite d10/d16/d23/d3a/d48/f85 [0,4194304] 0 2024-05-25T06:11:25.479 INFO:tasks.workunit.client.1.smithi203.stdout:5/548: read dc/d11/f14 [1020939,66394] 0 2024-05-25T06:11:25.480 INFO:tasks.workunit.client.1.smithi203.stdout:1/275: dread d14/f18 [0,4194304] 0 2024-05-25T06:11:25.481 INFO:tasks.workunit.client.1.smithi203.stdout:8/303: dwrite d2/d5/d18/d31/f50 [0,4194304] 0 2024-05-25T06:11:25.481 INFO:tasks.workunit.client.1.smithi203.stdout:1/276: chown d14/f54 485437 1 2024-05-25T06:11:25.482 INFO:tasks.workunit.client.1.smithi203.stdout:9/267: rmdir d0 39 2024-05-25T06:11:25.482 INFO:tasks.workunit.client.1.smithi203.stdout:8/304: chown d2/d5/d28/c48 230529997 1 2024-05-25T06:11:25.484 INFO:tasks.workunit.client.1.smithi203.stdout:3/404: rmdir d10/d16/d23/d46 39 2024-05-25T06:11:25.486 INFO:tasks.workunit.client.1.smithi203.stdout:5/549: symlink dc/d11/d42/d94/dad/lc3 0 2024-05-25T06:11:25.490 INFO:tasks.workunit.client.1.smithi203.stdout:8/305: creat d2/f61 x:0 0 0 2024-05-25T06:11:25.492 INFO:tasks.workunit.client.1.smithi203.stdout:3/405: rmdir d10/d16/d23/d45 39 2024-05-25T06:11:25.495 INFO:tasks.workunit.client.1.smithi203.stdout:5/550: dwrite dc/d11/d17/d45/f62 [0,4194304] 0 2024-05-25T06:11:25.497 INFO:tasks.workunit.client.1.smithi203.stdout:3/406: truncate d10/d22/f5b 1028990 0 2024-05-25T06:11:25.498 INFO:tasks.workunit.client.1.smithi203.stdout:9/268: dwrite d0/df/f1a [0,4194304] 0 2024-05-25T06:11:25.498 INFO:tasks.workunit.client.1.smithi203.stdout:5/551: symlink dc/d6a/lc4 0 2024-05-25T06:11:25.500 INFO:tasks.workunit.client.1.smithi203.stdout:9/269: chown d0/d17/d20/d29/c38 18 1 2024-05-25T06:11:25.500 INFO:tasks.workunit.client.1.smithi203.stdout:3/407: unlink d10/d22/l2e 0 2024-05-25T06:11:25.503 INFO:tasks.workunit.client.1.smithi203.stdout:8/306: link d2/d5/d10/d11/de/c5b d2/d5/d28/d38/c62 0 2024-05-25T06:11:25.503 INFO:tasks.workunit.client.1.smithi203.stdout:1/277: write d14/d23/f42 [1765929,17132] 0 2024-05-25T06:11:25.505 INFO:tasks.workunit.client.1.smithi203.stdout:5/552: dread - dc/d11/d17/d1a/d2d/f65 zero size 2024-05-25T06:11:25.506 INFO:tasks.workunit.client.1.smithi203.stdout:8/307: truncate d2/f2a 2320988 0 2024-05-25T06:11:25.509 INFO:tasks.workunit.client.1.smithi203.stdout:5/553: write dc/d11/d17/d44/fbe [5177032,13289] 0 2024-05-25T06:11:25.511 INFO:tasks.workunit.client.1.smithi203.stdout:9/270: dwrite d0/d17/d20/f3a [0,4194304] 0 2024-05-25T06:11:25.511 INFO:tasks.workunit.client.1.smithi203.stdout:3/408: dread d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:25.512 INFO:tasks.workunit.client.1.smithi203.stdout:9/271: write d0/df/f1a [2833898,66287] 0 2024-05-25T06:11:25.513 INFO:tasks.workunit.client.1.smithi203.stdout:1/278: getdents d14/d37 0 2024-05-25T06:11:25.513 INFO:tasks.workunit.client.1.smithi203.stdout:8/308: creat d2/d5/d10/d11/d59/f63 x:0 0 0 2024-05-25T06:11:25.514 INFO:tasks.workunit.client.1.smithi203.stdout:3/409: read d10/f20 [1561516,32031] 0 2024-05-25T06:11:25.517 INFO:tasks.workunit.client.1.smithi203.stdout:9/272: symlink d0/d17/d20/d29/l55 0 2024-05-25T06:11:25.519 INFO:tasks.workunit.client.1.smithi203.stdout:9/273: fsync d0/d17/f1b 0 2024-05-25T06:11:25.521 INFO:tasks.workunit.client.1.smithi203.stdout:8/309: mknod d2/d5/d10/c64 0 2024-05-25T06:11:25.524 INFO:tasks.workunit.client.1.smithi203.stdout:1/279: mknod d14/d57/c60 0 2024-05-25T06:11:25.524 INFO:tasks.workunit.client.1.smithi203.stdout:5/554: dwrite dc/d11/d17/d1a/f92 [0,4194304] 0 2024-05-25T06:11:25.525 INFO:tasks.workunit.client.1.smithi203.stdout:9/274: chown d0/l3 3493 1 2024-05-25T06:11:25.526 INFO:tasks.workunit.client.1.smithi203.stdout:3/410: dwrite d10/d16/d23/d3a/d48/f85 [0,4194304] 0 2024-05-25T06:11:25.526 INFO:tasks.workunit.client.1.smithi203.stdout:8/310: fdatasync d2/d5/d18/d31/f39 0 2024-05-25T06:11:25.529 INFO:tasks.workunit.client.1.smithi203.stdout:9/275: chown d0/df/d2d/d45/f49 6482860 1 2024-05-25T06:11:25.529 INFO:tasks.workunit.client.1.smithi203.stdout:1/280: read fc [603175,85217] 0 2024-05-25T06:11:25.531 INFO:tasks.workunit.client.1.smithi203.stdout:1/281: chown c7 31255377 1 2024-05-25T06:11:25.532 INFO:tasks.workunit.client.1.smithi203.stdout:3/411: creat d10/d16/d50/f87 x:0 0 0 2024-05-25T06:11:25.533 INFO:tasks.workunit.client.1.smithi203.stdout:8/311: symlink d2/d5/d18/d4f/l65 0 2024-05-25T06:11:25.536 INFO:tasks.workunit.client.1.smithi203.stdout:3/412: mkdir d10/d16/d23/d4a/d88 0 2024-05-25T06:11:25.536 INFO:tasks.workunit.client.1.smithi203.stdout:9/276: symlink d0/d17/d20/l56 0 2024-05-25T06:11:25.538 INFO:tasks.workunit.client.1.smithi203.stdout:8/312: rename d2/d5/d28/f5f to d2/d5/d18/d4f/f66 0 2024-05-25T06:11:25.538 INFO:tasks.workunit.client.1.smithi203.stdout:5/555: dwrite dc/d11/d17/d1a/f2e [4194304,4194304] 0 2024-05-25T06:11:25.541 INFO:tasks.workunit.client.1.smithi203.stdout:8/313: chown d2/d5/d18/d31/f5e 22749565 1 2024-05-25T06:11:25.543 INFO:tasks.workunit.client.1.smithi203.stdout:9/277: mkdir d0/d57 0 2024-05-25T06:11:25.543 INFO:tasks.workunit.client.1.smithi203.stdout:1/282: dwrite d14/d57/d1d/f2f [4194304,4194304] 0 2024-05-25T06:11:25.594 INFO:tasks.workunit.client.1.smithi203.stdout:5/556: mknod dc/d11/d17/d1a/d75/cc5 0 2024-05-25T06:11:25.594 INFO:tasks.workunit.client.1.smithi203.stdout:1/283: creat d14/d57/d1d/d3e/f61 x:0 0 0 2024-05-25T06:11:25.595 INFO:tasks.workunit.client.1.smithi203.stdout:8/314: rename d2/d5/d10/d11/d1f/c30 to d2/d5/d10/d11/d1f/c67 0 2024-05-25T06:11:25.597 INFO:tasks.workunit.client.1.smithi203.stdout:1/284: unlink d14/f4b 0 2024-05-25T06:11:25.599 INFO:tasks.workunit.client.1.smithi203.stdout:1/285: dread - d14/d57/d1d/d3e/f61 zero size 2024-05-25T06:11:25.599 INFO:tasks.workunit.client.1.smithi203.stdout:9/278: dwrite d0/f51 [0,4194304] 0 2024-05-25T06:11:25.600 INFO:tasks.workunit.client.1.smithi203.stdout:5/557: mknod dc/d11/d17/d1a/d75/cc6 0 2024-05-25T06:11:25.602 INFO:tasks.workunit.client.1.smithi203.stdout:8/315: mkdir d2/d5/d10/d45/d5d/d68 0 2024-05-25T06:11:25.603 INFO:tasks.workunit.client.1.smithi203.stdout:1/286: truncate d14/d23/d4c/f5d 168287 0 2024-05-25T06:11:25.603 INFO:tasks.workunit.client.1.smithi203.stdout:9/279: creat d0/df/d37/f58 x:0 0 0 2024-05-25T06:11:25.604 INFO:tasks.workunit.client.1.smithi203.stdout:5/558: creat dc/d11/d42/d94/fc7 x:0 0 0 2024-05-25T06:11:25.606 INFO:tasks.workunit.client.1.smithi203.stdout:1/287: chown d14/d57/d1d/f40 2 1 2024-05-25T06:11:25.607 INFO:tasks.workunit.client.1.smithi203.stdout:8/316: creat d2/d5/d10/d11/d59/f69 x:0 0 0 2024-05-25T06:11:25.610 INFO:tasks.workunit.client.1.smithi203.stdout:1/288: creat d14/d57/d1d/d3e/d36/f62 x:0 0 0 2024-05-25T06:11:25.610 INFO:tasks.workunit.client.1.smithi203.stdout:8/317: read - d2/d5/d10/d11/d59/f63 zero size 2024-05-25T06:11:25.613 INFO:tasks.workunit.client.1.smithi203.stdout:9/280: dwrite d0/d17/f1b [4194304,4194304] 0 2024-05-25T06:11:25.615 INFO:tasks.workunit.client.1.smithi203.stdout:8/318: write d2/d5/d10/f4d [1092263,65500] 0 2024-05-25T06:11:25.617 INFO:tasks.workunit.client.1.smithi203.stdout:8/319: write d2/d5/d10/d11/d59/f69 [467424,105580] 0 2024-05-25T06:11:25.618 INFO:tasks.workunit.client.1.smithi203.stdout:1/289: link d14/d57/d29/c3c d14/d23/d4c/c63 0 2024-05-25T06:11:25.620 INFO:tasks.workunit.client.1.smithi203.stdout:9/281: link d0/d17/f33 d0/d17/f59 0 2024-05-25T06:11:25.624 INFO:tasks.workunit.client.1.smithi203.stdout:9/282: creat d0/d17/d23/f5a x:0 0 0 2024-05-25T06:11:25.624 INFO:tasks.workunit.client.1.smithi203.stdout:8/320: mknod d2/d5/d10/d4a/c6a 0 2024-05-25T06:11:25.628 INFO:tasks.workunit.client.1.smithi203.stdout:8/321: fsync d2/d5/d2d/f54 0 2024-05-25T06:11:25.628 INFO:tasks.workunit.client.1.smithi203.stdout:9/283: write d0/df/d2b/f2f [168841,15603] 0 2024-05-25T06:11:25.631 INFO:tasks.workunit.client.1.smithi203.stdout:1/290: dwrite d14/f17 [4194304,4194304] 0 2024-05-25T06:11:25.631 INFO:tasks.workunit.client.1.smithi203.stdout:8/322: read d2/d5/d2d/f4b [837845,33268] 0 2024-05-25T06:11:25.634 INFO:tasks.workunit.client.1.smithi203.stdout:9/284: dread d0/df/f4a [0,4194304] 0 2024-05-25T06:11:25.636 INFO:tasks.workunit.client.1.smithi203.stdout:5/559: dwrite dc/d11/d17/d44/d73/d98/f9f [0,4194304] 0 2024-05-25T06:11:25.637 INFO:tasks.workunit.client.1.smithi203.stdout:1/291: mknod d14/d37/d5e/c64 0 2024-05-25T06:11:25.637 INFO:tasks.workunit.client.1.smithi203.stdout:8/323: readlink d2/d5/d10/d11/lb 0 2024-05-25T06:11:25.640 INFO:tasks.workunit.client.1.smithi203.stdout:8/324: stat d2/d5/d10/d11/d1f/f35 0 2024-05-25T06:11:25.641 INFO:tasks.workunit.client.1.smithi203.stdout:8/325: chown d2/d5/d28/c48 1683 1 2024-05-25T06:11:25.645 INFO:tasks.workunit.client.1.smithi203.stdout:1/292: rename d14/d57/d29/c5c to d14/d37/c65 0 2024-05-25T06:11:25.646 INFO:tasks.workunit.client.1.smithi203.stdout:8/326: dread d2/d5/fc [0,4194304] 0 2024-05-25T06:11:25.646 INFO:tasks.workunit.client.1.smithi203.stdout:8/327: readlink d2/l56 0 2024-05-25T06:11:25.649 INFO:tasks.workunit.client.1.smithi203.stdout:7/336: sync 2024-05-25T06:11:25.650 INFO:tasks.workunit.client.1.smithi203.stdout:0/245: sync 2024-05-25T06:11:25.651 INFO:tasks.workunit.client.1.smithi203.stdout:8/328: truncate d2/d5/d10/d11/d59/f69 1213851 0 2024-05-25T06:11:25.655 INFO:tasks.workunit.client.1.smithi203.stdout:1/293: dwrite d14/d57/d1d/f40 [0,4194304] 0 2024-05-25T06:11:25.660 INFO:tasks.workunit.client.1.smithi203.stdout:8/329: truncate d2/d5/d28/f3a 4461920 0 2024-05-25T06:11:25.668 INFO:tasks.workunit.client.1.smithi203.stdout:9/285: dwrite d0/d17/f59 [0,4194304] 0 2024-05-25T06:11:25.672 INFO:tasks.workunit.client.1.smithi203.stdout:7/337: dwrite d1/d9/d2c/d79/f81 [0,4194304] 0 2024-05-25T06:11:25.674 INFO:tasks.workunit.client.1.smithi203.stdout:8/330: symlink d2/d5/d10/d4a/l6b 0 2024-05-25T06:11:25.675 INFO:tasks.workunit.client.1.smithi203.stdout:0/246: dwrite dd/f44 [0,4194304] 0 2024-05-25T06:11:25.676 INFO:tasks.workunit.client.1.smithi203.stdout:5/560: dwrite dc/d11/d17/d1a/d22/f86 [0,4194304] 0 2024-05-25T06:11:25.677 INFO:tasks.workunit.client.1.smithi203.stdout:9/286: chown d0/df/d2d/l30 158588565 1 2024-05-25T06:11:25.677 INFO:tasks.workunit.client.1.smithi203.stdout:7/338: stat d1/d9/d2c/d3a/d4e/c57 0 2024-05-25T06:11:25.683 INFO:tasks.workunit.client.1.smithi203.stdout:9/287: creat d0/df/f5b x:0 0 0 2024-05-25T06:11:25.684 INFO:tasks.workunit.client.1.smithi203.stdout:1/294: write d14/f18 [669419,122508] 0 2024-05-25T06:11:25.686 INFO:tasks.workunit.client.1.smithi203.stdout:4/242: sync 2024-05-25T06:11:25.686 INFO:tasks.workunit.client.1.smithi203.stdout:2/344: sync 2024-05-25T06:11:25.688 INFO:tasks.workunit.client.1.smithi203.stdout:0/247: mkdir dd/d3d/d59 0 2024-05-25T06:11:25.689 INFO:tasks.workunit.client.1.smithi203.stdout:7/339: truncate d1/fa 988950 0 2024-05-25T06:11:25.692 INFO:tasks.workunit.client.1.smithi203.stdout:0/248: read dd/f44 [513330,68354] 0 2024-05-25T06:11:25.692 INFO:tasks.workunit.client.1.smithi203.stdout:2/345: chown d2/d38/d2f/f4f 443134 1 2024-05-25T06:11:25.693 INFO:tasks.workunit.client.1.smithi203.stdout:8/331: dwrite d2/d5/d10/d11/d59/f69 [0,4194304] 0 2024-05-25T06:11:25.695 INFO:tasks.workunit.client.1.smithi203.stdout:4/243: rmdir d0/d8/d1f 39 2024-05-25T06:11:25.695 INFO:tasks.workunit.client.1.smithi203.stdout:9/288: mkdir d0/d17/d20/d3c/d5c 0 2024-05-25T06:11:25.695 INFO:tasks.workunit.client.1.smithi203.stdout:4/244: chown d0/l2 14119716 1 2024-05-25T06:11:25.696 INFO:tasks.workunit.client.1.smithi203.stdout:7/340: dread d1/d9/f18 [0,4194304] 0 2024-05-25T06:11:25.697 INFO:tasks.workunit.client.1.smithi203.stdout:0/249: fdatasync dd/d10/d34/f47 0 2024-05-25T06:11:25.698 INFO:tasks.workunit.client.1.smithi203.stdout:1/295: link d14/d23/d4c/f5d d14/d57/d1d/f66 0 2024-05-25T06:11:25.702 INFO:tasks.workunit.client.1.smithi203.stdout:0/250: fdatasync dd/d10/f33 0 2024-05-25T06:11:25.703 INFO:tasks.workunit.client.1.smithi203.stdout:4/245: stat d0/d8/d1f/d26/d3b/l3e 0 2024-05-25T06:11:25.704 INFO:tasks.workunit.client.1.smithi203.stdout:1/296: creat d14/d57/d1d/f67 x:0 0 0 2024-05-25T06:11:25.707 INFO:tasks.workunit.client.1.smithi203.stdout:0/251: creat dd/f5a x:0 0 0 2024-05-25T06:11:25.707 INFO:tasks.workunit.client.1.smithi203.stdout:7/341: truncate d1/d9/d2c/d3a/d4e/f5b 599721 0 2024-05-25T06:11:25.708 INFO:tasks.workunit.client.1.smithi203.stdout:8/332: dwrite d2/d5/d18/d4f/f66 [0,4194304] 0 2024-05-25T06:11:25.710 INFO:tasks.workunit.client.1.smithi203.stdout:4/246: mkdir d0/d8/d1f/d26/d43 0 2024-05-25T06:11:25.712 INFO:tasks.workunit.client.1.smithi203.stdout:1/297: symlink d14/d23/d4c/l68 0 2024-05-25T06:11:25.712 INFO:tasks.workunit.client.1.smithi203.stdout:8/333: mknod d2/d5/d10/d11/d59/c6c 0 2024-05-25T06:11:25.715 INFO:tasks.workunit.client.1.smithi203.stdout:4/247: write d0/d9/d24/d27/f30 [3127824,93191] 0 2024-05-25T06:11:25.716 INFO:tasks.workunit.client.1.smithi203.stdout:9/289: dread d0/df/f14 [0,4194304] 0 2024-05-25T06:11:25.721 INFO:tasks.workunit.client.1.smithi203.stdout:8/334: symlink d2/d5/l6d 0 2024-05-25T06:11:25.723 INFO:tasks.workunit.client.1.smithi203.stdout:7/342: dwrite d1/d11/d3d/d4c/f63 [0,4194304] 0 2024-05-25T06:11:25.724 INFO:tasks.workunit.client.1.smithi203.stdout:4/248: creat d0/d8/f44 x:0 0 0 2024-05-25T06:11:25.724 INFO:tasks.workunit.client.1.smithi203.stdout:8/335: chown d2/d5/d10/d11/l20 75535 1 2024-05-25T06:11:25.726 INFO:tasks.workunit.client.1.smithi203.stdout:0/252: chown dd/f23 13 1 2024-05-25T06:11:25.726 INFO:tasks.workunit.client.1.smithi203.stdout:8/336: chown d2/d5/d10/d11/d59 2184 1 2024-05-25T06:11:25.726 INFO:tasks.workunit.client.1.smithi203.stdout:1/298: dread d14/d57/d29/f2b [0,4194304] 0 2024-05-25T06:11:25.729 INFO:tasks.workunit.client.1.smithi203.stdout:5/561: dread dc/d11/d17/d1a/d22/f90 [0,4194304] 0 2024-05-25T06:11:25.730 INFO:tasks.workunit.client.1.smithi203.stdout:8/337: truncate d2/d5/f29 1469164 0 2024-05-25T06:11:25.731 INFO:tasks.workunit.client.1.smithi203.stdout:5/562: write dc/d11/d17/f56 [357726,37193] 0 2024-05-25T06:11:25.732 INFO:tasks.workunit.client.1.smithi203.stdout:9/290: link d0/df/c12 d0/df/d37/c5d 0 2024-05-25T06:11:25.734 INFO:tasks.workunit.client.1.smithi203.stdout:2/346: dread d2/f6f [0,4194304] 0 2024-05-25T06:11:25.734 INFO:tasks.workunit.client.1.smithi203.stdout:8/338: stat d2/d5/d10/d11/de/f16 0 2024-05-25T06:11:25.735 INFO:tasks.workunit.client.1.smithi203.stdout:4/249: symlink d0/d9/d24/d27/l45 0 2024-05-25T06:11:25.739 INFO:tasks.workunit.client.1.smithi203.stdout:9/291: creat d0/d17/d20/d3c/f5e x:0 0 0 2024-05-25T06:11:25.739 INFO:tasks.workunit.client.1.smithi203.stdout:2/347: creat d2/d4/d10/d50/f82 x:0 0 0 2024-05-25T06:11:25.740 INFO:tasks.workunit.client.1.smithi203.stdout:5/563: mkdir dc/d11/d17/d1a/d72/dc8 0 2024-05-25T06:11:25.741 INFO:tasks.workunit.client.1.smithi203.stdout:7/343: dwrite d1/d11/d20/f23 [0,4194304] 0 2024-05-25T06:11:25.741 INFO:tasks.workunit.client.1.smithi203.stdout:1/299: rename d14/f35 to d14/f69 0 2024-05-25T06:11:25.744 INFO:tasks.workunit.client.1.smithi203.stdout:1/300: dread - d14/d57/d1d/f67 zero size 2024-05-25T06:11:25.744 INFO:tasks.workunit.client.1.smithi203.stdout:4/250: creat d0/d8/d3a/f46 x:0 0 0 2024-05-25T06:11:25.745 INFO:tasks.workunit.client.1.smithi203.stdout:8/339: mknod d2/d5/d28/d38/c6e 0 2024-05-25T06:11:25.745 INFO:tasks.workunit.client.1.smithi203.stdout:9/292: creat d0/df/f5f x:0 0 0 2024-05-25T06:11:25.746 INFO:tasks.workunit.client.1.smithi203.stdout:4/251: chown d0/d9/d11/f2f 0 1 2024-05-25T06:11:25.746 INFO:tasks.workunit.client.1.smithi203.stdout:8/340: chown d2/d5/d28/l46 20933328 1 2024-05-25T06:11:25.747 INFO:tasks.workunit.client.1.smithi203.stdout:7/344: fsync d1/f32 0 2024-05-25T06:11:25.749 INFO:tasks.workunit.client.1.smithi203.stdout:4/252: write d0/f16 [3833014,4469] 0 2024-05-25T06:11:25.750 INFO:tasks.workunit.client.1.smithi203.stdout:1/301: chown d14/d37/c65 14569489 1 2024-05-25T06:11:25.751 INFO:tasks.workunit.client.1.smithi203.stdout:9/293: chown d0/d17/d20/d3c/l3e 4104 1 2024-05-25T06:11:25.754 INFO:tasks.workunit.client.1.smithi203.stdout:5/564: dwrite dc/d11/f1c [0,4194304] 0 2024-05-25T06:11:25.755 INFO:tasks.workunit.client.1.smithi203.stdout:4/253: dread - d0/d9/d24/f40 zero size 2024-05-25T06:11:25.760 INFO:tasks.workunit.client.1.smithi203.stdout:5/565: fsync dc/d3c/d3f/f93 0 2024-05-25T06:11:25.761 INFO:tasks.workunit.client.1.smithi203.stdout:4/254: rmdir d0/d8/d1f 39 2024-05-25T06:11:25.761 INFO:tasks.workunit.client.1.smithi203.stdout:0/253: dwrite dd/d10/d34/f58 [0,4194304] 0 2024-05-25T06:11:25.763 INFO:tasks.workunit.client.1.smithi203.stdout:5/566: fdatasync dc/d11/d17/f38 0 2024-05-25T06:11:25.766 INFO:tasks.workunit.client.1.smithi203.stdout:4/255: mkdir d0/d9/d24/d47 0 2024-05-25T06:11:25.769 INFO:tasks.workunit.client.1.smithi203.stdout:1/302: dwrite d14/d57/d1d/f40 [0,4194304] 0 2024-05-25T06:11:25.769 INFO:tasks.workunit.client.1.smithi203.stdout:0/254: rmdir dd/d32/d37 39 2024-05-25T06:11:25.771 INFO:tasks.workunit.client.1.smithi203.stdout:5/567: unlink dc/d11/d17/d1a/d22/l4b 0 2024-05-25T06:11:25.779 INFO:tasks.workunit.client.1.smithi203.stdout:5/568: rmdir dc/d3c/d8b 0 2024-05-25T06:11:25.780 INFO:tasks.workunit.client.1.smithi203.stdout:9/294: write d0/d17/d20/f31 [883543,86650] 0 2024-05-25T06:11:25.782 INFO:tasks.workunit.client.1.smithi203.stdout:5/569: mkdir dc/d11/d17/d1a/d2d/d53/dc9 0 2024-05-25T06:11:25.783 INFO:tasks.workunit.client.1.smithi203.stdout:7/345: dwrite f0 [4194304,4194304] 0 2024-05-25T06:11:25.784 INFO:tasks.workunit.client.1.smithi203.stdout:8/341: dread d2/d5/d18/d31/f39 [0,4194304] 0 2024-05-25T06:11:25.786 INFO:tasks.workunit.client.1.smithi203.stdout:9/295: mknod d0/df/d2d/c60 0 2024-05-25T06:11:25.787 INFO:tasks.workunit.client.1.smithi203.stdout:5/570: mknod dc/d11/d18/dc0/cca 0 2024-05-25T06:11:25.791 INFO:tasks.workunit.client.1.smithi203.stdout:5/571: dread dc/d11/d17/d1a/d22/fb4 [0,4194304] 0 2024-05-25T06:11:25.792 INFO:tasks.workunit.client.1.smithi203.stdout:5/572: readlink dc/d11/d17/d1a/d2d/d53/l59 0 2024-05-25T06:11:25.792 INFO:tasks.workunit.client.1.smithi203.stdout:1/303: truncate d14/f17 2634280 0 2024-05-25T06:11:25.794 INFO:tasks.workunit.client.1.smithi203.stdout:1/304: mknod d14/d57/d1d/d3e/d36/c6a 0 2024-05-25T06:11:25.795 INFO:tasks.workunit.client.1.smithi203.stdout:5/573: mkdir dc/dcb 0 2024-05-25T06:11:25.796 INFO:tasks.workunit.client.1.smithi203.stdout:9/296: write d0/f24 [1720917,10672] 0 2024-05-25T06:11:25.799 INFO:tasks.workunit.client.1.smithi203.stdout:1/305: rename d14/d23/d4c to d14/d57/d6b 0 2024-05-25T06:11:25.805 INFO:tasks.workunit.client.1.smithi203.stdout:8/342: dread d2/d5/d10/d11/de/f16 [0,4194304] 0 2024-05-25T06:11:25.805 INFO:tasks.workunit.client.1.smithi203.stdout:1/306: link d14/c3a d14/d57/d1d/d3e/d36/c6c 0 2024-05-25T06:11:25.807 INFO:tasks.workunit.client.1.smithi203.stdout:5/574: truncate dc/d11/d17/d44/fbe 2887774 0 2024-05-25T06:11:25.807 INFO:tasks.workunit.client.1.smithi203.stdout:8/343: creat d2/d5/d18/d31/f6f x:0 0 0 2024-05-25T06:11:25.808 INFO:tasks.workunit.client.1.smithi203.stdout:1/307: rmdir d14/d37/d5e 39 2024-05-25T06:11:25.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/344: truncate d2/d5/d18/d31/f39 5081096 0 2024-05-25T06:11:25.813 INFO:tasks.workunit.client.1.smithi203.stdout:8/345: chown d2/d5/d18/d31/f34 22 1 2024-05-25T06:11:25.813 INFO:tasks.workunit.client.1.smithi203.stdout:5/575: dread dc/d11/d17/d1a/d72/f8c [0,4194304] 0 2024-05-25T06:11:25.814 INFO:tasks.workunit.client.1.smithi203.stdout:1/308: rename d14/d57/d6b/c63 to d14/d57/d1d/c6d 0 2024-05-25T06:11:25.815 INFO:tasks.workunit.client.1.smithi203.stdout:9/297: dread d0/f4 [0,4194304] 0 2024-05-25T06:11:25.815 INFO:tasks.workunit.client.1.smithi203.stdout:1/309: chown d14/d57/d1d/f2a 98981 1 2024-05-25T06:11:25.816 INFO:tasks.workunit.client.1.smithi203.stdout:8/346: creat d2/d5/d18/f70 x:0 0 0 2024-05-25T06:11:25.817 INFO:tasks.workunit.client.1.smithi203.stdout:5/576: mkdir dc/d11/d17/d1a/d2d/d4d/dcc 0 2024-05-25T06:11:25.822 INFO:tasks.workunit.client.1.smithi203.stdout:9/298: mkdir d0/d61 0 2024-05-25T06:11:25.824 INFO:tasks.workunit.client.1.smithi203.stdout:8/347: rmdir d2 39 2024-05-25T06:11:25.826 INFO:tasks.workunit.client.1.smithi203.stdout:1/310: truncate d14/f32 3829934 0 2024-05-25T06:11:25.828 INFO:tasks.workunit.client.1.smithi203.stdout:1/311: fsync d14/d57/d1d/d3e/d36/f39 0 2024-05-25T06:11:25.829 INFO:tasks.workunit.client.1.smithi203.stdout:9/299: dread d0/f21 [0,4194304] 0 2024-05-25T06:11:25.832 INFO:tasks.workunit.client.1.smithi203.stdout:1/312: creat d14/d57/d29/f6e x:0 0 0 2024-05-25T06:11:25.832 INFO:tasks.workunit.client.1.smithi203.stdout:5/577: truncate dc/f21 3134330 0 2024-05-25T06:11:25.832 INFO:tasks.workunit.client.1.smithi203.stdout:9/300: truncate d0/d17/d20/d29/f48 430033 0 2024-05-25T06:11:25.833 INFO:tasks.workunit.client.1.smithi203.stdout:9/301: dread - d0/df/d37/f58 zero size 2024-05-25T06:11:25.833 INFO:tasks.workunit.client.1.smithi203.stdout:8/348: dread d2/d5/d10/f53 [0,4194304] 0 2024-05-25T06:11:25.834 INFO:tasks.workunit.client.1.smithi203.stdout:5/578: getdents dc/dcb 0 2024-05-25T06:11:25.835 INFO:tasks.workunit.client.1.smithi203.stdout:9/302: dread d0/f1e [0,4194304] 0 2024-05-25T06:11:25.835 INFO:tasks.workunit.client.1.smithi203.stdout:8/349: mknod d2/d5/d18/d31/c71 0 2024-05-25T06:11:25.836 INFO:tasks.workunit.client.1.smithi203.stdout:5/579: read - dc/d11/d17/d1a/d75/f77 zero size 2024-05-25T06:11:25.837 INFO:tasks.workunit.client.1.smithi203.stdout:8/350: readlink d2/d5/l40 0 2024-05-25T06:11:25.838 INFO:tasks.workunit.client.1.smithi203.stdout:9/303: symlink d0/d17/d20/d3c/l62 0 2024-05-25T06:11:25.839 INFO:tasks.workunit.client.1.smithi203.stdout:8/351: creat d2/d5/d10/d11/d1f/f72 x:0 0 0 2024-05-25T06:11:25.840 INFO:tasks.workunit.client.1.smithi203.stdout:1/313: dwrite d14/d57/d29/f2b [4194304,4194304] 0 2024-05-25T06:11:25.841 INFO:tasks.workunit.client.1.smithi203.stdout:1/314: write d14/f25 [4986400,114040] 0 2024-05-25T06:11:25.843 INFO:tasks.workunit.client.1.smithi203.stdout:9/304: rename d0/l3 to d0/df/d2d/l63 0 2024-05-25T06:11:25.849 INFO:tasks.workunit.client.1.smithi203.stdout:8/352: dwrite d2/d5/d10/d11/d1f/f35 [0,4194304] 0 2024-05-25T06:11:25.851 INFO:tasks.workunit.client.1.smithi203.stdout:9/305: link d0/d17/d20/d3c/l3d d0/df/d37/d4f/l64 0 2024-05-25T06:11:25.855 INFO:tasks.workunit.client.1.smithi203.stdout:8/353: link d2/d5/d10/d11/de/f13 d2/d5/d10/d45/d5d/d68/f73 0 2024-05-25T06:11:25.856 INFO:tasks.workunit.client.1.smithi203.stdout:1/315: write f10 [10042695,122621] 0 2024-05-25T06:11:25.861 INFO:tasks.workunit.client.1.smithi203.stdout:9/306: getdents d0/d17/d23 0 2024-05-25T06:11:25.861 INFO:tasks.workunit.client.1.smithi203.stdout:1/316: read d14/d37/d5e/f55 [2994195,124916] 0 2024-05-25T06:11:25.863 INFO:tasks.workunit.client.1.smithi203.stdout:8/354: dwrite d2/d5/d28/f4c [0,4194304] 0 2024-05-25T06:11:25.863 INFO:tasks.workunit.client.1.smithi203.stdout:1/317: fsync d14/d57/d29/f2b 0 2024-05-25T06:11:25.866 INFO:tasks.workunit.client.1.smithi203.stdout:1/318: mknod d14/d57/d6b/c6f 0 2024-05-25T06:11:25.869 INFO:tasks.workunit.client.1.smithi203.stdout:8/355: link d2/d5/d10/d11/d1f/f35 d2/d5/d28/d38/d4e/f74 0 2024-05-25T06:11:25.871 INFO:tasks.workunit.client.1.smithi203.stdout:9/307: dread d0/d17/d20/f31 [0,4194304] 0 2024-05-25T06:11:25.871 INFO:tasks.workunit.client.1.smithi203.stdout:1/319: symlink d14/d5f/l70 0 2024-05-25T06:11:25.873 INFO:tasks.workunit.client.1.smithi203.stdout:9/308: unlink d0/f7 0 2024-05-25T06:11:25.875 INFO:tasks.workunit.client.1.smithi203.stdout:9/309: write d0/f51 [734975,95072] 0 2024-05-25T06:11:25.877 INFO:tasks.workunit.client.1.smithi203.stdout:8/356: dwrite d2/d5/d18/d4f/f66 [4194304,4194304] 0 2024-05-25T06:11:25.878 INFO:tasks.workunit.client.1.smithi203.stdout:9/310: creat d0/d61/f65 x:0 0 0 2024-05-25T06:11:25.881 INFO:tasks.workunit.client.1.smithi203.stdout:8/357: mkdir d2/d5/d10/d11/d1f/d75 0 2024-05-25T06:11:25.882 INFO:tasks.workunit.client.1.smithi203.stdout:9/311: mknod d0/d17/d20/d3c/c66 0 2024-05-25T06:11:25.884 INFO:tasks.workunit.client.1.smithi203.stdout:8/358: mkdir d2/d5/d10/d45/d5d/d76 0 2024-05-25T06:11:25.887 INFO:tasks.workunit.client.1.smithi203.stdout:9/312: link d0/df/d2d/c60 d0/d17/d20/d3c/c67 0 2024-05-25T06:11:25.890 INFO:tasks.workunit.client.1.smithi203.stdout:8/359: link d2/d5/d2d/f54 d2/d5/d18/d4f/f77 0 2024-05-25T06:11:25.891 INFO:tasks.workunit.client.1.smithi203.stdout:8/360: write d2/d5/f29 [902467,114180] 0 2024-05-25T06:11:25.892 INFO:tasks.workunit.client.1.smithi203.stdout:9/313: mknod d0/d57/c68 0 2024-05-25T06:11:25.892 INFO:tasks.workunit.client.1.smithi203.stdout:1/320: dread d14/d57/d1d/f30 [0,4194304] 0 2024-05-25T06:11:25.892 INFO:tasks.workunit.client.1.smithi203.stdout:8/361: chown d2/d5/d10/f4d 1909 1 2024-05-25T06:11:25.895 INFO:tasks.workunit.client.1.smithi203.stdout:8/362: mknod d2/d5/d10/d45/d5d/c78 0 2024-05-25T06:11:25.898 INFO:tasks.workunit.client.1.smithi203.stdout:1/321: dwrite d14/f25 [4194304,4194304] 0 2024-05-25T06:11:25.898 INFO:tasks.workunit.client.1.smithi203.stdout:1/322: chown d14/d37/l49 6 1 2024-05-25T06:11:25.899 INFO:tasks.workunit.client.1.smithi203.stdout:8/363: link d2/d5/d10/l24 d2/d5/d18/d31/l79 0 2024-05-25T06:11:25.900 INFO:tasks.workunit.client.1.smithi203.stdout:3/413: sync 2024-05-25T06:11:25.900 INFO:tasks.workunit.client.1.smithi203.stdout:6/351: sync 2024-05-25T06:11:25.900 INFO:tasks.workunit.client.1.smithi203.stdout:1/323: rmdir d14/d37/d5e 39 2024-05-25T06:11:25.900 INFO:tasks.workunit.client.1.smithi203.stdout:8/364: chown d2/d5/d2d 2378452 1 2024-05-25T06:11:25.902 INFO:tasks.workunit.client.1.smithi203.stdout:6/352: write d8/d70/d38/f39 [1239448,67671] 0 2024-05-25T06:11:25.903 INFO:tasks.workunit.client.1.smithi203.stdout:6/353: fdatasync d8/d11/d15/f59 0 2024-05-25T06:11:25.904 INFO:tasks.workunit.client.1.smithi203.stdout:3/414: creat d10/d16/d23/d38/f89 x:0 0 0 2024-05-25T06:11:25.904 INFO:tasks.workunit.client.1.smithi203.stdout:8/365: readlink d2/d5/d10/l24 0 2024-05-25T06:11:25.907 INFO:tasks.workunit.client.1.smithi203.stdout:3/415: symlink d10/d22/d2a/l8a 0 2024-05-25T06:11:25.907 INFO:tasks.workunit.client.1.smithi203.stdout:6/354: getdents d8/d5a 0 2024-05-25T06:11:25.908 INFO:tasks.workunit.client.1.smithi203.stdout:1/324: dread d14/d37/d5e/f55 [0,4194304] 0 2024-05-25T06:11:25.912 INFO:tasks.workunit.client.1.smithi203.stdout:6/355: rename d8/d11/d15/d28/d52/c45 to d8/d5a/c72 0 2024-05-25T06:11:25.914 INFO:tasks.workunit.client.1.smithi203.stdout:3/416: dwrite d10/d16/d23/d4a/f61 [0,4194304] 0 2024-05-25T06:11:25.915 INFO:tasks.workunit.client.1.smithi203.stdout:6/356: fdatasync d8/f12 0 2024-05-25T06:11:25.917 INFO:tasks.workunit.client.1.smithi203.stdout:1/325: dwrite d14/d57/f58 [0,4194304] 0 2024-05-25T06:11:25.919 INFO:tasks.workunit.client.1.smithi203.stdout:3/417: write d10/d22/d2a/f3b [3804198,4243] 0 2024-05-25T06:11:25.921 INFO:tasks.workunit.client.1.smithi203.stdout:1/326: mknod d14/d5f/c71 0 2024-05-25T06:11:25.923 INFO:tasks.workunit.client.1.smithi203.stdout:3/418: fdatasync d10/f21 0 2024-05-25T06:11:25.925 INFO:tasks.workunit.client.1.smithi203.stdout:6/357: dwrite d8/d3d/f44 [0,4194304] 0 2024-05-25T06:11:25.926 INFO:tasks.workunit.client.1.smithi203.stdout:1/327: unlink d14/d57/c28 0 2024-05-25T06:11:25.927 INFO:tasks.workunit.client.1.smithi203.stdout:6/358: write d8/d2a/f4a [347201,34252] 0 2024-05-25T06:11:25.931 INFO:tasks.workunit.client.1.smithi203.stdout:3/419: rmdir d10/d16/d23/d46 39 2024-05-25T06:11:25.931 INFO:tasks.workunit.client.1.smithi203.stdout:1/328: creat d14/d57/d1d/d3e/f72 x:0 0 0 2024-05-25T06:11:25.931 INFO:tasks.workunit.client.1.smithi203.stdout:8/366: write d2/d5/d10/d11/de/f13 [1536283,77371] 0 2024-05-25T06:11:25.932 INFO:tasks.workunit.client.1.smithi203.stdout:6/359: mkdir d8/d11/d15/d28/d52/d43/d73 0 2024-05-25T06:11:25.933 INFO:tasks.workunit.client.1.smithi203.stdout:3/420: write d10/d22/f42 [332497,84983] 0 2024-05-25T06:11:25.933 INFO:tasks.workunit.client.1.smithi203.stdout:8/367: fdatasync d2/d5/d18/d31/f3d 0 2024-05-25T06:11:25.934 INFO:tasks.workunit.client.1.smithi203.stdout:3/421: fdatasync d10/f1a 0 2024-05-25T06:11:25.934 INFO:tasks.workunit.client.1.smithi203.stdout:1/329: mknod d14/d57/d1d/c73 0 2024-05-25T06:11:25.936 INFO:tasks.workunit.client.1.smithi203.stdout:6/360: mknod d8/d5a/c74 0 2024-05-25T06:11:25.938 INFO:tasks.workunit.client.1.smithi203.stdout:6/361: chown d8/d2a/d63/f5b 6939670 1 2024-05-25T06:11:25.940 INFO:tasks.workunit.client.1.smithi203.stdout:3/422: dwrite d10/f51 [0,4194304] 0 2024-05-25T06:11:25.940 INFO:tasks.workunit.client.1.smithi203.stdout:8/368: creat d2/d5/d28/f7a x:0 0 0 2024-05-25T06:11:25.940 INFO:tasks.workunit.client.1.smithi203.stdout:6/362: rmdir d8/d11/d15/d28/d52/d43 39 2024-05-25T06:11:25.942 INFO:tasks.workunit.client.1.smithi203.stdout:1/330: rename d14/d57/d1d/c46 to d14/d37/c74 0 2024-05-25T06:11:25.943 INFO:tasks.workunit.client.1.smithi203.stdout:6/363: mkdir d8/d11/d1b/d75 0 2024-05-25T06:11:25.943 INFO:tasks.workunit.client.1.smithi203.stdout:6/364: fsync d8/d2a/f67 0 2024-05-25T06:11:25.944 INFO:tasks.workunit.client.1.smithi203.stdout:1/331: truncate d14/d57/d1d/d3e/f61 386253 0 2024-05-25T06:11:25.945 INFO:tasks.workunit.client.1.smithi203.stdout:3/423: rename d10/d16/d23/d3a/d48 to d10/d16/d50/d8b 0 2024-05-25T06:11:25.946 INFO:tasks.workunit.client.1.smithi203.stdout:8/369: mknod d2/d5/d10/d45/d5d/d76/c7b 0 2024-05-25T06:11:25.947 INFO:tasks.workunit.client.1.smithi203.stdout:3/424: dread d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:25.947 INFO:tasks.workunit.client.1.smithi203.stdout:6/365: chown d8/d11/d15/d28/d52/d43/c3b 5268 1 2024-05-25T06:11:25.947 INFO:tasks.workunit.client.1.smithi203.stdout:1/332: fdatasync d14/d57/d1d/f66 0 2024-05-25T06:11:25.948 INFO:tasks.workunit.client.1.smithi203.stdout:8/370: mknod d2/d5/d18/d4f/c7c 0 2024-05-25T06:11:25.948 INFO:tasks.workunit.client.1.smithi203.stdout:3/425: fdatasync d10/d16/d23/d67/f78 0 2024-05-25T06:11:25.949 INFO:tasks.workunit.client.1.smithi203.stdout:6/366: truncate d8/fb 753610 0 2024-05-25T06:11:25.950 INFO:tasks.workunit.client.1.smithi203.stdout:3/426: mknod d10/d16/d23/d3a/c8c 0 2024-05-25T06:11:25.951 INFO:tasks.workunit.client.1.smithi203.stdout:6/367: mkdir d8/d3d/d76 0 2024-05-25T06:11:25.952 INFO:tasks.workunit.client.1.smithi203.stdout:1/333: dread d14/d57/d1d/f2a [0,4194304] 0 2024-05-25T06:11:25.952 INFO:tasks.workunit.client.1.smithi203.stdout:6/368: rename d8 to d8/d2a/d6c/d77 22 2024-05-25T06:11:25.952 INFO:tasks.workunit.client.1.smithi203.stdout:1/334: stat d14/d23 0 2024-05-25T06:11:25.954 INFO:tasks.workunit.client.1.smithi203.stdout:1/335: mknod d14/d57/d1d/d3e/d36/c75 0 2024-05-25T06:11:25.967 INFO:tasks.workunit.client.1.smithi203.stdout:3/427: dread d10/d22/f5b [0,4194304] 0 2024-05-25T06:11:25.970 INFO:tasks.workunit.client.1.smithi203.stdout:3/428: creat d10/d16/d23/d4a/d88/f8d x:0 0 0 2024-05-25T06:11:25.972 INFO:tasks.workunit.client.1.smithi203.stdout:3/429: symlink d10/d16/d23/d6c/l8e 0 2024-05-25T06:11:25.973 INFO:tasks.workunit.client.1.smithi203.stdout:3/430: rmdir d10/d16/d23/d6c 39 2024-05-25T06:11:25.982 INFO:tasks.workunit.client.1.smithi203.stdout:8/371: rmdir d2/d5/d28 39 2024-05-25T06:11:25.983 INFO:tasks.workunit.client.1.smithi203.stdout:8/372: stat d2/d5/d10/d11/de/f13 0 2024-05-25T06:11:25.983 INFO:tasks.workunit.client.1.smithi203.stdout:1/336: getdents d14/d37 0 2024-05-25T06:11:25.984 INFO:tasks.workunit.client.1.smithi203.stdout:1/337: readlink d14/d37/l47 0 2024-05-25T06:11:25.985 INFO:tasks.workunit.client.1.smithi203.stdout:8/373: write d2/d5/d18/f1b [1966099,50224] 0 2024-05-25T06:11:25.986 INFO:tasks.workunit.client.1.smithi203.stdout:1/338: creat d14/d37/d5e/f76 x:0 0 0 2024-05-25T06:11:25.989 INFO:tasks.workunit.client.1.smithi203.stdout:1/339: write d14/d37/d5e/f5b [761466,4998] 0 2024-05-25T06:11:25.990 INFO:tasks.workunit.client.1.smithi203.stdout:3/431: dread d10/d16/d50/d8b/f52 [0,4194304] 0 2024-05-25T06:11:25.990 INFO:tasks.workunit.client.1.smithi203.stdout:8/374: creat d2/d5/d10/d45/d5d/d68/f7d x:0 0 0 2024-05-25T06:11:25.991 INFO:tasks.workunit.client.1.smithi203.stdout:1/340: chown d14/d23/f42 97 1 2024-05-25T06:11:25.993 INFO:tasks.workunit.client.1.smithi203.stdout:1/341: write d14/f18 [1436862,53747] 0 2024-05-25T06:11:25.994 INFO:tasks.workunit.client.1.smithi203.stdout:8/375: write d2/d5/d10/d11/d59/f63 [293829,43181] 0 2024-05-25T06:11:25.995 INFO:tasks.workunit.client.1.smithi203.stdout:3/432: creat d10/d16/f8f x:0 0 0 2024-05-25T06:11:25.996 INFO:tasks.workunit.client.1.smithi203.stdout:3/433: readlink d10/d22/l83 0 2024-05-25T06:11:25.998 INFO:tasks.workunit.client.1.smithi203.stdout:6/369: dwrite d8/d11/d15/f2e [0,4194304] 0 2024-05-25T06:11:25.998 INFO:tasks.workunit.client.1.smithi203.stdout:8/376: chown d2/d5/d2d/f3e 0 1 2024-05-25T06:11:25.998 INFO:tasks.workunit.client.1.smithi203.stdout:8/377: readlink d2/l56 0 2024-05-25T06:11:25.999 INFO:tasks.workunit.client.1.smithi203.stdout:6/370: write d8/d11/f19 [889671,110547] 0 2024-05-25T06:11:26.001 INFO:tasks.workunit.client.1.smithi203.stdout:6/371: fdatasync f2 0 2024-05-25T06:11:26.002 INFO:tasks.workunit.client.1.smithi203.stdout:6/372: readlink d8/l16 0 2024-05-25T06:11:26.005 INFO:tasks.workunit.client.1.smithi203.stdout:6/373: mknod d8/d11/d47/d60/c78 0 2024-05-25T06:11:26.006 INFO:tasks.workunit.client.1.smithi203.stdout:3/434: dwrite d10/d16/d23/d3a/f53 [0,4194304] 0 2024-05-25T06:11:26.007 INFO:tasks.workunit.client.1.smithi203.stdout:8/378: truncate d2/d5/f8 3810162 0 2024-05-25T06:11:26.008 INFO:tasks.workunit.client.1.smithi203.stdout:6/374: truncate d8/d3d/f6a 151855 0 2024-05-25T06:11:26.013 INFO:tasks.workunit.client.1.smithi203.stdout:3/435: dread - d10/d16/d23/d46/f4b zero size 2024-05-25T06:11:26.015 INFO:tasks.workunit.client.1.smithi203.stdout:6/375: getdents d8/d11/d47 0 2024-05-25T06:11:26.015 INFO:tasks.workunit.client.1.smithi203.stdout:8/379: dwrite d2/f14 [4194304,4194304] 0 2024-05-25T06:11:26.017 INFO:tasks.workunit.client.1.smithi203.stdout:3/436: mknod d10/d16/d50/d7d/c90 0 2024-05-25T06:11:26.021 INFO:tasks.workunit.client.1.smithi203.stdout:3/437: mknod d10/c91 0 2024-05-25T06:11:26.021 INFO:tasks.workunit.client.1.smithi203.stdout:8/380: dread d2/d5/d18/d31/f39 [4194304,4194304] 0 2024-05-25T06:11:26.023 INFO:tasks.workunit.client.1.smithi203.stdout:3/438: symlink d10/d16/d23/d46/l92 0 2024-05-25T06:11:26.026 INFO:tasks.workunit.client.1.smithi203.stdout:3/439: truncate d10/d16/d23/d4a/d88/f8d 571272 0 2024-05-25T06:11:26.027 INFO:tasks.workunit.client.1.smithi203.stdout:6/376: write d8/d11/d15/f18 [310411,49099] 0 2024-05-25T06:11:26.028 INFO:tasks.workunit.client.1.smithi203.stdout:8/381: getdents d2/d5 0 2024-05-25T06:11:26.029 INFO:tasks.workunit.client.1.smithi203.stdout:8/382: write d2/d5/d28/d38/f47 [827631,51664] 0 2024-05-25T06:11:26.030 INFO:tasks.workunit.client.1.smithi203.stdout:6/377: creat d8/d11/d15/f79 x:0 0 0 2024-05-25T06:11:26.030 INFO:tasks.workunit.client.1.smithi203.stdout:3/440: creat d10/d16/d23/d45/f93 x:0 0 0 2024-05-25T06:11:26.032 INFO:tasks.workunit.client.1.smithi203.stdout:8/383: unlink d2/d5/d28/c48 0 2024-05-25T06:11:26.036 INFO:tasks.workunit.client.1.smithi203.stdout:8/384: read d2/f14 [899687,53454] 0 2024-05-25T06:11:26.039 INFO:tasks.workunit.client.1.smithi203.stdout:8/385: write d2/d5/d28/d38/f3c [3749633,66608] 0 2024-05-25T06:11:26.041 INFO:tasks.workunit.client.1.smithi203.stdout:3/441: dwrite d10/d16/d23/d46/f79 [0,4194304] 0 2024-05-25T06:11:26.044 INFO:tasks.workunit.client.1.smithi203.stdout:1/342: dread d14/d57/d6b/f4f [0,4194304] 0 2024-05-25T06:11:26.044 INFO:tasks.workunit.client.1.smithi203.stdout:8/386: chown d2/d5/d10/d11/d1f/c67 157 1 2024-05-25T06:11:26.046 INFO:tasks.workunit.client.1.smithi203.stdout:3/442: write d10/d16/d23/d4a/f58 [798777,15854] 0 2024-05-25T06:11:26.047 INFO:tasks.workunit.client.1.smithi203.stdout:6/378: truncate d8/fc 722554 0 2024-05-25T06:11:26.047 INFO:tasks.workunit.client.1.smithi203.stdout:1/343: readlink d14/l27 0 2024-05-25T06:11:26.049 INFO:tasks.workunit.client.1.smithi203.stdout:8/387: chown d2/d5/d18/d4f/l65 1335479 1 2024-05-25T06:11:26.050 INFO:tasks.workunit.client.1.smithi203.stdout:8/388: truncate d2/d5/f29 1502032 0 2024-05-25T06:11:26.050 INFO:tasks.workunit.client.1.smithi203.stdout:1/344: mknod d14/d57/d1d/d3e/d36/c77 0 2024-05-25T06:11:26.051 INFO:tasks.workunit.client.1.smithi203.stdout:8/389: read d2/d5/fc [3942377,104615] 0 2024-05-25T06:11:26.051 INFO:tasks.workunit.client.1.smithi203.stdout:3/443: creat d10/d16/d23/d67/f94 x:0 0 0 2024-05-25T06:11:26.054 INFO:tasks.workunit.client.1.smithi203.stdout:1/345: rename d14/f69 to d14/d23/f78 0 2024-05-25T06:11:26.055 INFO:tasks.workunit.client.1.smithi203.stdout:8/390: mknod d2/d5/d10/d4a/c7e 0 2024-05-25T06:11:26.056 INFO:tasks.workunit.client.1.smithi203.stdout:6/379: getdents d8/d2a/d6c 0 2024-05-25T06:11:26.061 INFO:tasks.workunit.client.1.smithi203.stdout:3/444: truncate d10/d22/d2a/f4d 5179654 0 2024-05-25T06:11:26.061 INFO:tasks.workunit.client.1.smithi203.stdout:1/346: dwrite d14/f24 [4194304,4194304] 0 2024-05-25T06:11:26.063 INFO:tasks.workunit.client.1.smithi203.stdout:3/445: chown d10/d16/d50 874256 1 2024-05-25T06:11:26.063 INFO:tasks.workunit.client.1.smithi203.stdout:3/446: readlink d10/d22/d2a/l7a 0 2024-05-25T06:11:26.063 INFO:tasks.workunit.client.1.smithi203.stdout:8/391: rmdir d2/d5/d10/d45/d5d/d68 39 2024-05-25T06:11:26.067 INFO:tasks.workunit.client.1.smithi203.stdout:8/392: mknod d2/d5/d28/d38/c7f 0 2024-05-25T06:11:26.067 INFO:tasks.workunit.client.1.smithi203.stdout:3/447: dread d10/d16/d23/d4a/f58 [0,4194304] 0 2024-05-25T06:11:26.067 INFO:tasks.workunit.client.1.smithi203.stdout:3/448: stat d10/d16/d23 0 2024-05-25T06:11:26.069 INFO:tasks.workunit.client.1.smithi203.stdout:3/449: truncate d10/d16/d23/d45/f76 749852 0 2024-05-25T06:11:26.073 INFO:tasks.workunit.client.1.smithi203.stdout:3/450: rename d10/f31 to d10/d16/d23/d4a/d88/f95 0 2024-05-25T06:11:26.074 INFO:tasks.workunit.client.1.smithi203.stdout:8/393: creat d2/d5/d10/d45/d5d/f80 x:0 0 0 2024-05-25T06:11:26.074 INFO:tasks.workunit.client.1.smithi203.stdout:6/380: dwrite d8/f12 [0,4194304] 0 2024-05-25T06:11:26.077 INFO:tasks.workunit.client.1.smithi203.stdout:8/394: write d2/d5/d18/d31/f6f [608515,17863] 0 2024-05-25T06:11:26.080 INFO:tasks.workunit.client.1.smithi203.stdout:8/395: chown d2/d5/l6d 413545 1 2024-05-25T06:11:26.082 INFO:tasks.workunit.client.1.smithi203.stdout:3/451: dwrite d10/f51 [0,4194304] 0 2024-05-25T06:11:26.083 INFO:tasks.workunit.client.1.smithi203.stdout:6/381: mknod d8/d11/d1b/d75/c7a 0 2024-05-25T06:11:26.089 INFO:tasks.workunit.client.1.smithi203.stdout:6/382: truncate d8/fb 561361 0 2024-05-25T06:11:26.090 INFO:tasks.workunit.client.1.smithi203.stdout:8/396: dwrite d2/d5/d28/d38/f3c [0,4194304] 0 2024-05-25T06:11:26.091 INFO:tasks.workunit.client.1.smithi203.stdout:6/383: unlink d8/d2a/d6c/f6f 0 2024-05-25T06:11:26.094 INFO:tasks.workunit.client.1.smithi203.stdout:8/397: mkdir d2/d5/d28/d38/d81 0 2024-05-25T06:11:26.095 INFO:tasks.workunit.client.1.smithi203.stdout:6/384: creat d8/d2a/d6c/f7b x:0 0 0 2024-05-25T06:11:26.097 INFO:tasks.workunit.client.1.smithi203.stdout:6/385: write d8/d11/d1b/f4d [342148,63779] 0 2024-05-25T06:11:26.100 INFO:tasks.workunit.client.1.smithi203.stdout:6/386: creat d8/d70/f7c x:0 0 0 2024-05-25T06:11:26.101 INFO:tasks.workunit.client.1.smithi203.stdout:8/398: dwrite d2/d5/d18/f1b [0,4194304] 0 2024-05-25T06:11:26.107 INFO:tasks.workunit.client.1.smithi203.stdout:6/387: dwrite d8/d11/d15/d28/d52/d43/f32 [4194304,4194304] 0 2024-05-25T06:11:26.111 INFO:tasks.workunit.client.1.smithi203.stdout:6/388: symlink d8/d11/d15/l7d 0 2024-05-25T06:11:26.113 INFO:tasks.workunit.client.1.smithi203.stdout:3/452: dread d10/d22/d2a/f4d [0,4194304] 0 2024-05-25T06:11:26.116 INFO:tasks.workunit.client.1.smithi203.stdout:6/389: read f1 [2199422,120720] 0 2024-05-25T06:11:26.119 INFO:tasks.workunit.client.1.smithi203.stdout:6/390: rename d8/d70/f4f to d8/d29/f7e 0 2024-05-25T06:11:26.121 INFO:tasks.workunit.client.1.smithi203.stdout:6/391: dread d8/d2a/f4a [0,4194304] 0 2024-05-25T06:11:26.122 INFO:tasks.workunit.client.1.smithi203.stdout:6/392: write d8/f12 [1506707,37057] 0 2024-05-25T06:11:26.125 INFO:tasks.workunit.client.1.smithi203.stdout:6/393: rmdir d8/d3d/d76 0 2024-05-25T06:11:26.126 INFO:tasks.workunit.client.1.smithi203.stdout:6/394: write d8/f9 [4408014,113888] 0 2024-05-25T06:11:26.128 INFO:tasks.workunit.client.1.smithi203.stdout:6/395: truncate d8/d11/d15/f17 5014599 0 2024-05-25T06:11:26.129 INFO:tasks.workunit.client.1.smithi203.stdout:6/396: read d8/d11/d15/f59 [453522,103427] 0 2024-05-25T06:11:26.130 INFO:tasks.workunit.client.1.smithi203.stdout:6/397: dread - d8/d11/d15/d28/f5e zero size 2024-05-25T06:11:26.130 INFO:tasks.workunit.client.1.smithi203.stdout:6/398: chown d8/d11 7416 1 2024-05-25T06:11:26.133 INFO:tasks.workunit.client.1.smithi203.stdout:6/399: dread d8/d11/d1b/f69 [0,4194304] 0 2024-05-25T06:11:26.134 INFO:tasks.workunit.client.1.smithi203.stdout:6/400: stat d8/d5a/c72 0 2024-05-25T06:11:26.135 INFO:tasks.workunit.client.1.smithi203.stdout:6/401: mknod d8/d3d/c7f 0 2024-05-25T06:11:26.148 INFO:tasks.workunit.client.1.smithi203.stdout:6/402: dread f2 [0,4194304] 0 2024-05-25T06:11:26.148 INFO:tasks.workunit.client.1.smithi203.stdout:6/403: fsync d8/d11/d15/f35 0 2024-05-25T06:11:26.149 INFO:tasks.workunit.client.1.smithi203.stdout:6/404: readlink d8/l14 0 2024-05-25T06:11:26.149 INFO:tasks.workunit.client.1.smithi203.stdout:6/405: read f2 [713245,69991] 0 2024-05-25T06:11:26.155 INFO:tasks.workunit.client.1.smithi203.stdout:6/406: dwrite d8/d70/d38/f4c [0,4194304] 0 2024-05-25T06:11:26.158 INFO:tasks.workunit.client.1.smithi203.stdout:6/407: mkdir d8/d29/d80 0 2024-05-25T06:11:26.160 INFO:tasks.workunit.client.1.smithi203.stdout:6/408: rename d8/d11/d47 to d8/d11/d47/d60/d81 22 2024-05-25T06:11:26.181 INFO:tasks.workunit.client.1.smithi203.stdout:6/409: dread d8/d29/f4b [0,4194304] 0 2024-05-25T06:11:26.183 INFO:tasks.workunit.client.1.smithi203.stdout:6/410: creat d8/d3d/f82 x:0 0 0 2024-05-25T06:11:26.183 INFO:tasks.workunit.client.1.smithi203.stdout:6/411: read - d8/d70/f7c zero size 2024-05-25T06:11:26.184 INFO:tasks.workunit.client.1.smithi203.stdout:6/412: write d8/d11/d15/f42 [1278813,108635] 0 2024-05-25T06:11:26.185 INFO:tasks.workunit.client.1.smithi203.stdout:6/413: write f1 [4034584,89022] 0 2024-05-25T06:11:26.186 INFO:tasks.workunit.client.1.smithi203.stdout:6/414: read d8/d11/d1b/f26 [2478172,28888] 0 2024-05-25T06:11:26.187 INFO:tasks.workunit.client.1.smithi203.stdout:6/415: fsync d8/d70/d38/f39 0 2024-05-25T06:11:26.189 INFO:tasks.workunit.client.1.smithi203.stdout:6/416: creat d8/d11/d47/f83 x:0 0 0 2024-05-25T06:11:26.191 INFO:tasks.workunit.client.1.smithi203.stdout:6/417: creat d8/d11/d15/d28/d52/d43/f84 x:0 0 0 2024-05-25T06:11:26.194 INFO:tasks.workunit.client.1.smithi203.stdout:6/418: dread d8/d11/d15/f2e [0,4194304] 0 2024-05-25T06:11:26.196 INFO:tasks.workunit.client.1.smithi203.stdout:6/419: mknod d8/d29/d80/c85 0 2024-05-25T06:11:26.202 INFO:tasks.workunit.client.1.smithi203.stdout:6/420: write d8/fc [352148,14349] 0 2024-05-25T06:11:26.203 INFO:tasks.workunit.client.1.smithi203.stdout:6/421: truncate d8/d3d/f6a 344764 0 2024-05-25T06:11:26.203 INFO:tasks.workunit.client.1.smithi203.stdout:6/422: fdatasync d8/d3d/f44 0 2024-05-25T06:11:26.205 INFO:tasks.workunit.client.1.smithi203.stdout:6/423: symlink d8/d11/d15/l86 0 2024-05-25T06:11:26.206 INFO:tasks.workunit.client.1.smithi203.stdout:6/424: read d8/d70/d38/f4c [3640847,98370] 0 2024-05-25T06:11:26.207 INFO:tasks.workunit.client.1.smithi203.stdout:6/425: fdatasync d8/d11/d15/d28/d52/d43/f32 0 2024-05-25T06:11:26.211 INFO:tasks.workunit.client.1.smithi203.stdout:6/426: dread d8/d11/d15/f2e [0,4194304] 0 2024-05-25T06:11:26.278 INFO:tasks.workunit.client.1.smithi203.stdout:6/427: dwrite d8/d11/d15/d28/d52/d43/f32 [8388608,4194304] 0 2024-05-25T06:11:26.288 INFO:tasks.workunit.client.1.smithi203.stdout:6/428: dwrite d8/fb [0,4194304] 0 2024-05-25T06:11:26.289 INFO:tasks.workunit.client.1.smithi203.stdout:6/429: write d8/d11/d15/f42 [1960853,54122] 0 2024-05-25T06:11:26.290 INFO:tasks.workunit.client.1.smithi203.stdout:6/430: chown d8/d11/d1b/c41 40307 1 2024-05-25T06:11:26.291 INFO:tasks.workunit.client.1.smithi203.stdout:6/431: truncate d8/d29/f7e 4837694 0 2024-05-25T06:11:26.291 INFO:tasks.workunit.client.1.smithi203.stdout:6/432: fsync d8/d11/f19 0 2024-05-25T06:11:26.293 INFO:tasks.workunit.client.1.smithi203.stdout:6/433: write d8/d11/d15/d28/f1d [389696,2955] 0 2024-05-25T06:11:26.295 INFO:tasks.workunit.client.1.smithi203.stdout:6/434: mkdir d8/d2a/d6c/d87 0 2024-05-25T06:11:26.308 INFO:tasks.workunit.client.1.smithi203.stdout:6/435: dwrite d8/ff [0,4194304] 0 2024-05-25T06:11:26.312 INFO:tasks.workunit.client.1.smithi203.stdout:6/436: getdents d8/d70 0 2024-05-25T06:11:26.313 INFO:tasks.workunit.client.1.smithi203.stdout:6/437: write d8/d11/d15/d28/d52/d43/f84 [556184,115217] 0 2024-05-25T06:11:26.315 INFO:tasks.workunit.client.1.smithi203.stdout:6/438: mkdir d8/d29/d46/d88 0 2024-05-25T06:11:26.316 INFO:tasks.workunit.client.1.smithi203.stdout:6/439: dread - d8/d11/d15/d28/f5e zero size 2024-05-25T06:11:26.317 INFO:tasks.workunit.client.1.smithi203.stdout:6/440: read f2 [3559612,110749] 0 2024-05-25T06:11:26.319 INFO:tasks.workunit.client.1.smithi203.stdout:6/441: mknod d8/d11/d47/c89 0 2024-05-25T06:11:26.320 INFO:tasks.workunit.client.1.smithi203.stdout:6/442: creat d8/d11/d47/f8a x:0 0 0 2024-05-25T06:11:26.333 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:25 smithi203 ceph-mon[36030]: pgmap v13: 65 pgs: 65 active+clean; 466 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 33 MiB/s wr, 217 op/s 2024-05-25T06:11:26.344 INFO:tasks.workunit.client.1.smithi203.stdout:6/443: dread d8/d11/d15/f2b [0,4194304] 0 2024-05-25T06:11:26.349 INFO:tasks.workunit.client.1.smithi203.stdout:6/444: dread d8/d11/d15/f37 [0,4194304] 0 2024-05-25T06:11:26.349 INFO:tasks.workunit.client.1.smithi203.stdout:6/445: fdatasync d8/fd 0 2024-05-25T06:11:26.350 INFO:tasks.workunit.client.1.smithi203.stdout:6/446: write d8/d11/d15/f18 [594165,100013] 0 2024-05-25T06:11:26.351 INFO:tasks.workunit.client.1.smithi203.stdout:6/447: readlink d8/d2a/l3a 0 2024-05-25T06:11:26.351 INFO:tasks.workunit.client.1.smithi203.stdout:6/448: stat d8/d11/d15/f18 0 2024-05-25T06:11:26.351 INFO:tasks.workunit.client.1.smithi203.stdout:6/449: readlink d8/d11/d15/l33 0 2024-05-25T06:11:26.352 INFO:tasks.workunit.client.1.smithi203.stdout:6/450: fdatasync d8/d11/d15/f17 0 2024-05-25T06:11:26.354 INFO:tasks.workunit.client.1.smithi203.stdout:6/451: unlink d8/d3d/f82 0 2024-05-25T06:11:26.355 INFO:tasks.workunit.client.1.smithi203.stdout:6/452: mkdir d8/d5a/d8b 0 2024-05-25T06:11:26.360 INFO:tasks.workunit.client.1.smithi203.stdout:6/453: dwrite d8/d11/d47/d60/f57 [4194304,4194304] 0 2024-05-25T06:11:26.361 INFO:tasks.workunit.client.1.smithi203.stdout:6/454: write d8/d11/d47/f8a [311050,111640] 0 2024-05-25T06:11:26.363 INFO:tasks.workunit.client.1.smithi203.stdout:6/455: mknod d8/d5a/d8b/c8c 0 2024-05-25T06:11:26.366 INFO:tasks.workunit.client.1.smithi203.stdout:6/456: dread d8/d11/d15/f18 [0,4194304] 0 2024-05-25T06:11:26.370 INFO:tasks.workunit.client.1.smithi203.stdout:4/256: sync 2024-05-25T06:11:26.395 INFO:tasks.workunit.client.1.smithi203.stdout:2/348: sync 2024-05-25T06:11:26.396 INFO:tasks.workunit.client.1.smithi203.stdout:0/255: sync 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:0/256: rmdir dd/d10/d34 39 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:6/457: dwrite d8/d2a/d6c/f7b [0,4194304] 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:6/458: symlink d8/d29/d6e/l8d 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:2/349: dread d2/f6f [0,4194304] 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:4/257: dwrite d0/f38 [0,4194304] 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:6/459: creat d8/d11/f8e x:0 0 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:4/258: truncate d0/d9/d11/f2f 232367 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:0/257: read dd/d10/f33 [613310,120003] 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:6/460: mknod d8/d3d/c8f 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:2/350: dwrite d2/f8 [0,4194304] 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:4/259: mknod d0/d8/d1f/c48 0 2024-05-25T06:11:26.398 INFO:tasks.workunit.client.1.smithi203.stdout:0/258: truncate dd/d10/f2f 2242849 0 2024-05-25T06:11:26.399 INFO:tasks.workunit.client.1.smithi203.stdout:2/351: creat d2/d42/f83 x:0 0 0 2024-05-25T06:11:26.399 INFO:tasks.workunit.client.1.smithi203.stdout:4/260: creat d0/d8/d3a/f49 x:0 0 0 2024-05-25T06:11:26.400 INFO:tasks.workunit.client.1.smithi203.stdout:6/461: dwrite d8/d11/d47/f83 [0,4194304] 0 2024-05-25T06:11:26.401 INFO:tasks.workunit.client.1.smithi203.stdout:4/261: rmdir d0/d8/d1f/d26/d3b 39 2024-05-25T06:11:26.403 INFO:tasks.workunit.client.1.smithi203.stdout:4/262: rmdir d0/d9/d24 39 2024-05-25T06:11:26.407 INFO:tasks.workunit.client.1.smithi203.stdout:6/462: getdents d8/d5a/d8b 0 2024-05-25T06:11:26.409 INFO:tasks.workunit.client.1.smithi203.stdout:4/263: creat d0/d8/d1f/d26/f4a x:0 0 0 2024-05-25T06:11:26.409 INFO:tasks.workunit.client.1.smithi203.stdout:6/463: dread d8/d11/d15/f37 [0,4194304] 0 2024-05-25T06:11:26.412 INFO:tasks.workunit.client.1.smithi203.stdout:6/464: creat d8/d11/f90 x:0 0 0 2024-05-25T06:11:26.414 INFO:tasks.workunit.client.1.smithi203.stdout:6/465: write d8/d11/d47/d60/f57 [1292754,5352] 0 2024-05-25T06:11:26.415 INFO:tasks.workunit.client.1.smithi203.stdout:4/264: dwrite d0/f5 [0,4194304] 0 2024-05-25T06:11:26.416 INFO:tasks.workunit.client.1.smithi203.stdout:2/352: truncate d2/d1a/f27 3684633 0 2024-05-25T06:11:26.417 INFO:tasks.workunit.client.1.smithi203.stdout:6/466: chown d8/d5a/d8b/c8c 16 1 2024-05-25T06:11:26.423 INFO:tasks.workunit.client.1.smithi203.stdout:4/265: rmdir d0/d9/d24/d47 0 2024-05-25T06:11:26.425 INFO:tasks.workunit.client.1.smithi203.stdout:6/467: dwrite d8/d11/d15/f59 [0,4194304] 0 2024-05-25T06:11:26.428 INFO:tasks.workunit.client.1.smithi203.stdout:4/266: rmdir d0/d8/d1f 39 2024-05-25T06:11:26.429 INFO:tasks.workunit.client.1.smithi203.stdout:6/468: rename d8/l14 to d8/d5a/d8b/l91 0 2024-05-25T06:11:26.429 INFO:tasks.workunit.client.1.smithi203.stdout:4/267: dread - d0/d8/d3a/f46 zero size 2024-05-25T06:11:26.433 INFO:tasks.workunit.client.1.smithi203.stdout:6/469: rmdir d8/d11/d15 39 2024-05-25T06:11:26.434 INFO:tasks.workunit.client.1.smithi203.stdout:4/268: mknod d0/d9/d24/d27/c4b 0 2024-05-25T06:11:26.436 INFO:tasks.workunit.client.1.smithi203.stdout:6/470: fdatasync d8/d11/d15/f42 0 2024-05-25T06:11:26.439 INFO:tasks.workunit.client.1.smithi203.stdout:6/471: write d8/d11/d1b/f4d [729488,22378] 0 2024-05-25T06:11:26.440 INFO:tasks.workunit.client.1.smithi203.stdout:6/472: write d8/d11/d15/f42 [1113360,117105] 0 2024-05-25T06:11:26.442 INFO:tasks.workunit.client.1.smithi203.stdout:2/353: dread d2/d1a/d33/f71 [4194304,4194304] 0 2024-05-25T06:11:26.443 INFO:tasks.workunit.client.1.smithi203.stdout:4/269: dread d0/d9/d11/f22 [0,4194304] 0 2024-05-25T06:11:26.445 INFO:tasks.workunit.client.1.smithi203.stdout:6/473: rename d8/d11/d15/f35 to d8/d11/d15/d28/d52/d43/f92 0 2024-05-25T06:11:26.446 INFO:tasks.workunit.client.1.smithi203.stdout:4/270: creat d0/d9/d24/d27/f4c x:0 0 0 2024-05-25T06:11:26.448 INFO:tasks.workunit.client.1.smithi203.stdout:6/474: unlink d8/d11/d1b/f69 0 2024-05-25T06:11:26.449 INFO:tasks.workunit.client.1.smithi203.stdout:2/354: truncate d2/fe 1762487 0 2024-05-25T06:11:26.450 INFO:tasks.workunit.client.1.smithi203.stdout:2/355: chown d2/d4/d10/f5e 82 1 2024-05-25T06:11:26.451 INFO:tasks.workunit.client.1.smithi203.stdout:6/475: fsync d8/d11/d15/f18 0 2024-05-25T06:11:26.452 INFO:tasks.workunit.client.1.smithi203.stdout:4/271: rename d0/d9/d24/f28 to d0/d8/d1f/d26/d43/f4d 0 2024-05-25T06:11:26.458 INFO:tasks.workunit.client.1.smithi203.stdout:4/272: mknod d0/d8/d1f/d26/d3b/c4e 0 2024-05-25T06:11:26.461 INFO:tasks.workunit.client.1.smithi203.stdout:6/476: dwrite d8/fb [0,4194304] 0 2024-05-25T06:11:26.461 INFO:tasks.workunit.client.1.smithi203.stdout:4/273: read - d0/d9/d24/f40 zero size 2024-05-25T06:11:26.461 INFO:tasks.workunit.client.1.smithi203.stdout:2/356: dwrite d2/d4/d10/d50/f82 [0,4194304] 0 2024-05-25T06:11:26.467 INFO:tasks.workunit.client.1.smithi203.stdout:4/274: dread d0/f5 [0,4194304] 0 2024-05-25T06:11:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:25 smithi160 ceph-mon[29041]: pgmap v13: 65 pgs: 65 active+clean; 466 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 33 MiB/s wr, 217 op/s 2024-05-25T06:11:26.474 INFO:tasks.workunit.client.1.smithi203.stdout:5/580: sync 2024-05-25T06:11:26.474 INFO:tasks.workunit.client.1.smithi203.stdout:8/399: sync 2024-05-25T06:11:26.474 INFO:tasks.workunit.client.1.smithi203.stdout:1/347: sync 2024-05-25T06:11:26.474 INFO:tasks.workunit.client.1.smithi203.stdout:9/314: sync 2024-05-25T06:11:26.474 INFO:tasks.workunit.client.1.smithi203.stdout:3/453: sync 2024-05-25T06:11:26.475 INFO:tasks.workunit.client.1.smithi203.stdout:7/346: sync 2024-05-25T06:11:26.479 INFO:tasks.workunit.client.1.smithi203.stdout:8/400: write d2/d5/d18/f1b [981173,51434] 0 2024-05-25T06:11:26.492 INFO:tasks.workunit.client.1.smithi203.stdout:2/357: symlink d2/d4/d10/d50/l84 0 2024-05-25T06:11:26.495 INFO:tasks.workunit.client.1.smithi203.stdout:6/477: rename d8/d2a/d6c/f7b to d8/d11/d15/d28/d52/d43/f93 0 2024-05-25T06:11:26.496 INFO:tasks.workunit.client.1.smithi203.stdout:4/275: truncate d0/d9/f12 1088328 0 2024-05-25T06:11:26.498 INFO:tasks.workunit.client.1.smithi203.stdout:3/454: stat la 0 2024-05-25T06:11:26.500 INFO:tasks.workunit.client.1.smithi203.stdout:3/455: write d10/d22/f42 [555432,90347] 0 2024-05-25T06:11:26.501 INFO:tasks.workunit.client.1.smithi203.stdout:2/358: dwrite d2/f6c [0,4194304] 0 2024-05-25T06:11:26.503 INFO:tasks.workunit.client.1.smithi203.stdout:3/456: dread - d10/d16/d23/d3a/f80 zero size 2024-05-25T06:11:26.510 INFO:tasks.workunit.client.1.smithi203.stdout:4/276: symlink d0/d8/d1f/d26/d3b/l4f 0 2024-05-25T06:11:26.511 INFO:tasks.workunit.client.1.smithi203.stdout:4/277: fsync d0/d8/f44 0 2024-05-25T06:11:26.511 INFO:tasks.workunit.client.1.smithi203.stdout:2/359: dwrite d2/d4/d10/d32/f1b [0,4194304] 0 2024-05-25T06:11:26.513 INFO:tasks.workunit.client.1.smithi203.stdout:4/278: fdatasync d0/d9/f17 0 2024-05-25T06:11:26.515 INFO:tasks.workunit.client.1.smithi203.stdout:8/401: creat d2/d5/d10/d45/d5d/f82 x:0 0 0 2024-05-25T06:11:26.515 INFO:tasks.workunit.client.1.smithi203.stdout:3/457: creat d10/d16/d50/d8b/f96 x:0 0 0 2024-05-25T06:11:26.516 INFO:tasks.workunit.client.1.smithi203.stdout:5/581: link dc/d11/d17/d1a/d2d/d4d/l4f dc/d11/d17/d1a/d2d/d53/da0/lcd 0 2024-05-25T06:11:26.519 INFO:tasks.workunit.client.1.smithi203.stdout:9/315: rename d0/d17/d20/c25 to d0/df/d2d/c69 0 2024-05-25T06:11:26.520 INFO:tasks.workunit.client.1.smithi203.stdout:5/582: write dc/d11/f3e [4442031,16154] 0 2024-05-25T06:11:26.522 INFO:tasks.workunit.client.1.smithi203.stdout:5/583: write dc/d11/d17/d45/f5c [2640624,108848] 0 2024-05-25T06:11:26.524 INFO:tasks.workunit.client.1.smithi203.stdout:5/584: chown dc/d11/d17/d44 102058455 1 2024-05-25T06:11:26.524 INFO:tasks.workunit.client.1.smithi203.stdout:3/458: truncate d10/d16/d23/d45/f4e 742389 0 2024-05-25T06:11:26.525 INFO:tasks.workunit.client.1.smithi203.stdout:2/360: link d2/d4/d10/d58/f6d d2/d4/d10/d5f/f85 0 2024-05-25T06:11:26.527 INFO:tasks.workunit.client.1.smithi203.stdout:3/459: write d10/d16/d23/f6e [457144,15419] 0 2024-05-25T06:11:26.537 INFO:tasks.workunit.client.1.smithi203.stdout:6/478: dread d8/d11/d47/d60/f55 [0,4194304] 0 2024-05-25T06:11:26.538 INFO:tasks.workunit.client.1.smithi203.stdout:5/585: fdatasync dc/d11/d17/d1a/d22/f39 0 2024-05-25T06:11:26.539 INFO:tasks.workunit.client.1.smithi203.stdout:6/479: truncate d8/d11/f90 701409 0 2024-05-25T06:11:26.542 INFO:tasks.workunit.client.1.smithi203.stdout:5/586: dread dc/d11/d17/d1a/d2d/d53/da2/f79 [0,4194304] 0 2024-05-25T06:11:26.544 INFO:tasks.workunit.client.1.smithi203.stdout:6/480: link d8/d11/d15/f18 d8/d70/f94 0 2024-05-25T06:11:26.545 INFO:tasks.workunit.client.1.smithi203.stdout:6/481: chown d8/d11/d47/c89 668132993 1 2024-05-25T06:11:26.545 INFO:tasks.workunit.client.1.smithi203.stdout:1/348: rename d14/d57/d1d/c73 to d14/d57/c79 0 2024-05-25T06:11:26.549 INFO:tasks.workunit.client.1.smithi203.stdout:2/361: dread d2/d4/d10/d5f/f85 [0,4194304] 0 2024-05-25T06:11:26.550 INFO:tasks.workunit.client.1.smithi203.stdout:2/362: chown d2/d1a/d33/c3e 1109 1 2024-05-25T06:11:26.556 INFO:tasks.workunit.client.1.smithi203.stdout:1/349: getdents d14/d57/d1d/d3e/d36 0 2024-05-25T06:11:26.558 INFO:tasks.workunit.client.1.smithi203.stdout:2/363: dwrite d2/d38/d2f/f34 [4194304,4194304] 0 2024-05-25T06:11:26.559 INFO:tasks.workunit.client.1.smithi203.stdout:1/350: creat d14/d37/f7a x:0 0 0 2024-05-25T06:11:26.560 INFO:tasks.workunit.client.1.smithi203.stdout:2/364: read d2/d38/d2f/f61 [178314,70366] 0 2024-05-25T06:11:26.563 INFO:tasks.workunit.client.1.smithi203.stdout:2/365: chown d2/d4/d10/l40 14737 1 2024-05-25T06:11:26.564 INFO:tasks.workunit.client.1.smithi203.stdout:6/482: dread d8/d11/d15/f17 [0,4194304] 0 2024-05-25T06:11:26.565 INFO:tasks.workunit.client.1.smithi203.stdout:6/483: chown d8/f12 469976 1 2024-05-25T06:11:26.567 INFO:tasks.workunit.client.1.smithi203.stdout:4/279: rename d0/d9/d42 to d0/d9/d50 0 2024-05-25T06:11:26.568 INFO:tasks.workunit.client.1.smithi203.stdout:4/280: write d0/d9/d11/f3f [4383756,57970] 0 2024-05-25T06:11:26.569 INFO:tasks.workunit.client.1.smithi203.stdout:6/484: dread d8/d11/d15/d28/d52/d43/f84 [0,4194304] 0 2024-05-25T06:11:26.570 INFO:tasks.workunit.client.1.smithi203.stdout:6/485: readlink d8/d11/d15/l3c 0 2024-05-25T06:11:26.571 INFO:tasks.workunit.client.1.smithi203.stdout:6/486: chown d8/d11/d15/d28/d52/d43/f93 455318285 1 2024-05-25T06:11:26.572 INFO:tasks.workunit.client.1.smithi203.stdout:9/316: truncate d0/df/f1d 158868 0 2024-05-25T06:11:26.575 INFO:tasks.workunit.client.1.smithi203.stdout:6/487: mknod d8/d11/d15/d28/c95 0 2024-05-25T06:11:26.577 INFO:tasks.workunit.client.1.smithi203.stdout:6/488: chown d8/l16 746971 1 2024-05-25T06:11:26.577 INFO:tasks.workunit.client.1.smithi203.stdout:2/366: mknod d2/d4/c86 0 2024-05-25T06:11:26.579 INFO:tasks.workunit.client.1.smithi203.stdout:4/281: dwrite d0/d9/d11/f22 [0,4194304] 0 2024-05-25T06:11:26.583 INFO:tasks.workunit.client.1.smithi203.stdout:3/460: dwrite d10/d22/f5b [0,4194304] 0 2024-05-25T06:11:26.586 INFO:tasks.workunit.client.1.smithi203.stdout:1/351: getdents d14/d57/d1d 0 2024-05-25T06:11:26.586 INFO:tasks.workunit.client.1.smithi203.stdout:5/587: truncate dc/d11/d17/d1a/d22/f24 76815 0 2024-05-25T06:11:26.587 INFO:tasks.workunit.client.1.smithi203.stdout:4/282: fsync d0/f5 0 2024-05-25T06:11:26.591 INFO:tasks.workunit.client.1.smithi203.stdout:8/402: creat d2/d5/d10/f83 x:0 0 0 2024-05-25T06:11:26.593 INFO:tasks.workunit.client.1.smithi203.stdout:1/352: chown d14/d57/c79 863213 1 2024-05-25T06:11:26.593 INFO:tasks.workunit.client.1.smithi203.stdout:2/367: dread d2/f6c [0,4194304] 0 2024-05-25T06:11:26.595 INFO:tasks.workunit.client.1.smithi203.stdout:8/403: write d2/d5/d28/d38/f3c [2044816,83359] 0 2024-05-25T06:11:26.597 INFO:tasks.workunit.client.1.smithi203.stdout:5/588: write dc/d11/d17/d1a/d2d/f65 [1029707,57263] 0 2024-05-25T06:11:26.597 INFO:tasks.workunit.client.1.smithi203.stdout:8/404: stat d2/l56 0 2024-05-25T06:11:26.599 INFO:tasks.workunit.client.1.smithi203.stdout:2/368: rmdir d2/d38/d43/d60 39 2024-05-25T06:11:26.599 INFO:tasks.workunit.client.1.smithi203.stdout:9/317: dread d0/d17/d20/f46 [0,4194304] 0 2024-05-25T06:11:26.600 INFO:tasks.workunit.client.1.smithi203.stdout:1/353: creat d14/d57/d29/f7b x:0 0 0 2024-05-25T06:11:26.602 INFO:tasks.workunit.client.1.smithi203.stdout:6/489: link d8/d70/d38/f4c d8/d5a/d8b/f96 0 2024-05-25T06:11:26.603 INFO:tasks.workunit.client.1.smithi203.stdout:1/354: readlink d14/d5f/l70 0 2024-05-25T06:11:26.604 INFO:tasks.workunit.client.1.smithi203.stdout:3/461: dwrite d10/d16/d23/d67/f78 [0,4194304] 0 2024-05-25T06:11:26.604 INFO:tasks.workunit.client.1.smithi203.stdout:5/589: unlink dc/d3c/d3f/f93 0 2024-05-25T06:11:26.606 INFO:tasks.workunit.client.1.smithi203.stdout:6/490: chown d8/ff 892 1 2024-05-25T06:11:26.607 INFO:tasks.workunit.client.1.smithi203.stdout:9/318: chown d0/d17/d20/d3c/c67 115025649 1 2024-05-25T06:11:26.608 INFO:tasks.workunit.client.1.smithi203.stdout:3/462: write d10/d22/f86 [919249,95533] 0 2024-05-25T06:11:26.609 INFO:tasks.workunit.client.1.smithi203.stdout:3/463: stat f0 0 2024-05-25T06:11:26.610 INFO:tasks.workunit.client.1.smithi203.stdout:3/464: dread - d10/d16/d23/d67/f94 zero size 2024-05-25T06:11:26.612 INFO:tasks.workunit.client.1.smithi203.stdout:6/491: symlink d8/d29/d46/d88/l97 0 2024-05-25T06:11:26.613 INFO:tasks.workunit.client.1.smithi203.stdout:3/465: write d10/d16/d23/d67/f94 [39187,4846] 0 2024-05-25T06:11:26.615 INFO:tasks.workunit.client.1.smithi203.stdout:3/466: write d10/d22/f86 [582165,31669] 0 2024-05-25T06:11:26.615 INFO:tasks.workunit.client.1.smithi203.stdout:5/590: rename dc/d11/d17/d45/f5f to dc/d11/d42/fce 0 2024-05-25T06:11:26.616 INFO:tasks.workunit.client.1.smithi203.stdout:2/369: dread d2/d4/d10/d32/f1f [0,4194304] 0 2024-05-25T06:11:26.617 INFO:tasks.workunit.client.1.smithi203.stdout:5/591: chown dc/d11/d17/d1a/d72/dc8 126517684 1 2024-05-25T06:11:26.617 INFO:tasks.workunit.client.1.smithi203.stdout:6/492: symlink d8/d29/d80/l98 0 2024-05-25T06:11:26.619 INFO:tasks.workunit.client.1.smithi203.stdout:2/370: write d2/d4/d10/d5f/f81 [557919,41779] 0 2024-05-25T06:11:26.622 INFO:tasks.workunit.client.1.smithi203.stdout:6/493: creat d8/d11/d47/f99 x:0 0 0 2024-05-25T06:11:26.622 INFO:tasks.workunit.client.1.smithi203.stdout:9/319: getdents d0 0 2024-05-25T06:11:26.623 INFO:tasks.workunit.client.1.smithi203.stdout:3/467: rename d10/d16/d23/d67/f78 to d10/d16/d23/d4a/d88/f97 0 2024-05-25T06:11:26.625 INFO:tasks.workunit.client.1.smithi203.stdout:6/494: truncate f2 4369025 0 2024-05-25T06:11:26.625 INFO:tasks.workunit.client.1.smithi203.stdout:8/405: dread d2/d5/f44 [0,4194304] 0 2024-05-25T06:11:26.626 INFO:tasks.workunit.client.1.smithi203.stdout:3/468: symlink d10/d16/d50/d7d/l98 0 2024-05-25T06:11:26.627 INFO:tasks.workunit.client.1.smithi203.stdout:8/406: dread - d2/d5/d10/f83 zero size 2024-05-25T06:11:26.627 INFO:tasks.workunit.client.1.smithi203.stdout:6/495: creat d8/d3d/f9a x:0 0 0 2024-05-25T06:11:26.628 INFO:tasks.workunit.client.1.smithi203.stdout:3/469: mkdir d10/d22/d2a/d99 0 2024-05-25T06:11:26.629 INFO:tasks.workunit.client.1.smithi203.stdout:6/496: dread d8/d2a/f4a [0,4194304] 0 2024-05-25T06:11:26.638 INFO:tasks.workunit.client.1.smithi203.stdout:2/371: dwrite d2/d4/d10/d5f/f85 [0,4194304] 0 2024-05-25T06:11:26.640 INFO:tasks.workunit.client.1.smithi203.stdout:5/592: dread dc/f21 [0,4194304] 0 2024-05-25T06:11:26.641 INFO:tasks.workunit.client.1.smithi203.stdout:5/593: chown dc/d11/d42/d94/dad/db1 0 1 2024-05-25T06:11:26.642 INFO:tasks.workunit.client.1.smithi203.stdout:6/497: truncate d8/d11/d47/d60/f57 3489088 0 2024-05-25T06:11:26.643 INFO:tasks.workunit.client.1.smithi203.stdout:8/407: dread d2/d5/d10/d11/de/f1a [0,4194304] 0 2024-05-25T06:11:26.644 INFO:tasks.workunit.client.1.smithi203.stdout:3/470: truncate d10/d16/d23/d4a/d88/f95 691539 0 2024-05-25T06:11:26.645 INFO:tasks.workunit.client.1.smithi203.stdout:9/320: dread d0/d17/d20/d29/f32 [0,4194304] 0 2024-05-25T06:11:26.649 INFO:tasks.workunit.client.1.smithi203.stdout:9/321: fsync d0/d17/d20/f46 0 2024-05-25T06:11:26.651 INFO:tasks.workunit.client.1.smithi203.stdout:2/372: dwrite d2/f6f [0,4194304] 0 2024-05-25T06:11:26.652 INFO:tasks.workunit.client.1.smithi203.stdout:3/471: unlink d10/d16/d23/d3a/f65 0 2024-05-25T06:11:26.654 INFO:tasks.workunit.client.1.smithi203.stdout:2/373: chown d2/d4/d10/d32/l65 384695 1 2024-05-25T06:11:26.655 INFO:tasks.workunit.client.1.smithi203.stdout:9/322: rmdir d0/df/d37/d4f 39 2024-05-25T06:11:26.657 INFO:tasks.workunit.client.1.smithi203.stdout:8/408: mkdir d2/d5/d28/d38/d81/d84 0 2024-05-25T06:11:26.659 INFO:tasks.workunit.client.1.smithi203.stdout:8/409: chown d2/d5/d10/d11/de/f1a 37444 1 2024-05-25T06:11:26.662 INFO:tasks.workunit.client.1.smithi203.stdout:9/323: mknod d0/d17/d20/d3c/c6a 0 2024-05-25T06:11:26.662 INFO:tasks.workunit.client.1.smithi203.stdout:2/374: dwrite d2/f6f [4194304,4194304] 0 2024-05-25T06:11:26.664 INFO:tasks.workunit.client.1.smithi203.stdout:3/472: creat d10/f9a x:0 0 0 2024-05-25T06:11:26.665 INFO:tasks.workunit.client.1.smithi203.stdout:8/410: creat d2/d5/d10/d11/de/f85 x:0 0 0 2024-05-25T06:11:26.666 INFO:tasks.workunit.client.1.smithi203.stdout:9/324: creat d0/d17/d20/d29/f6b x:0 0 0 2024-05-25T06:11:26.671 INFO:tasks.workunit.client.1.smithi203.stdout:8/411: fsync d2/d5/d10/d11/d1f/f72 0 2024-05-25T06:11:26.671 INFO:tasks.workunit.client.1.smithi203.stdout:2/375: fsync d2/d1a/f27 0 2024-05-25T06:11:26.671 INFO:tasks.workunit.client.1.smithi203.stdout:3/473: creat d10/d16/d23/d45/f9b x:0 0 0 2024-05-25T06:11:26.672 INFO:tasks.workunit.client.1.smithi203.stdout:9/325: creat d0/d17/d23/f6c x:0 0 0 2024-05-25T06:11:26.674 INFO:tasks.workunit.client.1.smithi203.stdout:3/474: chown d10/d16/d23/d45/f4e 28 1 2024-05-25T06:11:26.675 INFO:tasks.workunit.client.1.smithi203.stdout:2/376: mkdir d2/d1a/d87 0 2024-05-25T06:11:26.677 INFO:tasks.workunit.client.1.smithi203.stdout:9/326: rename d0/df/c12 to d0/d17/c6d 0 2024-05-25T06:11:26.677 INFO:tasks.workunit.client.1.smithi203.stdout:5/594: dread dc/d11/d17/d45/f62 [0,4194304] 0 2024-05-25T06:11:26.678 INFO:tasks.workunit.client.1.smithi203.stdout:2/377: readlink d2/d4/l7 0 2024-05-25T06:11:26.678 INFO:tasks.workunit.client.1.smithi203.stdout:8/412: dwrite d2/d5/d10/f83 [0,4194304] 0 2024-05-25T06:11:26.680 INFO:tasks.workunit.client.1.smithi203.stdout:3/475: creat d10/d16/d50/d7d/f9c x:0 0 0 2024-05-25T06:11:26.681 INFO:tasks.workunit.client.1.smithi203.stdout:5/595: creat dc/daa/fcf x:0 0 0 2024-05-25T06:11:26.681 INFO:tasks.workunit.client.1.smithi203.stdout:8/413: symlink d2/d5/d10/l86 0 2024-05-25T06:11:26.683 INFO:tasks.workunit.client.1.smithi203.stdout:2/378: read d2/d4/f74 [3192470,10478] 0 2024-05-25T06:11:26.684 INFO:tasks.workunit.client.1.smithi203.stdout:8/414: read d2/d5/d28/d38/f47 [390607,112232] 0 2024-05-25T06:11:26.684 INFO:tasks.workunit.client.1.smithi203.stdout:5/596: mkdir dc/d11/d18/dc0/dd0 0 2024-05-25T06:11:26.685 INFO:tasks.workunit.client.1.smithi203.stdout:2/379: write d2/d4/d10/d32/f52 [1777360,26683] 0 2024-05-25T06:11:26.688 INFO:tasks.workunit.client.1.smithi203.stdout:9/327: rename d0/d61 to d0/d17/d20/d6e 0 2024-05-25T06:11:26.689 INFO:tasks.workunit.client.1.smithi203.stdout:5/597: unlink dc/d11/d17/d44/fbb 0 2024-05-25T06:11:26.692 INFO:tasks.workunit.client.1.smithi203.stdout:2/380: creat d2/d4/d10/d50/f88 x:0 0 0 2024-05-25T06:11:26.694 INFO:tasks.workunit.client.1.smithi203.stdout:8/415: creat d2/d5/d10/d45/d5d/d68/f87 x:0 0 0 2024-05-25T06:11:26.694 INFO:tasks.workunit.client.1.smithi203.stdout:2/381: mkdir d2/d1a/d89 0 2024-05-25T06:11:26.702 INFO:tasks.workunit.client.1.smithi203.stdout:3/476: dread f8 [0,4194304] 0 2024-05-25T06:11:26.705 INFO:tasks.workunit.client.1.smithi203.stdout:8/416: dwrite d2/d5/d18/d4f/f66 [4194304,4194304] 0 2024-05-25T06:11:26.705 INFO:tasks.workunit.client.1.smithi203.stdout:3/477: chown d10/d16/d23/d38 1 1 2024-05-25T06:11:26.707 INFO:tasks.workunit.client.1.smithi203.stdout:2/382: dwrite d2/d4/d10/d5f/f85 [0,4194304] 0 2024-05-25T06:11:26.709 INFO:tasks.workunit.client.1.smithi203.stdout:9/328: rename d0/df/f5b to d0/f6f 0 2024-05-25T06:11:26.710 INFO:tasks.workunit.client.1.smithi203.stdout:3/478: fsync d10/d16/d50/d7d/f9c 0 2024-05-25T06:11:26.712 INFO:tasks.workunit.client.1.smithi203.stdout:9/329: stat d0/f1e 0 2024-05-25T06:11:26.714 INFO:tasks.workunit.client.1.smithi203.stdout:5/598: dread dc/d11/d17/d1a/d22/f90 [0,4194304] 0 2024-05-25T06:11:26.716 INFO:tasks.workunit.client.1.smithi203.stdout:9/330: chown d0/f13 838 1 2024-05-25T06:11:26.717 INFO:tasks.workunit.client.1.smithi203.stdout:5/599: stat dc/d11/d17/d1a/d2d/d4d/d64/fa1 0 2024-05-25T06:11:26.722 INFO:tasks.workunit.client.1.smithi203.stdout:8/417: dread d2/d5/d10/d11/de/ff [0,4194304] 0 2024-05-25T06:11:26.726 INFO:tasks.workunit.client.1.smithi203.stdout:9/331: getdents d0/d17/d20 0 2024-05-25T06:11:26.727 INFO:tasks.workunit.client.1.smithi203.stdout:8/418: mkdir d2/d5/d28/d38/d81/d84/d88 0 2024-05-25T06:11:26.733 INFO:tasks.workunit.client.1.smithi203.stdout:8/419: getdents d2/d5/d2d 0 2024-05-25T06:11:26.734 INFO:tasks.workunit.client.1.smithi203.stdout:9/332: creat d0/df/d37/f70 x:0 0 0 2024-05-25T06:11:26.736 INFO:tasks.workunit.client.1.smithi203.stdout:3/479: dwrite d10/d16/d50/f64 [4194304,4194304] 0 2024-05-25T06:11:26.737 INFO:tasks.workunit.client.1.smithi203.stdout:8/420: fsync d2/d5/d10/f4d 0 2024-05-25T06:11:26.739 INFO:tasks.workunit.client.1.smithi203.stdout:3/480: chown d10/d16/d50/d8b/f85 59515823 1 2024-05-25T06:11:26.740 INFO:tasks.workunit.client.1.smithi203.stdout:5/600: dwrite dc/d11/d42/fce [0,4194304] 0 2024-05-25T06:11:26.742 INFO:tasks.workunit.client.1.smithi203.stdout:5/601: fsync dc/d11/f29 0 2024-05-25T06:11:26.744 INFO:tasks.workunit.client.1.smithi203.stdout:8/421: rmdir d2/d5/d10/d45/d5d/d76 39 2024-05-25T06:11:26.748 INFO:tasks.workunit.client.1.smithi203.stdout:2/383: dread d2/d4/d10/d5f/f25 [0,4194304] 0 2024-05-25T06:11:26.748 INFO:tasks.workunit.client.1.smithi203.stdout:8/422: mknod d2/d5/d2d/c89 0 2024-05-25T06:11:26.748 INFO:tasks.workunit.client.1.smithi203.stdout:3/481: dread f8 [0,4194304] 0 2024-05-25T06:11:26.752 INFO:tasks.workunit.client.1.smithi203.stdout:9/333: truncate d0/f19 4122933 0 2024-05-25T06:11:26.752 INFO:tasks.workunit.client.1.smithi203.stdout:3/482: creat d10/d16/d50/d7d/f9d x:0 0 0 2024-05-25T06:11:26.753 INFO:tasks.workunit.client.1.smithi203.stdout:8/423: mkdir d2/d5/d28/d38/d81/d84/d8a 0 2024-05-25T06:11:26.753 INFO:tasks.workunit.client.1.smithi203.stdout:8/424: stat d2/l56 0 2024-05-25T06:11:26.753 INFO:tasks.workunit.client.1.smithi203.stdout:9/334: chown d0/df/l35 0 1 2024-05-25T06:11:26.755 INFO:tasks.workunit.client.1.smithi203.stdout:9/335: read - d0/df/d37/f58 zero size 2024-05-25T06:11:26.756 INFO:tasks.workunit.client.1.smithi203.stdout:8/425: write d2/d5/d18/d4f/f66 [6008108,7947] 0 2024-05-25T06:11:26.756 INFO:tasks.workunit.client.1.smithi203.stdout:3/483: rename d10/d16/f40 to d10/d16/d50/f9e 0 2024-05-25T06:11:26.758 INFO:tasks.workunit.client.1.smithi203.stdout:9/336: mknod d0/df/c71 0 2024-05-25T06:11:26.758 INFO:tasks.workunit.client.1.smithi203.stdout:8/426: dread - d2/d5/d18/f70 zero size 2024-05-25T06:11:26.760 INFO:tasks.workunit.client.1.smithi203.stdout:2/384: dwrite d2/d1a/f27 [0,4194304] 0 2024-05-25T06:11:26.761 INFO:tasks.workunit.client.1.smithi203.stdout:3/484: creat d10/d16/d23/d4a/d88/f9f x:0 0 0 2024-05-25T06:11:26.763 INFO:tasks.workunit.client.1.smithi203.stdout:8/427: write d2/d5/d10/d11/de/f13 [2447198,110753] 0 2024-05-25T06:11:26.764 INFO:tasks.workunit.client.1.smithi203.stdout:2/385: truncate d2/d4/d10/d5f/f5b 1028689 0 2024-05-25T06:11:26.765 INFO:tasks.workunit.client.1.smithi203.stdout:3/485: symlink d10/d16/d71/la0 0 2024-05-25T06:11:26.767 INFO:tasks.workunit.client.1.smithi203.stdout:9/337: getdents d0/d17/d23 0 2024-05-25T06:11:26.771 INFO:tasks.workunit.client.1.smithi203.stdout:2/386: link d2/d4/d10/d50/f51 d2/d38/f8a 0 2024-05-25T06:11:26.773 INFO:tasks.workunit.client.1.smithi203.stdout:8/428: dwrite d2/d5/d10/d45/d5d/d68/f87 [0,4194304] 0 2024-05-25T06:11:26.775 INFO:tasks.workunit.client.1.smithi203.stdout:5/602: dread dc/d11/d17/f5d [0,4194304] 0 2024-05-25T06:11:26.776 INFO:tasks.workunit.client.1.smithi203.stdout:8/429: chown d2/d5/c23 45430465 1 2024-05-25T06:11:26.776 INFO:tasks.workunit.client.1.smithi203.stdout:2/387: creat d2/d1a/f8b x:0 0 0 2024-05-25T06:11:26.778 INFO:tasks.workunit.client.1.smithi203.stdout:5/603: dread - dc/d11/d17/d1a/d2d/d4d/d64/fa1 zero size 2024-05-25T06:11:26.780 INFO:tasks.workunit.client.1.smithi203.stdout:2/388: mkdir d2/d4/d10/d8c 0 2024-05-25T06:11:26.781 INFO:tasks.workunit.client.1.smithi203.stdout:5/604: stat dc/d11/d17/d1a/f32 0 2024-05-25T06:11:26.783 INFO:tasks.workunit.client.1.smithi203.stdout:2/389: write d2/d38/d2f/f61 [259379,59360] 0 2024-05-25T06:11:26.785 INFO:tasks.workunit.client.1.smithi203.stdout:5/605: mknod dc/d11/d17/d44/d73/d81/cd1 0 2024-05-25T06:11:26.786 INFO:tasks.workunit.client.1.smithi203.stdout:2/390: symlink d2/d4/d10/d32/l8d 0 2024-05-25T06:11:26.789 INFO:tasks.workunit.client.1.smithi203.stdout:8/430: fsync d2/d5/d28/f3a 0 2024-05-25T06:11:26.790 INFO:tasks.workunit.client.1.smithi203.stdout:5/606: dread dc/d11/d17/f5d [0,4194304] 0 2024-05-25T06:11:26.793 INFO:tasks.workunit.client.1.smithi203.stdout:8/431: creat d2/d5/d10/d45/d5d/d76/f8b x:0 0 0 2024-05-25T06:11:26.794 INFO:tasks.workunit.client.1.smithi203.stdout:8/432: chown d2/d5/d28 0 1 2024-05-25T06:11:26.799 INFO:tasks.workunit.client.1.smithi203.stdout:2/391: write d2/d4/d10/d50/f51 [5011848,115940] 0 2024-05-25T06:11:26.799 INFO:tasks.workunit.client.1.smithi203.stdout:5/607: write dc/d11/d18/f37 [4039893,6227] 0 2024-05-25T06:11:26.799 INFO:tasks.workunit.client.1.smithi203.stdout:3/486: dread d10/d16/f19 [0,4194304] 0 2024-05-25T06:11:26.800 INFO:tasks.workunit.client.1.smithi203.stdout:2/392: stat d2/d4/c3c 0 2024-05-25T06:11:26.800 INFO:tasks.workunit.client.1.smithi203.stdout:8/433: truncate d2/d5/d10/d11/de/f13 1962250 0 2024-05-25T06:11:26.802 INFO:tasks.workunit.client.1.smithi203.stdout:5/608: readlink dc/d11/d17/d1a/d2d/d4d/l4f 0 2024-05-25T06:11:26.803 INFO:tasks.workunit.client.1.smithi203.stdout:2/393: mkdir d2/d42/d8e 0 2024-05-25T06:11:26.805 INFO:tasks.workunit.client.1.smithi203.stdout:8/434: unlink d2/d5/l6d 0 2024-05-25T06:11:26.808 INFO:tasks.workunit.client.1.smithi203.stdout:7/347: sync 2024-05-25T06:11:26.808 INFO:tasks.workunit.client.1.smithi203.stdout:0/259: sync 2024-05-25T06:11:26.808 INFO:tasks.workunit.client.1.smithi203.stdout:5/609: mkdir dc/d11/d17/d1a/dd2 0 2024-05-25T06:11:26.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/435: mknod d2/d5/d28/c8c 0 2024-05-25T06:11:26.813 INFO:tasks.workunit.client.1.smithi203.stdout:3/487: dread d10/f51 [0,4194304] 0 2024-05-25T06:11:26.813 INFO:tasks.workunit.client.1.smithi203.stdout:2/394: link d2/d4/l7 d2/d4/l8f 0 2024-05-25T06:11:26.814 INFO:tasks.workunit.client.1.smithi203.stdout:2/395: chown d2/d4/l16 3115 1 2024-05-25T06:11:26.814 INFO:tasks.workunit.client.1.smithi203.stdout:3/488: truncate d10/d16/d23/d3a/f80 7829 0 2024-05-25T06:11:26.816 INFO:tasks.workunit.client.1.smithi203.stdout:8/436: unlink d2/d5/d10/d45/d5d/f80 0 2024-05-25T06:11:26.819 INFO:tasks.workunit.client.1.smithi203.stdout:0/260: rename dd/d10/f28 to dd/d45/f5b 0 2024-05-25T06:11:26.819 INFO:tasks.workunit.client.1.smithi203.stdout:2/396: dwrite d2/d1a/f27 [0,4194304] 0 2024-05-25T06:11:26.821 INFO:tasks.workunit.client.1.smithi203.stdout:2/397: write d2/d4/d10/d5f/f81 [310264,38152] 0 2024-05-25T06:11:26.826 INFO:tasks.workunit.client.1.smithi203.stdout:2/398: dread d2/d4/d10/d58/f6d [0,4194304] 0 2024-05-25T06:11:26.826 INFO:tasks.workunit.client.1.smithi203.stdout:7/348: write d1/d9/fc [2479989,100489] 0 2024-05-25T06:11:26.828 INFO:tasks.workunit.client.1.smithi203.stdout:5/610: link dc/d11/d17/d1a/d2d/f7d dc/d11/d17/d44/d73/d81/fd3 0 2024-05-25T06:11:26.830 INFO:tasks.workunit.client.1.smithi203.stdout:2/399: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:26.831 INFO:tasks.workunit.client.1.smithi203.stdout:8/437: creat d2/d5/d10/d11/d1f/d75/f8d x:0 0 0 2024-05-25T06:11:26.832 INFO:tasks.workunit.client.1.smithi203.stdout:0/261: mkdir dd/d5c 0 2024-05-25T06:11:26.833 INFO:tasks.workunit.client.1.smithi203.stdout:8/438: fdatasync d2/d5/d28/d38/f3c 0 2024-05-25T06:11:26.834 INFO:tasks.workunit.client.1.smithi203.stdout:7/349: rename d1/d9/l74 to d1/d9/d2c/d3a/d4e/l83 0 2024-05-25T06:11:26.835 INFO:tasks.workunit.client.1.smithi203.stdout:7/350: dread - d1/f4d zero size 2024-05-25T06:11:26.835 INFO:tasks.workunit.client.1.smithi203.stdout:8/439: write d2/d5/d10/d11/de/f1e [155533,122545] 0 2024-05-25T06:11:26.836 INFO:tasks.workunit.client.1.smithi203.stdout:5/611: dwrite dc/daa/fbc [0,4194304] 0 2024-05-25T06:11:26.839 INFO:tasks.workunit.client.1.smithi203.stdout:8/440: rmdir d2/d5/d10/d11/d59 39 2024-05-25T06:11:26.840 INFO:tasks.workunit.client.1.smithi203.stdout:7/351: mknod d1/d9/d60/c84 0 2024-05-25T06:11:26.841 INFO:tasks.workunit.client.1.smithi203.stdout:2/400: rmdir d2/d38/d2f/d69/d7a 0 2024-05-25T06:11:26.841 INFO:tasks.workunit.client.1.smithi203.stdout:2/401: chown d2/d1a/c20 361454 1 2024-05-25T06:11:26.842 INFO:tasks.workunit.client.1.smithi203.stdout:2/402: stat d2/d38/d2f/f31 0 2024-05-25T06:11:26.843 INFO:tasks.workunit.client.1.smithi203.stdout:2/403: write d2/d38/f6e [4277246,16222] 0 2024-05-25T06:11:26.843 INFO:tasks.workunit.client.1.smithi203.stdout:2/404: chown d2/d1a/d33/l76 9 1 2024-05-25T06:11:26.844 INFO:tasks.workunit.client.1.smithi203.stdout:5/612: rename dc/d11/d17/d44/d73/d81 to dc/d11/d17/d1a/d2d/d4d/d64/dd4 0 2024-05-25T06:11:26.846 INFO:tasks.workunit.client.1.smithi203.stdout:8/441: mknod d2/d5/d10/d11/de/c8e 0 2024-05-25T06:11:26.848 INFO:tasks.workunit.client.1.smithi203.stdout:7/352: symlink d1/d9/d2c/d36/d3b/l85 0 2024-05-25T06:11:26.849 INFO:tasks.workunit.client.1.smithi203.stdout:0/262: mkdir dd/d10/d34/d4c/d5d 0 2024-05-25T06:11:26.851 INFO:tasks.workunit.client.1.smithi203.stdout:2/405: rename d2/d38 to d2/d38/d43/d60/d6b/d90 22 2024-05-25T06:11:26.851 INFO:tasks.workunit.client.1.smithi203.stdout:5/613: mknod dc/d11/d17/d44/d73/cd5 0 2024-05-25T06:11:26.852 INFO:tasks.workunit.client.1.smithi203.stdout:7/353: symlink d1/d11/d3d/l86 0 2024-05-25T06:11:26.852 INFO:tasks.workunit.client.1.smithi203.stdout:5/614: readlink dc/d6a/l97 0 2024-05-25T06:11:26.852 INFO:tasks.workunit.client.1.smithi203.stdout:8/442: dwrite d2/d5/d10/d45/d5d/d68/f7d [0,4194304] 0 2024-05-25T06:11:26.854 INFO:tasks.workunit.client.1.smithi203.stdout:7/354: truncate d1/f4d 107530 0 2024-05-25T06:11:26.858 INFO:tasks.workunit.client.1.smithi203.stdout:5/615: creat dc/d11/d17/d1a/d2d/d53/da2/fd6 x:0 0 0 2024-05-25T06:11:26.859 INFO:tasks.workunit.client.1.smithi203.stdout:5/616: truncate dc/d11/d17/d44/f8f 600628 0 2024-05-25T06:11:26.863 INFO:tasks.workunit.client.1.smithi203.stdout:7/355: dread d1/d9/d60/f65 [0,4194304] 0 2024-05-25T06:11:26.864 INFO:tasks.workunit.client.1.smithi203.stdout:8/443: dread d2/d5/d10/f53 [0,4194304] 0 2024-05-25T06:11:26.866 INFO:tasks.workunit.client.1.smithi203.stdout:2/406: getdents d2/d4/d10/d58 0 2024-05-25T06:11:26.867 INFO:tasks.workunit.client.1.smithi203.stdout:7/356: fdatasync d1/d9/d2c/f5d 0 2024-05-25T06:11:26.869 INFO:tasks.workunit.client.1.smithi203.stdout:7/357: write d1/d9/d2c/d3a/d4e/f46 [888867,123015] 0 2024-05-25T06:11:26.870 INFO:tasks.workunit.client.1.smithi203.stdout:7/358: chown d1/d11/l13 89825794 1 2024-05-25T06:11:26.874 INFO:tasks.workunit.client.1.smithi203.stdout:0/263: truncate dd/d10/f41 3954454 0 2024-05-25T06:11:26.874 INFO:tasks.workunit.client.1.smithi203.stdout:2/407: dwrite d2/d1a/f7f [0,4194304] 0 2024-05-25T06:11:26.874 INFO:tasks.workunit.client.1.smithi203.stdout:5/617: truncate dc/d11/d17/f5d 1422275 0 2024-05-25T06:11:26.878 INFO:tasks.workunit.client.1.smithi203.stdout:2/408: mknod d2/d4/d10/d58/c91 0 2024-05-25T06:11:26.881 INFO:tasks.workunit.client.1.smithi203.stdout:7/359: dwrite d1/f4d [0,4194304] 0 2024-05-25T06:11:26.881 INFO:tasks.workunit.client.1.smithi203.stdout:7/360: chown d1/d9/d2c/d79 41055 1 2024-05-25T06:11:26.882 INFO:tasks.workunit.client.1.smithi203.stdout:7/361: stat d1/d9/f48 0 2024-05-25T06:11:26.886 INFO:tasks.workunit.client.1.smithi203.stdout:0/264: write dd/d45/f5b [1388040,57814] 0 2024-05-25T06:11:26.890 INFO:tasks.workunit.client.1.smithi203.stdout:7/362: dwrite d1/f1c [0,4194304] 0 2024-05-25T06:11:26.892 INFO:tasks.workunit.client.1.smithi203.stdout:7/363: mknod d1/c87 0 2024-05-25T06:11:26.898 INFO:tasks.workunit.client.1.smithi203.stdout:2/409: dread d2/fe [0,4194304] 0 2024-05-25T06:11:26.904 INFO:tasks.workunit.client.1.smithi203.stdout:7/364: truncate d1/d11/d3d/d4c/f63 3259431 0 2024-05-25T06:11:26.905 INFO:tasks.workunit.client.1.smithi203.stdout:5/618: dread dc/d11/d17/d1a/d2d/d4d/f7b [0,4194304] 0 2024-05-25T06:11:26.905 INFO:tasks.workunit.client.1.smithi203.stdout:2/410: dwrite d2/d4/d10/d32/f1b [4194304,4194304] 0 2024-05-25T06:11:26.906 INFO:tasks.workunit.client.1.smithi203.stdout:7/365: write d1/d9/d2c/f5d [882827,105436] 0 2024-05-25T06:11:26.911 INFO:tasks.workunit.client.1.smithi203.stdout:7/366: symlink d1/d11/d3d/l88 0 2024-05-25T06:11:26.911 INFO:tasks.workunit.client.1.smithi203.stdout:0/265: dread dd/d10/f2f [0,4194304] 0 2024-05-25T06:11:26.913 INFO:tasks.workunit.client.1.smithi203.stdout:5/619: dread dc/d11/d42/fce [0,4194304] 0 2024-05-25T06:11:26.916 INFO:tasks.workunit.client.1.smithi203.stdout:7/367: mkdir d1/d9/d2c/d62/d7a/d89 0 2024-05-25T06:11:26.918 INFO:tasks.workunit.client.1.smithi203.stdout:0/266: rename dd/c13 to dd/d3d/d59/c5e 0 2024-05-25T06:11:26.919 INFO:tasks.workunit.client.1.smithi203.stdout:2/411: truncate d2/d4/d10/d5f/f25 2647017 0 2024-05-25T06:11:26.920 INFO:tasks.workunit.client.1.smithi203.stdout:5/620: dwrite dc/d11/d42/d94/fc7 [0,4194304] 0 2024-05-25T06:11:26.922 INFO:tasks.workunit.client.1.smithi203.stdout:7/368: mknod d1/d9/d2c/d3a/d4e/d50/c8a 0 2024-05-25T06:11:26.922 INFO:tasks.workunit.client.1.smithi203.stdout:0/267: symlink dd/d3d/d59/l5f 0 2024-05-25T06:11:26.922 INFO:tasks.workunit.client.1.smithi203.stdout:2/412: stat d2/d42/c6a 0 2024-05-25T06:11:26.924 INFO:tasks.workunit.client.1.smithi203.stdout:0/268: rmdir dd/d10/d34 39 2024-05-25T06:11:26.924 INFO:tasks.workunit.client.1.smithi203.stdout:7/369: dread - d1/f78 zero size 2024-05-25T06:11:26.925 INFO:tasks.workunit.client.1.smithi203.stdout:2/413: write d2/d4/d10/d32/f28 [5813932,101038] 0 2024-05-25T06:11:26.926 INFO:tasks.workunit.client.1.smithi203.stdout:0/269: chown dd/d10/d34 370533279 1 2024-05-25T06:11:26.930 INFO:tasks.workunit.client.1.smithi203.stdout:0/270: symlink dd/d32/l60 0 2024-05-25T06:11:26.938 INFO:tasks.workunit.client.1.smithi203.stdout:7/370: truncate d1/d9/d2c/f42 3225463 0 2024-05-25T06:11:26.938 INFO:tasks.workunit.client.1.smithi203.stdout:0/271: dread dd/d3a/f3b [0,4194304] 0 2024-05-25T06:11:26.939 INFO:tasks.workunit.client.1.smithi203.stdout:7/371: write d1/d11/f30 [1396396,130975] 0 2024-05-25T06:11:26.939 INFO:tasks.workunit.client.1.smithi203.stdout:2/414: write d2/d4/d10/d5f/f25 [331014,35266] 0 2024-05-25T06:11:26.940 INFO:tasks.workunit.client.1.smithi203.stdout:2/415: stat d2/d4/l3b 0 2024-05-25T06:11:26.942 INFO:tasks.workunit.client.1.smithi203.stdout:7/372: symlink d1/d9/d2c/l8b 0 2024-05-25T06:11:26.942 INFO:tasks.workunit.client.1.smithi203.stdout:5/621: dread dc/d11/d17/d44/fbe [0,4194304] 0 2024-05-25T06:11:26.943 INFO:tasks.workunit.client.1.smithi203.stdout:5/622: chown dc/d11/d17/d1a/d2d/d53 6219 1 2024-05-25T06:11:26.944 INFO:tasks.workunit.client.1.smithi203.stdout:7/373: rmdir d1/d9/d2c/d3a/d4e/d50 39 2024-05-25T06:11:26.951 INFO:tasks.workunit.client.1.smithi203.stdout:5/623: dwrite dc/d11/d17/d1a/d2d/f65 [0,4194304] 0 2024-05-25T06:11:26.952 INFO:tasks.workunit.client.1.smithi203.stdout:7/374: dwrite d1/f38 [0,4194304] 0 2024-05-25T06:11:26.954 INFO:tasks.workunit.client.1.smithi203.stdout:5/624: symlink dc/d11/d42/d94/dad/db1/ld7 0 2024-05-25T06:11:26.956 INFO:tasks.workunit.client.1.smithi203.stdout:7/375: rename d1/d9/d2c/d3a/d4e/f46 to d1/d9/d2c/d62/d7a/d89/f8c 0 2024-05-25T06:11:26.957 INFO:tasks.workunit.client.1.smithi203.stdout:5/625: creat dc/d3c/d3f/fd8 x:0 0 0 2024-05-25T06:11:26.960 INFO:tasks.workunit.client.1.smithi203.stdout:7/376: creat d1/d9/d2c/d36/d66/f8d x:0 0 0 2024-05-25T06:11:26.961 INFO:tasks.workunit.client.1.smithi203.stdout:5/626: chown dc/d11/d42/lbd 1090097 1 2024-05-25T06:11:26.967 INFO:tasks.workunit.client.1.smithi203.stdout:5/627: dwrite dc/d11/d17/fba [0,4194304] 0 2024-05-25T06:11:26.972 INFO:tasks.workunit.client.1.smithi203.stdout:5/628: dread dc/d11/d42/d94/fc7 [0,4194304] 0 2024-05-25T06:11:26.974 INFO:tasks.workunit.client.1.smithi203.stdout:5/629: rmdir dc/d11/d17/d1a/d2d 39 2024-05-25T06:11:26.976 INFO:tasks.workunit.client.1.smithi203.stdout:5/630: creat dc/d11/d17/d44/d73/fd9 x:0 0 0 2024-05-25T06:11:26.978 INFO:tasks.workunit.client.1.smithi203.stdout:5/631: mkdir dc/d11/d17/d1a/dda 0 2024-05-25T06:11:26.979 INFO:tasks.workunit.client.1.smithi203.stdout:5/632: write dc/d11/d17/d1a/f2e [11049618,1014] 0 2024-05-25T06:11:26.980 INFO:tasks.workunit.client.1.smithi203.stdout:7/377: rmdir d1/d9/d2c/d3a 39 2024-05-25T06:11:26.981 INFO:tasks.workunit.client.1.smithi203.stdout:7/378: write d1/f4d [5241482,99470] 0 2024-05-25T06:11:26.982 INFO:tasks.workunit.client.1.smithi203.stdout:5/633: fsync dc/fe 0 2024-05-25T06:11:26.986 INFO:tasks.workunit.client.1.smithi203.stdout:7/379: dread d1/d9/d2c/d62/d7a/d89/f8c [0,4194304] 0 2024-05-25T06:11:26.987 INFO:tasks.workunit.client.1.smithi203.stdout:5/634: truncate f6 1725187 0 2024-05-25T06:11:26.989 INFO:tasks.workunit.client.1.smithi203.stdout:7/380: mknod d1/d9/d2c/d3a/d4e/c8e 0 2024-05-25T06:11:26.990 INFO:tasks.workunit.client.1.smithi203.stdout:5/635: getdents dc 0 2024-05-25T06:11:26.990 INFO:tasks.workunit.client.1.smithi203.stdout:7/381: readlink d1/d9/l16 0 2024-05-25T06:11:26.992 INFO:tasks.workunit.client.1.smithi203.stdout:7/382: creat d1/d9/d2c/d36/d54/f8f x:0 0 0 2024-05-25T06:11:26.997 INFO:tasks.workunit.client.1.smithi203.stdout:5/636: dread dc/d11/d17/d1a/d2d/f7d [0,4194304] 0 2024-05-25T06:11:26.998 INFO:tasks.workunit.client.1.smithi203.stdout:5/637: chown dc/d11/d17/d1a/d2d 167183 1 2024-05-25T06:11:26.998 INFO:tasks.workunit.client.1.smithi203.stdout:5/638: stat dc/d3c/d3f/fae 0 2024-05-25T06:11:27.008 INFO:tasks.workunit.client.1.smithi203.stdout:7/383: dread d1/d9/f2e [0,4194304] 0 2024-05-25T06:11:27.008 INFO:tasks.workunit.client.1.smithi203.stdout:5/639: dwrite dc/d11/d17/f5d [0,4194304] 0 2024-05-25T06:11:27.011 INFO:tasks.workunit.client.1.smithi203.stdout:7/384: rename d1/l73 to d1/d9/d2c/d62/d7a/l90 0 2024-05-25T06:11:27.013 INFO:tasks.workunit.client.1.smithi203.stdout:7/385: chown d1/d9/d2c/d3a/f5c 5865934 1 2024-05-25T06:11:27.013 INFO:tasks.workunit.client.1.smithi203.stdout:7/386: fdatasync d1/f4d 0 2024-05-25T06:11:27.013 INFO:tasks.workunit.client.1.smithi203.stdout:7/387: stat d1/l52 0 2024-05-25T06:11:27.016 INFO:tasks.workunit.client.1.smithi203.stdout:7/388: dread d1/d9/fc [0,4194304] 0 2024-05-25T06:11:27.021 INFO:tasks.workunit.client.1.smithi203.stdout:7/389: dwrite d1/d9/f29 [0,4194304] 0 2024-05-25T06:11:27.040 INFO:tasks.workunit.client.1.smithi203.stdout:4/283: sync 2024-05-25T06:11:27.041 INFO:tasks.workunit.client.1.smithi203.stdout:4/284: dread - d0/d8/d3a/f46 zero size 2024-05-25T06:11:27.042 INFO:tasks.workunit.client.1.smithi203.stdout:4/285: creat d0/d9/d24/d27/f51 x:0 0 0 2024-05-25T06:11:27.043 INFO:tasks.workunit.client.1.smithi203.stdout:4/286: fsync d0/d9/d24/f40 0 2024-05-25T06:11:27.048 INFO:tasks.workunit.client.1.smithi203.stdout:4/287: dwrite d0/d9/d24/d27/f51 [0,4194304] 0 2024-05-25T06:11:27.050 INFO:tasks.workunit.client.1.smithi203.stdout:4/288: mkdir d0/d8/d52 0 2024-05-25T06:11:27.058 INFO:tasks.workunit.client.1.smithi203.stdout:4/289: dwrite d0/d8/d1f/f2b [4194304,4194304] 0 2024-05-25T06:11:27.064 INFO:tasks.workunit.client.1.smithi203.stdout:4/290: dwrite d0/f16 [0,4194304] 0 2024-05-25T06:11:27.066 INFO:tasks.workunit.client.1.smithi203.stdout:4/291: write d0/d9/d11/f3f [4578586,90234] 0 2024-05-25T06:11:27.068 INFO:tasks.workunit.client.1.smithi203.stdout:4/292: read d0/d8/d1f/f2b [6108464,117198] 0 2024-05-25T06:11:27.073 INFO:tasks.workunit.client.1.smithi203.stdout:4/293: creat d0/d8/d3a/f53 x:0 0 0 2024-05-25T06:11:27.171 INFO:tasks.workunit.client.1.smithi203.stdout:6/498: sync 2024-05-25T06:11:27.171 INFO:tasks.workunit.client.1.smithi203.stdout:1/355: sync 2024-05-25T06:11:27.172 INFO:tasks.workunit.client.1.smithi203.stdout:6/499: readlink d8/d11/d15/l33 0 2024-05-25T06:11:27.173 INFO:tasks.workunit.client.1.smithi203.stdout:6/500: chown d8/d5a/d8b/f96 1202184 1 2024-05-25T06:11:27.175 INFO:tasks.workunit.client.1.smithi203.stdout:6/501: write d8/d11/d15/d28/d52/d43/f84 [1513686,106307] 0 2024-05-25T06:11:27.182 INFO:tasks.workunit.client.1.smithi203.stdout:1/356: dread d14/f17 [0,4194304] 0 2024-05-25T06:11:27.184 INFO:tasks.workunit.client.1.smithi203.stdout:1/357: mkdir d14/d7c 0 2024-05-25T06:11:27.186 INFO:tasks.workunit.client.1.smithi203.stdout:1/358: creat d14/d7c/f7d x:0 0 0 2024-05-25T06:11:27.199 INFO:tasks.workunit.client.1.smithi203.stdout:6/502: dread d8/fd [4194304,4194304] 0 2024-05-25T06:11:27.200 INFO:tasks.workunit.client.1.smithi203.stdout:6/503: chown d8/d11/d47/c89 5 1 2024-05-25T06:11:27.214 INFO:tasks.workunit.client.1.smithi203.stdout:6/504: dread d8/ff [0,4194304] 0 2024-05-25T06:11:27.216 INFO:tasks.workunit.client.1.smithi203.stdout:6/505: mkdir d8/d5a/d9b 0 2024-05-25T06:11:27.220 INFO:tasks.workunit.client.1.smithi203.stdout:6/506: link d8/d29/f4b d8/d29/d46/d88/f9c 0 2024-05-25T06:11:27.222 INFO:tasks.workunit.client.1.smithi203.stdout:6/507: symlink d8/d11/d47/d60/l9d 0 2024-05-25T06:11:27.242 INFO:tasks.workunit.client.1.smithi203.stdout:6/508: truncate d8/d11/d15/f2e 2880501 0 2024-05-25T06:11:27.243 INFO:tasks.workunit.client.1.smithi203.stdout:6/509: write d8/d11/d15/f79 [463514,122095] 0 2024-05-25T06:11:27.246 INFO:tasks.workunit.client.1.smithi203.stdout:6/510: getdents d8/d70 0 2024-05-25T06:11:27.260 INFO:tasks.workunit.client.1.smithi203.stdout:6/511: dread d8/f9 [0,4194304] 0 2024-05-25T06:11:27.260 INFO:tasks.workunit.client.1.smithi203.stdout:6/512: chown d8/d11/d15/f79 199243 1 2024-05-25T06:11:27.261 INFO:tasks.workunit.client.1.smithi203.stdout:6/513: write d8/d11/d15/f79 [1178568,36459] 0 2024-05-25T06:11:27.262 INFO:tasks.workunit.client.1.smithi203.stdout:6/514: fdatasync d8/d11/f90 0 2024-05-25T06:11:27.264 INFO:tasks.workunit.client.1.smithi203.stdout:6/515: creat d8/d29/d6e/f9e x:0 0 0 2024-05-25T06:11:27.264 INFO:tasks.workunit.client.1.smithi203.stdout:6/516: stat d8/d2a/d6c/d87 0 2024-05-25T06:11:27.266 INFO:tasks.workunit.client.1.smithi203.stdout:6/517: getdents d8/d5a 0 2024-05-25T06:11:27.267 INFO:tasks.workunit.client.1.smithi203.stdout:6/518: read - d8/d29/d6e/f9e zero size 2024-05-25T06:11:27.269 INFO:tasks.workunit.client.1.smithi203.stdout:6/519: getdents d8/d29/d46 0 2024-05-25T06:11:27.270 INFO:tasks.workunit.client.1.smithi203.stdout:6/520: mknod d8/d11/d15/d28/d52/d43/c9f 0 2024-05-25T06:11:27.275 INFO:tasks.workunit.client.1.smithi203.stdout:6/521: dwrite d8/d29/f4b [0,4194304] 0 2024-05-25T06:11:27.285 INFO:tasks.workunit.client.1.smithi203.stdout:2/416: sync 2024-05-25T06:11:27.285 INFO:tasks.workunit.client.1.smithi203.stdout:3/489: sync 2024-05-25T06:11:27.285 INFO:tasks.workunit.client.1.smithi203.stdout:0/272: sync 2024-05-25T06:11:27.286 INFO:tasks.workunit.client.1.smithi203.stdout:9/338: sync 2024-05-25T06:11:27.286 INFO:tasks.workunit.client.1.smithi203.stdout:8/444: sync 2024-05-25T06:11:27.286 INFO:tasks.workunit.client.1.smithi203.stdout:7/390: sync 2024-05-25T06:11:27.286 INFO:tasks.workunit.client.1.smithi203.stdout:5/640: sync 2024-05-25T06:11:27.286 INFO:tasks.workunit.client.1.smithi203.stdout:4/294: sync 2024-05-25T06:11:27.286 INFO:tasks.workunit.client.1.smithi203.stdout:1/359: sync 2024-05-25T06:11:27.288 INFO:tasks.workunit.client.1.smithi203.stdout:3/490: chown d10/d16/d23/l37 493 1 2024-05-25T06:11:27.288 INFO:tasks.workunit.client.1.smithi203.stdout:4/295: mkdir d0/d9/d11/d54 0 2024-05-25T06:11:27.289 INFO:tasks.workunit.client.1.smithi203.stdout:2/417: creat d2/d42/d4e/f92 x:0 0 0 2024-05-25T06:11:27.289 INFO:tasks.workunit.client.1.smithi203.stdout:1/360: mkdir d14/d57/d7e 0 2024-05-25T06:11:27.290 INFO:tasks.workunit.client.1.smithi203.stdout:0/273: rename dd/lf to dd/d3d/l61 0 2024-05-25T06:11:27.291 INFO:tasks.workunit.client.1.smithi203.stdout:5/641: mkdir dc/d11/d17/d1a/d75/ddb 0 2024-05-25T06:11:27.292 INFO:tasks.workunit.client.1.smithi203.stdout:1/361: write d14/d37/f7a [56643,91557] 0 2024-05-25T06:11:27.293 INFO:tasks.workunit.client.1.smithi203.stdout:2/418: truncate d2/d1a/f8b 301586 0 2024-05-25T06:11:27.295 INFO:tasks.workunit.client.1.smithi203.stdout:8/445: mkdir d2/d5/d28/d38/d81/d84/d8a/d8f 0 2024-05-25T06:11:27.295 INFO:tasks.workunit.client.1.smithi203.stdout:4/296: symlink d0/d8/d52/l55 0 2024-05-25T06:11:27.298 INFO:tasks.workunit.client.1.smithi203.stdout:7/391: dwrite d1/d9/d2c/d3a/f3e [0,4194304] 0 2024-05-25T06:11:27.298 INFO:tasks.workunit.client.1.smithi203.stdout:9/339: rename d0/df/f14 to d0/df/d2d/f72 0 2024-05-25T06:11:27.299 INFO:tasks.workunit.client.1.smithi203.stdout:4/297: write d0/d8/f23 [3319257,62732] 0 2024-05-25T06:11:27.302 INFO:tasks.workunit.client.1.smithi203.stdout:4/298: chown d0/d8/d1f/d26/d43/f4d 24 1 2024-05-25T06:11:27.302 INFO:tasks.workunit.client.1.smithi203.stdout:4/299: stat d0/d9/d11/d54 0 2024-05-25T06:11:27.302 INFO:tasks.workunit.client.1.smithi203.stdout:2/419: mknod d2/d4/c93 0 2024-05-25T06:11:27.302 INFO:tasks.workunit.client.1.smithi203.stdout:3/491: dread d10/d16/d23/d4a/d88/f95 [0,4194304] 0 2024-05-25T06:11:27.305 INFO:tasks.workunit.client.1.smithi203.stdout:3/492: write d10/d16/d23/d3a/f55 [1447974,87328] 0 2024-05-25T06:11:27.305 INFO:tasks.workunit.client.1.smithi203.stdout:1/362: mknod d14/d37/d5e/c7f 0 2024-05-25T06:11:27.305 INFO:tasks.workunit.client.1.smithi203.stdout:5/642: dread dc/d11/d17/d1a/f32 [0,4194304] 0 2024-05-25T06:11:27.308 INFO:tasks.workunit.client.1.smithi203.stdout:1/363: chown d14/l21 4022 1 2024-05-25T06:11:27.308 INFO:tasks.workunit.client.1.smithi203.stdout:9/340: chown d0/df/f1d 0 1 2024-05-25T06:11:27.310 INFO:tasks.workunit.client.1.smithi203.stdout:5/643: readlink dc/d11/d18/l88 0 2024-05-25T06:11:27.310 INFO:tasks.workunit.client.1.smithi203.stdout:1/364: chown d14/f18 575 1 2024-05-25T06:11:27.311 INFO:tasks.workunit.client.1.smithi203.stdout:2/420: mknod d2/d4/d10/d50/c94 0 2024-05-25T06:11:27.311 INFO:tasks.workunit.client.1.smithi203.stdout:4/300: dread d0/f16 [0,4194304] 0 2024-05-25T06:11:27.314 INFO:tasks.workunit.client.1.smithi203.stdout:2/421: fsync d2/d38/d2f/f61 0 2024-05-25T06:11:27.316 INFO:tasks.workunit.client.1.smithi203.stdout:9/341: rename d0/df/d2d/d45 to d0/df/d73 0 2024-05-25T06:11:27.316 INFO:tasks.workunit.client.1.smithi203.stdout:4/301: chown d0/d8/l35 906708381 1 2024-05-25T06:11:27.318 INFO:tasks.workunit.client.1.smithi203.stdout:4/302: fdatasync d0/f5 0 2024-05-25T06:11:27.320 INFO:tasks.workunit.client.1.smithi203.stdout:5/644: creat dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdc x:0 0 0 2024-05-25T06:11:27.327 INFO:tasks.workunit.client.1.smithi203.stdout:8/446: dwrite d2/d5/f44 [0,4194304] 0 2024-05-25T06:11:27.329 INFO:tasks.workunit.client.1.smithi203.stdout:1/365: dwrite d14/d37/f7a [0,4194304] 0 2024-05-25T06:11:27.329 INFO:tasks.workunit.client.1.smithi203.stdout:7/392: dread f0 [0,4194304] 0 2024-05-25T06:11:27.330 INFO:tasks.workunit.client.1.smithi203.stdout:4/303: rename d0/d9/f12 to d0/d8/d3a/f56 0 2024-05-25T06:11:27.331 INFO:tasks.workunit.client.1.smithi203.stdout:0/274: dread dd/f1f [0,4194304] 0 2024-05-25T06:11:27.333 INFO:tasks.workunit.client.1.smithi203.stdout:8/447: chown d2/d5/d28/f7a 1071653140 1 2024-05-25T06:11:27.334 INFO:tasks.workunit.client.1.smithi203.stdout:5/645: creat dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdd x:0 0 0 2024-05-25T06:11:27.338 INFO:tasks.workunit.client.1.smithi203.stdout:3/493: dread d10/d22/d2a/f4d [0,4194304] 0 2024-05-25T06:11:27.339 INFO:tasks.workunit.client.1.smithi203.stdout:8/448: write d2/d5/d18/d31/f6f [1169354,74320] 0 2024-05-25T06:11:27.339 INFO:tasks.workunit.client.1.smithi203.stdout:2/422: dwrite d2/d4/d10/d32/f28 [4194304,4194304] 0 2024-05-25T06:11:27.340 INFO:tasks.workunit.client.1.smithi203.stdout:9/342: truncate d0/df/d2b/f2f 2409216 0 2024-05-25T06:11:27.342 INFO:tasks.workunit.client.1.smithi203.stdout:0/275: mkdir dd/d3d/d4f/d62 0 2024-05-25T06:11:27.343 INFO:tasks.workunit.client.1.smithi203.stdout:4/304: chown d0/d8/d3a/f56 8 1 2024-05-25T06:11:27.343 INFO:tasks.workunit.client.1.smithi203.stdout:9/343: fsync d0/df/d37/f58 0 2024-05-25T06:11:27.343 INFO:tasks.workunit.client.1.smithi203.stdout:3/494: write d10/d16/d23/d46/f79 [2886286,13506] 0 2024-05-25T06:11:27.349 INFO:tasks.workunit.client.1.smithi203.stdout:9/344: chown d0/d17/d20/d3c 56 1 2024-05-25T06:11:27.351 INFO:tasks.workunit.client.1.smithi203.stdout:4/305: dread - d0/d8/d1f/d26/f2a zero size 2024-05-25T06:11:27.352 INFO:tasks.workunit.client.1.smithi203.stdout:3/495: rmdir d10/d16/d23/d67 39 2024-05-25T06:11:27.355 INFO:tasks.workunit.client.1.smithi203.stdout:7/393: dwrite d1/d9/f2e [4194304,4194304] 0 2024-05-25T06:11:27.357 INFO:tasks.workunit.client.1.smithi203.stdout:0/276: mkdir dd/d5c/d63 0 2024-05-25T06:11:27.358 INFO:tasks.workunit.client.1.smithi203.stdout:9/345: mkdir d0/d74 0 2024-05-25T06:11:27.362 INFO:tasks.workunit.client.1.smithi203.stdout:9/346: dread d0/d17/d20/f46 [4194304,4194304] 0 2024-05-25T06:11:27.363 INFO:tasks.workunit.client.1.smithi203.stdout:3/496: rmdir d10/d16/d23/d6c 39 2024-05-25T06:11:27.364 INFO:tasks.workunit.client.1.smithi203.stdout:0/277: dread dd/fe [0,4194304] 0 2024-05-25T06:11:27.365 INFO:tasks.workunit.client.1.smithi203.stdout:0/278: fsync dd/d10/f51 0 2024-05-25T06:11:27.366 INFO:tasks.workunit.client.1.smithi203.stdout:1/366: dread d14/d57/d6b/f4f [0,4194304] 0 2024-05-25T06:11:27.368 INFO:tasks.workunit.client.1.smithi203.stdout:4/306: getdents d0/d8/d10 0 2024-05-25T06:11:27.369 INFO:tasks.workunit.client.1.smithi203.stdout:2/423: dwrite d2/d4/d10/d5f/f85 [0,4194304] 0 2024-05-25T06:11:27.370 INFO:tasks.workunit.client.1.smithi203.stdout:7/394: truncate d1/d11/d20/f28 3346861 0 2024-05-25T06:11:27.371 INFO:tasks.workunit.client.1.smithi203.stdout:2/424: chown d2/d1a/d33/c3f 1669755774 1 2024-05-25T06:11:27.371 INFO:tasks.workunit.client.1.smithi203.stdout:1/367: creat d14/d57/d29/f80 x:0 0 0 2024-05-25T06:11:27.372 INFO:tasks.workunit.client.1.smithi203.stdout:3/497: mkdir d10/d16/da1 0 2024-05-25T06:11:27.375 INFO:tasks.workunit.client.1.smithi203.stdout:7/395: symlink d1/d9/d60/l91 0 2024-05-25T06:11:27.378 INFO:tasks.workunit.client.1.smithi203.stdout:2/425: write d2/d38/f7d [2114778,114535] 0 2024-05-25T06:11:27.378 INFO:tasks.workunit.client.1.smithi203.stdout:0/279: truncate dd/f15 343071 0 2024-05-25T06:11:27.381 INFO:tasks.workunit.client.1.smithi203.stdout:7/396: symlink d1/d9/l92 0 2024-05-25T06:11:27.383 INFO:tasks.workunit.client.1.smithi203.stdout:8/449: dread d2/d5/d28/d38/d4e/f74 [0,4194304] 0 2024-05-25T06:11:27.383 INFO:tasks.workunit.client.1.smithi203.stdout:2/426: mkdir d2/d1a/d95 0 2024-05-25T06:11:27.383 INFO:tasks.workunit.client.1.smithi203.stdout:3/498: dread d10/d16/d23/d45/f4e [0,4194304] 0 2024-05-25T06:11:27.385 INFO:tasks.workunit.client.1.smithi203.stdout:4/307: dwrite d0/d8/d1f/d26/f1e [8388608,4194304] 0 2024-05-25T06:11:27.385 INFO:tasks.workunit.client.1.smithi203.stdout:0/280: dread - dd/d32/f38 zero size 2024-05-25T06:11:27.393 INFO:tasks.workunit.client.1.smithi203.stdout:1/368: dwrite d14/d37/d5e/f5b [0,4194304] 0 2024-05-25T06:11:27.393 INFO:tasks.workunit.client.1.smithi203.stdout:4/308: rmdir d0/d9 39 2024-05-25T06:11:27.394 INFO:tasks.workunit.client.1.smithi203.stdout:8/450: rmdir d2/d5/d10/d11/d1f/d75 39 2024-05-25T06:11:27.399 INFO:tasks.workunit.client.1.smithi203.stdout:0/281: creat dd/f64 x:0 0 0 2024-05-25T06:11:27.399 INFO:tasks.workunit.client.1.smithi203.stdout:7/397: dwrite d1/f32 [0,4194304] 0 2024-05-25T06:11:27.400 INFO:tasks.workunit.client.1.smithi203.stdout:2/427: rename d2/d4/d10/d58/f64 to d2/d38/d43/d60/f96 0 2024-05-25T06:11:27.403 INFO:tasks.workunit.client.1.smithi203.stdout:9/347: dread d0/f16 [4194304,4194304] 0 2024-05-25T06:11:27.404 INFO:tasks.workunit.client.1.smithi203.stdout:8/451: mkdir d2/d5/d10/d45/d90 0 2024-05-25T06:11:27.405 INFO:tasks.workunit.client.1.smithi203.stdout:2/428: chown d2/d4/d10/d32/f3d 58963 1 2024-05-25T06:11:27.405 INFO:tasks.workunit.client.1.smithi203.stdout:7/398: read d1/d9/d2c/d3a/f3e [317747,41932] 0 2024-05-25T06:11:27.406 INFO:tasks.workunit.client.1.smithi203.stdout:0/282: rename dd/d10/c49 to dd/d3d/d4f/c65 0 2024-05-25T06:11:27.407 INFO:tasks.workunit.client.1.smithi203.stdout:9/348: mkdir d0/d17/d20/d6e/d75 0 2024-05-25T06:11:27.407 INFO:tasks.workunit.client.1.smithi203.stdout:3/499: dwrite d10/d16/d50/f64 [4194304,4194304] 0 2024-05-25T06:11:27.410 INFO:tasks.workunit.client.1.smithi203.stdout:0/283: mkdir dd/d3a/d66 0 2024-05-25T06:11:27.413 INFO:tasks.workunit.client.1.smithi203.stdout:2/429: creat d2/d1a/f97 x:0 0 0 2024-05-25T06:11:27.413 INFO:tasks.workunit.client.1.smithi203.stdout:4/309: write d0/d8/d1f/d26/f2a [52402,11975] 0 2024-05-25T06:11:27.415 INFO:tasks.workunit.client.1.smithi203.stdout:0/284: chown dd/d32/d37 19301 1 2024-05-25T06:11:27.415 INFO:tasks.workunit.client.1.smithi203.stdout:8/452: symlink d2/d5/d28/d38/d81/d84/d8a/d8f/l91 0 2024-05-25T06:11:27.415 INFO:tasks.workunit.client.1.smithi203.stdout:6/522: sync 2024-05-25T06:11:27.415 INFO:tasks.workunit.client.1.smithi203.stdout:5/646: sync 2024-05-25T06:11:27.416 INFO:tasks.workunit.client.1.smithi203.stdout:6/523: chown d8/d2a/d63 1 1 2024-05-25T06:11:27.418 INFO:tasks.workunit.client.1.smithi203.stdout:4/310: symlink d0/d8/d10/l57 0 2024-05-25T06:11:27.418 INFO:tasks.workunit.client.1.smithi203.stdout:8/453: fsync d2/d5/d10/d11/de/f1d 0 2024-05-25T06:11:27.419 INFO:tasks.workunit.client.1.smithi203.stdout:3/500: creat d10/fa2 x:0 0 0 2024-05-25T06:11:27.420 INFO:tasks.workunit.client.1.smithi203.stdout:0/285: creat dd/d10/d34/f67 x:0 0 0 2024-05-25T06:11:27.421 INFO:tasks.workunit.client.1.smithi203.stdout:3/501: fdatasync d10/d22/d2a/f81 0 2024-05-25T06:11:27.421 INFO:tasks.workunit.client.1.smithi203.stdout:8/454: read - d2/d5/d10/d11/d1f/f72 zero size 2024-05-25T06:11:27.421 INFO:tasks.workunit.client.1.smithi203.stdout:6/524: creat d8/fa0 x:0 0 0 2024-05-25T06:11:27.422 INFO:tasks.workunit.client.1.smithi203.stdout:4/311: unlink d0/d8/d1f/d26/d3b/l4f 0 2024-05-25T06:11:27.423 INFO:tasks.workunit.client.1.smithi203.stdout:0/286: creat dd/d5c/f68 x:0 0 0 2024-05-25T06:11:27.427 INFO:tasks.workunit.client.1.smithi203.stdout:4/312: creat d0/d8/d10/f58 x:0 0 0 2024-05-25T06:11:27.428 INFO:tasks.workunit.client.1.smithi203.stdout:4/313: write d0/d8/f44 [962883,101372] 0 2024-05-25T06:11:27.429 INFO:tasks.workunit.client.1.smithi203.stdout:6/525: truncate f2 1494050 0 2024-05-25T06:11:27.431 INFO:tasks.workunit.client.1.smithi203.stdout:3/502: rename d10/l36 to d10/d16/d23/d3a/la3 0 2024-05-25T06:11:27.432 INFO:tasks.workunit.client.1.smithi203.stdout:0/287: creat dd/d3a/d66/f69 x:0 0 0 2024-05-25T06:11:27.432 INFO:tasks.workunit.client.1.smithi203.stdout:3/503: dread - d10/d16/d23/d4a/d88/f9f zero size 2024-05-25T06:11:27.434 INFO:tasks.workunit.client.1.smithi203.stdout:6/526: creat d8/d70/fa1 x:0 0 0 2024-05-25T06:11:27.434 INFO:tasks.workunit.client.1.smithi203.stdout:7/399: truncate d1/f3 6987490 0 2024-05-25T06:11:27.436 INFO:tasks.workunit.client.1.smithi203.stdout:3/504: write d10/d16/d23/d45/f76 [1087393,116779] 0 2024-05-25T06:11:27.436 INFO:tasks.workunit.client.1.smithi203.stdout:7/400: chown d1/d9/d2c/f56 44 1 2024-05-25T06:11:27.437 INFO:tasks.workunit.client.1.smithi203.stdout:6/527: mknod d8/d2a/d6c/ca2 0 2024-05-25T06:11:27.439 INFO:tasks.workunit.client.1.smithi203.stdout:0/288: creat dd/d3d/d4f/d62/f6a x:0 0 0 2024-05-25T06:11:27.443 INFO:tasks.workunit.client.1.smithi203.stdout:7/401: truncate d1/d9/f76 1459248 0 2024-05-25T06:11:27.445 INFO:tasks.workunit.client.1.smithi203.stdout:6/528: mknod d8/d29/d80/ca3 0 2024-05-25T06:11:27.445 INFO:tasks.workunit.client.1.smithi203.stdout:5/647: dread dc/d11/d17/d1a/d72/f8c [0,4194304] 0 2024-05-25T06:11:27.447 INFO:tasks.workunit.client.1.smithi203.stdout:3/505: mknod d10/d16/d23/d4a/d88/ca4 0 2024-05-25T06:11:27.449 INFO:tasks.workunit.client.1.smithi203.stdout:7/402: chown d1/d9/f48 1009807770 1 2024-05-25T06:11:27.449 INFO:tasks.workunit.client.1.smithi203.stdout:2/430: dwrite d2/d4/d10/f21 [0,4194304] 0 2024-05-25T06:11:27.452 INFO:tasks.workunit.client.1.smithi203.stdout:4/314: dwrite d0/d9/d24/d27/f30 [4194304,4194304] 0 2024-05-25T06:11:27.452 INFO:tasks.workunit.client.1.smithi203.stdout:3/506: chown f1 1 1 2024-05-25T06:11:27.454 INFO:tasks.workunit.client.1.smithi203.stdout:6/529: creat d8/d29/d46/fa4 x:0 0 0 2024-05-25T06:11:27.455 INFO:tasks.workunit.client.1.smithi203.stdout:5/648: write dc/d11/d17/d44/d73/d98/fb3 [475352,117831] 0 2024-05-25T06:11:27.455 INFO:tasks.workunit.client.1.smithi203.stdout:8/455: dwrite d2/d5/fc [0,4194304] 0 2024-05-25T06:11:27.460 INFO:tasks.workunit.client.1.smithi203.stdout:4/315: mkdir d0/d8/d52/d59 0 2024-05-25T06:11:27.461 INFO:tasks.workunit.client.1.smithi203.stdout:6/530: mkdir d8/d29/d46/d88/da5 0 2024-05-25T06:11:27.464 INFO:tasks.workunit.client.1.smithi203.stdout:5/649: write dc/d11/d17/f5d [4405830,122458] 0 2024-05-25T06:11:27.467 INFO:tasks.workunit.client.1.smithi203.stdout:4/316: creat d0/d8/d1f/d26/d3b/f5a x:0 0 0 2024-05-25T06:11:27.468 INFO:tasks.workunit.client.1.smithi203.stdout:7/403: dwrite d1/d9/d2c/f56 [0,4194304] 0 2024-05-25T06:11:27.469 INFO:tasks.workunit.client.1.smithi203.stdout:8/456: symlink d2/d5/d10/d11/l92 0 2024-05-25T06:11:27.470 INFO:tasks.workunit.client.1.smithi203.stdout:7/404: read d1/d9/f2e [1073745,74708] 0 2024-05-25T06:11:27.470 INFO:tasks.workunit.client.1.smithi203.stdout:3/507: link d10/d16/d23/d3a/c8c d10/d16/d23/d4a/ca5 0 2024-05-25T06:11:27.471 INFO:tasks.workunit.client.1.smithi203.stdout:4/317: truncate d0/d8/d1f/d26/f4a 729252 0 2024-05-25T06:11:27.473 INFO:tasks.workunit.client.1.smithi203.stdout:7/405: chown d1/d11/d20/f23 2021623549 1 2024-05-25T06:11:27.473 INFO:tasks.workunit.client.1.smithi203.stdout:4/318: read d0/d8/d1f/d26/f20 [410050,1831] 0 2024-05-25T06:11:27.473 INFO:tasks.workunit.client.1.smithi203.stdout:8/457: fsync d2/d5/d18/d31/f39 0 2024-05-25T06:11:27.476 INFO:tasks.workunit.client.1.smithi203.stdout:5/650: link dc/d11/f12 dc/d11/d17/d1a/d22/fde 0 2024-05-25T06:11:27.477 INFO:tasks.workunit.client.1.smithi203.stdout:6/531: link d8/d11/d15/d28/c1c d8/d2a/ca6 0 2024-05-25T06:11:27.478 INFO:tasks.workunit.client.1.smithi203.stdout:3/508: mknod d10/d16/d50/d8b/ca6 0 2024-05-25T06:11:27.479 INFO:tasks.workunit.client.1.smithi203.stdout:8/458: dread d2/d5/d10/d11/d1f/f35 [0,4194304] 0 2024-05-25T06:11:27.480 INFO:tasks.workunit.client.1.smithi203.stdout:4/319: creat d0/d9/d11/f5b x:0 0 0 2024-05-25T06:11:27.481 INFO:tasks.workunit.client.1.smithi203.stdout:7/406: creat d1/d11/d3d/d4c/f93 x:0 0 0 2024-05-25T06:11:27.484 INFO:tasks.workunit.client.1.smithi203.stdout:0/289: dread dd/d3a/f46 [0,4194304] 0 2024-05-25T06:11:27.485 INFO:tasks.workunit.client.1.smithi203.stdout:0/290: chown dd 1 1 2024-05-25T06:11:27.485 INFO:tasks.workunit.client.1.smithi203.stdout:3/509: rmdir d10/d22 39 2024-05-25T06:11:27.487 INFO:tasks.workunit.client.1.smithi203.stdout:6/532: symlink d8/d5a/d9b/la7 0 2024-05-25T06:11:27.488 INFO:tasks.workunit.client.1.smithi203.stdout:4/320: getdents d0/d8/d52/d59 0 2024-05-25T06:11:27.489 INFO:tasks.workunit.client.1.smithi203.stdout:8/459: dwrite d2/d5/d10/f4d [8388608,4194304] 0 2024-05-25T06:11:27.490 INFO:tasks.workunit.client.1.smithi203.stdout:0/291: symlink dd/d3d/l6b 0 2024-05-25T06:11:27.491 INFO:tasks.workunit.client.1.smithi203.stdout:6/533: write d8/d3d/f9a [1027442,53807] 0 2024-05-25T06:11:27.492 INFO:tasks.workunit.client.1.smithi203.stdout:7/407: mknod d1/c94 0 2024-05-25T06:11:27.493 INFO:tasks.workunit.client.1.smithi203.stdout:3/510: creat d10/d16/d23/d4a/d88/fa7 x:0 0 0 2024-05-25T06:11:27.495 INFO:tasks.workunit.client.1.smithi203.stdout:0/292: write dd/d32/d37/f42 [2687818,10026] 0 2024-05-25T06:11:27.495 INFO:tasks.workunit.client.1.smithi203.stdout:3/511: chown d10/f20 0 1 2024-05-25T06:11:27.499 INFO:tasks.workunit.client.1.smithi203.stdout:5/651: dwrite dc/d11/f12 [0,4194304] 0 2024-05-25T06:11:27.502 INFO:tasks.workunit.client.1.smithi203.stdout:6/534: truncate d8/d5a/d8b/f96 2588345 0 2024-05-25T06:11:27.502 INFO:tasks.workunit.client.1.smithi203.stdout:8/460: mkdir d2/d5/d28/d38/d81/d84/d88/d93 0 2024-05-25T06:11:27.505 INFO:tasks.workunit.client.1.smithi203.stdout:0/293: rename dd/d10/l39 to dd/d3a/l6c 0 2024-05-25T06:11:27.507 INFO:tasks.workunit.client.1.smithi203.stdout:5/652: chown dc/d11/d17/d1a/l89 877209145 1 2024-05-25T06:11:27.508 INFO:tasks.workunit.client.1.smithi203.stdout:3/512: mknod d10/d22/d2a/d99/ca8 0 2024-05-25T06:11:27.508 INFO:tasks.workunit.client.1.smithi203.stdout:5/653: chown dc/d11/d17/f38 181855743 1 2024-05-25T06:11:27.510 INFO:tasks.workunit.client.1.smithi203.stdout:6/535: creat d8/d5a/d8b/fa8 x:0 0 0 2024-05-25T06:11:27.510 INFO:tasks.workunit.client.1.smithi203.stdout:5/654: fsync dc/d11/d18/f37 0 2024-05-25T06:11:27.512 INFO:tasks.workunit.client.1.smithi203.stdout:3/513: symlink d10/d16/d71/la9 0 2024-05-25T06:11:27.514 INFO:tasks.workunit.client.1.smithi203.stdout:6/536: mkdir d8/d2a/da9 0 2024-05-25T06:11:27.516 INFO:tasks.workunit.client.1.smithi203.stdout:0/294: creat dd/d10/f6d x:0 0 0 2024-05-25T06:11:27.517 INFO:tasks.workunit.client.1.smithi203.stdout:5/655: mknod dc/d11/d17/d1a/d2d/d53/da2/cdf 0 2024-05-25T06:11:27.518 INFO:tasks.workunit.client.1.smithi203.stdout:3/514: write d10/d16/d23/d45/f9b [814501,96538] 0 2024-05-25T06:11:27.521 INFO:tasks.workunit.client.1.smithi203.stdout:6/537: chown d8/d3d/c7f 99 1 2024-05-25T06:11:27.521 INFO:tasks.workunit.client.1.smithi203.stdout:8/461: dwrite d2/d5/d28/f7a [0,4194304] 0 2024-05-25T06:11:27.525 INFO:tasks.workunit.client.1.smithi203.stdout:0/295: mkdir dd/d3a/d66/d6e 0 2024-05-25T06:11:27.526 INFO:tasks.workunit.client.1.smithi203.stdout:4/321: dread d0/f38 [0,4194304] 0 2024-05-25T06:11:27.528 INFO:tasks.workunit.client.1.smithi203.stdout:7/408: dwrite d1/d9/d2c/d36/d3b/f47 [0,4194304] 0 2024-05-25T06:11:27.528 INFO:tasks.workunit.client.1.smithi203.stdout:3/515: getdents d10/d16/d71 0 2024-05-25T06:11:27.528 INFO:tasks.workunit.client.1.smithi203.stdout:0/296: chown dd/d10/f26 52120316 1 2024-05-25T06:11:27.530 INFO:tasks.workunit.client.1.smithi203.stdout:3/516: chown d10/c91 0 1 2024-05-25T06:11:27.530 INFO:tasks.workunit.client.1.smithi203.stdout:0/297: stat dd/d10/d34/d4c/d5d 0 2024-05-25T06:11:27.530 INFO:tasks.workunit.client.1.smithi203.stdout:8/462: creat d2/d5/d28/d38/d81/d84/f94 x:0 0 0 2024-05-25T06:11:27.531 INFO:tasks.workunit.client.1.smithi203.stdout:3/517: write d10/d22/f5b [3135796,106275] 0 2024-05-25T06:11:27.532 INFO:tasks.workunit.client.1.smithi203.stdout:5/656: rename dc/d11/d17/d1a/d2d/d4d/l6d to dc/d11/d42/d94/dad/le0 0 2024-05-25T06:11:27.533 INFO:tasks.workunit.client.1.smithi203.stdout:4/322: unlink d0/d9/d11/f3f 0 2024-05-25T06:11:27.537 INFO:tasks.workunit.client.1.smithi203.stdout:0/298: dread dd/d10/d34/f3c [0,4194304] 0 2024-05-25T06:11:27.538 INFO:tasks.workunit.client.1.smithi203.stdout:0/299: chown dd/f1f 1539 1 2024-05-25T06:11:27.540 INFO:tasks.workunit.client.1.smithi203.stdout:8/463: dwrite d2/d5/d18/f70 [0,4194304] 0 2024-05-25T06:11:27.542 INFO:tasks.workunit.client.1.smithi203.stdout:8/464: read d2/d5/f29 [507455,65947] 0 2024-05-25T06:11:27.547 INFO:tasks.workunit.client.1.smithi203.stdout:0/300: write dd/d10/d34/f3c [817383,86021] 0 2024-05-25T06:11:27.548 INFO:tasks.workunit.client.1.smithi203.stdout:8/465: chown d2/d5/d18/d31/c71 393102 1 2024-05-25T06:11:27.549 INFO:tasks.workunit.client.1.smithi203.stdout:8/466: chown d2/d5/f29 510131 1 2024-05-25T06:11:27.549 INFO:tasks.workunit.client.1.smithi203.stdout:3/518: getdents d10/d16/d23/d46 0 2024-05-25T06:11:27.553 INFO:tasks.workunit.client.1.smithi203.stdout:0/301: unlink dd/f15 0 2024-05-25T06:11:27.554 INFO:tasks.workunit.client.1.smithi203.stdout:8/467: mknod d2/d5/d18/d31/c95 0 2024-05-25T06:11:27.555 INFO:tasks.workunit.client.1.smithi203.stdout:3/519: creat d10/d16/d50/d8b/faa x:0 0 0 2024-05-25T06:11:27.557 INFO:tasks.workunit.client.1.smithi203.stdout:7/409: dwrite d1/d11/f21 [0,4194304] 0 2024-05-25T06:11:27.557 INFO:tasks.workunit.client.1.smithi203.stdout:0/302: write dd/f23 [888093,51464] 0 2024-05-25T06:11:27.558 INFO:tasks.workunit.client.1.smithi203.stdout:8/468: rmdir d2/d5/d10/d45/d5d 39 2024-05-25T06:11:27.558 INFO:tasks.workunit.client.1.smithi203.stdout:7/410: readlink d1/d11/d3d/l88 0 2024-05-25T06:11:27.562 INFO:tasks.workunit.client.1.smithi203.stdout:0/303: mknod dd/d5c/c6f 0 2024-05-25T06:11:27.564 INFO:tasks.workunit.client.1.smithi203.stdout:5/657: write dc/d11/f14 [2016343,84487] 0 2024-05-25T06:11:27.566 INFO:tasks.workunit.client.1.smithi203.stdout:0/304: read dd/d45/f5b [1104936,87175] 0 2024-05-25T06:11:27.566 INFO:tasks.workunit.client.1.smithi203.stdout:4/323: truncate d0/d9/d11/f22 1355058 0 2024-05-25T06:11:27.566 INFO:tasks.workunit.client.1.smithi203.stdout:5/658: stat dc/d11/f3e 0 2024-05-25T06:11:27.569 INFO:tasks.workunit.client.1.smithi203.stdout:6/538: dread d8/d29/f7e [0,4194304] 0 2024-05-25T06:11:27.569 INFO:tasks.workunit.client.1.smithi203.stdout:7/411: chown d1/d11/d3d/d4c/f63 82677165 1 2024-05-25T06:11:27.570 INFO:tasks.workunit.client.1.smithi203.stdout:5/659: fsync dc/d11/d17/d1a/fc1 0 2024-05-25T06:11:27.571 INFO:tasks.workunit.client.1.smithi203.stdout:7/412: write d1/f2 [1414655,106923] 0 2024-05-25T06:11:27.573 INFO:tasks.workunit.client.1.smithi203.stdout:3/520: creat d10/fab x:0 0 0 2024-05-25T06:11:27.579 INFO:tasks.workunit.client.1.smithi203.stdout:5/660: rmdir dc/d11/d42 39 2024-05-25T06:11:27.580 INFO:tasks.workunit.client.1.smithi203.stdout:5/661: chown dc/d11/fb5 432 1 2024-05-25T06:11:27.583 INFO:tasks.workunit.client.1.smithi203.stdout:3/521: symlink d10/d16/d50/lac 0 2024-05-25T06:11:27.583 INFO:tasks.workunit.client.1.smithi203.stdout:4/324: link d0/d9/d11/f2f d0/d8/d10/f5c 0 2024-05-25T06:11:27.585 INFO:tasks.workunit.client.1.smithi203.stdout:7/413: write d1/d9/d2c/f42 [615699,42095] 0 2024-05-25T06:11:27.586 INFO:tasks.workunit.client.1.smithi203.stdout:6/539: mkdir d8/d2a/d6c/d87/daa 0 2024-05-25T06:11:27.588 INFO:tasks.workunit.client.1.smithi203.stdout:7/414: write d1/d9/d2c/d79/f81 [1883860,87216] 0 2024-05-25T06:11:27.590 INFO:tasks.workunit.client.1.smithi203.stdout:8/469: dread d2/d5/d18/d31/f50 [0,4194304] 0 2024-05-25T06:11:27.596 INFO:tasks.workunit.client.1.smithi203.stdout:7/415: symlink d1/d9/d2c/d36/d3b/l95 0 2024-05-25T06:11:27.599 INFO:tasks.workunit.client.1.smithi203.stdout:8/470: dread - d2/d5/d10/d45/d5d/f5c zero size 2024-05-25T06:11:27.599 INFO:tasks.workunit.client.1.smithi203.stdout:5/662: dwrite dc/d11/d17/d1a/d22/fb4 [0,4194304] 0 2024-05-25T06:11:27.599 INFO:tasks.workunit.client.1.smithi203.stdout:7/416: stat d1/d11/d3d/l88 0 2024-05-25T06:11:27.600 INFO:tasks.workunit.client.1.smithi203.stdout:1/369: sync 2024-05-25T06:11:27.600 INFO:tasks.workunit.client.1.smithi203.stdout:3/522: truncate d10/d16/d23/d4a/f61 377897 0 2024-05-25T06:11:27.601 INFO:tasks.workunit.client.1.smithi203.stdout:6/540: dread d8/d11/d15/f42 [0,4194304] 0 2024-05-25T06:11:27.601 INFO:tasks.workunit.client.1.smithi203.stdout:4/325: rename d0/d9/d11/f22 to d0/d8/d3a/f5d 0 2024-05-25T06:11:27.604 INFO:tasks.workunit.client.1.smithi203.stdout:7/417: chown d1/d9/d2c/d3a/f3e 1331942171 1 2024-05-25T06:11:27.604 INFO:tasks.workunit.client.1.smithi203.stdout:3/523: fsync d10/d16/d50/d8b/f85 0 2024-05-25T06:11:27.605 INFO:tasks.workunit.client.1.smithi203.stdout:0/305: getdents dd/d10/d34/d4c 0 2024-05-25T06:11:27.605 INFO:tasks.workunit.client.1.smithi203.stdout:5/663: mknod dc/d11/d17/d1a/d72/ce1 0 2024-05-25T06:11:27.608 INFO:tasks.workunit.client.1.smithi203.stdout:4/326: creat d0/d8/d52/f5e x:0 0 0 2024-05-25T06:11:27.608 INFO:tasks.workunit.client.1.smithi203.stdout:8/471: mknod d2/d5/d10/d45/d90/c96 0 2024-05-25T06:11:27.611 INFO:tasks.workunit.client.1.smithi203.stdout:6/541: creat d8/d70/d38/fab x:0 0 0 2024-05-25T06:11:27.612 INFO:tasks.workunit.client.1.smithi203.stdout:0/306: fdatasync dd/d3a/f3b 0 2024-05-25T06:11:27.616 INFO:tasks.workunit.client.1.smithi203.stdout:8/472: unlink d2/d5/d28/d38/d81/d84/f94 0 2024-05-25T06:11:27.617 INFO:tasks.workunit.client.1.smithi203.stdout:5/664: symlink dc/d11/d17/d1a/d2d/d4d/dcc/le2 0 2024-05-25T06:11:27.619 INFO:tasks.workunit.client.1.smithi203.stdout:3/524: dwrite d10/d16/d23/d67/f94 [0,4194304] 0 2024-05-25T06:11:27.622 INFO:tasks.workunit.client.1.smithi203.stdout:8/473: write d2/d5/d18/d31/f34 [3015859,99702] 0 2024-05-25T06:11:27.625 INFO:tasks.workunit.client.1.smithi203.stdout:5/665: mknod dc/d11/d17/d1a/d22/ce3 0 2024-05-25T06:11:27.625 INFO:tasks.workunit.client.1.smithi203.stdout:4/327: dread d0/d8/d3a/f5d [0,4194304] 0 2024-05-25T06:11:27.625 INFO:tasks.workunit.client.1.smithi203.stdout:3/525: fdatasync d10/d16/d23/d3a/f53 0 2024-05-25T06:11:27.627 INFO:tasks.workunit.client.1.smithi203.stdout:6/542: dwrite d8/d11/d15/d28/f5e [0,4194304] 0 2024-05-25T06:11:27.627 INFO:tasks.workunit.client.1.smithi203.stdout:4/328: readlink d0/l2d 0 2024-05-25T06:11:27.632 INFO:tasks.workunit.client.1.smithi203.stdout:3/526: write d10/d16/d23/d45/f9b [812576,79183] 0 2024-05-25T06:11:27.633 INFO:tasks.workunit.client.1.smithi203.stdout:8/474: rename d2/d5/d18/d4f to d2/d5/d28/d38/d4e/d97 0 2024-05-25T06:11:27.634 INFO:tasks.workunit.client.1.smithi203.stdout:0/307: dwrite dd/d45/f5b [0,4194304] 0 2024-05-25T06:11:27.638 INFO:tasks.workunit.client.1.smithi203.stdout:4/329: creat d0/d9/d24/f5f x:0 0 0 2024-05-25T06:11:27.640 INFO:tasks.workunit.client.1.smithi203.stdout:8/475: rmdir d2/d5/d28/d38/d81/d84/d8a/d8f 39 2024-05-25T06:11:27.641 INFO:tasks.workunit.client.1.smithi203.stdout:0/308: write dd/d10/d34/f58 [5234920,109107] 0 2024-05-25T06:11:27.643 INFO:tasks.workunit.client.1.smithi203.stdout:6/543: dwrite d8/ff [0,4194304] 0 2024-05-25T06:11:27.646 INFO:tasks.workunit.client.1.smithi203.stdout:0/309: fsync dd/fe 0 2024-05-25T06:11:27.647 INFO:tasks.workunit.client.1.smithi203.stdout:8/476: chown d2/d5/d10/d11/de/f13 460857 1 2024-05-25T06:11:27.648 INFO:tasks.workunit.client.1.smithi203.stdout:4/330: mknod d0/d9/d50/c60 0 2024-05-25T06:11:27.649 INFO:tasks.workunit.client.1.smithi203.stdout:0/310: mkdir dd/d3a/d66/d6e/d70 0 2024-05-25T06:11:27.652 INFO:tasks.workunit.client.1.smithi203.stdout:8/477: creat d2/d5/d28/d38/d81/d84/f98 x:0 0 0 2024-05-25T06:11:27.657 INFO:tasks.workunit.client.1.smithi203.stdout:8/478: creat d2/f99 x:0 0 0 2024-05-25T06:11:27.658 INFO:tasks.workunit.client.1.smithi203.stdout:4/331: rmdir d0/d9/d41 0 2024-05-25T06:11:27.663 INFO:tasks.workunit.client.1.smithi203.stdout:4/332: creat d0/d9/f61 x:0 0 0 2024-05-25T06:11:27.663 INFO:tasks.workunit.client.1.smithi203.stdout:0/311: dwrite dd/f24 [0,4194304] 0 2024-05-25T06:11:27.666 INFO:tasks.workunit.client.1.smithi203.stdout:8/479: getdents d2/d5/d28/d38/d81/d84/d8a 0 2024-05-25T06:11:27.667 INFO:tasks.workunit.client.1.smithi203.stdout:4/333: symlink d0/l62 0 2024-05-25T06:11:27.668 INFO:tasks.workunit.client.1.smithi203.stdout:0/312: mknod dd/d32/c71 0 2024-05-25T06:11:27.668 INFO:tasks.workunit.client.1.smithi203.stdout:4/334: dread - d0/d9/d24/d27/f4c zero size 2024-05-25T06:11:27.669 INFO:tasks.workunit.client.1.smithi203.stdout:4/335: chown d0/d8/d1f 4234 1 2024-05-25T06:11:27.671 INFO:tasks.workunit.client.1.smithi203.stdout:8/480: creat d2/d5/d28/d38/d81/d84/f9a x:0 0 0 2024-05-25T06:11:27.672 INFO:tasks.workunit.client.1.smithi203.stdout:4/336: symlink d0/d9/l63 0 2024-05-25T06:11:27.674 INFO:tasks.workunit.client.1.smithi203.stdout:8/481: creat d2/d5/d28/d38/d4e/d97/f9b x:0 0 0 2024-05-25T06:11:27.677 INFO:tasks.workunit.client.1.smithi203.stdout:6/544: dread d8/d3d/f44 [0,4194304] 0 2024-05-25T06:11:27.677 INFO:tasks.workunit.client.1.smithi203.stdout:4/337: rename d0/d8/d1f/d26/f20 to d0/d9/f64 0 2024-05-25T06:11:27.678 INFO:tasks.workunit.client.1.smithi203.stdout:4/338: fdatasync d0/f16 0 2024-05-25T06:11:27.685 INFO:tasks.workunit.client.1.smithi203.stdout:6/545: dwrite d8/d11/d15/f42 [0,4194304] 0 2024-05-25T06:11:27.686 INFO:tasks.workunit.client.1.smithi203.stdout:4/339: dwrite d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:27.693 INFO:tasks.workunit.client.1.smithi203.stdout:4/340: fdatasync d0/d8/d1f/d26/d43/f4d 0 2024-05-25T06:11:27.698 INFO:tasks.workunit.client.1.smithi203.stdout:4/341: dread d0/d9/d24/d27/f51 [0,4194304] 0 2024-05-25T06:11:27.699 INFO:tasks.workunit.client.1.smithi203.stdout:8/482: dread d2/d5/d2d/f3e [0,4194304] 0 2024-05-25T06:11:27.704 INFO:tasks.workunit.client.1.smithi203.stdout:8/483: rename d2/d5/d2d/f54 to d2/d5/d10/d11/f9c 0 2024-05-25T06:11:27.706 INFO:tasks.workunit.client.1.smithi203.stdout:8/484: mkdir d2/d5/d28/d38/d81/d84/d88/d93/d9d 0 2024-05-25T06:11:27.707 INFO:tasks.workunit.client.1.smithi203.stdout:8/485: dread - d2/f61 zero size 2024-05-25T06:11:27.708 INFO:tasks.workunit.client.1.smithi203.stdout:8/486: dread - d2/d5/d10/d45/d5d/f82 zero size 2024-05-25T06:11:27.710 INFO:tasks.workunit.client.1.smithi203.stdout:8/487: readlink d2/d5/d10/l24 0 2024-05-25T06:11:27.711 INFO:tasks.workunit.client.1.smithi203.stdout:8/488: stat d2/d5/d28/d38/d81/d84/d8a/d8f 0 2024-05-25T06:11:27.712 INFO:tasks.workunit.client.1.smithi203.stdout:8/489: readlink d2/d5/d28/d38/d81/d84/d8a/d8f/l91 0 2024-05-25T06:11:27.717 INFO:tasks.workunit.client.1.smithi203.stdout:6/546: dread d8/d11/d15/f2e [0,4194304] 0 2024-05-25T06:11:27.720 INFO:tasks.workunit.client.1.smithi203.stdout:8/490: dwrite d2/d5/d18/d31/f34 [0,4194304] 0 2024-05-25T06:11:27.720 INFO:tasks.workunit.client.1.smithi203.stdout:4/342: dread d0/f3 [0,4194304] 0 2024-05-25T06:11:27.720 INFO:tasks.workunit.client.1.smithi203.stdout:6/547: mknod d8/d2a/d6c/d87/daa/cac 0 2024-05-25T06:11:27.721 INFO:tasks.workunit.client.1.smithi203.stdout:4/343: readlink d0/d8/l32 0 2024-05-25T06:11:27.724 INFO:tasks.workunit.client.1.smithi203.stdout:8/491: mkdir d2/d5/d28/d38/d81/d84/d88/d93/d9d/d9e 0 2024-05-25T06:11:27.724 INFO:tasks.workunit.client.1.smithi203.stdout:4/344: creat d0/d9/d24/f65 x:0 0 0 2024-05-25T06:11:27.725 INFO:tasks.workunit.client.1.smithi203.stdout:8/492: stat d2/d5/d10/d45/d90 0 2024-05-25T06:11:27.727 INFO:tasks.workunit.client.1.smithi203.stdout:4/345: fsync d0/d9/d24/d27/f51 0 2024-05-25T06:11:27.730 INFO:tasks.workunit.client.1.smithi203.stdout:4/346: unlink d0/d9/d24/d27/l3d 0 2024-05-25T06:11:27.732 INFO:tasks.workunit.client.1.smithi203.stdout:4/347: write d0/d8/d1f/d26/d43/f4d [574498,57186] 0 2024-05-25T06:11:27.732 INFO:tasks.workunit.client.1.smithi203.stdout:8/493: dwrite d2/d5/d18/f70 [0,4194304] 0 2024-05-25T06:11:27.735 INFO:tasks.workunit.client.1.smithi203.stdout:4/348: creat d0/d9/d11/f66 x:0 0 0 2024-05-25T06:11:27.737 INFO:tasks.workunit.client.1.smithi203.stdout:4/349: stat d0/d8/d10/l1a 0 2024-05-25T06:11:27.737 INFO:tasks.workunit.client.1.smithi203.stdout:8/494: write d2/d5/d10/d11/d59/f69 [2120400,15981] 0 2024-05-25T06:11:27.743 INFO:tasks.workunit.client.1.smithi203.stdout:8/495: read d2/d5/d10/d11/de/f13 [233900,64300] 0 2024-05-25T06:11:27.745 INFO:tasks.workunit.client.1.smithi203.stdout:4/350: getdents d0/d8/d1f/d26 0 2024-05-25T06:11:27.745 INFO:tasks.workunit.client.1.smithi203.stdout:6/548: dread d8/d11/d15/d28/d52/d43/f93 [0,4194304] 0 2024-05-25T06:11:27.746 INFO:tasks.workunit.client.1.smithi203.stdout:8/496: creat d2/d5/d10/d11/d59/f9f x:0 0 0 2024-05-25T06:11:27.747 INFO:tasks.workunit.client.1.smithi203.stdout:4/351: symlink d0/d8/d3a/l67 0 2024-05-25T06:11:27.747 INFO:tasks.workunit.client.1.smithi203.stdout:8/497: truncate d2/d5/d2d/f3e 4972377 0 2024-05-25T06:11:27.748 INFO:tasks.workunit.client.1.smithi203.stdout:6/549: fdatasync d8/d11/d15/f18 0 2024-05-25T06:11:27.749 INFO:tasks.workunit.client.1.smithi203.stdout:8/498: write d2/d5/d28/d38/f49 [183788,107826] 0 2024-05-25T06:11:27.750 INFO:tasks.workunit.client.1.smithi203.stdout:4/352: creat d0/d8/d1f/d26/d43/f68 x:0 0 0 2024-05-25T06:11:27.751 INFO:tasks.workunit.client.1.smithi203.stdout:6/550: mknod d8/d11/d15/d28/cad 0 2024-05-25T06:11:27.752 INFO:tasks.workunit.client.1.smithi203.stdout:8/499: mknod d2/d5/d28/d38/d81/ca0 0 2024-05-25T06:11:27.752 INFO:tasks.workunit.client.1.smithi203.stdout:8/500: read - d2/f99 zero size 2024-05-25T06:11:27.755 INFO:tasks.workunit.client.1.smithi203.stdout:8/501: mkdir d2/d5/d28/d38/d4e/d97/da1 0 2024-05-25T06:11:27.755 INFO:tasks.workunit.client.1.smithi203.stdout:6/551: dread d8/d2a/f4a [0,4194304] 0 2024-05-25T06:11:27.758 INFO:tasks.workunit.client.1.smithi203.stdout:6/552: creat d8/d2a/d6c/fae x:0 0 0 2024-05-25T06:11:27.760 INFO:tasks.workunit.client.1.smithi203.stdout:6/553: mkdir d8/d5a/d9b/daf 0 2024-05-25T06:11:27.760 INFO:tasks.workunit.client.1.smithi203.stdout:8/502: write d2/d5/d2d/f4b [4355478,45771] 0 2024-05-25T06:11:27.762 INFO:tasks.workunit.client.1.smithi203.stdout:8/503: chown d2/d5/d18/d31/f39 26 1 2024-05-25T06:11:27.764 INFO:tasks.workunit.client.1.smithi203.stdout:6/554: dread d8/d3d/f44 [0,4194304] 0 2024-05-25T06:11:27.765 INFO:tasks.workunit.client.1.smithi203.stdout:8/504: creat d2/d5/d10/d11/de/fa2 x:0 0 0 2024-05-25T06:11:27.765 INFO:tasks.workunit.client.1.smithi203.stdout:4/353: dread d0/d9/d24/d27/f30 [0,4194304] 0 2024-05-25T06:11:27.766 INFO:tasks.workunit.client.1.smithi203.stdout:8/505: stat d2/d5/d10 0 2024-05-25T06:11:27.767 INFO:tasks.workunit.client.1.smithi203.stdout:6/555: creat d8/d11/d1b/fb0 x:0 0 0 2024-05-25T06:11:27.767 INFO:tasks.workunit.client.1.smithi203.stdout:4/354: dread - d0/d9/d24/f5f zero size 2024-05-25T06:11:27.767 INFO:tasks.workunit.client.1.smithi203.stdout:8/506: dread - d2/d5/d10/d11/d59/f9f zero size 2024-05-25T06:11:27.768 INFO:tasks.workunit.client.1.smithi203.stdout:8/507: stat d2/d5/d28/d38/d4e/f51 0 2024-05-25T06:11:27.769 INFO:tasks.workunit.client.1.smithi203.stdout:6/556: symlink d8/d11/d15/lb1 0 2024-05-25T06:11:27.769 INFO:tasks.workunit.client.1.smithi203.stdout:4/355: mknod d0/d9/d24/c69 0 2024-05-25T06:11:27.770 INFO:tasks.workunit.client.1.smithi203.stdout:8/508: creat d2/d5/d2d/fa3 x:0 0 0 2024-05-25T06:11:27.771 INFO:tasks.workunit.client.1.smithi203.stdout:4/356: mkdir d0/d9/d11/d6a 0 2024-05-25T06:11:27.773 INFO:tasks.workunit.client.1.smithi203.stdout:9/349: sync 2024-05-25T06:11:27.773 INFO:tasks.workunit.client.1.smithi203.stdout:2/431: sync 2024-05-25T06:11:27.773 INFO:tasks.workunit.client.1.smithi203.stdout:6/557: dread d8/d3d/f9a [0,4194304] 0 2024-05-25T06:11:27.773 INFO:tasks.workunit.client.1.smithi203.stdout:8/509: write d2/d5/d10/d45/d5d/f5c [409680,54321] 0 2024-05-25T06:11:27.776 INFO:tasks.workunit.client.1.smithi203.stdout:4/357: rename d0/d9/d24/d27 to d0/d9/d50/d6b 0 2024-05-25T06:11:27.777 INFO:tasks.workunit.client.1.smithi203.stdout:4/358: chown d0/l2d 7 1 2024-05-25T06:11:27.777 INFO:tasks.workunit.client.1.smithi203.stdout:8/510: symlink d2/d5/d10/la4 0 2024-05-25T06:11:27.779 INFO:tasks.workunit.client.1.smithi203.stdout:9/350: creat d0/d57/f76 x:0 0 0 2024-05-25T06:11:27.779 INFO:tasks.workunit.client.1.smithi203.stdout:4/359: chown d0/d9/d50/d6b/c4b 25 1 2024-05-25T06:11:27.782 INFO:tasks.workunit.client.1.smithi203.stdout:8/511: creat d2/d5/d28/d38/d81/d84/fa5 x:0 0 0 2024-05-25T06:11:27.783 INFO:tasks.workunit.client.1.smithi203.stdout:6/558: dwrite d8/d70/f7c [0,4194304] 0 2024-05-25T06:11:27.784 INFO:tasks.workunit.client.1.smithi203.stdout:9/351: mknod d0/d17/d20/d3c/c77 0 2024-05-25T06:11:27.784 INFO:tasks.workunit.client.1.smithi203.stdout:4/360: mknod d0/d9/d50/d6b/c6c 0 2024-05-25T06:11:27.786 INFO:tasks.workunit.client.1.smithi203.stdout:8/512: fsync d2/d5/d10/d45/d5d/d68/f73 0 2024-05-25T06:11:27.787 INFO:tasks.workunit.client.1.smithi203.stdout:8/513: fsync d2/d5/d10/f83 0 2024-05-25T06:11:27.788 INFO:tasks.workunit.client.1.smithi203.stdout:6/559: dread d8/d11/d15/f2e [0,4194304] 0 2024-05-25T06:11:27.788 INFO:tasks.workunit.client.1.smithi203.stdout:9/352: read d0/df/d2d/f72 [654391,17903] 0 2024-05-25T06:11:27.789 INFO:tasks.workunit.client.1.smithi203.stdout:2/432: dread d2/d6/f26 [0,4194304] 0 2024-05-25T06:11:27.792 INFO:tasks.workunit.client.1.smithi203.stdout:8/514: read d2/d5/d28/d38/d4e/d97/f66 [5959139,123037] 0 2024-05-25T06:11:27.793 INFO:tasks.workunit.client.1.smithi203.stdout:8/515: fdatasync d2/d5/d18/d31/f34 0 2024-05-25T06:11:27.793 INFO:tasks.workunit.client.1.smithi203.stdout:6/560: mknod d8/d29/cb2 0 2024-05-25T06:11:27.793 INFO:tasks.workunit.client.1.smithi203.stdout:9/353: fsync d0/f39 0 2024-05-25T06:11:27.795 INFO:tasks.workunit.client.1.smithi203.stdout:2/433: write d2/f6c [1751883,36496] 0 2024-05-25T06:11:27.796 INFO:tasks.workunit.client.1.smithi203.stdout:4/361: rmdir d0/d9/d11/d6a 0 2024-05-25T06:11:27.796 INFO:tasks.workunit.client.1.smithi203.stdout:2/434: chown d2/d4/d10/d58 1 1 2024-05-25T06:11:27.798 INFO:tasks.workunit.client.1.smithi203.stdout:6/561: mknod d8/d29/d80/cb3 0 2024-05-25T06:11:27.800 INFO:tasks.workunit.client.1.smithi203.stdout:8/516: dread - d2/d5/d10/d11/d1f/d75/f8d zero size 2024-05-25T06:11:27.800 INFO:tasks.workunit.client.1.smithi203.stdout:2/435: mknod d2/d1a/c98 0 2024-05-25T06:11:27.801 INFO:tasks.workunit.client.1.smithi203.stdout:4/362: creat d0/d8/d3a/f6d x:0 0 0 2024-05-25T06:11:27.804 INFO:tasks.workunit.client.1.smithi203.stdout:2/436: stat d2/d4/l8f 0 2024-05-25T06:11:27.804 INFO:tasks.workunit.client.1.smithi203.stdout:6/562: write d8/d11/d15/d28/d52/d43/f93 [1043177,120832] 0 2024-05-25T06:11:27.805 INFO:tasks.workunit.client.1.smithi203.stdout:8/517: creat d2/d5/d28/d38/fa6 x:0 0 0 2024-05-25T06:11:27.806 INFO:tasks.workunit.client.1.smithi203.stdout:6/563: write d8/f12 [242191,77382] 0 2024-05-25T06:11:27.806 INFO:tasks.workunit.client.1.smithi203.stdout:2/437: symlink d2/d6/l99 0 2024-05-25T06:11:27.807 INFO:tasks.workunit.client.1.smithi203.stdout:2/438: stat d2/d4/c3c 0 2024-05-25T06:11:27.810 INFO:tasks.workunit.client.1.smithi203.stdout:4/363: dwrite d0/d9/d24/f65 [0,4194304] 0 2024-05-25T06:11:27.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/518: chown d2/d5/d28/d38/d4e/d97/l65 3 1 2024-05-25T06:11:27.811 INFO:tasks.workunit.client.1.smithi203.stdout:6/564: unlink d8/d11/d47/d60/c78 0 2024-05-25T06:11:27.814 INFO:tasks.workunit.client.1.smithi203.stdout:4/364: mknod d0/d8/d3a/c6e 0 2024-05-25T06:11:27.814 INFO:tasks.workunit.client.1.smithi203.stdout:8/519: symlink d2/d5/d10/d45/d5d/d68/la7 0 2024-05-25T06:11:27.815 INFO:tasks.workunit.client.1.smithi203.stdout:9/354: truncate d0/df/f1a 239330 0 2024-05-25T06:11:27.816 INFO:tasks.workunit.client.1.smithi203.stdout:2/439: dread d2/d4/d10/d58/f6d [0,4194304] 0 2024-05-25T06:11:27.816 INFO:tasks.workunit.client.1.smithi203.stdout:6/565: creat d8/d2a/d6c/d87/daa/fb4 x:0 0 0 2024-05-25T06:11:27.818 INFO:tasks.workunit.client.1.smithi203.stdout:2/440: write d2/d42/f83 [495483,107030] 0 2024-05-25T06:11:27.818 INFO:tasks.workunit.client.1.smithi203.stdout:8/520: mkdir d2/d5/d10/d11/de/da8 0 2024-05-25T06:11:27.819 INFO:tasks.workunit.client.1.smithi203.stdout:8/521: fsync d2/d5/d10/d45/d5d/f5c 0 2024-05-25T06:11:27.820 INFO:tasks.workunit.client.1.smithi203.stdout:6/566: fsync d8/d11/d15/f2b 0 2024-05-25T06:11:27.821 INFO:tasks.workunit.client.1.smithi203.stdout:6/567: stat d8/d11/d15/d28/d52/d43/c61 0 2024-05-25T06:11:27.821 INFO:tasks.workunit.client.1.smithi203.stdout:6/568: readlink d8/d29/d6e/l8d 0 2024-05-25T06:11:27.822 INFO:tasks.workunit.client.1.smithi203.stdout:8/522: mknod d2/ca9 0 2024-05-25T06:11:27.822 INFO:tasks.workunit.client.1.smithi203.stdout:6/569: write d8/d70/f7c [4496698,41969] 0 2024-05-25T06:11:27.823 INFO:tasks.workunit.client.1.smithi203.stdout:6/570: stat d8/d2a/d6c/ca2 0 2024-05-25T06:11:27.824 INFO:tasks.workunit.client.1.smithi203.stdout:2/441: getdents d2/d1a/d33 0 2024-05-25T06:11:27.825 INFO:tasks.workunit.client.1.smithi203.stdout:2/442: write d2/d4/d10/f79 [2034691,81385] 0 2024-05-25T06:11:27.826 INFO:tasks.workunit.client.1.smithi203.stdout:6/571: creat d8/d11/d15/d28/d52/d43/d73/fb5 x:0 0 0 2024-05-25T06:11:27.827 INFO:tasks.workunit.client.1.smithi203.stdout:2/443: mknod d2/d1a/d87/c9a 0 2024-05-25T06:11:27.828 INFO:tasks.workunit.client.1.smithi203.stdout:2/444: write d2/d38/f6e [1296912,70948] 0 2024-05-25T06:11:27.829 INFO:tasks.workunit.client.1.smithi203.stdout:2/445: chown d2/d1a/d33/d80 11615486 1 2024-05-25T06:11:27.830 INFO:tasks.workunit.client.1.smithi203.stdout:2/446: dread - d2/d1a/f97 zero size 2024-05-25T06:11:27.831 INFO:tasks.workunit.client.1.smithi203.stdout:6/572: rename d8/f65 to d8/d11/fb6 0 2024-05-25T06:11:27.835 INFO:tasks.workunit.client.1.smithi203.stdout:6/573: chown f2 3736292 1 2024-05-25T06:11:27.836 INFO:tasks.workunit.client.1.smithi203.stdout:9/355: dread d0/df/f1a [0,4194304] 0 2024-05-25T06:11:27.838 INFO:tasks.workunit.client.1.smithi203.stdout:6/574: mkdir d8/d11/d15/d28/d52/d43/d73/db7 0 2024-05-25T06:11:27.839 INFO:tasks.workunit.client.1.smithi203.stdout:4/365: dread d0/d8/d1f/f2b [4194304,4194304] 0 2024-05-25T06:11:27.840 INFO:tasks.workunit.client.1.smithi203.stdout:6/575: read d8/fd [1045178,30714] 0 2024-05-25T06:11:27.841 INFO:tasks.workunit.client.1.smithi203.stdout:4/366: creat d0/d8/d52/f6f x:0 0 0 2024-05-25T06:11:27.843 INFO:tasks.workunit.client.1.smithi203.stdout:6/576: rename d8/ff to d8/d11/d47/fb8 0 2024-05-25T06:11:27.844 INFO:tasks.workunit.client.1.smithi203.stdout:6/577: readlink d8/d11/l21 0 2024-05-25T06:11:27.845 INFO:tasks.workunit.client.1.smithi203.stdout:2/447: dread d2/d38/d2f/f31 [0,4194304] 0 2024-05-25T06:11:27.846 INFO:tasks.workunit.client.1.smithi203.stdout:4/367: rename d0/d9/d11/c33 to d0/d9/d11/d54/c70 0 2024-05-25T06:11:27.846 INFO:tasks.workunit.client.1.smithi203.stdout:6/578: mknod d8/d3d/cb9 0 2024-05-25T06:11:27.848 INFO:tasks.workunit.client.1.smithi203.stdout:4/368: stat d0/d8/c1b 0 2024-05-25T06:11:27.849 INFO:tasks.workunit.client.1.smithi203.stdout:4/369: read d0/f3 [1781582,60278] 0 2024-05-25T06:11:27.851 INFO:tasks.workunit.client.1.smithi203.stdout:6/579: unlink d8/d11/d47/d60/f57 0 2024-05-25T06:11:27.851 INFO:tasks.workunit.client.1.smithi203.stdout:2/448: getdents d2/d4/d10/d32 0 2024-05-25T06:11:27.853 INFO:tasks.workunit.client.1.smithi203.stdout:6/580: dread - d8/d11/d15/d28/d52/d43/d73/fb5 zero size 2024-05-25T06:11:27.854 INFO:tasks.workunit.client.1.smithi203.stdout:9/356: dread d0/d17/d20/f44 [0,4194304] 0 2024-05-25T06:11:27.854 INFO:tasks.workunit.client.1.smithi203.stdout:2/449: creat d2/d4/d10/f9b x:0 0 0 2024-05-25T06:11:27.857 INFO:tasks.workunit.client.1.smithi203.stdout:6/581: getdents d8/d11 0 2024-05-25T06:11:27.858 INFO:tasks.workunit.client.1.smithi203.stdout:4/370: dwrite d0/d8/d3a/f49 [0,4194304] 0 2024-05-25T06:11:27.859 INFO:tasks.workunit.client.1.smithi203.stdout:4/371: write d0/d9/f61 [829071,5071] 0 2024-05-25T06:11:27.860 INFO:tasks.workunit.client.1.smithi203.stdout:2/450: rename d2/d1a/d33/c3e to d2/d1a/d89/c9c 0 2024-05-25T06:11:27.860 INFO:tasks.workunit.client.1.smithi203.stdout:9/357: mkdir d0/d17/d20/d6e/d75/d78 0 2024-05-25T06:11:27.863 INFO:tasks.workunit.client.1.smithi203.stdout:2/451: fsync d2/d4/d10/f21 0 2024-05-25T06:11:27.863 INFO:tasks.workunit.client.1.smithi203.stdout:6/582: symlink d8/d2a/da9/lba 0 2024-05-25T06:11:27.865 INFO:tasks.workunit.client.1.smithi203.stdout:2/452: stat d2/d38/d2f/f31 0 2024-05-25T06:11:27.866 INFO:tasks.workunit.client.1.smithi203.stdout:9/358: creat d0/df/d37/f79 x:0 0 0 2024-05-25T06:11:27.867 INFO:tasks.workunit.client.1.smithi203.stdout:6/583: write d8/d11/d47/d60/f55 [2876908,19695] 0 2024-05-25T06:11:27.868 INFO:tasks.workunit.client.1.smithi203.stdout:9/359: chown d0/df/f4a 41133377 1 2024-05-25T06:11:27.868 INFO:tasks.workunit.client.1.smithi203.stdout:2/453: mknod d2/c9d 0 2024-05-25T06:11:27.869 INFO:tasks.workunit.client.1.smithi203.stdout:4/372: dwrite d0/d9/f61 [0,4194304] 0 2024-05-25T06:11:27.869 INFO:tasks.workunit.client.1.smithi203.stdout:2/454: write d2/f6f [8282798,27195] 0 2024-05-25T06:11:27.870 INFO:tasks.workunit.client.1.smithi203.stdout:6/584: symlink d8/d29/lbb 0 2024-05-25T06:11:27.874 INFO:tasks.workunit.client.1.smithi203.stdout:9/360: mkdir d0/d17/d20/d6e/d75/d78/d7a 0 2024-05-25T06:11:27.876 INFO:tasks.workunit.client.1.smithi203.stdout:4/373: dread d0/d9/d24/f65 [0,4194304] 0 2024-05-25T06:11:27.878 INFO:tasks.workunit.client.1.smithi203.stdout:6/585: dread d8/d70/f94 [0,4194304] 0 2024-05-25T06:11:27.879 INFO:tasks.workunit.client.1.smithi203.stdout:4/374: creat d0/d8/d52/f71 x:0 0 0 2024-05-25T06:11:27.881 INFO:tasks.workunit.client.1.smithi203.stdout:6/586: creat d8/d11/d15/fbc x:0 0 0 2024-05-25T06:11:27.882 INFO:tasks.workunit.client.1.smithi203.stdout:4/375: symlink d0/d9/d11/d54/l72 0 2024-05-25T06:11:27.882 INFO:tasks.workunit.client.1.smithi203.stdout:9/361: link d0/df/c71 d0/df/d2d/c7b 0 2024-05-25T06:11:27.883 INFO:tasks.workunit.client.1.smithi203.stdout:6/587: mknod d8/d11/cbd 0 2024-05-25T06:11:27.885 INFO:tasks.workunit.client.1.smithi203.stdout:6/588: fdatasync d8/d3d/f6a 0 2024-05-25T06:11:27.885 INFO:tasks.workunit.client.1.smithi203.stdout:4/376: symlink d0/d9/d24/l73 0 2024-05-25T06:11:27.885 INFO:tasks.workunit.client.1.smithi203.stdout:9/362: dread - d0/f52 zero size 2024-05-25T06:11:27.886 INFO:tasks.workunit.client.1.smithi203.stdout:2/455: dwrite d2/d38/d2f/f4f [0,4194304] 0 2024-05-25T06:11:27.887 INFO:tasks.workunit.client.1.smithi203.stdout:6/589: truncate d8/fc 1061159 0 2024-05-25T06:11:27.889 INFO:tasks.workunit.client.1.smithi203.stdout:4/377: symlink d0/d9/d11/l74 0 2024-05-25T06:11:27.892 INFO:tasks.workunit.client.1.smithi203.stdout:9/363: creat d0/df/d2d/f7c x:0 0 0 2024-05-25T06:11:27.893 INFO:tasks.workunit.client.1.smithi203.stdout:4/378: creat d0/d8/d1f/d26/d43/f75 x:0 0 0 2024-05-25T06:11:27.894 INFO:tasks.workunit.client.1.smithi203.stdout:6/590: creat d8/d11/d47/fbe x:0 0 0 2024-05-25T06:11:27.896 INFO:tasks.workunit.client.1.smithi203.stdout:4/379: fsync d0/d9/d11/f2f 0 2024-05-25T06:11:27.899 INFO:tasks.workunit.client.1.smithi203.stdout:9/364: dwrite d0/d17/d20/f3a [0,4194304] 0 2024-05-25T06:11:27.903 INFO:tasks.workunit.client.1.smithi203.stdout:6/591: link d8/d3d/f9a d8/d11/d47/fbf 0 2024-05-25T06:11:27.903 INFO:tasks.workunit.client.1.smithi203.stdout:6/592: dread - d8/d2a/f67 zero size 2024-05-25T06:11:27.905 INFO:tasks.workunit.client.1.smithi203.stdout:2/456: truncate d2/d4/d10/d32/f1b 5286055 0 2024-05-25T06:11:27.908 INFO:tasks.workunit.client.1.smithi203.stdout:2/457: mkdir d2/d1a/d87/d9e 0 2024-05-25T06:11:27.910 INFO:tasks.workunit.client.1.smithi203.stdout:2/458: symlink d2/d42/l9f 0 2024-05-25T06:11:27.912 INFO:tasks.workunit.client.1.smithi203.stdout:6/593: dwrite d8/d11/d47/f83 [4194304,4194304] 0 2024-05-25T06:11:27.913 INFO:tasks.workunit.client.1.smithi203.stdout:4/380: dwrite d0/d9/d11/f2f [0,4194304] 0 2024-05-25T06:11:27.914 INFO:tasks.workunit.client.1.smithi203.stdout:6/594: fsync d8/d2a/d6c/fae 0 2024-05-25T06:11:27.917 INFO:tasks.workunit.client.1.smithi203.stdout:2/459: mkdir d2/d4/d10/d58/da0 0 2024-05-25T06:11:27.919 INFO:tasks.workunit.client.1.smithi203.stdout:6/595: chown d8/d11/d15/d28/d52/d43/c9f 519658 1 2024-05-25T06:11:27.920 INFO:tasks.workunit.client.1.smithi203.stdout:4/381: mkdir d0/d9/d11/d76 0 2024-05-25T06:11:27.920 INFO:tasks.workunit.client.1.smithi203.stdout:9/365: write d0/df/d2d/f72 [1724809,123068] 0 2024-05-25T06:11:27.922 INFO:tasks.workunit.client.1.smithi203.stdout:9/366: fsync d0/df/d73/f49 0 2024-05-25T06:11:27.922 INFO:tasks.workunit.client.1.smithi203.stdout:6/596: symlink d8/d5a/d9b/daf/lc0 0 2024-05-25T06:11:27.923 INFO:tasks.workunit.client.1.smithi203.stdout:4/382: creat d0/d8/d3a/f77 x:0 0 0 2024-05-25T06:11:27.925 INFO:tasks.workunit.client.1.smithi203.stdout:6/597: write d8/d11/d1b/f4d [1725241,70395] 0 2024-05-25T06:11:27.925 INFO:tasks.workunit.client.1.smithi203.stdout:9/367: creat d0/d17/d20/d3c/f7d x:0 0 0 2024-05-25T06:11:27.927 INFO:tasks.workunit.client.1.smithi203.stdout:4/383: mknod d0/d8/c78 0 2024-05-25T06:11:27.928 INFO:tasks.workunit.client.1.smithi203.stdout:4/384: write d0/d9/f17 [1207312,42961] 0 2024-05-25T06:11:27.929 INFO:tasks.workunit.client.1.smithi203.stdout:9/368: unlink d0/d17/c6d 0 2024-05-25T06:11:27.931 INFO:tasks.workunit.client.1.smithi203.stdout:2/460: dread d2/d38/f6e [0,4194304] 0 2024-05-25T06:11:27.933 INFO:tasks.workunit.client.1.smithi203.stdout:2/461: creat d2/d1a/d95/fa1 x:0 0 0 2024-05-25T06:11:27.933 INFO:tasks.workunit.client.1.smithi203.stdout:9/369: truncate d0/df/d2b/f2f 2371492 0 2024-05-25T06:11:27.936 INFO:tasks.workunit.client.1.smithi203.stdout:4/385: dwrite d0/d8/d1f/d26/f4a [0,4194304] 0 2024-05-25T06:11:27.936 INFO:tasks.workunit.client.1.smithi203.stdout:9/370: write d0/d17/d20/f46 [2428603,120384] 0 2024-05-25T06:11:27.937 INFO:tasks.workunit.client.1.smithi203.stdout:4/386: fsync d0/d8/d52/f71 0 2024-05-25T06:11:27.938 INFO:tasks.workunit.client.1.smithi203.stdout:9/371: stat d0/df/l54 0 2024-05-25T06:11:27.939 INFO:tasks.workunit.client.1.smithi203.stdout:9/372: read - d0/df/d2d/f7c zero size 2024-05-25T06:11:27.944 INFO:tasks.workunit.client.1.smithi203.stdout:4/387: rename d0/d9/d24 to d0/d9/d50/d79 0 2024-05-25T06:11:27.945 INFO:tasks.workunit.client.1.smithi203.stdout:4/388: chown d0/d9/d11/l74 861785 1 2024-05-25T06:11:27.945 INFO:tasks.workunit.client.1.smithi203.stdout:9/373: mknod d0/d17/d20/d6e/d75/c7e 0 2024-05-25T06:11:27.947 INFO:tasks.workunit.client.1.smithi203.stdout:4/389: mkdir d0/d8/d52/d59/d7a 0 2024-05-25T06:11:27.947 INFO:tasks.workunit.client.1.smithi203.stdout:9/374: dread d0/df/f4a [0,4194304] 0 2024-05-25T06:11:27.949 INFO:tasks.workunit.client.1.smithi203.stdout:9/375: fsync d0/f24 0 2024-05-25T06:11:27.949 INFO:tasks.workunit.client.1.smithi203.stdout:4/390: creat d0/d8/d1f/d26/d3b/f7b x:0 0 0 2024-05-25T06:11:27.949 INFO:tasks.workunit.client.1.smithi203.stdout:9/376: chown d0/f13 1827 1 2024-05-25T06:11:27.951 INFO:tasks.workunit.client.1.smithi203.stdout:9/377: chown d0/f51 1924786935 1 2024-05-25T06:11:27.953 INFO:tasks.workunit.client.1.smithi203.stdout:4/391: dread d0/d9/f17 [0,4194304] 0 2024-05-25T06:11:27.954 INFO:tasks.workunit.client.1.smithi203.stdout:4/392: dread - d0/d9/d50/d79/f5f zero size 2024-05-25T06:11:27.956 INFO:tasks.workunit.client.1.smithi203.stdout:4/393: symlink d0/d8/d52/l7c 0 2024-05-25T06:11:27.957 INFO:tasks.workunit.client.1.smithi203.stdout:9/378: dwrite d0/f21 [0,4194304] 0 2024-05-25T06:11:27.958 INFO:tasks.workunit.client.1.smithi203.stdout:4/394: write d0/d8/d52/f5e [86386,117451] 0 2024-05-25T06:11:27.960 INFO:tasks.workunit.client.1.smithi203.stdout:4/395: read d0/d9/f64 [20711,70753] 0 2024-05-25T06:11:27.961 INFO:tasks.workunit.client.1.smithi203.stdout:9/379: rename d0/d17/d23/l28 to d0/l7f 0 2024-05-25T06:11:28.005 INFO:tasks.workunit.client.1.smithi203.stdout:1/370: sync 2024-05-25T06:11:28.026 INFO:tasks.workunit.client.1.smithi203.stdout:4/396: dwrite d0/f16 [0,4194304] 0 2024-05-25T06:11:28.035 INFO:tasks.workunit.client.1.smithi203.stdout:2/462: write d2/d4/d10/d32/f1b [4408425,68678] 0 2024-05-25T06:11:28.037 INFO:tasks.workunit.client.1.smithi203.stdout:2/463: write d2/d1a/f7f [4347425,111594] 0 2024-05-25T06:11:28.037 INFO:tasks.workunit.client.1.smithi203.stdout:1/371: dread d14/f32 [0,4194304] 0 2024-05-25T06:11:28.037 INFO:tasks.workunit.client.1.smithi203.stdout:2/464: chown d2/d4/l12 10512569 1 2024-05-25T06:11:28.041 INFO:tasks.workunit.client.1.smithi203.stdout:1/372: truncate d14/d57/d29/f2b 5974336 0 2024-05-25T06:11:28.043 INFO:tasks.workunit.client.1.smithi203.stdout:1/373: write d14/d57/d1d/f2f [5946605,108253] 0 2024-05-25T06:11:28.044 INFO:tasks.workunit.client.1.smithi203.stdout:2/465: dwrite d2/d38/f29 [0,4194304] 0 2024-05-25T06:11:28.045 INFO:tasks.workunit.client.1.smithi203.stdout:2/466: chown d2/d38/f6e 108 1 2024-05-25T06:11:28.046 INFO:tasks.workunit.client.1.smithi203.stdout:1/374: mkdir d14/d81 0 2024-05-25T06:11:28.047 INFO:tasks.workunit.client.1.smithi203.stdout:2/467: fsync d2/d4/d10/d32/f52 0 2024-05-25T06:11:28.048 INFO:tasks.workunit.client.1.smithi203.stdout:2/468: fdatasync d2/d38/f1d 0 2024-05-25T06:11:28.054 INFO:tasks.workunit.client.1.smithi203.stdout:2/469: dwrite d2/d4/d10/d32/f52 [0,4194304] 0 2024-05-25T06:11:28.058 INFO:tasks.workunit.client.1.smithi203.stdout:9/380: creat d0/df/d37/d4f/f80 x:0 0 0 2024-05-25T06:11:28.059 INFO:tasks.workunit.client.1.smithi203.stdout:9/381: chown d0/d17/d20/d6e/d75/d78/d7a 25866 1 2024-05-25T06:11:28.059 INFO:tasks.workunit.client.1.smithi203.stdout:9/382: stat d0/f21 0 2024-05-25T06:11:28.063 INFO:tasks.workunit.client.1.smithi203.stdout:1/375: rmdir d14/d57/d29 39 2024-05-25T06:11:28.066 INFO:tasks.workunit.client.1.smithi203.stdout:1/376: creat d14/d37/d5e/f82 x:0 0 0 2024-05-25T06:11:28.068 INFO:tasks.workunit.client.1.smithi203.stdout:1/377: mknod d14/d37/d5e/c83 0 2024-05-25T06:11:28.068 INFO:tasks.workunit.client.1.smithi203.stdout:1/378: readlink d14/d57/d6b/l68 0 2024-05-25T06:11:28.069 INFO:tasks.workunit.client.1.smithi203.stdout:2/470: truncate d2/d1a/d33/f71 4377551 0 2024-05-25T06:11:28.069 INFO:tasks.workunit.client.1.smithi203.stdout:2/471: dread - d2/d4/d10/d50/f88 zero size 2024-05-25T06:11:28.070 INFO:tasks.workunit.client.1.smithi203.stdout:1/379: mknod d14/c84 0 2024-05-25T06:11:28.071 INFO:tasks.workunit.client.1.smithi203.stdout:1/380: chown d14/d57/d1d/f2a 216115 1 2024-05-25T06:11:28.073 INFO:tasks.workunit.client.1.smithi203.stdout:1/381: creat d14/d37/f85 x:0 0 0 2024-05-25T06:11:28.073 INFO:tasks.workunit.client.1.smithi203.stdout:1/382: dread - d14/d7c/f7d zero size 2024-05-25T06:11:28.077 INFO:tasks.workunit.client.1.smithi203.stdout:9/383: dread d0/f13 [0,4194304] 0 2024-05-25T06:11:28.078 INFO:tasks.workunit.client.1.smithi203.stdout:1/383: rename d14/d37/l47 to d14/d37/d5e/l86 0 2024-05-25T06:11:28.078 INFO:tasks.workunit.client.1.smithi203.stdout:2/472: write d2/d6/f46 [59056,2459] 0 2024-05-25T06:11:28.079 INFO:tasks.workunit.client.1.smithi203.stdout:9/384: creat d0/d17/f81 x:0 0 0 2024-05-25T06:11:28.082 INFO:tasks.workunit.client.1.smithi203.stdout:2/473: creat d2/d4/d10/d5f/fa2 x:0 0 0 2024-05-25T06:11:28.082 INFO:tasks.workunit.client.1.smithi203.stdout:2/474: read - d2/d4/d10/d5f/fa2 zero size 2024-05-25T06:11:28.086 INFO:tasks.workunit.client.1.smithi203.stdout:1/384: write d14/d57/d6b/f5d [482436,107633] 0 2024-05-25T06:11:28.086 INFO:tasks.workunit.client.1.smithi203.stdout:9/385: write d0/df/d37/f70 [697029,63536] 0 2024-05-25T06:11:28.087 INFO:tasks.workunit.client.1.smithi203.stdout:2/475: mkdir d2/d38/d43/d60/da3 0 2024-05-25T06:11:28.088 INFO:tasks.workunit.client.1.smithi203.stdout:2/476: chown d2/d4/d10/d50/f82 12239321 1 2024-05-25T06:11:28.091 INFO:tasks.workunit.client.1.smithi203.stdout:9/386: link d0/d17/d20/d29/c4d d0/df/d37/c82 0 2024-05-25T06:11:28.093 INFO:tasks.workunit.client.1.smithi203.stdout:9/387: symlink d0/l83 0 2024-05-25T06:11:28.096 INFO:tasks.workunit.client.1.smithi203.stdout:9/388: truncate d0/f36 2955610 0 2024-05-25T06:11:28.102 INFO:tasks.workunit.client.1.smithi203.stdout:9/389: dwrite d0/d17/d20/f44 [0,4194304] 0 2024-05-25T06:11:28.102 INFO:tasks.workunit.client.1.smithi203.stdout:9/390: write d0/d17/f4b [8161940,74974] 0 2024-05-25T06:11:28.106 INFO:tasks.workunit.client.1.smithi203.stdout:9/391: getdents d0/d17/d20/d3c/d5c 0 2024-05-25T06:11:28.107 INFO:tasks.workunit.client.1.smithi203.stdout:9/392: dread - d0/d17/d23/f5a zero size 2024-05-25T06:11:28.110 INFO:tasks.workunit.client.1.smithi203.stdout:9/393: rename d0/df/f1a to d0/d17/d20/d6e/d75/d78/f84 0 2024-05-25T06:11:28.126 INFO:tasks.workunit.client.1.smithi203.stdout:9/394: dread d0/df/d2b/f2f [0,4194304] 0 2024-05-25T06:11:28.127 INFO:tasks.workunit.client.1.smithi203.stdout:9/395: write d0/d17/d20/d3c/f7d [570218,89288] 0 2024-05-25T06:11:28.129 INFO:tasks.workunit.client.1.smithi203.stdout:9/396: symlink d0/df/d37/l85 0 2024-05-25T06:11:28.130 INFO:tasks.workunit.client.1.smithi203.stdout:9/397: dread - d0/d17/d23/f6c zero size 2024-05-25T06:11:28.130 INFO:tasks.workunit.client.1.smithi203.stdout:9/398: write d0/f24 [3581025,21648] 0 2024-05-25T06:11:28.131 INFO:tasks.workunit.client.1.smithi203.stdout:9/399: stat d0/d17/d20/d3c/f7d 0 2024-05-25T06:11:28.132 INFO:tasks.workunit.client.1.smithi203.stdout:9/400: dread - d0/df/d37/d4f/f80 zero size 2024-05-25T06:11:28.134 INFO:tasks.workunit.client.1.smithi203.stdout:9/401: truncate d0/df/f1d 347759 0 2024-05-25T06:11:28.135 INFO:tasks.workunit.client.1.smithi203.stdout:9/402: creat d0/d57/f86 x:0 0 0 2024-05-25T06:11:28.136 INFO:tasks.workunit.client.1.smithi203.stdout:9/403: creat d0/df/d73/f87 x:0 0 0 2024-05-25T06:11:28.137 INFO:tasks.workunit.client.1.smithi203.stdout:9/404: write d0/d17/f33 [4733939,50746] 0 2024-05-25T06:11:28.138 INFO:tasks.workunit.client.1.smithi203.stdout:9/405: chown d0/d17/d20/d3c/l3e 96676 1 2024-05-25T06:11:28.139 INFO:tasks.workunit.client.1.smithi203.stdout:9/406: mknod d0/df/d73/c88 0 2024-05-25T06:11:28.143 INFO:tasks.workunit.client.1.smithi203.stdout:9/407: link d0/l1 d0/df/d2d/l89 0 2024-05-25T06:11:28.145 INFO:tasks.workunit.client.1.smithi203.stdout:9/408: creat d0/d17/d20/d29/f8a x:0 0 0 2024-05-25T06:11:28.146 INFO:tasks.workunit.client.1.smithi203.stdout:9/409: symlink d0/df/d2b/l8b 0 2024-05-25T06:11:28.161 INFO:tasks.workunit.client.1.smithi203.stdout:9/410: dread d0/f19 [0,4194304] 0 2024-05-25T06:11:28.163 INFO:tasks.workunit.client.1.smithi203.stdout:9/411: mknod d0/df/d2b/c8c 0 2024-05-25T06:11:28.168 INFO:tasks.workunit.client.1.smithi203.stdout:5/666: sync 2024-05-25T06:11:28.168 INFO:tasks.workunit.client.1.smithi203.stdout:7/418: sync 2024-05-25T06:11:28.170 INFO:tasks.workunit.client.1.smithi203.stdout:7/419: symlink d1/d9/d2c/d3a/l96 0 2024-05-25T06:11:28.171 INFO:tasks.workunit.client.1.smithi203.stdout:7/420: fsync d1/d9/d2c/d3a/f4b 0 2024-05-25T06:11:28.171 INFO:tasks.workunit.client.1.smithi203.stdout:5/667: mkdir dc/d11/d17/d1a/d72/de4 0 2024-05-25T06:11:28.173 INFO:tasks.workunit.client.1.smithi203.stdout:5/668: write dc/d11/d17/d45/f46 [3827237,81180] 0 2024-05-25T06:11:28.173 INFO:tasks.workunit.client.1.smithi203.stdout:7/421: mknod d1/d9/d2c/d36/d3b/c97 0 2024-05-25T06:11:28.175 INFO:tasks.workunit.client.1.smithi203.stdout:5/669: creat dc/daa/fe5 x:0 0 0 2024-05-25T06:11:28.178 INFO:tasks.workunit.client.1.smithi203.stdout:9/412: dread d0/d17/f4b [4194304,4194304] 0 2024-05-25T06:11:28.186 INFO:tasks.workunit.client.1.smithi203.stdout:5/670: dread dc/d11/d17/f31 [0,4194304] 0 2024-05-25T06:11:28.187 INFO:tasks.workunit.client.1.smithi203.stdout:5/671: write dc/daa/fbc [2963554,75943] 0 2024-05-25T06:11:28.190 INFO:tasks.workunit.client.1.smithi203.stdout:5/672: link dc/d11/d17/f31 dc/fe6 0 2024-05-25T06:11:28.268 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:27 smithi203 ceph-mon[36030]: pgmap v14: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 129 MiB/s wr, 418 op/s 2024-05-25T06:11:28.297 INFO:tasks.workunit.client.1.smithi203.stdout:3/527: sync 2024-05-25T06:11:28.298 INFO:tasks.workunit.client.1.smithi203.stdout:3/528: write d10/d16/d50/f84 [393544,8597] 0 2024-05-25T06:11:28.306 INFO:tasks.workunit.client.1.smithi203.stdout:3/529: rmdir d10/d16/d23/d38 39 2024-05-25T06:11:28.310 INFO:tasks.workunit.client.1.smithi203.stdout:9/413: truncate d0/f21 1533432 0 2024-05-25T06:11:28.310 INFO:tasks.workunit.client.1.smithi203.stdout:9/414: chown d0/d17/d20/d29/l55 114184 1 2024-05-25T06:11:28.311 INFO:tasks.workunit.client.1.smithi203.stdout:9/415: creat d0/d17/d20/d6e/f8d x:0 0 0 2024-05-25T06:11:28.313 INFO:tasks.workunit.client.1.smithi203.stdout:5/673: symlink dc/d11/d17/d1a/d2d/d53/le7 0 2024-05-25T06:11:28.314 INFO:tasks.workunit.client.1.smithi203.stdout:5/674: fdatasync dc/d11/d17/d1a/d2d/d4d/d64/f9e 0 2024-05-25T06:11:28.319 INFO:tasks.workunit.client.1.smithi203.stdout:9/416: truncate d0/d17/d20/f31 1658 0 2024-05-25T06:11:28.319 INFO:tasks.workunit.client.1.smithi203.stdout:5/675: dwrite dc/d11/f1c [0,4194304] 0 2024-05-25T06:11:28.323 INFO:tasks.workunit.client.1.smithi203.stdout:9/417: creat d0/d17/d23/f8e x:0 0 0 2024-05-25T06:11:28.323 INFO:tasks.workunit.client.1.smithi203.stdout:9/418: chown d0/d17/d20/d3c/l3e 1922 1 2024-05-25T06:11:28.326 INFO:tasks.workunit.client.1.smithi203.stdout:9/419: mkdir d0/d57/d8f 0 2024-05-25T06:11:28.327 INFO:tasks.workunit.client.1.smithi203.stdout:5/676: getdents dc/daa 0 2024-05-25T06:11:28.329 INFO:tasks.workunit.client.1.smithi203.stdout:5/677: dread dc/d11/d17/d44/f8f [0,4194304] 0 2024-05-25T06:11:28.332 INFO:tasks.workunit.client.1.smithi203.stdout:5/678: creat dc/d11/d42/d94/dad/db1/fe8 x:0 0 0 2024-05-25T06:11:28.334 INFO:tasks.workunit.client.1.smithi203.stdout:5/679: fsync dc/d11/d17/d1a/d2d/d4d/f83 0 2024-05-25T06:11:28.339 INFO:tasks.workunit.client.1.smithi203.stdout:5/680: write dc/d11/d17/d1a/d2d/d4d/d64/dd4/fd3 [428739,33882] 0 2024-05-25T06:11:28.345 INFO:tasks.workunit.client.1.smithi203.stdout:5/681: truncate dc/d11/d17/d45/f52 6898342 0 2024-05-25T06:11:28.428 INFO:tasks.workunit.client.1.smithi203.stdout:8/523: sync 2024-05-25T06:11:28.429 INFO:tasks.workunit.client.1.smithi203.stdout:6/598: sync 2024-05-25T06:11:28.429 INFO:tasks.workunit.client.1.smithi203.stdout:0/313: sync 2024-05-25T06:11:28.429 INFO:tasks.workunit.client.1.smithi203.stdout:0/314: fsync dd/f24 0 2024-05-25T06:11:28.429 INFO:tasks.workunit.client.1.smithi203.stdout:6/599: chown d8/d29/d80/c85 0 1 2024-05-25T06:11:28.430 INFO:tasks.workunit.client.1.smithi203.stdout:6/600: chown d8/d2a/c50 5951 1 2024-05-25T06:11:28.430 INFO:tasks.workunit.client.1.smithi203.stdout:8/524: chown d2/d5/d18/d31/c43 0 1 2024-05-25T06:11:28.432 INFO:tasks.workunit.client.1.smithi203.stdout:6/601: mkdir d8/d2a/d6c/dc1 0 2024-05-25T06:11:28.435 INFO:tasks.workunit.client.1.smithi203.stdout:0/315: creat dd/f72 x:0 0 0 2024-05-25T06:11:28.437 INFO:tasks.workunit.client.1.smithi203.stdout:8/525: creat d2/d5/d10/d45/faa x:0 0 0 2024-05-25T06:11:28.439 INFO:tasks.workunit.client.1.smithi203.stdout:8/526: symlink d2/d5/d10/lab 0 2024-05-25T06:11:28.447 INFO:tasks.workunit.client.1.smithi203.stdout:6/602: write d8/d11/d15/f17 [7190,111209] 0 2024-05-25T06:11:28.449 INFO:tasks.workunit.client.1.smithi203.stdout:0/316: truncate dd/d3a/f4b 3577869 0 2024-05-25T06:11:28.450 INFO:tasks.workunit.client.1.smithi203.stdout:6/603: rmdir d8/d2a/d6c/d87/daa 39 2024-05-25T06:11:28.451 INFO:tasks.workunit.client.1.smithi203.stdout:0/317: creat dd/d5c/f73 x:0 0 0 2024-05-25T06:11:28.452 INFO:tasks.workunit.client.1.smithi203.stdout:6/604: symlink d8/d11/d15/lc2 0 2024-05-25T06:11:28.454 INFO:tasks.workunit.client.1.smithi203.stdout:0/318: creat dd/d3d/d4f/d62/f74 x:0 0 0 2024-05-25T06:11:28.455 INFO:tasks.workunit.client.1.smithi203.stdout:6/605: creat d8/d11/d1b/fc3 x:0 0 0 2024-05-25T06:11:28.455 INFO:tasks.workunit.client.1.smithi203.stdout:0/319: mkdir dd/d3d/d75 0 2024-05-25T06:11:28.456 INFO:tasks.workunit.client.1.smithi203.stdout:6/606: write d8/d29/f7e [3652063,1384] 0 2024-05-25T06:11:28.458 INFO:tasks.workunit.client.1.smithi203.stdout:6/607: chown d8/d11/d15/d28/c1c 1416 1 2024-05-25T06:11:28.458 INFO:tasks.workunit.client.1.smithi203.stdout:0/320: dread dd/d10/f4a [0,4194304] 0 2024-05-25T06:11:28.459 INFO:tasks.workunit.client.1.smithi203.stdout:6/608: write d8/d29/f7e [299851,88394] 0 2024-05-25T06:11:28.461 INFO:tasks.workunit.client.1.smithi203.stdout:6/609: symlink d8/d2a/d63/lc4 0 2024-05-25T06:11:28.464 INFO:tasks.workunit.client.1.smithi203.stdout:6/610: dread d8/d11/d15/f2e [0,4194304] 0 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/10: sync 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/11: stat - no entries 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/12: dread - no filename 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/13: unlink - no file 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/14: dread - no filename 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/15: rename - no filename 2024-05-25T06:11:28.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/16: fdatasync - no filename 2024-05-25T06:11:28.468 INFO:tasks.workunit.client.0.smithi160.stdout:0/17: creat f0 x:0 0 0 2024-05-25T06:11:28.468 INFO:tasks.workunit.client.0.smithi160.stdout:0/18: readlink - no filename 2024-05-25T06:11:28.469 INFO:tasks.workunit.client.0.smithi160.stdout:0/19: chown f0 7 1 2024-05-25T06:11:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:27 smithi160 ceph-mon[29041]: pgmap v14: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 129 MiB/s wr, 418 op/s 2024-05-25T06:11:28.470 INFO:tasks.workunit.client.0.smithi160.stdout:0/20: mknod c1 0 2024-05-25T06:11:28.471 INFO:tasks.workunit.client.1.smithi203.stdout:6/611: dwrite d8/fa0 [0,4194304] 0 2024-05-25T06:11:28.471 INFO:tasks.workunit.client.0.smithi160.stdout:0/21: dread - f0 zero size 2024-05-25T06:11:28.473 INFO:tasks.workunit.client.1.smithi203.stdout:0/321: dread dd/f44 [0,4194304] 0 2024-05-25T06:11:28.477 INFO:tasks.workunit.client.1.smithi203.stdout:0/322: getdents dd/d3d/d4f/d62 0 2024-05-25T06:11:28.479 INFO:tasks.workunit.client.1.smithi203.stdout:6/612: dwrite d8/d11/d47/fbf [0,4194304] 0 2024-05-25T06:11:28.480 INFO:tasks.workunit.client.1.smithi203.stdout:0/323: truncate dd/d3d/d4f/d62/f74 861105 0 2024-05-25T06:11:28.480 INFO:tasks.workunit.client.1.smithi203.stdout:6/613: readlink d8/d2a/d63/lc4 0 2024-05-25T06:11:28.483 INFO:tasks.workunit.client.1.smithi203.stdout:0/324: creat dd/d3a/f76 x:0 0 0 2024-05-25T06:11:28.484 INFO:tasks.workunit.client.1.smithi203.stdout:6/614: fsync d8/d11/d15/d28/d52/d43/f92 0 2024-05-25T06:11:28.484 INFO:tasks.workunit.client.1.smithi203.stdout:6/615: readlink d8/d11/d15/lb1 0 2024-05-25T06:11:28.485 INFO:tasks.workunit.client.1.smithi203.stdout:0/325: symlink dd/d3a/d66/d6e/l77 0 2024-05-25T06:11:28.496 INFO:tasks.workunit.client.1.smithi203.stdout:2/477: sync 2024-05-25T06:11:28.496 INFO:tasks.workunit.client.1.smithi203.stdout:1/385: sync 2024-05-25T06:11:28.496 INFO:tasks.workunit.client.1.smithi203.stdout:7/422: sync 2024-05-25T06:11:28.496 INFO:tasks.workunit.client.1.smithi203.stdout:4/397: sync 2024-05-25T06:11:28.498 INFO:tasks.workunit.client.1.smithi203.stdout:0/326: dwrite dd/d10/f41 [0,4194304] 0 2024-05-25T06:11:28.500 INFO:tasks.workunit.client.1.smithi203.stdout:7/423: mknod d1/d11/d20/c98 0 2024-05-25T06:11:28.500 INFO:tasks.workunit.client.1.smithi203.stdout:2/478: creat d2/d38/d43/d60/fa4 x:0 0 0 2024-05-25T06:11:28.500 INFO:tasks.workunit.client.1.smithi203.stdout:1/386: mknod d14/d37/c87 0 2024-05-25T06:11:28.501 INFO:tasks.workunit.client.1.smithi203.stdout:0/327: mkdir dd/d3d/d4f/d62/d78 0 2024-05-25T06:11:28.505 INFO:tasks.workunit.client.1.smithi203.stdout:2/479: dread d2/d6/f26 [0,4194304] 0 2024-05-25T06:11:28.505 INFO:tasks.workunit.client.1.smithi203.stdout:0/328: rmdir dd/d45 39 2024-05-25T06:11:28.505 INFO:tasks.workunit.client.1.smithi203.stdout:0/329: dread - dd/f64 zero size 2024-05-25T06:11:28.506 INFO:tasks.workunit.client.1.smithi203.stdout:6/616: dread d8/d11/f23 [0,4194304] 0 2024-05-25T06:11:28.506 INFO:tasks.workunit.client.1.smithi203.stdout:1/387: symlink d14/d57/l88 0 2024-05-25T06:11:28.507 INFO:tasks.workunit.client.1.smithi203.stdout:4/398: getdents d0/d9/d11/d54 0 2024-05-25T06:11:28.508 INFO:tasks.workunit.client.1.smithi203.stdout:1/388: truncate d14/d57/d6b/f5d 1005495 0 2024-05-25T06:11:28.509 INFO:tasks.workunit.client.1.smithi203.stdout:4/399: fdatasync d0/d8/d3a/f6d 0 2024-05-25T06:11:28.513 INFO:tasks.workunit.client.1.smithi203.stdout:1/389: creat d14/d57/d1d/f89 x:0 0 0 2024-05-25T06:11:28.514 INFO:tasks.workunit.client.1.smithi203.stdout:4/400: creat d0/d8/d1f/d26/d43/f7d x:0 0 0 2024-05-25T06:11:28.514 INFO:tasks.workunit.client.1.smithi203.stdout:2/480: getdents d2/d1a/d95 0 2024-05-25T06:11:28.515 INFO:tasks.workunit.client.1.smithi203.stdout:1/390: chown d14/d23/f78 694470538 1 2024-05-25T06:11:28.516 INFO:tasks.workunit.client.1.smithi203.stdout:6/617: getdents d8/d11/d15/d28/d52/d43 0 2024-05-25T06:11:28.516 INFO:tasks.workunit.client.1.smithi203.stdout:0/330: dwrite dd/f5a [0,4194304] 0 2024-05-25T06:11:28.519 INFO:tasks.workunit.client.1.smithi203.stdout:7/424: dwrite d1/f24 [0,4194304] 0 2024-05-25T06:11:28.519 INFO:tasks.workunit.client.1.smithi203.stdout:6/618: chown d8/d3d/f44 13107235 1 2024-05-25T06:11:28.521 INFO:tasks.workunit.client.1.smithi203.stdout:4/401: unlink d0/d8/d1f/d26/f2e 0 2024-05-25T06:11:28.521 INFO:tasks.workunit.client.1.smithi203.stdout:6/619: chown d8/d11/d1b/c41 1588411 1 2024-05-25T06:11:28.522 INFO:tasks.workunit.client.1.smithi203.stdout:7/425: write d1/d11/d3d/d4c/f93 [435024,40210] 0 2024-05-25T06:11:28.523 INFO:tasks.workunit.client.1.smithi203.stdout:1/391: write d14/f17 [3601556,83585] 0 2024-05-25T06:11:28.525 INFO:tasks.workunit.client.1.smithi203.stdout:1/392: chown d14/c15 4449644 1 2024-05-25T06:11:28.526 INFO:tasks.workunit.client.1.smithi203.stdout:6/620: mknod d8/d29/cc5 0 2024-05-25T06:11:28.526 INFO:tasks.workunit.client.1.smithi203.stdout:0/331: creat dd/d3d/d75/f79 x:0 0 0 2024-05-25T06:11:28.528 INFO:tasks.workunit.client.1.smithi203.stdout:1/393: chown d14/f18 2 1 2024-05-25T06:11:28.530 INFO:tasks.workunit.client.1.smithi203.stdout:0/332: fsync dd/d3d/d4f/f50 0 2024-05-25T06:11:28.531 INFO:tasks.workunit.client.1.smithi203.stdout:2/481: dwrite d2/d1a/f8b [0,4194304] 0 2024-05-25T06:11:28.533 INFO:tasks.workunit.client.1.smithi203.stdout:6/621: write d8/d11/fb6 [998254,103595] 0 2024-05-25T06:11:28.534 INFO:tasks.workunit.client.1.smithi203.stdout:2/482: chown d2/d38/d2f/f4f 61265757 1 2024-05-25T06:11:28.535 INFO:tasks.workunit.client.1.smithi203.stdout:7/426: rename d1/d9/d60/c84 to d1/d11/c99 0 2024-05-25T06:11:28.535 INFO:tasks.workunit.client.1.smithi203.stdout:3/530: sync 2024-05-25T06:11:28.535 INFO:tasks.workunit.client.1.smithi203.stdout:9/420: sync 2024-05-25T06:11:28.536 INFO:tasks.workunit.client.1.smithi203.stdout:5/682: sync 2024-05-25T06:11:28.536 INFO:tasks.workunit.client.1.smithi203.stdout:8/527: sync 2024-05-25T06:11:28.537 INFO:tasks.workunit.client.1.smithi203.stdout:1/394: creat d14/d81/f8a x:0 0 0 2024-05-25T06:11:28.537 INFO:tasks.workunit.client.1.smithi203.stdout:0/333: creat dd/d3d/d48/f7a x:0 0 0 2024-05-25T06:11:28.540 INFO:tasks.workunit.client.1.smithi203.stdout:2/483: write d2/d1a/d95/fa1 [810341,40531] 0 2024-05-25T06:11:28.541 INFO:tasks.workunit.client.1.smithi203.stdout:1/395: fsync d14/f18 0 2024-05-25T06:11:28.541 INFO:tasks.workunit.client.1.smithi203.stdout:4/402: dwrite d0/d8/d3a/f53 [0,4194304] 0 2024-05-25T06:11:28.542 INFO:tasks.workunit.client.1.smithi203.stdout:8/528: fdatasync d2/d5/d10/d45/d5d/f82 0 2024-05-25T06:11:28.545 INFO:tasks.workunit.client.1.smithi203.stdout:9/421: creat d0/d17/d23/f90 x:0 0 0 2024-05-25T06:11:28.549 INFO:tasks.workunit.client.1.smithi203.stdout:4/403: rmdir d0/d9/d50/d79 39 2024-05-25T06:11:28.550 INFO:tasks.workunit.client.1.smithi203.stdout:8/529: write d2/d5/d28/f3a [4241226,88672] 0 2024-05-25T06:11:28.550 INFO:tasks.workunit.client.1.smithi203.stdout:9/422: chown d0/d17/d20/d6e/d75/c7e 557539919 1 2024-05-25T06:11:28.551 INFO:tasks.workunit.client.1.smithi203.stdout:2/484: mknod d2/d38/d43/d60/ca5 0 2024-05-25T06:11:28.552 INFO:tasks.workunit.client.1.smithi203.stdout:7/427: getdents d1/d9/d2c/d62 0 2024-05-25T06:11:28.553 INFO:tasks.workunit.client.1.smithi203.stdout:3/531: dread d10/d16/d23/d45/f4e [0,4194304] 0 2024-05-25T06:11:28.555 INFO:tasks.workunit.client.1.smithi203.stdout:2/485: mkdir d2/d4/da6 0 2024-05-25T06:11:28.555 INFO:tasks.workunit.client.1.smithi203.stdout:5/683: link dc/d11/d18/c1d dc/daa/ce9 0 2024-05-25T06:11:28.557 INFO:tasks.workunit.client.1.smithi203.stdout:8/530: rmdir d2/d5/d10/d11/d1f/d75 39 2024-05-25T06:11:28.557 INFO:tasks.workunit.client.1.smithi203.stdout:1/396: link d14/l27 d14/d57/d1d/l8b 0 2024-05-25T06:11:28.558 INFO:tasks.workunit.client.1.smithi203.stdout:7/428: mkdir d1/d11/d3d/d9a 0 2024-05-25T06:11:28.558 INFO:tasks.workunit.client.1.smithi203.stdout:0/334: dwrite dd/d10/f41 [4194304,4194304] 0 2024-05-25T06:11:28.559 INFO:tasks.workunit.client.1.smithi203.stdout:4/404: write d0/d8/d1f/d26/f1e [6112847,97979] 0 2024-05-25T06:11:28.559 INFO:tasks.workunit.client.1.smithi203.stdout:3/532: creat d10/d16/d50/d8b/fad x:0 0 0 2024-05-25T06:11:28.560 INFO:tasks.workunit.client.1.smithi203.stdout:2/486: creat d2/d38/d2f/fa7 x:0 0 0 2024-05-25T06:11:28.561 INFO:tasks.workunit.client.1.smithi203.stdout:1/397: stat d14/d37 0 2024-05-25T06:11:28.563 INFO:tasks.workunit.client.1.smithi203.stdout:8/531: fsync d2/d5/d28/d38/f49 0 2024-05-25T06:11:28.564 INFO:tasks.workunit.client.1.smithi203.stdout:6/622: truncate d8/d11/d1b/f4d 787434 0 2024-05-25T06:11:28.565 INFO:tasks.workunit.client.1.smithi203.stdout:2/487: read d2/d4/d10/d5f/f25 [683532,55527] 0 2024-05-25T06:11:28.565 INFO:tasks.workunit.client.1.smithi203.stdout:7/429: unlink d1/d11/f2b 0 2024-05-25T06:11:28.566 INFO:tasks.workunit.client.1.smithi203.stdout:9/423: dwrite d0/f24 [0,4194304] 0 2024-05-25T06:11:28.567 INFO:tasks.workunit.client.1.smithi203.stdout:4/405: chown d0/d9/d50/d79/l73 3111430 1 2024-05-25T06:11:28.567 INFO:tasks.workunit.client.1.smithi203.stdout:5/684: symlink dc/d11/d17/d1a/d2d/lea 0 2024-05-25T06:11:28.570 INFO:tasks.workunit.client.1.smithi203.stdout:3/533: creat d10/d16/d23/d67/fae x:0 0 0 2024-05-25T06:11:28.571 INFO:tasks.workunit.client.1.smithi203.stdout:1/398: read - d14/d57/d29/f7b zero size 2024-05-25T06:11:28.571 INFO:tasks.workunit.client.1.smithi203.stdout:8/532: mkdir d2/d5/d18/d31/dac 0 2024-05-25T06:11:28.573 INFO:tasks.workunit.client.1.smithi203.stdout:8/533: dread - d2/f99 zero size 2024-05-25T06:11:28.574 INFO:tasks.workunit.client.1.smithi203.stdout:7/430: mkdir d1/d9/d2c/d62/d9b 0 2024-05-25T06:11:28.576 INFO:tasks.workunit.client.1.smithi203.stdout:4/406: fdatasync d0/d9/d50/d6b/f30 0 2024-05-25T06:11:28.576 INFO:tasks.workunit.client.1.smithi203.stdout:8/534: mknod d2/cad 0 2024-05-25T06:11:28.578 INFO:tasks.workunit.client.1.smithi203.stdout:1/399: dread - d14/d57/d1d/d3e/f72 zero size 2024-05-25T06:11:28.579 INFO:tasks.workunit.client.1.smithi203.stdout:0/335: getdents dd/d3d/d48 0 2024-05-25T06:11:28.579 INFO:tasks.workunit.client.1.smithi203.stdout:5/685: truncate dc/f80 1476804 0 2024-05-25T06:11:28.580 INFO:tasks.workunit.client.1.smithi203.stdout:4/407: dread - d0/d8/d3a/f6d zero size 2024-05-25T06:11:28.580 INFO:tasks.workunit.client.1.smithi203.stdout:8/535: mkdir d2/d5/dae 0 2024-05-25T06:11:28.582 INFO:tasks.workunit.client.1.smithi203.stdout:2/488: dwrite d2/d38/d2f/f34 [4194304,4194304] 0 2024-05-25T06:11:28.583 INFO:tasks.workunit.client.1.smithi203.stdout:7/431: truncate d1/fa 1762945 0 2024-05-25T06:11:28.584 INFO:tasks.workunit.client.1.smithi203.stdout:1/400: symlink d14/d37/d5e/l8c 0 2024-05-25T06:11:28.585 INFO:tasks.workunit.client.1.smithi203.stdout:0/336: unlink dd/d10/f2f 0 2024-05-25T06:11:28.585 INFO:tasks.workunit.client.1.smithi203.stdout:8/536: dread - d2/d5/d2d/fa3 zero size 2024-05-25T06:11:28.587 INFO:tasks.workunit.client.1.smithi203.stdout:5/686: symlink dc/d11/d42/leb 0 2024-05-25T06:11:28.587 INFO:tasks.workunit.client.1.smithi203.stdout:3/534: dwrite d10/d16/d50/f84 [0,4194304] 0 2024-05-25T06:11:28.589 INFO:tasks.workunit.client.1.smithi203.stdout:7/432: fsync d1/f31 0 2024-05-25T06:11:28.593 INFO:tasks.workunit.client.0.smithi160.stdout:3/0: sync 2024-05-25T06:11:28.593 INFO:tasks.workunit.client.0.smithi160.stdout:3/1: dread - no filename 2024-05-25T06:11:28.594 INFO:tasks.workunit.client.1.smithi203.stdout:2/489: fsync d2/d4/d10/d5f/f7b 0 2024-05-25T06:11:28.595 INFO:tasks.workunit.client.1.smithi203.stdout:5/687: rmdir dc/d11/d42/d94/dad/db1 39 2024-05-25T06:11:28.595 INFO:tasks.workunit.client.1.smithi203.stdout:1/401: write d14/d57/d1d/d3e/f61 [345109,86920] 0 2024-05-25T06:11:28.596 INFO:tasks.workunit.client.1.smithi203.stdout:9/424: dwrite d0/df/d37/f58 [0,4194304] 0 2024-05-25T06:11:28.596 INFO:tasks.workunit.client.0.smithi160.stdout:3/2: creat f0 x:0 0 0 2024-05-25T06:11:28.597 INFO:tasks.workunit.client.1.smithi203.stdout:7/433: dread - d1/d9/d2c/d3a/f4b zero size 2024-05-25T06:11:28.598 INFO:tasks.workunit.client.1.smithi203.stdout:8/537: chown d2/d5/d10/f53 1626997 1 2024-05-25T06:11:28.599 INFO:tasks.workunit.client.1.smithi203.stdout:4/408: truncate d0/d8/d1f/d26/d43/f75 707936 0 2024-05-25T06:11:28.600 INFO:tasks.workunit.client.0.smithi160.stdout:3/3: symlink l1 0 2024-05-25T06:11:28.600 INFO:tasks.workunit.client.0.smithi160.stdout:3/4: read - f0 zero size 2024-05-25T06:11:28.601 INFO:tasks.workunit.client.0.smithi160.stdout:3/5: truncate f0 627105 0 2024-05-25T06:11:28.602 INFO:tasks.workunit.client.0.smithi160.stdout:3/6: chown f0 14648 1 2024-05-25T06:11:28.604 INFO:tasks.workunit.client.1.smithi203.stdout:2/490: symlink d2/d4/d10/la8 0 2024-05-25T06:11:28.604 INFO:tasks.workunit.client.0.smithi160.stdout:3/7: dread f0 [0,4194304] 0 2024-05-25T06:11:28.606 INFO:tasks.workunit.client.1.smithi203.stdout:8/538: write d2/d5/d28/d38/f3c [1374792,30904] 0 2024-05-25T06:11:28.606 INFO:tasks.workunit.client.1.smithi203.stdout:1/402: creat d14/d57/d1d/d3e/d36/f8d x:0 0 0 2024-05-25T06:11:28.607 INFO:tasks.workunit.client.1.smithi203.stdout:4/409: write d0/d8/d3a/f53 [4656547,45613] 0 2024-05-25T06:11:28.607 INFO:tasks.workunit.client.1.smithi203.stdout:5/688: creat dc/d11/d17/d1a/d2d/d53/fec x:0 0 0 2024-05-25T06:11:28.610 INFO:tasks.workunit.client.1.smithi203.stdout:6/623: write d8/d70/f94 [603057,102401] 0 2024-05-25T06:11:28.614 INFO:tasks.workunit.client.1.smithi203.stdout:8/539: chown d2/d5/d28/d38/d81 519473 1 2024-05-25T06:11:28.615 INFO:tasks.workunit.client.1.smithi203.stdout:0/337: dread dd/d10/f33 [0,4194304] 0 2024-05-25T06:11:28.617 INFO:tasks.workunit.client.1.smithi203.stdout:9/425: creat d0/f91 x:0 0 0 2024-05-25T06:11:28.618 INFO:tasks.workunit.client.1.smithi203.stdout:7/434: truncate d1/d9/d2c/d62/d7a/d89/f8c 1314076 0 2024-05-25T06:11:28.621 INFO:tasks.workunit.client.1.smithi203.stdout:3/535: dwrite d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:28.622 INFO:tasks.workunit.client.1.smithi203.stdout:1/403: creat d14/d57/d6b/f8e x:0 0 0 2024-05-25T06:11:28.623 INFO:tasks.workunit.client.1.smithi203.stdout:5/689: unlink dc/d11/d17/d1a/d2d/f7d 0 2024-05-25T06:11:28.624 INFO:tasks.workunit.client.1.smithi203.stdout:6/624: mknod d8/d29/d80/cc6 0 2024-05-25T06:11:28.624 INFO:tasks.workunit.client.1.smithi203.stdout:3/536: stat d10/d22/f57 0 2024-05-25T06:11:28.625 INFO:tasks.workunit.client.1.smithi203.stdout:8/540: dread - d2/d5/d28/d38/d81/d84/f98 zero size 2024-05-25T06:11:28.628 INFO:tasks.workunit.client.1.smithi203.stdout:6/625: chown d8/d3d/c8f 399225 1 2024-05-25T06:11:28.630 INFO:tasks.workunit.client.1.smithi203.stdout:4/410: creat d0/d8/d10/f7e x:0 0 0 2024-05-25T06:11:28.631 INFO:tasks.workunit.client.1.smithi203.stdout:4/411: chown d0/d9/d11/d76 40 1 2024-05-25T06:11:28.632 INFO:tasks.workunit.client.1.smithi203.stdout:9/426: rmdir d0/d57 39 2024-05-25T06:11:28.635 INFO:tasks.workunit.client.1.smithi203.stdout:4/412: dread - d0/d8/d1f/d26/d43/f68 zero size 2024-05-25T06:11:28.636 INFO:tasks.workunit.client.1.smithi203.stdout:1/404: stat d14/d37/d5e/l86 0 2024-05-25T06:11:28.637 INFO:tasks.workunit.client.1.smithi203.stdout:5/690: mkdir dc/d11/d17/d1a/d2d/d4d/ded 0 2024-05-25T06:11:28.638 INFO:tasks.workunit.client.1.smithi203.stdout:0/338: dwrite dd/d3d/d4f/f50 [4194304,4194304] 0 2024-05-25T06:11:28.639 INFO:tasks.workunit.client.1.smithi203.stdout:3/537: rename d10/d16/d23/d4a/d88/f97 to d10/d16/d23/d46/faf 0 2024-05-25T06:11:28.640 INFO:tasks.workunit.client.1.smithi203.stdout:1/405: dread - d14/d57/d6b/f8e zero size 2024-05-25T06:11:28.644 INFO:tasks.workunit.client.1.smithi203.stdout:2/491: dread d2/d4/d10/f21 [0,4194304] 0 2024-05-25T06:11:28.645 INFO:tasks.workunit.client.1.smithi203.stdout:6/626: creat d8/d29/d46/d88/da5/fc7 x:0 0 0 2024-05-25T06:11:28.646 INFO:tasks.workunit.client.1.smithi203.stdout:9/427: mkdir d0/df/d2d/d92 0 2024-05-25T06:11:28.649 INFO:tasks.workunit.client.1.smithi203.stdout:6/627: chown d8/d70 14235762 1 2024-05-25T06:11:28.654 INFO:tasks.workunit.client.1.smithi203.stdout:7/435: truncate d1/d9/d2c/d36/d3b/f47 2545689 0 2024-05-25T06:11:28.657 INFO:tasks.workunit.client.1.smithi203.stdout:5/691: write dc/fe6 [4495906,15250] 0 2024-05-25T06:11:28.659 INFO:tasks.workunit.client.1.smithi203.stdout:8/541: dwrite d2/f14 [0,4194304] 0 2024-05-25T06:11:28.660 INFO:tasks.workunit.client.1.smithi203.stdout:4/413: dwrite d0/d8/d3a/f77 [0,4194304] 0 2024-05-25T06:11:28.660 INFO:tasks.workunit.client.1.smithi203.stdout:7/436: dread - d1/d11/f80 zero size 2024-05-25T06:11:28.665 INFO:tasks.workunit.client.1.smithi203.stdout:9/428: mkdir d0/df/d37/d4f/d93 0 2024-05-25T06:11:28.667 INFO:tasks.workunit.client.1.smithi203.stdout:4/414: write d0/f5 [8206696,28204] 0 2024-05-25T06:11:28.668 INFO:tasks.workunit.client.1.smithi203.stdout:2/492: dwrite d2/d38/d43/d60/f96 [0,4194304] 0 2024-05-25T06:11:28.671 INFO:tasks.workunit.client.1.smithi203.stdout:9/429: chown d0/f24 0 1 2024-05-25T06:11:28.674 INFO:tasks.workunit.client.1.smithi203.stdout:6/628: dwrite d8/d70/f94 [0,4194304] 0 2024-05-25T06:11:28.675 INFO:tasks.workunit.client.1.smithi203.stdout:0/339: truncate dd/d32/d37/f42 3290839 0 2024-05-25T06:11:28.678 INFO:tasks.workunit.client.1.smithi203.stdout:1/406: rename d14/d57/d1d/d3e/l4d to d14/d57/l8f 0 2024-05-25T06:11:28.681 INFO:tasks.workunit.client.0.smithi160.stdout:3/8: rename l1 to l2 0 2024-05-25T06:11:28.683 INFO:tasks.workunit.client.1.smithi203.stdout:5/692: write dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdc [574422,22406] 0 2024-05-25T06:11:28.683 INFO:tasks.workunit.client.1.smithi203.stdout:0/340: truncate f9 3362841 0 2024-05-25T06:11:28.683 INFO:tasks.workunit.client.1.smithi203.stdout:4/415: creat d0/d8/d10/f7f x:0 0 0 2024-05-25T06:11:28.685 INFO:tasks.workunit.client.1.smithi203.stdout:8/542: creat d2/d5/d10/d45/d5d/faf x:0 0 0 2024-05-25T06:11:28.687 INFO:tasks.workunit.client.1.smithi203.stdout:1/407: creat d14/d57/d1d/d3e/f90 x:0 0 0 2024-05-25T06:11:28.689 INFO:tasks.workunit.client.1.smithi203.stdout:7/437: write d1/d9/d2c/d3a/f3e [346432,72757] 0 2024-05-25T06:11:28.689 INFO:tasks.workunit.client.1.smithi203.stdout:8/543: readlink d2/d5/d10/la4 0 2024-05-25T06:11:28.690 INFO:tasks.workunit.client.1.smithi203.stdout:3/538: dwrite d10/d16/d50/f9e [0,4194304] 0 2024-05-25T06:11:28.690 INFO:tasks.workunit.client.1.smithi203.stdout:0/341: rmdir dd/d3a/d66/d6e 39 2024-05-25T06:11:28.692 INFO:tasks.workunit.client.1.smithi203.stdout:2/493: dwrite d2/d4/d10/d5f/f7b [0,4194304] 0 2024-05-25T06:11:28.692 INFO:tasks.workunit.client.1.smithi203.stdout:7/438: fsync d1/f31 0 2024-05-25T06:11:28.694 INFO:tasks.workunit.client.1.smithi203.stdout:5/693: creat dc/d11/d17/d44/d73/d98/fee x:0 0 0 2024-05-25T06:11:28.695 INFO:tasks.workunit.client.1.smithi203.stdout:0/342: readlink dd/d10/l3f 0 2024-05-25T06:11:28.701 INFO:tasks.workunit.client.1.smithi203.stdout:4/416: rename d0/d9/d50/d6b/c4b to d0/d9/d11/d54/c80 0 2024-05-25T06:11:28.704 INFO:tasks.workunit.client.1.smithi203.stdout:4/417: stat d0/d8/d3a/c6e 0 2024-05-25T06:11:28.708 INFO:tasks.workunit.client.1.smithi203.stdout:6/629: truncate d8/fa0 925195 0 2024-05-25T06:11:28.708 INFO:tasks.workunit.client.1.smithi203.stdout:4/418: stat d0/f3 0 2024-05-25T06:11:28.711 INFO:tasks.workunit.client.1.smithi203.stdout:3/539: dwrite d10/d22/f33 [0,4194304] 0 2024-05-25T06:11:28.712 INFO:tasks.workunit.client.1.smithi203.stdout:7/439: symlink d1/d9/d2c/d62/l9c 0 2024-05-25T06:11:28.717 INFO:tasks.workunit.client.1.smithi203.stdout:1/408: rename d14/d37/d5e/f5b to d14/d7c/f91 0 2024-05-25T06:11:28.717 INFO:tasks.workunit.client.1.smithi203.stdout:7/440: chown d1/d9/d2c/d3a 343 1 2024-05-25T06:11:28.719 INFO:tasks.workunit.client.1.smithi203.stdout:5/694: dwrite dc/d11/d17/d45/f62 [0,4194304] 0 2024-05-25T06:11:28.720 INFO:tasks.workunit.client.1.smithi203.stdout:2/494: dread d2/d4/d10/d5f/f7b [0,4194304] 0 2024-05-25T06:11:28.721 INFO:tasks.workunit.client.1.smithi203.stdout:3/540: creat d10/d16/d50/d8b/fb0 x:0 0 0 2024-05-25T06:11:28.721 INFO:tasks.workunit.client.1.smithi203.stdout:6/630: truncate d8/d11/f8e 307471 0 2024-05-25T06:11:28.723 INFO:tasks.workunit.client.1.smithi203.stdout:4/419: getdents d0/d8/d52/d59/d7a 0 2024-05-25T06:11:28.725 INFO:tasks.workunit.client.1.smithi203.stdout:1/409: symlink d14/d57/d1d/d3e/d36/l92 0 2024-05-25T06:11:28.725 INFO:tasks.workunit.client.1.smithi203.stdout:4/420: write d0/d8/d10/f5c [2137785,25966] 0 2024-05-25T06:11:28.727 INFO:tasks.workunit.client.1.smithi203.stdout:5/695: chown dc/d11/d17/d1a/d75/la6 698195 1 2024-05-25T06:11:28.728 INFO:tasks.workunit.client.1.smithi203.stdout:8/544: link d2/d5/d10/c64 d2/d5/d10/d45/d5d/cb0 0 2024-05-25T06:11:28.730 INFO:tasks.workunit.client.1.smithi203.stdout:3/541: mknod d10/d22/cb1 0 2024-05-25T06:11:28.732 INFO:tasks.workunit.client.1.smithi203.stdout:4/421: mkdir d0/d8/d3a/d81 0 2024-05-25T06:11:28.736 INFO:tasks.workunit.client.1.smithi203.stdout:3/542: symlink d10/d16/d50/lb2 0 2024-05-25T06:11:28.737 INFO:tasks.workunit.client.1.smithi203.stdout:5/696: mknod dc/d11/d17/d1a/d2d/d4d/cef 0 2024-05-25T06:11:28.737 INFO:tasks.workunit.client.1.smithi203.stdout:8/545: rmdir d2/d5/d28/d38/d81/d84/d8a/d8f 39 2024-05-25T06:11:28.741 INFO:tasks.workunit.client.1.smithi203.stdout:6/631: dwrite d8/d70/d38/fab [0,4194304] 0 2024-05-25T06:11:28.746 INFO:tasks.workunit.client.1.smithi203.stdout:4/422: rename d0/d9/d50/d6b/l45 to d0/d9/d11/d54/l82 0 2024-05-25T06:11:28.747 INFO:tasks.workunit.client.1.smithi203.stdout:3/543: mkdir d10/d16/d50/d8b/db3 0 2024-05-25T06:11:28.751 INFO:tasks.workunit.client.1.smithi203.stdout:1/410: dwrite d14/d37/d5e/f82 [0,4194304] 0 2024-05-25T06:11:28.752 INFO:tasks.workunit.client.1.smithi203.stdout:4/423: chown d0/d8/d10/l1a 190504 1 2024-05-25T06:11:28.754 INFO:tasks.workunit.client.1.smithi203.stdout:7/441: dread d1/d9/f18 [0,4194304] 0 2024-05-25T06:11:28.760 INFO:tasks.workunit.client.1.smithi203.stdout:4/424: mkdir d0/d8/d52/d59/d7a/d83 0 2024-05-25T06:11:28.764 INFO:tasks.workunit.client.1.smithi203.stdout:6/632: dwrite d8/fd [0,4194304] 0 2024-05-25T06:11:28.764 INFO:tasks.workunit.client.1.smithi203.stdout:1/411: rename d14/d57/d29/c3c to d14/d57/d29/c93 0 2024-05-25T06:11:28.766 INFO:tasks.workunit.client.1.smithi203.stdout:2/495: dread d2/d4/d10/d32/f3d [0,4194304] 0 2024-05-25T06:11:28.767 INFO:tasks.workunit.client.1.smithi203.stdout:8/546: truncate d2/f14 6220760 0 2024-05-25T06:11:28.768 INFO:tasks.workunit.client.1.smithi203.stdout:3/544: dwrite d10/d16/d23/d45/f9b [0,4194304] 0 2024-05-25T06:11:28.768 INFO:tasks.workunit.client.0.smithi160.stdout:9/15: sync 2024-05-25T06:11:28.769 INFO:tasks.workunit.client.0.smithi160.stdout:2/3: sync 2024-05-25T06:11:28.769 INFO:tasks.workunit.client.0.smithi160.stdout:9/16: chown l1 1553848 1 2024-05-25T06:11:28.769 INFO:tasks.workunit.client.0.smithi160.stdout:9/17: stat l1 0 2024-05-25T06:11:28.769 INFO:tasks.workunit.client.0.smithi160.stdout:9/18: chown l1 0 1 2024-05-25T06:11:28.769 INFO:tasks.workunit.client.0.smithi160.stdout:9/19: stat l1 0 2024-05-25T06:11:28.771 INFO:tasks.workunit.client.1.smithi203.stdout:7/442: dread d1/d9/d2c/d62/d7a/d89/f8c [0,4194304] 0 2024-05-25T06:11:28.771 INFO:tasks.workunit.client.1.smithi203.stdout:6/633: fdatasync d8/d11/d15/d28/d52/d43/f32 0 2024-05-25T06:11:28.775 INFO:tasks.workunit.client.1.smithi203.stdout:3/545: fsync d10/d16/d23/d38/f3e 0 2024-05-25T06:11:28.776 INFO:tasks.workunit.client.1.smithi203.stdout:4/425: read d0/d9/d50/d6b/f30 [721040,69251] 0 2024-05-25T06:11:28.777 INFO:tasks.workunit.client.1.smithi203.stdout:6/634: chown d8/d3d/c7f 738089961 1 2024-05-25T06:11:28.777 INFO:tasks.workunit.client.1.smithi203.stdout:5/697: dread dc/d11/d17/d1a/d2d/f65 [0,4194304] 0 2024-05-25T06:11:28.778 INFO:tasks.workunit.client.0.smithi160.stdout:2/4: link f0 f1 0 2024-05-25T06:11:28.778 INFO:tasks.workunit.client.0.smithi160.stdout:2/5: chown f0 6 1 2024-05-25T06:11:28.779 INFO:tasks.workunit.client.1.smithi203.stdout:4/426: readlink d0/d8/d10/l57 0 2024-05-25T06:11:28.780 INFO:tasks.workunit.client.0.smithi160.stdout:9/20: link f2 f3 0 2024-05-25T06:11:28.780 INFO:tasks.workunit.client.1.smithi203.stdout:3/546: write d10/d16/f8f [910471,66158] 0 2024-05-25T06:11:28.780 INFO:tasks.workunit.client.0.smithi160.stdout:9/21: read f3 [112002,32187] 0 2024-05-25T06:11:28.784 INFO:tasks.workunit.client.1.smithi203.stdout:6/635: write d8/d11/d15/f59 [1458930,69679] 0 2024-05-25T06:11:28.785 INFO:tasks.workunit.client.0.smithi160.stdout:9/22: fdatasync f2 0 2024-05-25T06:11:28.787 INFO:tasks.workunit.client.1.smithi203.stdout:8/547: dread d2/d5/d18/d31/f3d [4194304,4194304] 0 2024-05-25T06:11:28.787 INFO:tasks.workunit.client.1.smithi203.stdout:5/698: creat dc/d11/d42/d94/ff0 x:0 0 0 2024-05-25T06:11:28.787 INFO:tasks.workunit.client.0.smithi160.stdout:9/23: mkdir d4 0 2024-05-25T06:11:28.787 INFO:tasks.workunit.client.0.smithi160.stdout:9/24: fdatasync f3 0 2024-05-25T06:11:28.788 INFO:tasks.workunit.client.1.smithi203.stdout:2/496: dwrite d2/d4/d10/d5f/f5b [0,4194304] 0 2024-05-25T06:11:28.788 INFO:tasks.workunit.client.1.smithi203.stdout:4/427: mknod d0/d8/d52/d59/d7a/c84 0 2024-05-25T06:11:28.792 INFO:tasks.workunit.client.1.smithi203.stdout:3/547: mknod d10/d16/d23/d38/cb4 0 2024-05-25T06:11:28.792 INFO:tasks.workunit.client.1.smithi203.stdout:5/699: write dc/d11/d42/d94/fa8 [1043385,37419] 0 2024-05-25T06:11:28.793 INFO:tasks.workunit.client.0.smithi160.stdout:9/25: creat d4/f5 x:0 0 0 2024-05-25T06:11:28.795 INFO:tasks.workunit.client.0.smithi160.stdout:9/26: mknod d4/c6 0 2024-05-25T06:11:28.796 INFO:tasks.workunit.client.0.smithi160.stdout:9/27: chown l1 550 1 2024-05-25T06:11:28.797 INFO:tasks.workunit.client.1.smithi203.stdout:8/548: readlink d2/d5/d10/d11/de/l17 0 2024-05-25T06:11:28.798 INFO:tasks.workunit.client.1.smithi203.stdout:4/428: truncate d0/d8/d1f/d26/d43/f75 1109314 0 2024-05-25T06:11:28.799 INFO:tasks.workunit.client.1.smithi203.stdout:6/636: dwrite d8/d3d/f6a [0,4194304] 0 2024-05-25T06:11:28.801 INFO:tasks.workunit.client.1.smithi203.stdout:5/700: write dc/daa/fbc [946010,63399] 0 2024-05-25T06:11:28.809 INFO:tasks.workunit.client.1.smithi203.stdout:6/637: dwrite d8/d2a/d6c/fae [0,4194304] 0 2024-05-25T06:11:28.835 INFO:tasks.workunit.client.1.smithi203.stdout:8/549: fsync d2/d5/d10/d11/de/f13 0 2024-05-25T06:11:28.837 INFO:tasks.workunit.client.1.smithi203.stdout:4/429: rename d0/d8/lc to d0/d9/d11/l85 0 2024-05-25T06:11:28.838 INFO:tasks.workunit.client.1.smithi203.stdout:7/443: write d1/d9/d2c/d36/f40 [389244,28157] 0 2024-05-25T06:11:28.841 INFO:tasks.workunit.client.1.smithi203.stdout:7/444: write d1/d11/d3d/d4c/f63 [4032913,62933] 0 2024-05-25T06:11:28.842 INFO:tasks.workunit.client.1.smithi203.stdout:8/550: creat d2/d5/d18/fb1 x:0 0 0 2024-05-25T06:11:28.848 INFO:tasks.workunit.client.1.smithi203.stdout:3/548: rename d10/fab to d10/fb5 0 2024-05-25T06:11:28.848 INFO:tasks.workunit.client.1.smithi203.stdout:5/701: dwrite dc/fe [0,4194304] 0 2024-05-25T06:11:28.848 INFO:tasks.workunit.client.1.smithi203.stdout:3/549: chown d10/d16/d50/d8b 3426828 1 2024-05-25T06:11:28.850 INFO:tasks.workunit.client.1.smithi203.stdout:3/550: fdatasync d10/d16/d50/d8b/f85 0 2024-05-25T06:11:28.850 INFO:tasks.workunit.client.1.smithi203.stdout:7/445: rmdir d1/d11 39 2024-05-25T06:11:28.851 INFO:tasks.workunit.client.1.smithi203.stdout:3/551: chown d10/d16/d23/d46/f4b 15400 1 2024-05-25T06:11:28.852 INFO:tasks.workunit.client.1.smithi203.stdout:7/446: write d1/f2 [1064490,11969] 0 2024-05-25T06:11:28.853 INFO:tasks.workunit.client.1.smithi203.stdout:4/430: symlink d0/d8/d52/d59/d7a/d83/l86 0 2024-05-25T06:11:28.853 INFO:tasks.workunit.client.1.smithi203.stdout:3/552: chown d10/d16/c3c 6875751 1 2024-05-25T06:11:28.855 INFO:tasks.workunit.client.1.smithi203.stdout:4/431: read d0/d8/d1f/d26/f1e [4379292,828] 0 2024-05-25T06:11:28.856 INFO:tasks.workunit.client.1.smithi203.stdout:6/638: truncate d8/d11/d47/f83 2519637 0 2024-05-25T06:11:28.856 INFO:tasks.workunit.client.1.smithi203.stdout:6/639: stat d8/d2a/ca6 0 2024-05-25T06:11:28.857 INFO:tasks.workunit.client.1.smithi203.stdout:8/551: mknod d2/d5/d28/d38/d81/d84/d88/d93/d9d/cb2 0 2024-05-25T06:11:28.858 INFO:tasks.workunit.client.1.smithi203.stdout:5/702: symlink dc/d11/d42/lf1 0 2024-05-25T06:11:28.862 INFO:tasks.workunit.client.1.smithi203.stdout:3/553: creat d10/d22/d2a/d99/fb6 x:0 0 0 2024-05-25T06:11:28.864 INFO:tasks.workunit.client.1.smithi203.stdout:8/552: creat d2/d5/d28/d38/d81/fb3 x:0 0 0 2024-05-25T06:11:28.865 INFO:tasks.workunit.client.1.smithi203.stdout:5/703: symlink dc/d11/d17/d1a/d2d/d4d/dcc/lf2 0 2024-05-25T06:11:28.867 INFO:tasks.workunit.client.1.smithi203.stdout:4/432: rename d0/d9/d11/d54/c80 to d0/d8/d3a/c87 0 2024-05-25T06:11:28.868 INFO:tasks.workunit.client.1.smithi203.stdout:7/447: link d1/d11/d3d/f7f d1/d9/d2c/d36/f9d 0 2024-05-25T06:11:28.869 INFO:tasks.workunit.client.1.smithi203.stdout:8/553: mknod d2/d5/d28/d38/d81/d84/cb4 0 2024-05-25T06:11:28.869 INFO:tasks.workunit.client.1.smithi203.stdout:4/433: read d0/d8/f23 [730059,22735] 0 2024-05-25T06:11:28.869 INFO:tasks.workunit.client.1.smithi203.stdout:8/554: fsync d2/f99 0 2024-05-25T06:11:28.870 INFO:tasks.workunit.client.1.smithi203.stdout:6/640: rename d8/d29/d46/d88/l97 to d8/d29/lc8 0 2024-05-25T06:11:28.871 INFO:tasks.workunit.client.1.smithi203.stdout:5/704: mknod dc/d11/d17/d1a/d75/ddb/cf3 0 2024-05-25T06:11:28.871 INFO:tasks.workunit.client.1.smithi203.stdout:7/448: symlink d1/d9/d2c/d36/d66/l9e 0 2024-05-25T06:11:28.872 INFO:tasks.workunit.client.1.smithi203.stdout:5/705: readlink dc/d11/d17/d1a/d2d/d53/l59 0 2024-05-25T06:11:28.875 INFO:tasks.workunit.client.1.smithi203.stdout:7/449: symlink d1/d9/d2c/d62/l9f 0 2024-05-25T06:11:28.875 INFO:tasks.workunit.client.1.smithi203.stdout:6/641: read d8/d5a/d8b/f96 [299521,81283] 0 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:1/28: sync 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:7/13: sync 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:1/29: rmdir - no directory 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:6/23: sync 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:4/108: sync 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:8/80: sync 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:0/22: sync 2024-05-25T06:11:28.878 INFO:tasks.workunit.client.0.smithi160.stdout:5/28: sync 2024-05-25T06:11:28.879 INFO:tasks.workunit.client.0.smithi160.stdout:5/29: dread - fa zero size 2024-05-25T06:11:28.879 INFO:tasks.workunit.client.1.smithi203.stdout:5/706: creat dc/d11/d17/d1a/d2d/d4d/dcc/ff4 x:0 0 0 2024-05-25T06:11:28.880 INFO:tasks.workunit.client.1.smithi203.stdout:7/450: rmdir d1/d9/d2c/d3a/d4e 39 2024-05-25T06:11:28.880 INFO:tasks.workunit.client.0.smithi160.stdout:1/30: write f1 [775053,20200] 0 2024-05-25T06:11:28.881 INFO:tasks.workunit.client.0.smithi160.stdout:5/30: write f1 [1755482,92978] 0 2024-05-25T06:11:28.881 INFO:tasks.workunit.client.1.smithi203.stdout:6/642: dread d8/f2c [0,4194304] 0 2024-05-25T06:11:28.882 INFO:tasks.workunit.client.1.smithi203.stdout:6/643: readlink d8/d11/d47/d60/l6d 0 2024-05-25T06:11:28.886 INFO:tasks.workunit.client.1.smithi203.stdout:5/707: mkdir dc/d3c/df5 0 2024-05-25T06:11:28.887 INFO:tasks.workunit.client.0.smithi160.stdout:1/31: dread f5 [0,4194304] 0 2024-05-25T06:11:28.887 INFO:tasks.workunit.client.0.smithi160.stdout:8/81: dwrite d5/d9/f13 [0,4194304] 0 2024-05-25T06:11:28.888 INFO:tasks.workunit.client.0.smithi160.stdout:8/82: chown d5/d7/c12 246 1 2024-05-25T06:11:28.889 INFO:tasks.workunit.client.1.smithi203.stdout:6/644: dwrite d8/d11/d47/d60/f55 [0,4194304] 0 2024-05-25T06:11:28.889 INFO:tasks.workunit.client.1.smithi203.stdout:7/451: creat d1/d9/d2c/d62/d9b/fa0 x:0 0 0 2024-05-25T06:11:28.894 INFO:tasks.workunit.client.1.smithi203.stdout:5/708: dread dc/d11/d17/f31 [0,4194304] 0 2024-05-25T06:11:28.895 INFO:tasks.workunit.client.1.smithi203.stdout:6/645: mknod d8/d5a/cc9 0 2024-05-25T06:11:28.896 INFO:tasks.workunit.client.0.smithi160.stdout:1/32: dread f1 [0,4194304] 0 2024-05-25T06:11:28.897 INFO:tasks.workunit.client.0.smithi160.stdout:1/33: rmdir - no directory 2024-05-25T06:11:28.899 INFO:tasks.workunit.client.1.smithi203.stdout:8/555: write d2/d5/d10/d11/de/f1a [771970,93185] 0 2024-05-25T06:11:28.903 INFO:tasks.workunit.client.0.smithi160.stdout:7/14: dread f1 [0,4194304] 0 2024-05-25T06:11:28.903 INFO:tasks.workunit.client.1.smithi203.stdout:6/646: rename d8/d5a/d8b/l91 to d8/d3d/lca 0 2024-05-25T06:11:28.904 INFO:tasks.workunit.client.0.smithi160.stdout:4/109: rmdir d4/d6 39 2024-05-25T06:11:28.904 INFO:tasks.workunit.client.1.smithi203.stdout:4/434: dwrite d0/d8/d3a/f5d [0,4194304] 0 2024-05-25T06:11:28.905 INFO:tasks.workunit.client.1.smithi203.stdout:8/556: creat d2/d5/d10/d11/d1f/fb5 x:0 0 0 2024-05-25T06:11:28.906 INFO:tasks.workunit.client.1.smithi203.stdout:6/647: mkdir d8/d11/d1b/d75/dcb 0 2024-05-25T06:11:28.907 INFO:tasks.workunit.client.0.smithi160.stdout:0/23: rename f0 to f2 0 2024-05-25T06:11:28.912 INFO:tasks.workunit.client.1.smithi203.stdout:5/709: dread f1 [0,4194304] 0 2024-05-25T06:11:28.914 INFO:tasks.workunit.client.0.smithi160.stdout:5/31: mkdir db 0 2024-05-25T06:11:28.915 INFO:tasks.workunit.client.1.smithi203.stdout:5/710: fdatasync dc/d11/d17/d1a/d22/fde 0 2024-05-25T06:11:28.917 INFO:tasks.workunit.client.1.smithi203.stdout:4/435: getdents d0/d8/d52/d59/d7a 0 2024-05-25T06:11:28.919 INFO:tasks.workunit.client.1.smithi203.stdout:4/436: chown d0 756223352 1 2024-05-25T06:11:28.920 INFO:tasks.workunit.client.1.smithi203.stdout:5/711: write dc/d11/d17/f54 [641812,71341] 0 2024-05-25T06:11:28.920 INFO:tasks.workunit.client.0.smithi160.stdout:1/34: unlink l0 0 2024-05-25T06:11:28.922 INFO:tasks.workunit.client.0.smithi160.stdout:5/32: dwrite f9 [0,4194304] 0 2024-05-25T06:11:28.922 INFO:tasks.workunit.client.0.smithi160.stdout:6/24: creat d5/dc/fd x:0 0 0 2024-05-25T06:11:28.922 INFO:tasks.workunit.client.1.smithi203.stdout:6/648: dread d8/d70/d38/fab [0,4194304] 0 2024-05-25T06:11:28.923 INFO:tasks.workunit.client.1.smithi203.stdout:8/557: dwrite d2/d5/d28/d38/d81/d84/fa5 [0,4194304] 0 2024-05-25T06:11:28.924 INFO:tasks.workunit.client.1.smithi203.stdout:6/649: stat d8/d3d/c53 0 2024-05-25T06:11:28.926 INFO:tasks.workunit.client.1.smithi203.stdout:6/650: write d8/d70/f94 [1663521,21616] 0 2024-05-25T06:11:28.928 INFO:tasks.workunit.client.0.smithi160.stdout:2/6: sync 2024-05-25T06:11:28.932 INFO:tasks.workunit.client.0.smithi160.stdout:6/25: dwrite d5/f7 [0,4194304] 0 2024-05-25T06:11:28.932 INFO:tasks.workunit.client.0.smithi160.stdout:6/26: stat d5/dc/fd 0 2024-05-25T06:11:28.934 INFO:tasks.workunit.client.0.smithi160.stdout:6/27: read - d5/dc/fd zero size 2024-05-25T06:11:28.936 INFO:tasks.workunit.client.1.smithi203.stdout:5/712: creat dc/d6a/ff6 x:0 0 0 2024-05-25T06:11:28.936 INFO:tasks.workunit.client.1.smithi203.stdout:7/452: dread d1/d9/f2e [0,4194304] 0 2024-05-25T06:11:28.937 INFO:tasks.workunit.client.1.smithi203.stdout:6/651: mkdir d8/d5a/d9b/dcc 0 2024-05-25T06:11:28.937 INFO:tasks.workunit.client.1.smithi203.stdout:4/437: dwrite d0/d8/d1f/d26/d3b/f7b [0,4194304] 0 2024-05-25T06:11:28.937 INFO:tasks.workunit.client.1.smithi203.stdout:8/558: chown d2/d5/d10/d11/d1f/d75/f8d 311601 1 2024-05-25T06:11:28.937 INFO:tasks.workunit.client.0.smithi160.stdout:1/35: creat f9 x:0 0 0 2024-05-25T06:11:28.937 INFO:tasks.workunit.client.0.smithi160.stdout:1/36: truncate f7 585223 0 2024-05-25T06:11:28.938 INFO:tasks.workunit.client.0.smithi160.stdout:4/110: creat d4/d6/f22 x:0 0 0 2024-05-25T06:11:28.941 INFO:tasks.workunit.client.1.smithi203.stdout:4/438: unlink d0/l2d 0 2024-05-25T06:11:28.946 INFO:tasks.workunit.client.1.smithi203.stdout:6/652: creat d8/d11/d1b/d75/dcb/fcd x:0 0 0 2024-05-25T06:11:28.950 INFO:tasks.workunit.client.0.smithi160.stdout:7/15: link c3 c5 0 2024-05-25T06:11:28.953 INFO:tasks.workunit.client.0.smithi160.stdout:5/33: mkdir db/dc 0 2024-05-25T06:11:28.954 INFO:tasks.workunit.client.0.smithi160.stdout:5/34: dread - fa zero size 2024-05-25T06:11:28.954 INFO:tasks.workunit.client.1.smithi203.stdout:7/453: creat d1/d9/d2c/fa1 x:0 0 0 2024-05-25T06:11:28.955 INFO:tasks.workunit.client.1.smithi203.stdout:5/713: dwrite dc/d11/d17/d1a/d2d/d4d/f83 [0,4194304] 0 2024-05-25T06:11:28.956 INFO:tasks.workunit.client.0.smithi160.stdout:4/111: symlink d4/d6/l23 0 2024-05-25T06:11:28.956 INFO:tasks.workunit.client.1.smithi203.stdout:4/439: creat d0/d9/d50/d6b/f88 x:0 0 0 2024-05-25T06:11:28.956 INFO:tasks.workunit.client.1.smithi203.stdout:7/454: chown d1/d9/d2c/d62/d7a 655 1 2024-05-25T06:11:28.957 INFO:tasks.workunit.client.1.smithi203.stdout:8/559: getdents d2/d5/d2d 0 2024-05-25T06:11:28.960 INFO:tasks.workunit.client.1.smithi203.stdout:7/455: fsync d1/d9/fe 0 2024-05-25T06:11:28.962 INFO:tasks.workunit.client.1.smithi203.stdout:8/560: mkdir d2/d5/d10/d4a/db6 0 2024-05-25T06:11:28.962 INFO:tasks.workunit.client.0.smithi160.stdout:5/35: creat db/dc/fd x:0 0 0 2024-05-25T06:11:28.963 INFO:tasks.workunit.client.0.smithi160.stdout:5/36: stat f5 0 2024-05-25T06:11:28.963 INFO:tasks.workunit.client.0.smithi160.stdout:5/37: dread - db/dc/fd zero size 2024-05-25T06:11:28.963 INFO:tasks.workunit.client.1.smithi203.stdout:8/561: write d2/d5/d10/f52 [593139,123027] 0 2024-05-25T06:11:28.965 INFO:tasks.workunit.client.0.smithi160.stdout:4/112: unlink d4/d6/f13 0 2024-05-25T06:11:28.965 INFO:tasks.workunit.client.1.smithi203.stdout:4/440: mknod d0/d8/d1f/d26/c89 0 2024-05-25T06:11:28.965 INFO:tasks.workunit.client.0.smithi160.stdout:4/113: fsync d4/f11 0 2024-05-25T06:11:28.967 INFO:tasks.workunit.client.1.smithi203.stdout:5/714: creat dc/d11/d42/d94/dad/ff7 x:0 0 0 2024-05-25T06:11:28.967 INFO:tasks.workunit.client.1.smithi203.stdout:8/562: symlink d2/d5/d28/d38/d4e/lb7 0 2024-05-25T06:11:28.968 INFO:tasks.workunit.client.1.smithi203.stdout:5/715: stat dc/d11/d42/fce 0 2024-05-25T06:11:28.970 INFO:tasks.workunit.client.1.smithi203.stdout:4/441: rmdir d0/d8/d52/d59/d7a 39 2024-05-25T06:11:28.970 INFO:tasks.workunit.client.1.smithi203.stdout:5/716: truncate dc/d11/d17/f38 1584039 0 2024-05-25T06:11:28.971 INFO:tasks.workunit.client.1.smithi203.stdout:8/563: unlink d2/d5/d18/f70 0 2024-05-25T06:11:28.971 INFO:tasks.workunit.client.1.smithi203.stdout:7/456: link d1/l4 d1/d9/d2c/d3a/la2 0 2024-05-25T06:11:28.971 INFO:tasks.workunit.client.0.smithi160.stdout:5/38: link f5 db/dc/fe 0 2024-05-25T06:11:28.972 INFO:tasks.workunit.client.0.smithi160.stdout:4/114: chown d4/l14 33 1 2024-05-25T06:11:28.973 INFO:tasks.workunit.client.1.smithi203.stdout:7/457: chown d1/d11/l2f 13112 1 2024-05-25T06:11:28.974 INFO:tasks.workunit.client.1.smithi203.stdout:4/442: creat d0/d9/d50/d6b/f8a x:0 0 0 2024-05-25T06:11:28.974 INFO:tasks.workunit.client.0.smithi160.stdout:4/115: rmdir d4/d1c 39 2024-05-25T06:11:28.975 INFO:tasks.workunit.client.0.smithi160.stdout:5/39: dread db/dc/fe [0,4194304] 0 2024-05-25T06:11:28.976 INFO:tasks.workunit.client.1.smithi203.stdout:6/653: truncate d8/d70/d38/fab 1269803 0 2024-05-25T06:11:28.977 INFO:tasks.workunit.client.0.smithi160.stdout:4/116: creat d4/d6/d21/f24 x:0 0 0 2024-05-25T06:11:28.978 INFO:tasks.workunit.client.0.smithi160.stdout:5/40: symlink db/lf 0 2024-05-25T06:11:28.978 INFO:tasks.workunit.client.0.smithi160.stdout:4/117: fdatasync d4/d6/f7 0 2024-05-25T06:11:28.979 INFO:tasks.workunit.client.1.smithi203.stdout:4/443: mknod d0/d8/d52/c8b 0 2024-05-25T06:11:28.982 INFO:tasks.workunit.client.0.smithi160.stdout:4/118: creat d4/d6/d15/f25 x:0 0 0 2024-05-25T06:11:28.982 INFO:tasks.workunit.client.1.smithi203.stdout:5/717: rename dc/f21 to dc/d11/d17/ff8 0 2024-05-25T06:11:28.983 INFO:tasks.workunit.client.1.smithi203.stdout:8/564: readlink d2/d5/d28/d38/d81/d84/d8a/d8f/l91 0 2024-05-25T06:11:28.985 INFO:tasks.workunit.client.0.smithi160.stdout:5/41: dwrite f4 [0,4194304] 0 2024-05-25T06:11:28.986 INFO:tasks.workunit.client.0.smithi160.stdout:4/119: creat d4/f26 x:0 0 0 2024-05-25T06:11:28.986 INFO:tasks.workunit.client.1.smithi203.stdout:6/654: mkdir d8/d5a/d9b/dcc/dce 0 2024-05-25T06:11:28.988 INFO:tasks.workunit.client.1.smithi203.stdout:8/565: mknod d2/d5/d10/d11/d59/cb8 0 2024-05-25T06:11:28.989 INFO:tasks.workunit.client.1.smithi203.stdout:4/444: dwrite d0/f16 [4194304,4194304] 0 2024-05-25T06:11:29.001 INFO:tasks.workunit.client.1.smithi203.stdout:4/445: truncate d0/f3 4217621 0 2024-05-25T06:11:29.004 INFO:tasks.workunit.client.1.smithi203.stdout:7/458: dwrite d1/fa [0,4194304] 0 2024-05-25T06:11:29.004 INFO:tasks.workunit.client.1.smithi203.stdout:5/718: rmdir dc/d11/d17 39 2024-05-25T06:11:29.007 INFO:tasks.workunit.client.1.smithi203.stdout:6/655: dwrite d8/d2a/f4a [0,4194304] 0 2024-05-25T06:11:29.008 INFO:tasks.workunit.client.1.smithi203.stdout:4/446: creat d0/d8/d1f/d26/d3b/f8c x:0 0 0 2024-05-25T06:11:29.009 INFO:tasks.workunit.client.1.smithi203.stdout:5/719: mkdir dc/d6a/df9 0 2024-05-25T06:11:29.010 INFO:tasks.workunit.client.1.smithi203.stdout:4/447: chown d0/d9/d50 6544596 1 2024-05-25T06:11:29.014 INFO:tasks.workunit.client.1.smithi203.stdout:5/720: write dc/fe6 [4181313,96183] 0 2024-05-25T06:11:29.016 INFO:tasks.workunit.client.1.smithi203.stdout:7/459: link d1/c2a d1/d9/d2c/d36/d66/ca3 0 2024-05-25T06:11:29.020 INFO:tasks.workunit.client.1.smithi203.stdout:8/566: write d2/d5/d10/d11/de/f13 [605900,25748] 0 2024-05-25T06:11:29.021 INFO:tasks.workunit.client.1.smithi203.stdout:5/721: getdents dc/dcb 0 2024-05-25T06:11:29.023 INFO:tasks.workunit.client.1.smithi203.stdout:4/448: dread d0/d8/d3a/f77 [0,4194304] 0 2024-05-25T06:11:29.026 INFO:tasks.workunit.client.1.smithi203.stdout:7/460: dread d1/d9/d2c/f56 [4194304,4194304] 0 2024-05-25T06:11:29.027 INFO:tasks.workunit.client.1.smithi203.stdout:4/449: creat d0/d8/f8d x:0 0 0 2024-05-25T06:11:29.029 INFO:tasks.workunit.client.1.smithi203.stdout:7/461: chown d1/d9/d2c/d3a/d4e/l67 2718164 1 2024-05-25T06:11:29.031 INFO:tasks.workunit.client.1.smithi203.stdout:5/722: rename dc/d11/l33 to dc/d11/d42/d94/dad/lfa 0 2024-05-25T06:11:29.032 INFO:tasks.workunit.client.1.smithi203.stdout:6/656: dread f2 [0,4194304] 0 2024-05-25T06:11:29.033 INFO:tasks.workunit.client.1.smithi203.stdout:7/462: write f0 [6091636,29589] 0 2024-05-25T06:11:29.034 INFO:tasks.workunit.client.1.smithi203.stdout:7/463: chown d1/f31 7664 1 2024-05-25T06:11:29.035 INFO:tasks.workunit.client.1.smithi203.stdout:5/723: chown dc/d11/d42/d94/dad/lc3 710 1 2024-05-25T06:11:29.035 INFO:tasks.workunit.client.1.smithi203.stdout:8/567: rename d2/d5/d28/d38/d81/d84/fa5 to d2/d5/d28/d38/d81/d84/d88/d93/fb9 0 2024-05-25T06:11:29.038 INFO:tasks.workunit.client.1.smithi203.stdout:7/464: mkdir d1/d11/d3d/d9a/da4 0 2024-05-25T06:11:29.039 INFO:tasks.workunit.client.1.smithi203.stdout:8/568: creat d2/d5/d10/d4a/fba x:0 0 0 2024-05-25T06:11:29.040 INFO:tasks.workunit.client.1.smithi203.stdout:7/465: rmdir d1/d11/d3d/d9a 39 2024-05-25T06:11:29.041 INFO:tasks.workunit.client.1.smithi203.stdout:5/724: symlink dc/d11/d17/d1a/d2d/d4d/d64/lfb 0 2024-05-25T06:11:29.046 INFO:tasks.workunit.client.1.smithi203.stdout:8/569: dread d2/d5/d10/d11/d59/f63 [0,4194304] 0 2024-05-25T06:11:29.048 INFO:tasks.workunit.client.1.smithi203.stdout:5/725: creat dc/d11/ffc x:0 0 0 2024-05-25T06:11:29.050 INFO:tasks.workunit.client.1.smithi203.stdout:8/570: rename d2/d5/d10/d11/d1f/f35 to d2/d5/d28/d38/d4e/fbb 0 2024-05-25T06:11:29.051 INFO:tasks.workunit.client.1.smithi203.stdout:5/726: symlink dc/d11/d17/d44/lfd 0 2024-05-25T06:11:29.051 INFO:tasks.workunit.client.1.smithi203.stdout:4/450: dread d0/f5 [0,4194304] 0 2024-05-25T06:11:29.053 INFO:tasks.workunit.client.1.smithi203.stdout:7/466: write d1/d9/d2c/d36/f9d [1004166,87675] 0 2024-05-25T06:11:29.054 INFO:tasks.workunit.client.1.smithi203.stdout:8/571: write d2/d5/d10/d45/d5d/d76/f8b [65444,53637] 0 2024-05-25T06:11:29.057 INFO:tasks.workunit.client.1.smithi203.stdout:6/657: dread d8/f12 [0,4194304] 0 2024-05-25T06:11:29.059 INFO:tasks.workunit.client.1.smithi203.stdout:4/451: creat d0/d8/d1f/d26/f8e x:0 0 0 2024-05-25T06:11:29.060 INFO:tasks.workunit.client.1.smithi203.stdout:7/467: symlink d1/d11/d3d/la5 0 2024-05-25T06:11:29.063 INFO:tasks.workunit.client.1.smithi203.stdout:6/658: rename d8/d29 to d8/d11/d1b/d75/dcb/dcf 0 2024-05-25T06:11:29.063 INFO:tasks.workunit.client.1.smithi203.stdout:9/430: sync 2024-05-25T06:11:29.064 INFO:tasks.workunit.client.1.smithi203.stdout:5/727: symlink dc/d11/d42/d94/dad/lfe 0 2024-05-25T06:11:29.067 INFO:tasks.workunit.client.1.smithi203.stdout:9/431: chown d0/df/d73/f53 1680 1 2024-05-25T06:11:29.069 INFO:tasks.workunit.client.1.smithi203.stdout:4/452: unlink d0/d8/c31 0 2024-05-25T06:11:29.069 INFO:tasks.workunit.client.1.smithi203.stdout:6/659: write d8/d11/f19 [2341211,86210] 0 2024-05-25T06:11:29.075 INFO:tasks.workunit.client.1.smithi203.stdout:9/432: dread - d0/d57/f86 zero size 2024-05-25T06:11:29.076 INFO:tasks.workunit.client.1.smithi203.stdout:7/468: dwrite d1/d11/d3d/d4c/f93 [0,4194304] 0 2024-05-25T06:11:29.077 INFO:tasks.workunit.client.1.smithi203.stdout:5/728: rename dc/d11/f14 to dc/d3c/fff 0 2024-05-25T06:11:29.078 INFO:tasks.workunit.client.1.smithi203.stdout:6/660: creat d8/d11/d15/fd0 x:0 0 0 2024-05-25T06:11:29.083 INFO:tasks.workunit.client.1.smithi203.stdout:8/572: dread d2/d5/d10/d11/de/ff [4194304,4194304] 0 2024-05-25T06:11:29.083 INFO:tasks.workunit.client.1.smithi203.stdout:9/433: creat d0/d17/d20/d6e/d75/d78/f94 x:0 0 0 2024-05-25T06:11:29.084 INFO:tasks.workunit.client.1.smithi203.stdout:8/573: stat d2/d5/d2d/f3e 0 2024-05-25T06:11:29.085 INFO:tasks.workunit.client.1.smithi203.stdout:6/661: symlink d8/d2a/da9/ld1 0 2024-05-25T06:11:29.085 INFO:tasks.workunit.client.1.smithi203.stdout:9/434: chown d0/d17/f1b 197293962 1 2024-05-25T06:11:29.087 INFO:tasks.workunit.client.1.smithi203.stdout:6/662: read - d8/d11/d15/fd0 zero size 2024-05-25T06:11:29.089 INFO:tasks.workunit.client.1.smithi203.stdout:6/663: chown d8/d3d/c53 401 1 2024-05-25T06:11:29.090 INFO:tasks.workunit.client.1.smithi203.stdout:5/729: link dc/d11/d17/d1a/d2d/d53/da2/l6c dc/l100 0 2024-05-25T06:11:29.093 INFO:tasks.workunit.client.1.smithi203.stdout:9/435: getdents d0/d17/d20/d3c 0 2024-05-25T06:11:29.094 INFO:tasks.workunit.client.1.smithi203.stdout:6/664: rename d8/d11/d1b/f26 to d8/d11/d1b/d75/dcb/dcf/d46/fd2 0 2024-05-25T06:11:29.096 INFO:tasks.workunit.client.1.smithi203.stdout:8/574: dread d2/d5/d18/d31/f6f [0,4194304] 0 2024-05-25T06:11:29.103 INFO:tasks.workunit.client.1.smithi203.stdout:6/665: fsync d8/d5a/d8b/f96 0 2024-05-25T06:11:29.104 INFO:tasks.workunit.client.1.smithi203.stdout:5/730: dwrite dc/fe [0,4194304] 0 2024-05-25T06:11:29.105 INFO:tasks.workunit.client.1.smithi203.stdout:4/453: dread d0/d9/d50/d6b/f51 [0,4194304] 0 2024-05-25T06:11:29.108 INFO:tasks.workunit.client.1.smithi203.stdout:6/666: fdatasync d8/d2a/d6c/d87/daa/fb4 0 2024-05-25T06:11:29.110 INFO:tasks.workunit.client.1.smithi203.stdout:4/454: write d0/d9/d50/d6b/f51 [1120731,19587] 0 2024-05-25T06:11:29.113 INFO:tasks.workunit.client.1.smithi203.stdout:5/731: write dc/d11/d17/d1a/d2d/d4d/f83 [4223753,56454] 0 2024-05-25T06:11:29.117 INFO:tasks.workunit.client.1.smithi203.stdout:5/732: chown dc/d11/d42/leb 445314891 1 2024-05-25T06:11:29.118 INFO:tasks.workunit.client.1.smithi203.stdout:8/575: getdents d2/d5/d10/d45/d5d/d76 0 2024-05-25T06:11:29.120 INFO:tasks.workunit.client.1.smithi203.stdout:9/436: dwrite d0/d17/d20/d3c/f43 [0,4194304] 0 2024-05-25T06:11:29.120 INFO:tasks.workunit.client.1.smithi203.stdout:1/412: sync 2024-05-25T06:11:29.120 INFO:tasks.workunit.client.1.smithi203.stdout:0/343: sync 2024-05-25T06:11:29.120 INFO:tasks.workunit.client.1.smithi203.stdout:2/497: sync 2024-05-25T06:11:29.120 INFO:tasks.workunit.client.1.smithi203.stdout:3/554: sync 2024-05-25T06:11:29.120 INFO:tasks.workunit.client.0.smithi160.stdout:2/7: sync 2024-05-25T06:11:29.121 INFO:tasks.workunit.client.0.smithi160.stdout:2/8: chown f1 2674 1 2024-05-25T06:11:29.123 INFO:tasks.workunit.client.1.smithi203.stdout:2/498: stat d2/d1a/d33 0 2024-05-25T06:11:29.125 INFO:tasks.workunit.client.1.smithi203.stdout:3/555: readlink d10/d16/d23/l37 0 2024-05-25T06:11:29.125 INFO:tasks.workunit.client.1.smithi203.stdout:9/437: readlink d0/df/l54 0 2024-05-25T06:11:29.130 INFO:tasks.workunit.client.1.smithi203.stdout:8/576: mknod d2/d5/d18/d31/cbc 0 2024-05-25T06:11:29.130 INFO:tasks.workunit.client.1.smithi203.stdout:4/455: dwrite d0/d8/d3a/f53 [0,4194304] 0 2024-05-25T06:11:29.131 INFO:tasks.workunit.client.1.smithi203.stdout:0/344: write dd/d3a/f3b [837220,100613] 0 2024-05-25T06:11:29.134 INFO:tasks.workunit.client.1.smithi203.stdout:1/413: mkdir d14/d57/d29/d94 0 2024-05-25T06:11:29.134 INFO:tasks.workunit.client.1.smithi203.stdout:0/345: dread - dd/d32/f38 zero size 2024-05-25T06:11:29.137 INFO:tasks.workunit.client.0.smithi160.stdout:3/9: getdents . 0 2024-05-25T06:11:29.138 INFO:tasks.workunit.client.1.smithi203.stdout:8/577: chown d2/d5/d10/d11/de/f1d 59416 1 2024-05-25T06:11:29.139 INFO:tasks.workunit.client.1.smithi203.stdout:5/733: dwrite dc/d11/d17/d1a/d2d/d4d/f83 [0,4194304] 0 2024-05-25T06:11:29.141 INFO:tasks.workunit.client.1.smithi203.stdout:6/667: write d8/d11/d15/f2e [3163614,20779] 0 2024-05-25T06:11:29.141 INFO:tasks.workunit.client.1.smithi203.stdout:1/414: stat d14/d57/d1d/d3e/d36/l92 0 2024-05-25T06:11:29.141 INFO:tasks.workunit.client.1.smithi203.stdout:2/499: creat d2/d42/fa9 x:0 0 0 2024-05-25T06:11:29.141 INFO:tasks.workunit.client.1.smithi203.stdout:3/556: rename d10/d16/d50/d8b/f96 to d10/d16/d23/d67/fb7 0 2024-05-25T06:11:29.141 INFO:tasks.workunit.client.0.smithi160.stdout:3/10: fdatasync f0 0 2024-05-25T06:11:29.144 INFO:tasks.workunit.client.0.smithi160.stdout:2/9: fsync f1 0 2024-05-25T06:11:29.145 INFO:tasks.workunit.client.0.smithi160.stdout:2/10: read f0 [560959,129982] 0 2024-05-25T06:11:29.146 INFO:tasks.workunit.client.1.smithi203.stdout:0/346: fdatasync dd/f23 0 2024-05-25T06:11:29.147 INFO:tasks.workunit.client.1.smithi203.stdout:3/557: write d10/d16/d50/d8b/faa [193342,109360] 0 2024-05-25T06:11:29.147 INFO:tasks.workunit.client.0.smithi160.stdout:4/120: sync 2024-05-25T06:11:29.147 INFO:tasks.workunit.client.0.smithi160.stdout:3/11: creat f3 x:0 0 0 2024-05-25T06:11:29.147 INFO:tasks.workunit.client.1.smithi203.stdout:6/668: chown d8/d11/d47/d60/c64 0 1 2024-05-25T06:11:29.149 INFO:tasks.workunit.client.1.smithi203.stdout:4/456: mknod d0/d9/d11/c8f 0 2024-05-25T06:11:29.149 INFO:tasks.workunit.client.1.smithi203.stdout:3/558: chown le 172 1 2024-05-25T06:11:29.151 INFO:tasks.workunit.client.0.smithi160.stdout:2/11: mknod c2 0 2024-05-25T06:11:29.155 INFO:tasks.workunit.client.1.smithi203.stdout:0/347: dread - dd/d3d/d75/f79 zero size 2024-05-25T06:11:29.158 INFO:tasks.workunit.client.0.smithi160.stdout:9/28: rmdir d4 39 2024-05-25T06:11:29.159 INFO:tasks.workunit.client.1.smithi203.stdout:1/415: mknod d14/d81/c95 0 2024-05-25T06:11:29.160 INFO:tasks.workunit.client.1.smithi203.stdout:6/669: mkdir d8/d11/d15/dd3 0 2024-05-25T06:11:29.161 INFO:tasks.workunit.client.1.smithi203.stdout:0/348: creat dd/d10/d34/d4c/f7b x:0 0 0 2024-05-25T06:11:29.161 INFO:tasks.workunit.client.0.smithi160.stdout:0/24: getdents . 0 2024-05-25T06:11:29.162 INFO:tasks.workunit.client.1.smithi203.stdout:3/559: mkdir d10/d16/d23/d67/db8 0 2024-05-25T06:11:29.163 INFO:tasks.workunit.client.1.smithi203.stdout:6/670: write d8/d11/d1b/d75/dcb/dcf/d46/fa4 [670550,50102] 0 2024-05-25T06:11:29.163 INFO:tasks.workunit.client.1.smithi203.stdout:4/457: rename d0/d8/d1f/d26/d3b/l3e to d0/d9/d50/d6b/l90 0 2024-05-25T06:11:29.164 INFO:tasks.workunit.client.1.smithi203.stdout:5/734: dread dc/d11/d17/d1a/d22/f24 [0,4194304] 0 2024-05-25T06:11:29.165 INFO:tasks.workunit.client.0.smithi160.stdout:0/25: write f2 [337820,5143] 0 2024-05-25T06:11:29.167 INFO:tasks.workunit.client.0.smithi160.stdout:9/29: dwrite f2 [0,4194304] 0 2024-05-25T06:11:29.168 INFO:tasks.workunit.client.1.smithi203.stdout:9/438: dread d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:29.168 INFO:tasks.workunit.client.0.smithi160.stdout:8/83: write d5/d9/f13 [4800575,72106] 0 2024-05-25T06:11:29.169 INFO:tasks.workunit.client.1.smithi203.stdout:9/439: read - d0/d57/f86 zero size 2024-05-25T06:11:29.170 INFO:tasks.workunit.client.0.smithi160.stdout:0/26: symlink l3 0 2024-05-25T06:11:29.172 INFO:tasks.workunit.client.0.smithi160.stdout:8/84: unlink f3 0 2024-05-25T06:11:29.173 INFO:tasks.workunit.client.0.smithi160.stdout:9/30: rmdir d4 39 2024-05-25T06:11:29.175 INFO:tasks.workunit.client.1.smithi203.stdout:4/458: mkdir d0/d8/d10/d91 0 2024-05-25T06:11:29.177 INFO:tasks.workunit.client.1.smithi203.stdout:5/735: mkdir dc/d11/d17/d1a/d2d/d4d/d101 0 2024-05-25T06:11:29.179 INFO:tasks.workunit.client.0.smithi160.stdout:8/85: dwrite d5/d9/f13 [0,4194304] 0 2024-05-25T06:11:29.180 INFO:tasks.workunit.client.1.smithi203.stdout:6/671: dwrite d8/d5a/d8b/fa8 [0,4194304] 0 2024-05-25T06:11:29.181 INFO:tasks.workunit.client.0.smithi160.stdout:6/28: rename d5/dc to d5/de 0 2024-05-25T06:11:29.183 INFO:tasks.workunit.client.1.smithi203.stdout:3/560: creat d10/d16/d23/fb9 x:0 0 0 2024-05-25T06:11:29.183 INFO:tasks.workunit.client.0.smithi160.stdout:1/37: truncate f1 948001 0 2024-05-25T06:11:29.185 INFO:tasks.workunit.client.0.smithi160.stdout:1/38: chown f9 3202 1 2024-05-25T06:11:29.185 INFO:tasks.workunit.client.1.smithi203.stdout:6/672: dread - d8/d2a/f67 zero size 2024-05-25T06:11:29.185 INFO:tasks.workunit.client.1.smithi203.stdout:8/578: write d2/d5/d10/f4d [3575817,107513] 0 2024-05-25T06:11:29.185 INFO:tasks.workunit.client.0.smithi160.stdout:8/86: stat d5/fe 0 2024-05-25T06:11:29.186 INFO:tasks.workunit.client.0.smithi160.stdout:9/31: fdatasync f3 0 2024-05-25T06:11:29.188 INFO:tasks.workunit.client.0.smithi160.stdout:8/87: read d5/d9/f13 [2948389,46122] 0 2024-05-25T06:11:29.189 INFO:tasks.workunit.client.1.smithi203.stdout:2/500: dread d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:29.190 INFO:tasks.workunit.client.1.smithi203.stdout:2/501: readlink d2/d38/d2f/l47 0 2024-05-25T06:11:29.192 INFO:tasks.workunit.client.0.smithi160.stdout:6/29: dwrite f2 [0,4194304] 0 2024-05-25T06:11:29.193 INFO:tasks.workunit.client.1.smithi203.stdout:6/673: stat d8/d11/d1b/d75/dcb/dcf/d80/c85 0 2024-05-25T06:11:29.196 INFO:tasks.workunit.client.1.smithi203.stdout:3/561: symlink d10/d22/lba 0 2024-05-25T06:11:29.196 INFO:tasks.workunit.client.1.smithi203.stdout:2/502: chown d2/d4/d10/d5f/l39 6 1 2024-05-25T06:11:29.200 INFO:tasks.workunit.client.1.smithi203.stdout:1/416: write d14/f32 [1060072,43549] 0 2024-05-25T06:11:29.203 INFO:tasks.workunit.client.1.smithi203.stdout:6/674: creat d8/d5a/d9b/fd4 x:0 0 0 2024-05-25T06:11:29.205 INFO:tasks.workunit.client.1.smithi203.stdout:2/503: write d2/d4/d10/d5f/f5b [4698962,24888] 0 2024-05-25T06:11:29.205 INFO:tasks.workunit.client.1.smithi203.stdout:3/562: rmdir d10/d22/d2a/d99 39 2024-05-25T06:11:29.207 INFO:tasks.workunit.client.0.smithi160.stdout:7/16: write f1 [1112941,57912] 0 2024-05-25T06:11:29.208 INFO:tasks.workunit.client.0.smithi160.stdout:9/32: rename d4/f5 to d4/f7 0 2024-05-25T06:11:29.211 INFO:tasks.workunit.client.1.smithi203.stdout:8/579: link d2/d5/d10/d45/d5d/d68/f7d d2/d5/d28/d38/d4e/d97/fbd 0 2024-05-25T06:11:29.213 INFO:tasks.workunit.client.1.smithi203.stdout:5/736: link dc/d11/d17/ff8 dc/d11/d17/d1a/d2d/d4d/dcc/f102 0 2024-05-25T06:11:29.214 INFO:tasks.workunit.client.0.smithi160.stdout:7/17: dwrite f1 [0,4194304] 0 2024-05-25T06:11:29.215 INFO:tasks.workunit.client.1.smithi203.stdout:2/504: unlink d2/c72 0 2024-05-25T06:11:29.216 INFO:tasks.workunit.client.1.smithi203.stdout:8/580: chown d2/ca9 37888040 1 2024-05-25T06:11:29.216 INFO:tasks.workunit.client.1.smithi203.stdout:6/675: chown d8/d11/d1b/d75/dcb/dcf/lc8 289241 1 2024-05-25T06:11:29.217 INFO:tasks.workunit.client.1.smithi203.stdout:2/505: write d2/d38/f7d [2523080,19230] 0 2024-05-25T06:11:29.219 INFO:tasks.workunit.client.1.smithi203.stdout:1/417: mknod d14/d57/d7e/c96 0 2024-05-25T06:11:29.219 INFO:tasks.workunit.client.0.smithi160.stdout:7/18: chown c5 138458 1 2024-05-25T06:11:29.219 INFO:tasks.workunit.client.0.smithi160.stdout:4/121: rmdir d4 39 2024-05-25T06:11:29.220 INFO:tasks.workunit.client.1.smithi203.stdout:3/563: dwrite d10/d16/d50/d7d/f9c [0,4194304] 0 2024-05-25T06:11:29.220 INFO:tasks.workunit.client.1.smithi203.stdout:9/440: dread d0/d17/f59 [0,4194304] 0 2024-05-25T06:11:29.221 INFO:tasks.workunit.client.1.smithi203.stdout:0/349: dread dd/d10/f41 [0,4194304] 0 2024-05-25T06:11:29.222 INFO:tasks.workunit.client.0.smithi160.stdout:7/19: mkdir d6 0 2024-05-25T06:11:29.224 INFO:tasks.workunit.client.0.smithi160.stdout:5/42: truncate f6 3770983 0 2024-05-25T06:11:29.228 INFO:tasks.workunit.client.1.smithi203.stdout:0/350: write dd/d3a/d66/f69 [380105,128061] 0 2024-05-25T06:11:29.229 INFO:tasks.workunit.client.0.smithi160.stdout:9/33: dwrite d4/f7 [0,4194304] 0 2024-05-25T06:11:29.229 INFO:tasks.workunit.client.0.smithi160.stdout:4/122: fdatasync d4/d6/fa 0 2024-05-25T06:11:29.234 INFO:tasks.workunit.client.1.smithi203.stdout:3/564: creat d10/d16/d23/d46/fbb x:0 0 0 2024-05-25T06:11:29.235 INFO:tasks.workunit.client.0.smithi160.stdout:7/20: dread f1 [0,4194304] 0 2024-05-25T06:11:29.235 INFO:tasks.workunit.client.1.smithi203.stdout:1/418: mkdir d14/d57/d97 0 2024-05-25T06:11:29.236 INFO:tasks.workunit.client.0.smithi160.stdout:5/43: dwrite f1 [0,4194304] 0 2024-05-25T06:11:29.237 INFO:tasks.workunit.client.1.smithi203.stdout:2/506: dread d2/d4/d10/f21 [0,4194304] 0 2024-05-25T06:11:29.238 INFO:tasks.workunit.client.1.smithi203.stdout:1/419: fdatasync d14/f1a 0 2024-05-25T06:11:29.238 INFO:tasks.workunit.client.1.smithi203.stdout:4/459: dwrite d0/d8/d3a/f56 [0,4194304] 0 2024-05-25T06:11:29.240 INFO:tasks.workunit.client.0.smithi160.stdout:4/123: creat d4/d6/d21/f27 x:0 0 0 2024-05-25T06:11:29.243 INFO:tasks.workunit.client.1.smithi203.stdout:1/420: write d14/d57/d1d/f67 [526770,105967] 0 2024-05-25T06:11:29.243 INFO:tasks.workunit.client.1.smithi203.stdout:5/737: truncate dc/d11/f29 2921398 0 2024-05-25T06:11:29.244 INFO:tasks.workunit.client.1.smithi203.stdout:4/460: write d0/d8/d3a/f53 [620933,91284] 0 2024-05-25T06:11:29.245 INFO:tasks.workunit.client.1.smithi203.stdout:6/676: getdents d8/d2a/d6c/dc1 0 2024-05-25T06:11:29.247 INFO:tasks.workunit.client.1.smithi203.stdout:9/441: dread d0/df/d2b/f2f [0,4194304] 0 2024-05-25T06:11:29.248 INFO:tasks.workunit.client.0.smithi160.stdout:7/21: fdatasync f1 0 2024-05-25T06:11:29.248 INFO:tasks.workunit.client.0.smithi160.stdout:9/34: dread d4/f7 [0,4194304] 0 2024-05-25T06:11:29.254 INFO:tasks.workunit.client.0.smithi160.stdout:0/27: getdents . 0 2024-05-25T06:11:29.256 INFO:tasks.workunit.client.1.smithi203.stdout:2/507: dwrite d2/d38/d43/d60/fa4 [0,4194304] 0 2024-05-25T06:11:29.258 INFO:tasks.workunit.client.1.smithi203.stdout:6/677: symlink d8/d70/ld5 0 2024-05-25T06:11:29.259 INFO:tasks.workunit.client.0.smithi160.stdout:2/12: dwrite f0 [0,4194304] 0 2024-05-25T06:11:29.260 INFO:tasks.workunit.client.1.smithi203.stdout:1/421: fdatasync d14/d57/d29/f2b 0 2024-05-25T06:11:29.261 INFO:tasks.workunit.client.1.smithi203.stdout:0/351: creat dd/d32/f7c x:0 0 0 2024-05-25T06:11:29.261 INFO:tasks.workunit.client.0.smithi160.stdout:0/28: write f2 [514893,34058] 0 2024-05-25T06:11:29.262 INFO:tasks.workunit.client.0.smithi160.stdout:8/88: truncate d5/d9/f13 3654069 0 2024-05-25T06:11:29.263 INFO:tasks.workunit.client.1.smithi203.stdout:3/565: read d10/d16/d50/d8b/f85 [3250089,129637] 0 2024-05-25T06:11:29.265 INFO:tasks.workunit.client.1.smithi203.stdout:5/738: dread - dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdd zero size 2024-05-25T06:11:29.266 INFO:tasks.workunit.client.1.smithi203.stdout:0/352: fsync dd/d3a/d66/f69 0 2024-05-25T06:11:29.266 INFO:tasks.workunit.client.1.smithi203.stdout:4/461: dwrite d0/d9/d50/d6b/f88 [0,4194304] 0 2024-05-25T06:11:29.267 INFO:tasks.workunit.client.1.smithi203.stdout:3/566: dread - d10/d16/d23/d46/f4b zero size 2024-05-25T06:11:29.267 INFO:tasks.workunit.client.1.smithi203.stdout:5/739: fsync dc/fe6 0 2024-05-25T06:11:29.267 INFO:tasks.workunit.client.0.smithi160.stdout:4/124: dwrite d4/d6/d21/f24 [0,4194304] 0 2024-05-25T06:11:29.268 INFO:tasks.workunit.client.0.smithi160.stdout:9/35: rename f3 to d4/f8 0 2024-05-25T06:11:29.272 INFO:tasks.workunit.client.0.smithi160.stdout:0/29: symlink l4 0 2024-05-25T06:11:29.272 INFO:tasks.workunit.client.0.smithi160.stdout:2/13: rename f1 to f3 0 2024-05-25T06:11:29.272 INFO:tasks.workunit.client.1.smithi203.stdout:7/469: sync 2024-05-25T06:11:29.275 INFO:tasks.workunit.client.1.smithi203.stdout:8/581: write d2/d5/d10/d11/f9c [687023,4256] 0 2024-05-25T06:11:29.277 INFO:tasks.workunit.client.0.smithi160.stdout:7/22: symlink d6/l7 0 2024-05-25T06:11:29.277 INFO:tasks.workunit.client.1.smithi203.stdout:0/353: readlink dd/l43 0 2024-05-25T06:11:29.279 INFO:tasks.workunit.client.1.smithi203.stdout:9/442: write d0/f1e [565760,48432] 0 2024-05-25T06:11:29.280 INFO:tasks.workunit.client.0.smithi160.stdout:2/14: read f0 [2934082,56879] 0 2024-05-25T06:11:29.281 INFO:tasks.workunit.client.0.smithi160.stdout:6/30: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:29.283 INFO:tasks.workunit.client.0.smithi160.stdout:8/89: fsync d5/fe 0 2024-05-25T06:11:29.285 INFO:tasks.workunit.client.1.smithi203.stdout:1/422: symlink d14/d57/d97/l98 0 2024-05-25T06:11:29.286 INFO:tasks.workunit.client.0.smithi160.stdout:3/12: dwrite f0 [0,4194304] 0 2024-05-25T06:11:29.286 INFO:tasks.workunit.client.1.smithi203.stdout:3/567: mknod d10/d16/d23/d67/db8/cbc 0 2024-05-25T06:11:29.287 INFO:tasks.workunit.client.0.smithi160.stdout:4/125: rename d4/d6/d15 to d4/d6/d28 0 2024-05-25T06:11:29.288 INFO:tasks.workunit.client.1.smithi203.stdout:4/462: mknod d0/d8/c92 0 2024-05-25T06:11:29.288 INFO:tasks.workunit.client.0.smithi160.stdout:9/36: fsync d4/f7 0 2024-05-25T06:11:29.290 INFO:tasks.workunit.client.0.smithi160.stdout:4/126: chown d4/d6/c8 195917042 1 2024-05-25T06:11:29.291 INFO:tasks.workunit.client.1.smithi203.stdout:2/508: dread d2/d4/f74 [0,4194304] 0 2024-05-25T06:11:29.291 INFO:tasks.workunit.client.1.smithi203.stdout:1/423: truncate d14/d57/d1d/d3e/f90 901263 0 2024-05-25T06:11:29.292 INFO:tasks.workunit.client.0.smithi160.stdout:7/23: rename c5 to d6/c8 0 2024-05-25T06:11:29.293 INFO:tasks.workunit.client.1.smithi203.stdout:4/463: write d0/d8/d3a/f49 [1014645,127621] 0 2024-05-25T06:11:29.294 INFO:tasks.workunit.client.1.smithi203.stdout:2/509: stat d2/d38/l48 0 2024-05-25T06:11:29.295 INFO:tasks.workunit.client.1.smithi203.stdout:0/354: creat dd/d3d/d4f/d62/d78/f7d x:0 0 0 2024-05-25T06:11:29.296 INFO:tasks.workunit.client.1.smithi203.stdout:6/678: truncate d8/d70/f94 356972 0 2024-05-25T06:11:29.298 INFO:tasks.workunit.client.0.smithi160.stdout:2/15: write f0 [3735302,69188] 0 2024-05-25T06:11:29.299 INFO:tasks.workunit.client.0.smithi160.stdout:3/13: symlink l4 0 2024-05-25T06:11:29.299 INFO:tasks.workunit.client.1.smithi203.stdout:9/443: dwrite d0/d17/d23/f90 [0,4194304] 0 2024-05-25T06:11:29.300 INFO:tasks.workunit.client.1.smithi203.stdout:6/679: dread - d8/d11/d1b/d75/dcb/fcd zero size 2024-05-25T06:11:29.301 INFO:tasks.workunit.client.0.smithi160.stdout:6/31: creat d5/de/ff x:0 0 0 2024-05-25T06:11:29.302 INFO:tasks.workunit.client.1.smithi203.stdout:9/444: write d0/f1e [988647,15538] 0 2024-05-25T06:11:29.302 INFO:tasks.workunit.client.0.smithi160.stdout:0/30: dwrite f2 [0,4194304] 0 2024-05-25T06:11:29.304 INFO:tasks.workunit.client.0.smithi160.stdout:3/14: truncate f3 966095 0 2024-05-25T06:11:29.305 INFO:tasks.workunit.client.1.smithi203.stdout:5/740: rename dc/d11/d17/d44/d73 to dc/d11/d17/d1a/d103 0 2024-05-25T06:11:29.305 INFO:tasks.workunit.client.1.smithi203.stdout:9/445: chown d0/df/d2b/f2f 108329 1 2024-05-25T06:11:29.306 INFO:tasks.workunit.client.0.smithi160.stdout:5/44: dwrite db/dc/fe [0,4194304] 0 2024-05-25T06:11:29.306 INFO:tasks.workunit.client.0.smithi160.stdout:2/16: unlink c2 0 2024-05-25T06:11:29.307 INFO:tasks.workunit.client.1.smithi203.stdout:7/470: creat d1/d11/d3d/fa6 x:0 0 0 2024-05-25T06:11:29.308 INFO:tasks.workunit.client.0.smithi160.stdout:4/127: mknod d4/c29 0 2024-05-25T06:11:29.309 INFO:tasks.workunit.client.1.smithi203.stdout:0/355: creat dd/d10/d34/d4c/f7e x:0 0 0 2024-05-25T06:11:29.309 INFO:tasks.workunit.client.1.smithi203.stdout:7/471: chown d1/d9/f18 265 1 2024-05-25T06:11:29.311 INFO:tasks.workunit.client.1.smithi203.stdout:2/510: mkdir d2/d1a/d89/daa 0 2024-05-25T06:11:29.312 INFO:tasks.workunit.client.1.smithi203.stdout:2/511: readlink d2/d4/d10/d5f/l39 0 2024-05-25T06:11:29.313 INFO:tasks.workunit.client.0.smithi160.stdout:0/31: creat f5 x:0 0 0 2024-05-25T06:11:29.313 INFO:tasks.workunit.client.0.smithi160.stdout:1/39: dwrite f1 [0,4194304] 0 2024-05-25T06:11:29.318 INFO:tasks.workunit.client.1.smithi203.stdout:1/424: creat d14/f99 x:0 0 0 2024-05-25T06:11:29.318 INFO:tasks.workunit.client.0.smithi160.stdout:6/32: symlink d5/de/l10 0 2024-05-25T06:11:29.321 INFO:tasks.workunit.client.0.smithi160.stdout:4/128: read d4/d6/fa [705006,92027] 0 2024-05-25T06:11:29.322 INFO:tasks.workunit.client.1.smithi203.stdout:6/680: creat d8/d5a/d9b/dcc/dce/fd6 x:0 0 0 2024-05-25T06:11:29.322 INFO:tasks.workunit.client.1.smithi203.stdout:8/582: getdents d2/d5/d10/d45/d90 0 2024-05-25T06:11:29.323 INFO:tasks.workunit.client.0.smithi160.stdout:3/15: mknod c5 0 2024-05-25T06:11:29.324 INFO:tasks.workunit.client.1.smithi203.stdout:5/741: creat dc/d11/d17/d1a/d2d/d4d/ded/f104 x:0 0 0 2024-05-25T06:11:29.326 INFO:tasks.workunit.client.0.smithi160.stdout:9/37: dwrite d4/f8 [0,4194304] 0 2024-05-25T06:11:29.329 INFO:tasks.workunit.client.0.smithi160.stdout:0/32: mknod c6 0 2024-05-25T06:11:29.330 INFO:tasks.workunit.client.1.smithi203.stdout:3/568: link d10/d16/d23/c82 d10/cbd 0 2024-05-25T06:11:29.330 INFO:tasks.workunit.client.1.smithi203.stdout:3/569: chown d10/d16/d71/la9 0 1 2024-05-25T06:11:29.331 INFO:tasks.workunit.client.1.smithi203.stdout:9/446: rename d0/d17/d20/d29/c4d to d0/d17/d20/c95 0 2024-05-25T06:11:29.333 INFO:tasks.workunit.client.0.smithi160.stdout:8/90: truncate d5/fe 451753 0 2024-05-25T06:11:29.335 INFO:tasks.workunit.client.1.smithi203.stdout:1/425: creat d14/d57/d1d/d3e/f9a x:0 0 0 2024-05-25T06:11:29.337 INFO:tasks.workunit.client.0.smithi160.stdout:1/40: dwrite f9 [0,4194304] 0 2024-05-25T06:11:29.338 INFO:tasks.workunit.client.1.smithi203.stdout:0/356: truncate dd/f23 5093344 0 2024-05-25T06:11:29.338 INFO:tasks.workunit.client.0.smithi160.stdout:6/33: dwrite d5/fa [0,4194304] 0 2024-05-25T06:11:29.339 INFO:tasks.workunit.client.1.smithi203.stdout:0/357: chown dd/d3d/l6b 668862 1 2024-05-25T06:11:29.340 INFO:tasks.workunit.client.1.smithi203.stdout:2/512: link d2/d42/fa9 d2/d38/d2f/fab 0 2024-05-25T06:11:29.341 INFO:tasks.workunit.client.0.smithi160.stdout:3/16: rename f3 to f6 0 2024-05-25T06:11:29.341 INFO:tasks.workunit.client.1.smithi203.stdout:2/513: chown d2/d1a/c98 112 1 2024-05-25T06:11:29.343 INFO:tasks.workunit.client.1.smithi203.stdout:2/514: write d2/d38/f8a [2248407,77907] 0 2024-05-25T06:11:29.343 INFO:tasks.workunit.client.0.smithi160.stdout:6/34: readlink l4 0 2024-05-25T06:11:29.344 INFO:tasks.workunit.client.1.smithi203.stdout:7/472: creat d1/fa7 x:0 0 0 2024-05-25T06:11:29.346 INFO:tasks.workunit.client.1.smithi203.stdout:9/447: mkdir d0/df/d2b/d96 0 2024-05-25T06:11:29.347 INFO:tasks.workunit.client.1.smithi203.stdout:9/448: fdatasync d0/f52 0 2024-05-25T06:11:29.348 INFO:tasks.workunit.client.1.smithi203.stdout:0/358: unlink dd/d10/d34/f47 0 2024-05-25T06:11:29.349 INFO:tasks.workunit.client.0.smithi160.stdout:6/35: creat d5/f11 x:0 0 0 2024-05-25T06:11:29.351 INFO:tasks.workunit.client.1.smithi203.stdout:3/570: mknod d10/cbe 0 2024-05-25T06:11:29.352 INFO:tasks.workunit.client.0.smithi160.stdout:4/129: creat d4/f2a x:0 0 0 2024-05-25T06:11:29.353 INFO:tasks.workunit.client.0.smithi160.stdout:4/130: chown d4/f11 11887 1 2024-05-25T06:11:29.356 INFO:tasks.workunit.client.0.smithi160.stdout:7/24: link d6/c8 d6/c9 0 2024-05-25T06:11:29.357 INFO:tasks.workunit.client.0.smithi160.stdout:8/91: rmdir d5/d7/d8/d15 0 2024-05-25T06:11:29.359 INFO:tasks.workunit.client.0.smithi160.stdout:6/36: dwrite d5/fa [0,4194304] 0 2024-05-25T06:11:29.359 INFO:tasks.workunit.client.0.smithi160.stdout:8/92: dread - d5/d7/f19 zero size 2024-05-25T06:11:29.360 INFO:tasks.workunit.client.0.smithi160.stdout:4/131: mknod d4/d1a/c2b 0 2024-05-25T06:11:29.360 INFO:tasks.workunit.client.0.smithi160.stdout:8/93: chown d5/d7/c12 164489 1 2024-05-25T06:11:29.360 INFO:tasks.workunit.client.0.smithi160.stdout:8/94: chown d5 1 1 2024-05-25T06:11:29.361 INFO:tasks.workunit.client.1.smithi203.stdout:2/515: write d2/d4/f2e [5223291,105304] 0 2024-05-25T06:11:29.361 INFO:tasks.workunit.client.0.smithi160.stdout:4/132: write d4/f2a [436959,58382] 0 2024-05-25T06:11:29.363 INFO:tasks.workunit.client.0.smithi160.stdout:6/37: creat d5/de/f12 x:0 0 0 2024-05-25T06:11:29.365 INFO:tasks.workunit.client.1.smithi203.stdout:0/359: dread dd/d10/d34/f3c [0,4194304] 0 2024-05-25T06:11:29.365 INFO:tasks.workunit.client.0.smithi160.stdout:8/95: creat d5/f1a x:0 0 0 2024-05-25T06:11:29.366 INFO:tasks.workunit.client.1.smithi203.stdout:3/571: unlink la 0 2024-05-25T06:11:29.367 INFO:tasks.workunit.client.0.smithi160.stdout:8/96: stat c0 0 2024-05-25T06:11:29.367 INFO:tasks.workunit.client.0.smithi160.stdout:7/25: rename d6/c9 to d6/ca 0 2024-05-25T06:11:29.371 INFO:tasks.workunit.client.1.smithi203.stdout:2/516: truncate d2/d4/d10/d32/f1b 6271114 0 2024-05-25T06:11:29.372 INFO:tasks.workunit.client.0.smithi160.stdout:2/17: truncate f0 1547542 0 2024-05-25T06:11:29.372 INFO:tasks.workunit.client.1.smithi203.stdout:4/464: dwrite d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:29.373 INFO:tasks.workunit.client.1.smithi203.stdout:2/517: chown d2/d6/l99 256 1 2024-05-25T06:11:29.374 INFO:tasks.workunit.client.0.smithi160.stdout:8/97: rmdir d5/d7/d8 39 2024-05-25T06:11:29.376 INFO:tasks.workunit.client.0.smithi160.stdout:3/17: write f0 [5065008,1847] 0 2024-05-25T06:11:29.376 INFO:tasks.workunit.client.1.smithi203.stdout:9/449: dwrite d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:29.376 INFO:tasks.workunit.client.1.smithi203.stdout:8/583: getdents d2/d5/d28/d38/d4e 0 2024-05-25T06:11:29.377 INFO:tasks.workunit.client.0.smithi160.stdout:4/133: dwrite d4/f26 [0,4194304] 0 2024-05-25T06:11:29.377 INFO:tasks.workunit.client.0.smithi160.stdout:6/38: dwrite d5/de/ff [0,4194304] 0 2024-05-25T06:11:29.378 INFO:tasks.workunit.client.0.smithi160.stdout:7/26: dread f1 [0,4194304] 0 2024-05-25T06:11:29.379 INFO:tasks.workunit.client.0.smithi160.stdout:1/41: truncate f9 2933891 0 2024-05-25T06:11:29.381 INFO:tasks.workunit.client.1.smithi203.stdout:3/572: creat d10/d16/d23/d46/fbf x:0 0 0 2024-05-25T06:11:29.385 INFO:tasks.workunit.client.0.smithi160.stdout:4/134: dread d4/d6/fa [0,4194304] 0 2024-05-25T06:11:29.386 INFO:tasks.workunit.client.1.smithi203.stdout:3/573: stat d10/d16/d50/d8b/fb0 0 2024-05-25T06:11:29.386 INFO:tasks.workunit.client.1.smithi203.stdout:5/742: link dc/d11/d17/d1a/f27 dc/d11/f105 0 2024-05-25T06:11:29.387 INFO:tasks.workunit.client.0.smithi160.stdout:4/135: fsync d4/f2a 0 2024-05-25T06:11:29.387 INFO:tasks.workunit.client.0.smithi160.stdout:3/18: rename c5 to c7 0 2024-05-25T06:11:29.388 INFO:tasks.workunit.client.1.smithi203.stdout:0/360: creat dd/d3a/f7f x:0 0 0 2024-05-25T06:11:29.389 INFO:tasks.workunit.client.1.smithi203.stdout:2/518: creat d2/d4/d10/fac x:0 0 0 2024-05-25T06:11:29.390 INFO:tasks.workunit.client.1.smithi203.stdout:4/465: dwrite d0/d8/d3a/f46 [0,4194304] 0 2024-05-25T06:11:29.391 INFO:tasks.workunit.client.1.smithi203.stdout:3/574: write d10/d16/d50/d7d/f9c [1658298,31337] 0 2024-05-25T06:11:29.392 INFO:tasks.workunit.client.0.smithi160.stdout:8/98: dread d5/d7/d8/f16 [4194304,4194304] 0 2024-05-25T06:11:29.392 INFO:tasks.workunit.client.1.smithi203.stdout:0/361: stat dd/d10/d34 0 2024-05-25T06:11:29.394 INFO:tasks.workunit.client.1.smithi203.stdout:8/584: rename d2/d5/d10/d11/d59/f69 to d2/d5/d2d/fbe 0 2024-05-25T06:11:29.394 INFO:tasks.workunit.client.0.smithi160.stdout:7/27: stat d6/c8 0 2024-05-25T06:11:29.395 INFO:tasks.workunit.client.1.smithi203.stdout:7/473: link d1/c12 d1/d9/d60/ca8 0 2024-05-25T06:11:29.396 INFO:tasks.workunit.client.0.smithi160.stdout:4/136: readlink d4/d1a/l20 0 2024-05-25T06:11:29.398 INFO:tasks.workunit.client.0.smithi160.stdout:2/18: truncate f0 2442990 0 2024-05-25T06:11:29.398 INFO:tasks.workunit.client.1.smithi203.stdout:3/575: read d10/d16/d23/d38/f3e [1768440,122279] 0 2024-05-25T06:11:29.403 INFO:tasks.workunit.client.0.smithi160.stdout:7/28: write f1 [2169062,126832] 0 2024-05-25T06:11:29.403 INFO:tasks.workunit.client.0.smithi160.stdout:4/137: write d4/d6/fe [2031500,79957] 0 2024-05-25T06:11:29.405 INFO:tasks.workunit.client.1.smithi203.stdout:9/450: mkdir d0/df/d2d/d97 0 2024-05-25T06:11:29.407 INFO:tasks.workunit.client.1.smithi203.stdout:0/362: mknod dd/d10/d34/d4c/c80 0 2024-05-25T06:11:29.410 INFO:tasks.workunit.client.0.smithi160.stdout:1/42: dread f2 [0,4194304] 0 2024-05-25T06:11:29.411 INFO:tasks.workunit.client.0.smithi160.stdout:5/45: dwrite f6 [4194304,4194304] 0 2024-05-25T06:11:29.411 INFO:tasks.workunit.client.0.smithi160.stdout:2/19: chown f0 42878793 1 2024-05-25T06:11:29.412 INFO:tasks.workunit.client.0.smithi160.stdout:8/99: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:29.412 INFO:tasks.workunit.client.1.smithi203.stdout:7/474: dwrite d1/d9/d2c/d36/d66/f8d [0,4194304] 0 2024-05-25T06:11:29.413 INFO:tasks.workunit.client.0.smithi160.stdout:5/46: rename db to db/d10 22 2024-05-25T06:11:29.414 INFO:tasks.workunit.client.0.smithi160.stdout:5/47: fsync f4 0 2024-05-25T06:11:29.416 INFO:tasks.workunit.client.1.smithi203.stdout:3/576: dwrite d10/d16/d23/d4a/d88/f8d [0,4194304] 0 2024-05-25T06:11:29.417 INFO:tasks.workunit.client.1.smithi203.stdout:2/519: mknod d2/d38/d43/d60/da3/cad 0 2024-05-25T06:11:29.417 INFO:tasks.workunit.client.0.smithi160.stdout:1/43: creat fa x:0 0 0 2024-05-25T06:11:29.418 INFO:tasks.workunit.client.0.smithi160.stdout:4/138: dread d4/d6/d21/f24 [0,4194304] 0 2024-05-25T06:11:29.419 INFO:tasks.workunit.client.0.smithi160.stdout:1/44: dread - fa zero size 2024-05-25T06:11:29.421 INFO:tasks.workunit.client.0.smithi160.stdout:1/45: read f7 [499980,129135] 0 2024-05-25T06:11:29.421 INFO:tasks.workunit.client.1.smithi203.stdout:4/466: mkdir d0/d8/d3a/d81/d93 0 2024-05-25T06:11:29.422 INFO:tasks.workunit.client.0.smithi160.stdout:4/139: write d4/f11 [5151020,26847] 0 2024-05-25T06:11:29.422 INFO:tasks.workunit.client.0.smithi160.stdout:5/48: creat db/f11 x:0 0 0 2024-05-25T06:11:29.423 INFO:tasks.workunit.client.0.smithi160.stdout:1/46: truncate fa 427508 0 2024-05-25T06:11:29.425 INFO:tasks.workunit.client.0.smithi160.stdout:1/47: stat f2 0 2024-05-25T06:11:29.427 INFO:tasks.workunit.client.1.smithi203.stdout:5/743: truncate dc/d3c/d3f/f4e 4162845 0 2024-05-25T06:11:29.431 INFO:tasks.workunit.client.1.smithi203.stdout:0/363: symlink dd/d45/l81 0 2024-05-25T06:11:29.432 INFO:tasks.workunit.client.0.smithi160.stdout:8/100: creat d5/d7/f1b x:0 0 0 2024-05-25T06:11:29.432 INFO:tasks.workunit.client.1.smithi203.stdout:2/520: rename d2/d4/d10/d50 to d2/d38/d43/d60/d6b/dae 0 2024-05-25T06:11:29.432 INFO:tasks.workunit.client.1.smithi203.stdout:0/364: chown dd/d10/f33 240180343 1 2024-05-25T06:11:29.433 INFO:tasks.workunit.client.1.smithi203.stdout:3/577: dread d10/f4f [0,4194304] 0 2024-05-25T06:11:29.434 INFO:tasks.workunit.client.0.smithi160.stdout:8/101: stat d5/d7 0 2024-05-25T06:11:29.434 INFO:tasks.workunit.client.1.smithi203.stdout:4/467: write d0/d9/d50/d79/f65 [3445573,85061] 0 2024-05-25T06:11:29.435 INFO:tasks.workunit.client.0.smithi160.stdout:7/29: fsync f1 0 2024-05-25T06:11:29.435 INFO:tasks.workunit.client.0.smithi160.stdout:2/20: fsync f3 0 2024-05-25T06:11:29.435 INFO:tasks.workunit.client.1.smithi203.stdout:7/475: mkdir d1/da9 0 2024-05-25T06:11:29.437 INFO:tasks.workunit.client.1.smithi203.stdout:6/681: truncate d8/d70/f94 308916 0 2024-05-25T06:11:29.438 INFO:tasks.workunit.client.0.smithi160.stdout:5/49: dwrite f9 [0,4194304] 0 2024-05-25T06:11:29.438 INFO:tasks.workunit.client.0.smithi160.stdout:4/140: dwrite d4/f26 [0,4194304] 0 2024-05-25T06:11:29.444 INFO:tasks.workunit.client.0.smithi160.stdout:8/102: rename d5/d7/f1b to d5/d9/f1c 0 2024-05-25T06:11:29.447 INFO:tasks.workunit.client.1.smithi203.stdout:7/476: creat d1/d9/d2c/d36/d3b/faa x:0 0 0 2024-05-25T06:11:29.448 INFO:tasks.workunit.client.0.smithi160.stdout:4/141: chown d4/d6/l16 71404 1 2024-05-25T06:11:29.448 INFO:tasks.workunit.client.1.smithi203.stdout:6/682: creat d8/d11/d1b/d75/fd7 x:0 0 0 2024-05-25T06:11:29.449 INFO:tasks.workunit.client.0.smithi160.stdout:8/103: stat d5/d7/d8/f16 0 2024-05-25T06:11:29.450 INFO:tasks.workunit.client.0.smithi160.stdout:2/21: dwrite f0 [0,4194304] 0 2024-05-25T06:11:29.452 INFO:tasks.workunit.client.0.smithi160.stdout:5/50: dread db/dc/fe [0,4194304] 0 2024-05-25T06:11:29.454 INFO:tasks.workunit.client.0.smithi160.stdout:4/142: creat d4/d6/f2c x:0 0 0 2024-05-25T06:11:29.454 INFO:tasks.workunit.client.0.smithi160.stdout:5/51: truncate f6 8916340 0 2024-05-25T06:11:29.456 INFO:tasks.workunit.client.0.smithi160.stdout:7/30: dread f1 [0,4194304] 0 2024-05-25T06:11:29.458 INFO:tasks.workunit.client.0.smithi160.stdout:2/22: rename f3 to f4 0 2024-05-25T06:11:29.459 INFO:tasks.workunit.client.0.smithi160.stdout:2/23: chown f4 15275 1 2024-05-25T06:11:29.460 INFO:tasks.workunit.client.0.smithi160.stdout:2/24: chown f4 92216242 1 2024-05-25T06:11:29.461 INFO:tasks.workunit.client.0.smithi160.stdout:7/31: creat d6/fb x:0 0 0 2024-05-25T06:11:29.463 INFO:tasks.workunit.client.0.smithi160.stdout:8/104: dread d5/d7/d8/f16 [4194304,4194304] 0 2024-05-25T06:11:29.464 INFO:tasks.workunit.client.0.smithi160.stdout:7/32: mkdir d6/dc 0 2024-05-25T06:11:29.465 INFO:tasks.workunit.client.0.smithi160.stdout:5/52: dwrite f9 [0,4194304] 0 2024-05-25T06:11:29.466 INFO:tasks.workunit.client.0.smithi160.stdout:4/143: link d4/cf d4/c2d 0 2024-05-25T06:11:29.466 INFO:tasks.workunit.client.0.smithi160.stdout:8/105: write d5/d7/d8/f16 [6606238,56326] 0 2024-05-25T06:11:29.467 INFO:tasks.workunit.client.0.smithi160.stdout:7/33: mknod d6/dc/cd 0 2024-05-25T06:11:29.471 INFO:tasks.workunit.client.0.smithi160.stdout:7/34: symlink d6/le 0 2024-05-25T06:11:29.471 INFO:tasks.workunit.client.0.smithi160.stdout:8/106: mknod d5/d7/c1d 0 2024-05-25T06:11:29.471 INFO:tasks.workunit.client.1.smithi203.stdout:5/744: mkdir dc/d11/d106 0 2024-05-25T06:11:29.471 INFO:tasks.workunit.client.1.smithi203.stdout:8/585: dread d2/d5/d10/d45/d5d/d68/f7d [0,4194304] 0 2024-05-25T06:11:29.474 INFO:tasks.workunit.client.0.smithi160.stdout:5/53: dread f6 [4194304,4194304] 0 2024-05-25T06:11:29.474 INFO:tasks.workunit.client.1.smithi203.stdout:0/365: rename dd/d10/d34/d4c/f7b to dd/d3a/d66/d6e/f82 0 2024-05-25T06:11:29.475 INFO:tasks.workunit.client.0.smithi160.stdout:7/35: creat d6/dc/ff x:0 0 0 2024-05-25T06:11:29.475 INFO:tasks.workunit.client.0.smithi160.stdout:7/36: stat d6 0 2024-05-25T06:11:29.475 INFO:tasks.workunit.client.1.smithi203.stdout:8/586: fdatasync d2/d5/d10/d11/de/fa2 0 2024-05-25T06:11:29.476 INFO:tasks.workunit.client.0.smithi160.stdout:7/37: read f1 [3216757,86937] 0 2024-05-25T06:11:29.477 INFO:tasks.workunit.client.0.smithi160.stdout:8/107: mkdir d5/d9/d14/d1e 0 2024-05-25T06:11:29.479 INFO:tasks.workunit.client.0.smithi160.stdout:5/54: mkdir db/d12 0 2024-05-25T06:11:29.479 INFO:tasks.workunit.client.1.smithi203.stdout:4/468: creat d0/d9/f94 x:0 0 0 2024-05-25T06:11:29.481 INFO:tasks.workunit.client.0.smithi160.stdout:2/25: rename f4 to f5 0 2024-05-25T06:11:29.482 INFO:tasks.workunit.client.1.smithi203.stdout:7/477: creat d1/d11/d3d/d9a/fab x:0 0 0 2024-05-25T06:11:29.482 INFO:tasks.workunit.client.0.smithi160.stdout:7/38: mknod d6/c10 0 2024-05-25T06:11:29.483 INFO:tasks.workunit.client.1.smithi203.stdout:8/587: dread - d2/d5/d10/d11/de/f85 zero size 2024-05-25T06:11:29.483 INFO:tasks.workunit.client.1.smithi203.stdout:5/745: unlink dc/d11/d17/d1a/d2d/d53/da2/f79 0 2024-05-25T06:11:29.485 INFO:tasks.workunit.client.1.smithi203.stdout:6/683: dwrite d8/d70/fa1 [0,4194304] 0 2024-05-25T06:11:29.486 INFO:tasks.workunit.client.0.smithi160.stdout:5/55: mkdir db/dc/d13 0 2024-05-25T06:11:29.489 INFO:tasks.workunit.client.1.smithi203.stdout:6/684: chown d8/d70/d38/f4c 4800682 1 2024-05-25T06:11:29.490 INFO:tasks.workunit.client.1.smithi203.stdout:8/588: write d2/d5/d10/f83 [2579799,78903] 0 2024-05-25T06:11:29.491 INFO:tasks.workunit.client.0.smithi160.stdout:7/39: dwrite d6/fb [0,4194304] 0 2024-05-25T06:11:29.491 INFO:tasks.workunit.client.1.smithi203.stdout:5/746: chown dc/d11/d42/d94/dad/lc3 1898532 1 2024-05-25T06:11:29.493 INFO:tasks.workunit.client.1.smithi203.stdout:0/366: link dd/d10/f41 dd/d3d/d59/f83 0 2024-05-25T06:11:29.495 INFO:tasks.workunit.client.1.smithi203.stdout:8/589: chown d2/d5/d28/d38/d4e/f51 1 1 2024-05-25T06:11:29.496 INFO:tasks.workunit.client.1.smithi203.stdout:9/451: dwrite d0/f21 [0,4194304] 0 2024-05-25T06:11:29.499 INFO:tasks.workunit.client.0.smithi160.stdout:8/108: rename d5/d9/d14 to d5/d7/d8/d1f 0 2024-05-25T06:11:29.500 INFO:tasks.workunit.client.1.smithi203.stdout:7/478: dwrite d1/d11/d3d/f7f [0,4194304] 0 2024-05-25T06:11:29.500 INFO:tasks.workunit.client.0.smithi160.stdout:5/56: rmdir db 39 2024-05-25T06:11:29.501 INFO:tasks.workunit.client.1.smithi203.stdout:6/685: mkdir d8/d2a/d6c/dc1/dd8 0 2024-05-25T06:11:29.501 INFO:tasks.workunit.client.0.smithi160.stdout:7/40: symlink d6/l11 0 2024-05-25T06:11:29.503 INFO:tasks.workunit.client.0.smithi160.stdout:8/109: stat d5/d7/c1d 0 2024-05-25T06:11:29.504 INFO:tasks.workunit.client.0.smithi160.stdout:2/26: dwrite f5 [4194304,4194304] 0 2024-05-25T06:11:29.504 INFO:tasks.workunit.client.1.smithi203.stdout:9/452: write d0/d17/f59 [353876,97841] 0 2024-05-25T06:11:29.505 INFO:tasks.workunit.client.1.smithi203.stdout:8/590: mkdir d2/d5/d18/dbf 0 2024-05-25T06:11:29.507 INFO:tasks.workunit.client.0.smithi160.stdout:2/27: chown f0 641 1 2024-05-25T06:11:29.508 INFO:tasks.workunit.client.0.smithi160.stdout:2/28: read f5 [6657768,59752] 0 2024-05-25T06:11:29.509 INFO:tasks.workunit.client.1.smithi203.stdout:8/591: truncate d2/d5/d28/f4c 4433984 0 2024-05-25T06:11:29.509 INFO:tasks.workunit.client.1.smithi203.stdout:7/479: stat d1/d9/d2c/d3a/d4e/d50/c8a 0 2024-05-25T06:11:29.511 INFO:tasks.workunit.client.0.smithi160.stdout:7/41: dread d6/fb [0,4194304] 0 2024-05-25T06:11:29.512 INFO:tasks.workunit.client.1.smithi203.stdout:8/592: chown d2/d5/d28/d38/d81/d84/cb4 4 1 2024-05-25T06:11:29.512 INFO:tasks.workunit.client.1.smithi203.stdout:6/686: creat d8/d11/d47/d60/fd9 x:0 0 0 2024-05-25T06:11:29.513 INFO:tasks.workunit.client.0.smithi160.stdout:7/42: chown f1 760 1 2024-05-25T06:11:29.514 INFO:tasks.workunit.client.0.smithi160.stdout:5/57: symlink db/l14 0 2024-05-25T06:11:29.515 INFO:tasks.workunit.client.1.smithi203.stdout:9/453: rmdir d0/df 39 2024-05-25T06:11:29.516 INFO:tasks.workunit.client.0.smithi160.stdout:7/43: creat d6/f12 x:0 0 0 2024-05-25T06:11:29.519 INFO:tasks.workunit.client.0.smithi160.stdout:8/110: rename d5/d7/d8/d1f/l18 to d5/d9/l20 0 2024-05-25T06:11:29.525 INFO:tasks.workunit.client.1.smithi203.stdout:9/454: creat d0/d17/d20/d6e/d75/f98 x:0 0 0 2024-05-25T06:11:29.525 INFO:tasks.workunit.client.0.smithi160.stdout:5/58: link f9 db/dc/f15 0 2024-05-25T06:11:29.525 INFO:tasks.workunit.client.0.smithi160.stdout:7/44: dwrite d6/dc/ff [0,4194304] 0 2024-05-25T06:11:29.525 INFO:tasks.workunit.client.0.smithi160.stdout:7/45: stat d6/l11 0 2024-05-25T06:11:29.528 INFO:tasks.workunit.client.0.smithi160.stdout:8/111: symlink d5/d7/d8/l21 0 2024-05-25T06:11:29.528 INFO:tasks.workunit.client.1.smithi203.stdout:7/480: dwrite d1/d9/d2c/d3a/d4e/f6c [0,4194304] 0 2024-05-25T06:11:29.529 INFO:tasks.workunit.client.0.smithi160.stdout:8/112: write d5/f1a [213620,85034] 0 2024-05-25T06:11:29.530 INFO:tasks.workunit.client.1.smithi203.stdout:8/593: dwrite d2/d5/d10/d45/d5d/d68/f73 [0,4194304] 0 2024-05-25T06:11:29.530 INFO:tasks.workunit.client.0.smithi160.stdout:7/46: symlink d6/dc/l13 0 2024-05-25T06:11:29.531 INFO:tasks.workunit.client.1.smithi203.stdout:7/481: write d1/f4d [513411,126119] 0 2024-05-25T06:11:29.532 INFO:tasks.workunit.client.0.smithi160.stdout:7/47: chown d6/dc/cd 259 1 2024-05-25T06:11:29.533 INFO:tasks.workunit.client.0.smithi160.stdout:8/113: write d5/fb [2037993,122113] 0 2024-05-25T06:11:29.534 INFO:tasks.workunit.client.1.smithi203.stdout:9/455: chown d0/d17/d20/d6e/d75/f98 911145 1 2024-05-25T06:11:29.536 INFO:tasks.workunit.client.0.smithi160.stdout:7/48: rename d6/dc/cd to d6/dc/c14 0 2024-05-25T06:11:29.537 INFO:tasks.workunit.client.1.smithi203.stdout:8/594: dread d2/d5/d2d/f3e [4194304,4194304] 0 2024-05-25T06:11:29.540 INFO:tasks.workunit.client.0.smithi160.stdout:7/49: creat d6/dc/f15 x:0 0 0 2024-05-25T06:11:29.540 INFO:tasks.workunit.client.0.smithi160.stdout:8/114: creat d5/d7/d8/d1f/f22 x:0 0 0 2024-05-25T06:11:29.541 INFO:tasks.workunit.client.1.smithi203.stdout:8/595: chown d2/d5/d10/c57 28034922 1 2024-05-25T06:11:29.543 INFO:tasks.workunit.client.0.smithi160.stdout:7/50: truncate d6/dc/f15 1032315 0 2024-05-25T06:11:29.545 INFO:tasks.workunit.client.0.smithi160.stdout:8/115: dread d5/fe [0,4194304] 0 2024-05-25T06:11:29.546 INFO:tasks.workunit.client.1.smithi203.stdout:8/596: dread - d2/d5/d10/d11/d1f/d75/f8d zero size 2024-05-25T06:11:29.546 INFO:tasks.workunit.client.1.smithi203.stdout:4/469: dwrite d0/f3 [0,4194304] 0 2024-05-25T06:11:29.547 INFO:tasks.workunit.client.0.smithi160.stdout:7/51: dread f1 [0,4194304] 0 2024-05-25T06:11:29.547 INFO:tasks.workunit.client.0.smithi160.stdout:7/52: chown d6 4 1 2024-05-25T06:11:29.549 INFO:tasks.workunit.client.1.smithi203.stdout:9/456: link d0/d17/d20/d3c/l3d d0/d17/d20/d29/l99 0 2024-05-25T06:11:29.549 INFO:tasks.workunit.client.1.smithi203.stdout:4/470: dread - d0/d9/d50/d79/f40 zero size 2024-05-25T06:11:29.553 INFO:tasks.workunit.client.1.smithi203.stdout:6/687: dread d8/d11/d15/d28/d52/d43/f93 [0,4194304] 0 2024-05-25T06:11:29.553 INFO:tasks.workunit.client.1.smithi203.stdout:6/688: stat d8/d70/ld5 0 2024-05-25T06:11:29.554 INFO:tasks.workunit.client.0.smithi160.stdout:7/53: dwrite d6/dc/ff [0,4194304] 0 2024-05-25T06:11:29.555 INFO:tasks.workunit.client.1.smithi203.stdout:8/597: symlink d2/d5/d10/d45/d5d/lc0 0 2024-05-25T06:11:29.556 INFO:tasks.workunit.client.1.smithi203.stdout:9/457: mknod d0/df/d37/c9a 0 2024-05-25T06:11:29.560 INFO:tasks.workunit.client.1.smithi203.stdout:0/367: dread dd/d32/d37/f42 [0,4194304] 0 2024-05-25T06:11:29.562 INFO:tasks.workunit.client.0.smithi160.stdout:2/29: fdatasync f0 0 2024-05-25T06:11:29.562 INFO:tasks.workunit.client.1.smithi203.stdout:6/689: rename d8/d11/fb6 to d8/d11/d1b/d75/fda 0 2024-05-25T06:11:29.566 INFO:tasks.workunit.client.0.smithi160.stdout:2/30: write f0 [2731403,103268] 0 2024-05-25T06:11:29.566 INFO:tasks.workunit.client.0.smithi160.stdout:2/31: readlink - no filename 2024-05-25T06:11:29.566 INFO:tasks.workunit.client.0.smithi160.stdout:2/32: readlink - no filename 2024-05-25T06:11:29.566 INFO:tasks.workunit.client.0.smithi160.stdout:2/33: rmdir - no directory 2024-05-25T06:11:29.567 INFO:tasks.workunit.client.0.smithi160.stdout:5/59: getdents db 0 2024-05-25T06:11:29.567 INFO:tasks.workunit.client.0.smithi160.stdout:2/34: chown f5 6680811 1 2024-05-25T06:11:29.567 INFO:tasks.workunit.client.0.smithi160.stdout:5/60: chown db/dc/fe 102089 1 2024-05-25T06:11:29.572 INFO:tasks.workunit.client.0.smithi160.stdout:5/61: dread f5 [0,4194304] 0 2024-05-25T06:11:29.572 INFO:tasks.workunit.client.0.smithi160.stdout:7/54: getdents d6/dc 0 2024-05-25T06:11:29.574 INFO:tasks.workunit.client.1.smithi203.stdout:9/458: creat d0/d57/d8f/f9b x:0 0 0 2024-05-25T06:11:29.574 INFO:tasks.workunit.client.0.smithi160.stdout:2/35: dwrite f5 [4194304,4194304] 0 2024-05-25T06:11:29.575 INFO:tasks.workunit.client.0.smithi160.stdout:2/36: read f0 [548649,40391] 0 2024-05-25T06:11:29.575 INFO:tasks.workunit.client.1.smithi203.stdout:8/598: creat d2/d5/d28/d38/d4e/d97/fc1 x:0 0 0 2024-05-25T06:11:29.576 INFO:tasks.workunit.client.0.smithi160.stdout:7/55: dread d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:29.576 INFO:tasks.workunit.client.0.smithi160.stdout:7/56: readlink d6/l11 0 2024-05-25T06:11:29.578 INFO:tasks.workunit.client.1.smithi203.stdout:9/459: readlink d0/df/d2d/l89 0 2024-05-25T06:11:29.579 INFO:tasks.workunit.client.1.smithi203.stdout:0/368: truncate dd/d10/f31 4319931 0 2024-05-25T06:11:29.580 INFO:tasks.workunit.client.0.smithi160.stdout:2/37: symlink l6 0 2024-05-25T06:11:29.581 INFO:tasks.workunit.client.0.smithi160.stdout:5/62: rmdir db/d12 0 2024-05-25T06:11:29.582 INFO:tasks.workunit.client.0.smithi160.stdout:5/63: dread - f8 zero size 2024-05-25T06:11:29.583 INFO:tasks.workunit.client.0.smithi160.stdout:7/57: rmdir d6 39 2024-05-25T06:11:29.588 INFO:tasks.workunit.client.1.smithi203.stdout:6/690: truncate d8/d11/d1b/d75/dcb/dcf/d6e/f9e 14198 0 2024-05-25T06:11:29.589 INFO:tasks.workunit.client.1.smithi203.stdout:5/747: dwrite dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:29.589 INFO:tasks.workunit.client.0.smithi160.stdout:7/58: creat d6/dc/f16 x:0 0 0 2024-05-25T06:11:29.589 INFO:tasks.workunit.client.1.smithi203.stdout:8/599: rename d2/d5/d28/d38/d81/d84 to d2/d5/d18/d31/dac/dc2 0 2024-05-25T06:11:29.591 INFO:tasks.workunit.client.1.smithi203.stdout:0/369: mkdir dd/d3a/d66/d6e/d70/d84 0 2024-05-25T06:11:29.592 INFO:tasks.workunit.client.1.smithi203.stdout:7/482: dread d1/f1c [0,4194304] 0 2024-05-25T06:11:29.592 INFO:tasks.workunit.client.0.smithi160.stdout:2/38: truncate f5 4680376 0 2024-05-25T06:11:29.592 INFO:tasks.workunit.client.0.smithi160.stdout:5/64: dwrite f5 [0,4194304] 0 2024-05-25T06:11:29.593 INFO:tasks.workunit.client.1.smithi203.stdout:6/691: chown d8/d11/d15/d28/d52/d43/f93 248 1 2024-05-25T06:11:29.594 INFO:tasks.workunit.client.0.smithi160.stdout:5/65: write f4 [4894713,127711] 0 2024-05-25T06:11:29.595 INFO:tasks.workunit.client.0.smithi160.stdout:7/59: creat d6/dc/f17 x:0 0 0 2024-05-25T06:11:29.596 INFO:tasks.workunit.client.1.smithi203.stdout:0/370: dread dd/fe [0,4194304] 0 2024-05-25T06:11:29.598 INFO:tasks.workunit.client.1.smithi203.stdout:6/692: fdatasync d8/d3d/f6a 0 2024-05-25T06:11:29.598 INFO:tasks.workunit.client.1.smithi203.stdout:7/483: write d1/d9/d2c/d79/f81 [1197342,88339] 0 2024-05-25T06:11:29.598 INFO:tasks.workunit.client.1.smithi203.stdout:0/371: dread - dd/f64 zero size 2024-05-25T06:11:29.599 INFO:tasks.workunit.client.1.smithi203.stdout:9/460: dwrite d0/f52 [0,4194304] 0 2024-05-25T06:11:29.602 INFO:tasks.workunit.client.1.smithi203.stdout:8/600: dread d2/d5/d28/d38/f3c [4194304,4194304] 0 2024-05-25T06:11:29.604 INFO:tasks.workunit.client.1.smithi203.stdout:8/601: dread - d2/d5/d28/d38/d4e/d97/fc1 zero size 2024-05-25T06:11:29.607 INFO:tasks.workunit.client.1.smithi203.stdout:6/693: chown d8/d3d/lca 81556920 1 2024-05-25T06:11:29.611 INFO:tasks.workunit.client.1.smithi203.stdout:7/484: dread - d1/d11/f41 zero size 2024-05-25T06:11:29.612 INFO:tasks.workunit.client.1.smithi203.stdout:7/485: stat d1/d9/d2c/d36/d54 0 2024-05-25T06:11:29.613 INFO:tasks.workunit.client.1.smithi203.stdout:7/486: chown d1/f31 23719 1 2024-05-25T06:11:29.613 INFO:tasks.workunit.client.1.smithi203.stdout:5/748: unlink dc/d11/d17/d1a/l2a 0 2024-05-25T06:11:29.615 INFO:tasks.workunit.client.1.smithi203.stdout:5/749: chown dc/fe6 0 1 2024-05-25T06:11:29.616 INFO:tasks.workunit.client.1.smithi203.stdout:8/602: rename d2/d5/d10/f83 to d2/d5/d18/d31/dac/dc2/d88/fc3 0 2024-05-25T06:11:29.616 INFO:tasks.workunit.client.0.smithi160.stdout:7/60: creat d6/dc/f18 x:0 0 0 2024-05-25T06:11:29.617 INFO:tasks.workunit.client.0.smithi160.stdout:5/66: symlink db/dc/d13/l16 0 2024-05-25T06:11:29.617 INFO:tasks.workunit.client.1.smithi203.stdout:6/694: symlink d8/d11/d1b/d75/dcb/dcf/d46/ldb 0 2024-05-25T06:11:29.618 INFO:tasks.workunit.client.1.smithi203.stdout:8/603: write d2/d5/d18/d31/f5e [1234478,78102] 0 2024-05-25T06:11:29.618 INFO:tasks.workunit.client.0.smithi160.stdout:5/67: write fa [373592,87436] 0 2024-05-25T06:11:29.618 INFO:tasks.workunit.client.0.smithi160.stdout:5/68: chown f6 1108 1 2024-05-25T06:11:29.620 INFO:tasks.workunit.client.1.smithi203.stdout:5/750: rmdir dc/d11/d17 39 2024-05-25T06:11:29.620 INFO:tasks.workunit.client.1.smithi203.stdout:6/695: dread f1 [0,4194304] 0 2024-05-25T06:11:29.623 INFO:tasks.workunit.client.1.smithi203.stdout:8/604: symlink d2/d5/d2d/lc4 0 2024-05-25T06:11:29.623 INFO:tasks.workunit.client.1.smithi203.stdout:0/372: link dd/c30 dd/d3a/d66/d6e/c85 0 2024-05-25T06:11:29.625 INFO:tasks.workunit.client.1.smithi203.stdout:8/605: chown d2/d5/d10/d11 420913 1 2024-05-25T06:11:29.627 INFO:tasks.workunit.client.1.smithi203.stdout:7/487: getdents d1/d9/d2c/d62/d7a/d89 0 2024-05-25T06:11:29.627 INFO:tasks.workunit.client.1.smithi203.stdout:9/461: dwrite d0/fb [8388608,4194304] 0 2024-05-25T06:11:29.628 INFO:tasks.workunit.client.1.smithi203.stdout:7/488: fsync d1/fa 0 2024-05-25T06:11:29.629 INFO:tasks.workunit.client.1.smithi203.stdout:8/606: chown d2/d5/d10/d11/lb 77349 1 2024-05-25T06:11:29.629 INFO:tasks.workunit.client.1.smithi203.stdout:6/696: mkdir d8/d11/d15/d28/d52/d43/d73/db7/ddc 0 2024-05-25T06:11:29.630 INFO:tasks.workunit.client.1.smithi203.stdout:6/697: read - d8/d2a/f67 zero size 2024-05-25T06:11:29.631 INFO:tasks.workunit.client.1.smithi203.stdout:6/698: fdatasync d8/d5a/d9b/fd4 0 2024-05-25T06:11:29.632 INFO:tasks.workunit.client.1.smithi203.stdout:0/373: mkdir dd/d3a/d66/d86 0 2024-05-25T06:11:29.635 INFO:tasks.workunit.client.1.smithi203.stdout:9/462: unlink d0/df/l35 0 2024-05-25T06:11:29.637 INFO:tasks.workunit.client.1.smithi203.stdout:7/489: mkdir d1/d9/d60/dac 0 2024-05-25T06:11:29.640 INFO:tasks.workunit.client.1.smithi203.stdout:8/607: write d2/d5/d2d/f3e [788753,126128] 0 2024-05-25T06:11:29.640 INFO:tasks.workunit.client.1.smithi203.stdout:5/751: unlink dc/d11/d17/d1a/d2d/d4d/d64/dd4/c82 0 2024-05-25T06:11:29.641 INFO:tasks.workunit.client.1.smithi203.stdout:6/699: dwrite d8/d11/d1b/fb0 [0,4194304] 0 2024-05-25T06:11:29.646 INFO:tasks.workunit.client.1.smithi203.stdout:0/374: truncate dd/f24 206547 0 2024-05-25T06:11:29.650 INFO:tasks.workunit.client.1.smithi203.stdout:6/700: unlink d8/d11/cbd 0 2024-05-25T06:11:29.650 INFO:tasks.workunit.client.1.smithi203.stdout:6/701: readlink l5 0 2024-05-25T06:11:29.651 INFO:tasks.workunit.client.1.smithi203.stdout:7/490: dwrite d1/f4d [4194304,4194304] 0 2024-05-25T06:11:29.653 INFO:tasks.workunit.client.1.smithi203.stdout:9/463: rename d0/df/d37/c5d to d0/d17/d20/d6e/d75/d78/d7a/c9c 0 2024-05-25T06:11:29.653 INFO:tasks.workunit.client.1.smithi203.stdout:5/752: fsync dc/d11/d17/d1a/d2d/d4d/f7b 0 2024-05-25T06:11:29.654 INFO:tasks.workunit.client.1.smithi203.stdout:0/375: creat dd/d3d/d4f/f87 x:0 0 0 2024-05-25T06:11:29.656 INFO:tasks.workunit.client.1.smithi203.stdout:0/376: dread - dd/d5c/f68 zero size 2024-05-25T06:11:29.657 INFO:tasks.workunit.client.1.smithi203.stdout:7/491: chown d1/d9/l16 331 1 2024-05-25T06:11:29.657 INFO:tasks.workunit.client.1.smithi203.stdout:8/608: creat d2/d5/d10/d45/d5d/fc5 x:0 0 0 2024-05-25T06:11:29.658 INFO:tasks.workunit.client.1.smithi203.stdout:0/377: chown dd 1 1 2024-05-25T06:11:29.660 INFO:tasks.workunit.client.1.smithi203.stdout:9/464: rename d0/df/l1c to d0/d17/d20/l9d 0 2024-05-25T06:11:29.661 INFO:tasks.workunit.client.1.smithi203.stdout:7/492: truncate d1/d9/d2c/d3a/f4b 166409 0 2024-05-25T06:11:29.661 INFO:tasks.workunit.client.1.smithi203.stdout:8/609: truncate d2/d5/d28/d38/f49 591512 0 2024-05-25T06:11:29.661 INFO:tasks.workunit.client.1.smithi203.stdout:9/465: fdatasync d0/d17/d20/d6e/f8d 0 2024-05-25T06:11:29.663 INFO:tasks.workunit.client.1.smithi203.stdout:0/378: mkdir dd/d3a/d66/d88 0 2024-05-25T06:11:29.663 INFO:tasks.workunit.client.1.smithi203.stdout:9/466: rmdir d0/d57 39 2024-05-25T06:11:29.666 INFO:tasks.workunit.client.1.smithi203.stdout:5/753: rename dc/d6a to dc/d11/d18/dc0/d107 0 2024-05-25T06:11:29.671 INFO:tasks.workunit.client.1.smithi203.stdout:6/702: write d8/d11/d15/f2b [2454173,12816] 0 2024-05-25T06:11:29.671 INFO:tasks.workunit.client.1.smithi203.stdout:5/754: creat dc/d11/d17/d1a/d2d/d4d/d64/f108 x:0 0 0 2024-05-25T06:11:29.672 INFO:tasks.workunit.client.1.smithi203.stdout:6/703: chown d8/d11/d47/fb8 76 1 2024-05-25T06:11:29.673 INFO:tasks.workunit.client.1.smithi203.stdout:6/704: stat d8/d11/d15/l86 0 2024-05-25T06:11:29.674 INFO:tasks.workunit.client.1.smithi203.stdout:9/467: unlink d0/df/f1d 0 2024-05-25T06:11:29.674 INFO:tasks.workunit.client.1.smithi203.stdout:0/379: mknod dd/d3a/d66/d88/c89 0 2024-05-25T06:11:29.676 INFO:tasks.workunit.client.1.smithi203.stdout:0/380: chown dd/c30 0 1 2024-05-25T06:11:29.678 INFO:tasks.workunit.client.1.smithi203.stdout:7/493: getdents d1/d9/d2c/d36 0 2024-05-25T06:11:29.680 INFO:tasks.workunit.client.1.smithi203.stdout:9/468: rename d0/d17/f59 to d0/d57/d8f/f9e 0 2024-05-25T06:11:29.681 INFO:tasks.workunit.client.0.smithi160.stdout:3/19: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.0.smithi160.stdout:9/38: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.0.smithi160.stdout:1/48: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.0.smithi160.stdout:8/116: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.0.smithi160.stdout:0/33: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.0.smithi160.stdout:6/39: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.0.smithi160.stdout:4/144: sync 2024-05-25T06:11:29.682 INFO:tasks.workunit.client.1.smithi203.stdout:0/381: creat dd/d3d/d4f/f8a x:0 0 0 2024-05-25T06:11:29.683 INFO:tasks.workunit.client.0.smithi160.stdout:3/20: readlink l4 0 2024-05-25T06:11:29.683 INFO:tasks.workunit.client.0.smithi160.stdout:1/49: chown f1 35147 1 2024-05-25T06:11:29.684 INFO:tasks.workunit.client.1.smithi203.stdout:8/610: dread d2/d5/d18/f1b [0,4194304] 0 2024-05-25T06:11:29.684 INFO:tasks.workunit.client.0.smithi160.stdout:1/50: readlink l6 0 2024-05-25T06:11:29.684 INFO:tasks.workunit.client.0.smithi160.stdout:0/34: write f5 [1027566,94022] 0 2024-05-25T06:11:29.684 INFO:tasks.workunit.client.0.smithi160.stdout:4/145: write d4/f26 [2706523,55621] 0 2024-05-25T06:11:29.684 INFO:tasks.workunit.client.0.smithi160.stdout:3/21: chown f0 13936 1 2024-05-25T06:11:29.685 INFO:tasks.workunit.client.1.smithi203.stdout:7/494: readlink d1/d11/l1a 0 2024-05-25T06:11:29.685 INFO:tasks.workunit.client.1.smithi203.stdout:8/611: chown d2/d5/d10/d45/d5d/fc5 343053 1 2024-05-25T06:11:29.685 INFO:tasks.workunit.client.1.smithi203.stdout:8/612: fsync d2/d5/d10/d11/d1f/fb5 0 2024-05-25T06:11:29.686 INFO:tasks.workunit.client.1.smithi203.stdout:9/469: mknod d0/df/d73/c9f 0 2024-05-25T06:11:29.687 INFO:tasks.workunit.client.1.smithi203.stdout:0/382: mknod dd/d3d/c8b 0 2024-05-25T06:11:29.690 INFO:tasks.workunit.client.0.smithi160.stdout:4/146: dwrite d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:29.691 INFO:tasks.workunit.client.0.smithi160.stdout:4/147: rename d4 to d4/d6/d21/d2e 22 2024-05-25T06:11:29.691 INFO:tasks.workunit.client.0.smithi160.stdout:4/148: chown d4/d1a 926842 1 2024-05-25T06:11:29.708 INFO:tasks.workunit.client.0.smithi160.stdout:7/61: creat d6/dc/f19 x:0 0 0 2024-05-25T06:11:29.708 INFO:tasks.workunit.client.0.smithi160.stdout:2/39: rename f0 to f7 0 2024-05-25T06:11:29.709 INFO:tasks.workunit.client.0.smithi160.stdout:5/69: mknod db/dc/d13/c17 0 2024-05-25T06:11:29.710 INFO:tasks.workunit.client.0.smithi160.stdout:6/40: unlink c3 0 2024-05-25T06:11:29.711 INFO:tasks.workunit.client.0.smithi160.stdout:8/117: creat d5/d7/d8/f23 x:0 0 0 2024-05-25T06:11:29.712 INFO:tasks.workunit.client.0.smithi160.stdout:8/118: stat d5/d9/fc 0 2024-05-25T06:11:29.712 INFO:tasks.workunit.client.0.smithi160.stdout:3/22: write f6 [411886,41666] 0 2024-05-25T06:11:29.712 INFO:tasks.workunit.client.0.smithi160.stdout:4/149: unlink d4/d6/c8 0 2024-05-25T06:11:29.713 INFO:tasks.workunit.client.0.smithi160.stdout:3/23: chown f6 6 1 2024-05-25T06:11:29.714 INFO:tasks.workunit.client.0.smithi160.stdout:6/41: dread d5/fb [0,4194304] 0 2024-05-25T06:11:29.714 INFO:tasks.workunit.client.0.smithi160.stdout:5/70: fsync f9 0 2024-05-25T06:11:29.715 INFO:tasks.workunit.client.0.smithi160.stdout:6/42: fdatasync d5/fa 0 2024-05-25T06:11:29.717 INFO:tasks.workunit.client.0.smithi160.stdout:8/119: creat d5/d9/f24 x:0 0 0 2024-05-25T06:11:29.718 INFO:tasks.workunit.client.0.smithi160.stdout:8/120: truncate d5/d9/fc 969048 0 2024-05-25T06:11:29.719 INFO:tasks.workunit.client.0.smithi160.stdout:2/40: link f5 f8 0 2024-05-25T06:11:29.720 INFO:tasks.workunit.client.0.smithi160.stdout:3/24: creat f8 x:0 0 0 2024-05-25T06:11:29.724 INFO:tasks.workunit.client.0.smithi160.stdout:5/71: symlink db/dc/d13/l18 0 2024-05-25T06:11:29.724 INFO:tasks.workunit.client.0.smithi160.stdout:5/72: chown f4 0 1 2024-05-25T06:11:29.724 INFO:tasks.workunit.client.0.smithi160.stdout:6/43: symlink d5/l13 0 2024-05-25T06:11:29.725 INFO:tasks.workunit.client.0.smithi160.stdout:7/62: link d6/l11 d6/l1a 0 2024-05-25T06:11:29.725 INFO:tasks.workunit.client.0.smithi160.stdout:6/44: read d5/f7 [1398960,129618] 0 2024-05-25T06:11:29.725 INFO:tasks.workunit.client.0.smithi160.stdout:7/63: dread - d6/dc/f18 zero size 2024-05-25T06:11:29.726 INFO:tasks.workunit.client.0.smithi160.stdout:4/150: mknod d4/d1c/c2f 0 2024-05-25T06:11:29.726 INFO:tasks.workunit.client.0.smithi160.stdout:7/64: dread - d6/dc/f17 zero size 2024-05-25T06:11:29.727 INFO:tasks.workunit.client.0.smithi160.stdout:6/45: write d5/de/ff [4021279,16217] 0 2024-05-25T06:11:29.727 INFO:tasks.workunit.client.0.smithi160.stdout:8/121: creat d5/d7/d8/f25 x:0 0 0 2024-05-25T06:11:29.727 INFO:tasks.workunit.client.0.smithi160.stdout:2/41: creat f9 x:0 0 0 2024-05-25T06:11:29.728 INFO:tasks.workunit.client.0.smithi160.stdout:8/122: chown d5/d7/l10 1227146450 1 2024-05-25T06:11:29.731 INFO:tasks.workunit.client.1.smithi203.stdout:9/470: read d0/f16 [8167174,19181] 0 2024-05-25T06:11:29.732 INFO:tasks.workunit.client.1.smithi203.stdout:8/613: creat d2/d5/d10/fc6 x:0 0 0 2024-05-25T06:11:29.736 INFO:tasks.workunit.client.0.smithi160.stdout:4/151: symlink d4/d6/d28/l30 0 2024-05-25T06:11:29.737 INFO:tasks.workunit.client.1.smithi203.stdout:7/495: mkdir d1/d11/d3d/dad 0 2024-05-25T06:11:29.737 INFO:tasks.workunit.client.1.smithi203.stdout:5/755: dwrite dc/f10 [0,4194304] 0 2024-05-25T06:11:29.738 INFO:tasks.workunit.client.0.smithi160.stdout:6/46: creat d5/f14 x:0 0 0 2024-05-25T06:11:29.738 INFO:tasks.workunit.client.0.smithi160.stdout:6/47: fdatasync d5/de/f12 0 2024-05-25T06:11:29.742 INFO:tasks.workunit.client.0.smithi160.stdout:8/123: rename d5/d7/d8/f23 to d5/d9/f26 0 2024-05-25T06:11:29.745 INFO:tasks.workunit.client.1.smithi203.stdout:9/471: creat d0/df/d2b/d96/fa0 x:0 0 0 2024-05-25T06:11:29.745 INFO:tasks.workunit.client.0.smithi160.stdout:4/152: mknod d4/d1c/c31 0 2024-05-25T06:11:29.746 INFO:tasks.workunit.client.1.smithi203.stdout:5/756: stat dc/daa/fe5 0 2024-05-25T06:11:29.747 INFO:tasks.workunit.client.0.smithi160.stdout:6/48: creat d5/f15 x:0 0 0 2024-05-25T06:11:29.749 INFO:tasks.workunit.client.1.smithi203.stdout:9/472: creat d0/df/d73/fa1 x:0 0 0 2024-05-25T06:11:29.750 INFO:tasks.workunit.client.1.smithi203.stdout:5/757: unlink dc/d11/d17/fba 0 2024-05-25T06:11:29.750 INFO:tasks.workunit.client.1.smithi203.stdout:9/473: write d0/d17/d23/f90 [3740606,620] 0 2024-05-25T06:11:29.750 INFO:tasks.workunit.client.0.smithi160.stdout:8/124: mkdir d5/d7/d8/d27 0 2024-05-25T06:11:29.751 INFO:tasks.workunit.client.1.smithi203.stdout:7/496: mknod d1/d9/cae 0 2024-05-25T06:11:29.752 INFO:tasks.workunit.client.0.smithi160.stdout:4/153: creat d4/d6/d21/f32 x:0 0 0 2024-05-25T06:11:29.753 INFO:tasks.workunit.client.0.smithi160.stdout:4/154: write d4/d6/d28/f25 [719388,109693] 0 2024-05-25T06:11:29.754 INFO:tasks.workunit.client.1.smithi203.stdout:5/758: creat dc/d11/d17/d1a/d72/f109 x:0 0 0 2024-05-25T06:11:29.755 INFO:tasks.workunit.client.0.smithi160.stdout:6/49: mkdir d5/de/d16 0 2024-05-25T06:11:29.757 INFO:tasks.workunit.client.0.smithi160.stdout:3/25: link c7 c9 0 2024-05-25T06:11:29.759 INFO:tasks.workunit.client.1.smithi203.stdout:5/759: mkdir dc/d11/d42/d94/d10a 0 2024-05-25T06:11:29.759 INFO:tasks.workunit.client.0.smithi160.stdout:6/50: creat d5/de/f17 x:0 0 0 2024-05-25T06:11:29.760 INFO:tasks.workunit.client.1.smithi203.stdout:0/383: dwrite dd/d3a/f4b [4194304,4194304] 0 2024-05-25T06:11:29.761 INFO:tasks.workunit.client.0.smithi160.stdout:6/51: chown d5/de/ff 3198980 1 2024-05-25T06:11:29.762 INFO:tasks.workunit.client.0.smithi160.stdout:4/155: dwrite d4/d6/d21/f32 [0,4194304] 0 2024-05-25T06:11:29.762 INFO:tasks.workunit.client.1.smithi203.stdout:7/497: dwrite d1/d9/d2c/d79/f81 [4194304,4194304] 0 2024-05-25T06:11:29.763 INFO:tasks.workunit.client.0.smithi160.stdout:6/52: chown d5/de/l10 4527897 1 2024-05-25T06:11:29.763 INFO:tasks.workunit.client.0.smithi160.stdout:8/125: unlink d5/d7/d8/d1f/c17 0 2024-05-25T06:11:29.764 INFO:tasks.workunit.client.0.smithi160.stdout:3/26: dread f0 [0,4194304] 0 2024-05-25T06:11:29.768 INFO:tasks.workunit.client.1.smithi203.stdout:5/760: creat dc/d11/d17/d1a/d2d/f10b x:0 0 0 2024-05-25T06:11:29.771 INFO:tasks.workunit.client.1.smithi203.stdout:0/384: truncate dd/d32/f54 517810 0 2024-05-25T06:11:29.772 INFO:tasks.workunit.client.1.smithi203.stdout:5/761: mkdir dc/daa/d10c 0 2024-05-25T06:11:29.773 INFO:tasks.workunit.client.0.smithi160.stdout:6/53: mkdir d5/de/d18 0 2024-05-25T06:11:29.773 INFO:tasks.workunit.client.0.smithi160.stdout:6/54: dread - d5/de/fd zero size 2024-05-25T06:11:29.775 INFO:tasks.workunit.client.0.smithi160.stdout:6/55: truncate d5/de/fd 844272 0 2024-05-25T06:11:29.776 INFO:tasks.workunit.client.0.smithi160.stdout:3/27: rename f6 to fa 0 2024-05-25T06:11:29.777 INFO:tasks.workunit.client.1.smithi203.stdout:0/385: mkdir dd/d8c 0 2024-05-25T06:11:29.779 INFO:tasks.workunit.client.1.smithi203.stdout:5/762: creat dc/d11/d42/d94/d10a/f10d x:0 0 0 2024-05-25T06:11:29.779 INFO:tasks.workunit.client.1.smithi203.stdout:7/498: dwrite d1/d9/d2c/f42 [0,4194304] 0 2024-05-25T06:11:29.780 INFO:tasks.workunit.client.1.smithi203.stdout:0/386: chown dd/c36 41 1 2024-05-25T06:11:29.781 INFO:tasks.workunit.client.1.smithi203.stdout:5/763: write dc/d11/f1c [1476198,63654] 0 2024-05-25T06:11:29.782 INFO:tasks.workunit.client.0.smithi160.stdout:6/56: symlink d5/l19 0 2024-05-25T06:11:29.782 INFO:tasks.workunit.client.1.smithi203.stdout:5/764: chown dc/d11/d17/d1a/d2d/d4d/d64 0 1 2024-05-25T06:11:29.783 INFO:tasks.workunit.client.0.smithi160.stdout:6/57: rename d5/de/d18 to d5/de/d18/d1a 22 2024-05-25T06:11:29.783 INFO:tasks.workunit.client.1.smithi203.stdout:5/765: chown dc/d11/c25 166 1 2024-05-25T06:11:29.784 INFO:tasks.workunit.client.0.smithi160.stdout:6/58: chown d5/de/ff 1 1 2024-05-25T06:11:29.784 INFO:tasks.workunit.client.0.smithi160.stdout:8/126: mkdir d5/d7/d8/d1f/d1e/d28 0 2024-05-25T06:11:29.785 INFO:tasks.workunit.client.1.smithi203.stdout:7/499: creat d1/d9/d2c/d36/d3b/faf x:0 0 0 2024-05-25T06:11:29.785 INFO:tasks.workunit.client.1.smithi203.stdout:0/387: symlink dd/d5c/l8d 0 2024-05-25T06:11:29.786 INFO:tasks.workunit.client.1.smithi203.stdout:5/766: dread - dc/d11/d17/d1a/d2d/d4d/d64/f108 zero size 2024-05-25T06:11:29.789 INFO:tasks.workunit.client.0.smithi160.stdout:3/28: unlink l2 0 2024-05-25T06:11:29.789 INFO:tasks.workunit.client.0.smithi160.stdout:3/29: readlink l4 0 2024-05-25T06:11:29.789 INFO:tasks.workunit.client.1.smithi203.stdout:7/500: truncate d1/d9/f14 701951 0 2024-05-25T06:11:29.790 INFO:tasks.workunit.client.1.smithi203.stdout:7/501: fdatasync d1/d11/f30 0 2024-05-25T06:11:29.792 INFO:tasks.workunit.client.1.smithi203.stdout:7/502: chown d1/d9/d2c/f42 984450835 1 2024-05-25T06:11:29.792 INFO:tasks.workunit.client.1.smithi203.stdout:7/503: fsync f0 0 2024-05-25T06:11:29.792 INFO:tasks.workunit.client.0.smithi160.stdout:4/156: getdents d4/d1a 0 2024-05-25T06:11:29.793 INFO:tasks.workunit.client.0.smithi160.stdout:6/59: mkdir d5/de/d16/d1b 0 2024-05-25T06:11:29.798 INFO:tasks.workunit.client.1.smithi203.stdout:5/767: dread dc/d11/d17/d1a/d2d/d4d/d64/dd4/fd3 [0,4194304] 0 2024-05-25T06:11:29.801 INFO:tasks.workunit.client.0.smithi160.stdout:4/157: rename d4/d6/fa to d4/f33 0 2024-05-25T06:11:29.801 INFO:tasks.workunit.client.1.smithi203.stdout:5/768: creat dc/d11/d17/d1a/d2d/f10e x:0 0 0 2024-05-25T06:11:29.801 INFO:tasks.workunit.client.0.smithi160.stdout:6/60: link l4 d5/de/l1c 0 2024-05-25T06:11:29.802 INFO:tasks.workunit.client.0.smithi160.stdout:6/61: dread - d5/de/f12 zero size 2024-05-25T06:11:29.802 INFO:tasks.workunit.client.1.smithi203.stdout:5/769: write dc/d11/d17/d1a/d103/d98/fee [655589,106487] 0 2024-05-25T06:11:29.803 INFO:tasks.workunit.client.0.smithi160.stdout:6/62: write f2 [3813661,63274] 0 2024-05-25T06:11:29.804 INFO:tasks.workunit.client.1.smithi203.stdout:5/770: dread - dc/d3c/d3f/fae zero size 2024-05-25T06:11:29.804 INFO:tasks.workunit.client.0.smithi160.stdout:4/158: rmdir d4/d1a 39 2024-05-25T06:11:29.804 INFO:tasks.workunit.client.0.smithi160.stdout:6/63: write d5/de/fd [1218385,88401] 0 2024-05-25T06:11:29.806 INFO:tasks.workunit.client.1.smithi203.stdout:7/504: write d1/d11/d20/f28 [1815187,39505] 0 2024-05-25T06:11:29.809 INFO:tasks.workunit.client.0.smithi160.stdout:4/159: stat d4/d1a 0 2024-05-25T06:11:29.810 INFO:tasks.workunit.client.0.smithi160.stdout:4/160: fdatasync d4/d6/d28/f17 0 2024-05-25T06:11:29.810 INFO:tasks.workunit.client.1.smithi203.stdout:7/505: write d1/d9/d2c/d79/f81 [7736015,87487] 0 2024-05-25T06:11:29.811 INFO:tasks.workunit.client.0.smithi160.stdout:6/64: dwrite d5/de/f17 [0,4194304] 0 2024-05-25T06:11:29.811 INFO:tasks.workunit.client.0.smithi160.stdout:4/161: chown d4/d6/d28 241384 1 2024-05-25T06:11:29.813 INFO:tasks.workunit.client.0.smithi160.stdout:6/65: write d5/fa [1274036,125294] 0 2024-05-25T06:11:29.813 INFO:tasks.workunit.client.1.smithi203.stdout:7/506: readlink d1/l52 0 2024-05-25T06:11:29.813 INFO:tasks.workunit.client.1.smithi203.stdout:0/388: dwrite dd/d3d/d59/f83 [0,4194304] 0 2024-05-25T06:11:29.813 INFO:tasks.workunit.client.0.smithi160.stdout:4/162: write d4/d6/d21/f32 [1256399,7644] 0 2024-05-25T06:11:29.814 INFO:tasks.workunit.client.0.smithi160.stdout:6/66: dread - d5/de/f12 zero size 2024-05-25T06:11:29.819 INFO:tasks.workunit.client.1.smithi203.stdout:5/771: dwrite dc/d3c/d3f/fd8 [0,4194304] 0 2024-05-25T06:11:29.820 INFO:tasks.workunit.client.0.smithi160.stdout:6/67: mknod d5/de/d18/c1d 0 2024-05-25T06:11:29.821 INFO:tasks.workunit.client.0.smithi160.stdout:4/163: dwrite d4/f26 [4194304,4194304] 0 2024-05-25T06:11:29.822 INFO:tasks.workunit.client.1.smithi203.stdout:0/389: unlink dd/d10/f51 0 2024-05-25T06:11:29.823 INFO:tasks.workunit.client.0.smithi160.stdout:4/164: write d4/d6/d28/f25 [1019862,63126] 0 2024-05-25T06:11:29.828 INFO:tasks.workunit.client.1.smithi203.stdout:5/772: fdatasync dc/f80 0 2024-05-25T06:11:29.830 INFO:tasks.workunit.client.0.smithi160.stdout:4/165: chown d4/c12 98461692 1 2024-05-25T06:11:29.834 INFO:tasks.workunit.client.0.smithi160.stdout:6/68: dwrite d5/f7 [0,4194304] 0 2024-05-25T06:11:29.836 INFO:tasks.workunit.client.0.smithi160.stdout:6/69: write d5/f14 [1023501,88189] 0 2024-05-25T06:11:29.837 INFO:tasks.workunit.client.1.smithi203.stdout:7/507: getdents d1/d11/d3d 0 2024-05-25T06:11:29.837 INFO:tasks.workunit.client.0.smithi160.stdout:6/70: chown d5/l13 2 1 2024-05-25T06:11:29.838 INFO:tasks.workunit.client.1.smithi203.stdout:7/508: chown d1/d9/d2c/d36/d3b/c5e 216704 1 2024-05-25T06:11:29.839 INFO:tasks.workunit.client.0.smithi160.stdout:4/166: rename d4/d6/d28/l30 to d4/l34 0 2024-05-25T06:11:29.840 INFO:tasks.workunit.client.1.smithi203.stdout:5/773: rename dc/d11/d17/d45/f46 to dc/d11/d17/d1a/d2d/d53/f10f 0 2024-05-25T06:11:29.841 INFO:tasks.workunit.client.0.smithi160.stdout:6/71: creat d5/f1e x:0 0 0 2024-05-25T06:11:29.842 INFO:tasks.workunit.client.0.smithi160.stdout:6/72: write d5/fa [3334222,30753] 0 2024-05-25T06:11:29.843 INFO:tasks.workunit.client.1.smithi203.stdout:7/509: chown d1/f3 25037 1 2024-05-25T06:11:29.844 INFO:tasks.workunit.client.0.smithi160.stdout:4/167: mkdir d4/d6/d35 0 2024-05-25T06:11:29.844 INFO:tasks.workunit.client.1.smithi203.stdout:7/510: chown d1/d9/d2c/d3a/d4e/f5b 158 1 2024-05-25T06:11:29.846 INFO:tasks.workunit.client.0.smithi160.stdout:6/73: mknod d5/de/c1f 0 2024-05-25T06:11:29.848 INFO:tasks.workunit.client.0.smithi160.stdout:4/168: dread d4/d6/d28/f1e [0,4194304] 0 2024-05-25T06:11:29.849 INFO:tasks.workunit.client.1.smithi203.stdout:7/511: unlink d1/d11/d3d/f6e 0 2024-05-25T06:11:29.851 INFO:tasks.workunit.client.0.smithi160.stdout:4/169: symlink d4/d6/d21/l36 0 2024-05-25T06:11:29.855 INFO:tasks.workunit.client.1.smithi203.stdout:0/390: truncate dd/d3d/d59/f83 3460191 0 2024-05-25T06:11:29.855 INFO:tasks.workunit.client.0.smithi160.stdout:4/170: rename d4/d6/d28/l18 to d4/d6/d21/l37 0 2024-05-25T06:11:29.855 INFO:tasks.workunit.client.0.smithi160.stdout:4/171: stat d4/d1c/c31 0 2024-05-25T06:11:29.857 INFO:tasks.workunit.client.0.smithi160.stdout:4/172: creat d4/d1a/f38 x:0 0 0 2024-05-25T06:11:29.858 INFO:tasks.workunit.client.0.smithi160.stdout:4/173: stat d4/d6/d21/f32 0 2024-05-25T06:11:29.858 INFO:tasks.workunit.client.0.smithi160.stdout:4/174: fsync d4/d6/d21/f27 0 2024-05-25T06:11:29.860 INFO:tasks.workunit.client.0.smithi160.stdout:4/175: read d4/d6/d28/f17 [697353,111765] 0 2024-05-25T06:11:29.860 INFO:tasks.workunit.client.0.smithi160.stdout:4/176: stat d4/d6 0 2024-05-25T06:11:29.861 INFO:tasks.workunit.client.0.smithi160.stdout:4/177: chown d4/d6/fe 3677 1 2024-05-25T06:11:29.861 INFO:tasks.workunit.client.1.smithi203.stdout:5/774: dwrite dc/d11/d17/d44/f8f [0,4194304] 0 2024-05-25T06:11:29.863 INFO:tasks.workunit.client.1.smithi203.stdout:5/775: chown dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdc 32 1 2024-05-25T06:11:29.864 INFO:tasks.workunit.client.0.smithi160.stdout:4/178: dread d4/f26 [0,4194304] 0 2024-05-25T06:11:29.866 INFO:tasks.workunit.client.0.smithi160.stdout:4/179: write d4/d6/f22 [482026,84935] 0 2024-05-25T06:11:29.869 INFO:tasks.workunit.client.1.smithi203.stdout:5/776: mknod dc/d11/d17/d1a/d22/c110 0 2024-05-25T06:11:29.869 INFO:tasks.workunit.client.0.smithi160.stdout:6/74: fsync d5/f7 0 2024-05-25T06:11:29.871 INFO:tasks.workunit.client.1.smithi203.stdout:5/777: symlink dc/d11/d17/d45/l111 0 2024-05-25T06:11:29.871 INFO:tasks.workunit.client.0.smithi160.stdout:6/75: mknod d5/c20 0 2024-05-25T06:11:29.872 INFO:tasks.workunit.client.0.smithi160.stdout:6/76: rename d5/de/d16 to d5/de/d16/d21 22 2024-05-25T06:11:29.873 INFO:tasks.workunit.client.0.smithi160.stdout:2/42: truncate f5 1878541 0 2024-05-25T06:11:29.874 INFO:tasks.workunit.client.0.smithi160.stdout:6/77: creat d5/de/f22 x:0 0 0 2024-05-25T06:11:29.874 INFO:tasks.workunit.client.0.smithi160.stdout:9/39: sync 2024-05-25T06:11:29.875 INFO:tasks.workunit.client.1.smithi203.stdout:5/778: getdents dc/d11/d17/d1a/d2d/d53 0 2024-05-25T06:11:29.875 INFO:tasks.workunit.client.0.smithi160.stdout:2/43: mknod ca 0 2024-05-25T06:11:29.875 INFO:tasks.workunit.client.0.smithi160.stdout:9/40: read d4/f7 [2981126,129131] 0 2024-05-25T06:11:29.876 INFO:tasks.workunit.client.1.smithi203.stdout:5/779: fdatasync dc/fe 0 2024-05-25T06:11:29.878 INFO:tasks.workunit.client.0.smithi160.stdout:6/78: symlink d5/de/l23 0 2024-05-25T06:11:29.878 INFO:tasks.workunit.client.1.smithi203.stdout:5/780: unlink dc/d11/d17/d1a/d22/f86 0 2024-05-25T06:11:29.880 INFO:tasks.workunit.client.0.smithi160.stdout:9/41: dread f2 [0,4194304] 0 2024-05-25T06:11:29.881 INFO:tasks.workunit.client.0.smithi160.stdout:9/42: rmdir d4 39 2024-05-25T06:11:29.885 INFO:tasks.workunit.client.0.smithi160.stdout:9/43: dread f2 [0,4194304] 0 2024-05-25T06:11:29.886 INFO:tasks.workunit.client.0.smithi160.stdout:5/73: dwrite db/dc/f15 [0,4194304] 0 2024-05-25T06:11:29.894 INFO:tasks.workunit.client.0.smithi160.stdout:5/74: dwrite db/dc/f15 [4194304,4194304] 0 2024-05-25T06:11:29.894 INFO:tasks.workunit.client.1.smithi203.stdout:5/781: dread dc/d3c/d3f/f4e [0,4194304] 0 2024-05-25T06:11:29.898 INFO:tasks.workunit.client.1.smithi203.stdout:5/782: mknod dc/d11/d17/d1a/d2d/d53/da2/c112 0 2024-05-25T06:11:29.901 INFO:tasks.workunit.client.1.smithi203.stdout:5/783: symlink dc/d11/d17/d1a/d2d/d53/da2/l113 0 2024-05-25T06:11:29.916 INFO:tasks.workunit.client.1.smithi203.stdout:2/521: sync 2024-05-25T06:11:29.916 INFO:tasks.workunit.client.1.smithi203.stdout:4/471: sync 2024-05-25T06:11:29.916 INFO:tasks.workunit.client.1.smithi203.stdout:3/578: sync 2024-05-25T06:11:29.916 INFO:tasks.workunit.client.1.smithi203.stdout:1/426: sync 2024-05-25T06:11:29.918 INFO:tasks.workunit.client.1.smithi203.stdout:2/522: chown d2/d1a/c2c 857 1 2024-05-25T06:11:29.919 INFO:tasks.workunit.client.1.smithi203.stdout:2/523: chown d2/d4/d10/l15 86609775 1 2024-05-25T06:11:29.920 INFO:tasks.workunit.client.1.smithi203.stdout:1/427: rmdir d14/d57/d7e 39 2024-05-25T06:11:29.920 INFO:tasks.workunit.client.0.smithi160.stdout:9/44: mkdir d4/d9 0 2024-05-25T06:11:29.921 INFO:tasks.workunit.client.1.smithi203.stdout:3/579: rmdir d10/d22/d2a 39 2024-05-25T06:11:29.921 INFO:tasks.workunit.client.1.smithi203.stdout:1/428: chown d14/d23/f42 0 1 2024-05-25T06:11:29.921 INFO:tasks.workunit.client.1.smithi203.stdout:4/472: mknod d0/d8/d1f/d26/d3b/c95 0 2024-05-25T06:11:29.922 INFO:tasks.workunit.client.0.smithi160.stdout:5/75: mkdir db/dc/d13/d19 0 2024-05-25T06:11:29.924 INFO:tasks.workunit.client.0.smithi160.stdout:5/76: fdatasync f7 0 2024-05-25T06:11:29.924 INFO:tasks.workunit.client.1.smithi203.stdout:2/524: creat d2/d42/d4e/d68/faf x:0 0 0 2024-05-25T06:11:29.926 INFO:tasks.workunit.client.1.smithi203.stdout:4/473: creat d0/d8/d3a/d81/f96 x:0 0 0 2024-05-25T06:11:29.928 INFO:tasks.workunit.client.1.smithi203.stdout:1/429: link d14/d81/f8a d14/d57/d29/f9b 0 2024-05-25T06:11:29.928 INFO:tasks.workunit.client.1.smithi203.stdout:2/525: mkdir d2/d1a/db0 0 2024-05-25T06:11:29.929 INFO:tasks.workunit.client.1.smithi203.stdout:5/784: write dc/d11/d17/d1a/d22/f24 [739867,26856] 0 2024-05-25T06:11:29.930 INFO:tasks.workunit.client.1.smithi203.stdout:4/474: creat d0/d8/d1f/d26/d3b/f97 x:0 0 0 2024-05-25T06:11:29.931 INFO:tasks.workunit.client.1.smithi203.stdout:3/580: dwrite d10/d16/f44 [0,4194304] 0 2024-05-25T06:11:29.934 INFO:tasks.workunit.client.0.smithi160.stdout:3/30: rename fa to fb 0 2024-05-25T06:11:29.936 INFO:tasks.workunit.client.1.smithi203.stdout:1/430: mknod d14/d81/c9c 0 2024-05-25T06:11:29.937 INFO:tasks.workunit.client.1.smithi203.stdout:3/581: creat d10/d16/d23/d4a/d88/fc0 x:0 0 0 2024-05-25T06:11:29.937 INFO:tasks.workunit.client.0.smithi160.stdout:3/31: mknod cc 0 2024-05-25T06:11:29.938 INFO:tasks.workunit.client.1.smithi203.stdout:1/431: dread - d14/d57/d1d/d3e/d36/f62 zero size 2024-05-25T06:11:29.939 INFO:tasks.workunit.client.1.smithi203.stdout:1/432: read f10 [5251005,15731] 0 2024-05-25T06:11:29.940 INFO:tasks.workunit.client.0.smithi160.stdout:3/32: read fb [887730,125685] 0 2024-05-25T06:11:29.940 INFO:tasks.workunit.client.1.smithi203.stdout:5/785: fdatasync dc/d11/d17/d45/f52 0 2024-05-25T06:11:29.941 INFO:tasks.workunit.client.0.smithi160.stdout:6/79: getdents d5/de/d16 0 2024-05-25T06:11:29.945 INFO:tasks.workunit.client.1.smithi203.stdout:3/582: unlink d10/d16/d23/d46/c72 0 2024-05-25T06:11:29.947 INFO:tasks.workunit.client.0.smithi160.stdout:8/127: dwrite d5/fe [0,4194304] 0 2024-05-25T06:11:29.947 INFO:tasks.workunit.client.1.smithi203.stdout:4/475: rmdir d0/d8/d1f/d26 39 2024-05-25T06:11:29.949 INFO:tasks.workunit.client.1.smithi203.stdout:3/583: dread - d10/d22/d2a/f75 zero size 2024-05-25T06:11:29.950 INFO:tasks.workunit.client.0.smithi160.stdout:8/128: creat d5/f29 x:0 0 0 2024-05-25T06:11:29.951 INFO:tasks.workunit.client.0.smithi160.stdout:6/80: link d5/de/l23 d5/de/d16/l24 0 2024-05-25T06:11:29.954 INFO:tasks.workunit.client.0.smithi160.stdout:6/81: mknod d5/de/d16/c25 0 2024-05-25T06:11:29.955 INFO:tasks.workunit.client.1.smithi203.stdout:4/476: unlink d0/l2 0 2024-05-25T06:11:29.957 INFO:tasks.workunit.client.1.smithi203.stdout:2/526: truncate d2/d4/d10/d5f/f81 534571 0 2024-05-25T06:11:29.959 INFO:tasks.workunit.client.0.smithi160.stdout:8/129: getdents d5/d7/d8/d1f 0 2024-05-25T06:11:29.960 INFO:tasks.workunit.client.1.smithi203.stdout:2/527: mknod d2/d4/d10/d58/da0/cb1 0 2024-05-25T06:11:29.961 INFO:tasks.workunit.client.1.smithi203.stdout:1/433: dread d14/d37/d5e/f55 [0,4194304] 0 2024-05-25T06:11:29.962 INFO:tasks.workunit.client.0.smithi160.stdout:6/82: link d5/de/f17 d5/f26 0 2024-05-25T06:11:29.962 INFO:tasks.workunit.client.1.smithi203.stdout:2/528: write d2/d38/f8a [3658935,8039] 0 2024-05-25T06:11:29.964 INFO:tasks.workunit.client.1.smithi203.stdout:1/434: read d14/d57/d1d/f40 [3437129,103770] 0 2024-05-25T06:11:29.964 INFO:tasks.workunit.client.1.smithi203.stdout:3/584: write d10/d16/d23/d4a/d88/f95 [1243170,97111] 0 2024-05-25T06:11:29.964 INFO:tasks.workunit.client.1.smithi203.stdout:1/435: chown d14/d23 22 1 2024-05-25T06:11:29.965 INFO:tasks.workunit.client.0.smithi160.stdout:6/83: read d5/de/f17 [1167169,9211] 0 2024-05-25T06:11:29.966 INFO:tasks.workunit.client.1.smithi203.stdout:2/529: mknod d2/d38/d2f/cb2 0 2024-05-25T06:11:29.966 INFO:tasks.workunit.client.0.smithi160.stdout:6/84: chown d5/fb 20 1 2024-05-25T06:11:29.967 INFO:tasks.workunit.client.1.smithi203.stdout:2/530: chown d2/d38/d43/d60/ca5 19 1 2024-05-25T06:11:29.968 INFO:tasks.workunit.client.0.smithi160.stdout:8/130: symlink d5/d7/l2a 0 2024-05-25T06:11:29.969 INFO:tasks.workunit.client.1.smithi203.stdout:2/531: stat d2/d4/d10/d5f 0 2024-05-25T06:11:29.969 INFO:tasks.workunit.client.1.smithi203.stdout:1/436: unlink d14/d57/d1d/d3e/d36/c6a 0 2024-05-25T06:11:29.970 INFO:tasks.workunit.client.0.smithi160.stdout:6/85: symlink d5/de/l27 0 2024-05-25T06:11:29.971 INFO:tasks.workunit.client.0.smithi160.stdout:9/45: getdents d4 0 2024-05-25T06:11:29.971 INFO:tasks.workunit.client.1.smithi203.stdout:3/585: truncate d10/d16/d23/d67/fb7 1001434 0 2024-05-25T06:11:29.972 INFO:tasks.workunit.client.1.smithi203.stdout:1/437: write d14/f32 [22743,38964] 0 2024-05-25T06:11:29.974 INFO:tasks.workunit.client.1.smithi203.stdout:1/438: stat d14/d57/d1d/d3e/d36/c75 0 2024-05-25T06:11:29.974 INFO:tasks.workunit.client.0.smithi160.stdout:8/131: dwrite d5/fe [0,4194304] 0 2024-05-25T06:11:29.975 INFO:tasks.workunit.client.0.smithi160.stdout:9/46: unlink d4/f7 0 2024-05-25T06:11:29.975 INFO:tasks.workunit.client.1.smithi203.stdout:2/532: symlink d2/d4/da6/lb3 0 2024-05-25T06:11:29.976 INFO:tasks.workunit.client.1.smithi203.stdout:2/533: chown d2/d38/d43/d60/da3 0 1 2024-05-25T06:11:29.977 INFO:tasks.workunit.client.1.smithi203.stdout:4/477: dread d0/d9/d11/f2f [0,4194304] 0 2024-05-25T06:11:29.978 INFO:tasks.workunit.client.0.smithi160.stdout:5/77: truncate f6 9478277 0 2024-05-25T06:11:29.980 INFO:tasks.workunit.client.1.smithi203.stdout:2/534: read d2/d38/d2f/f31 [2797047,127562] 0 2024-05-25T06:11:29.980 INFO:tasks.workunit.client.0.smithi160.stdout:9/47: creat d4/fa x:0 0 0 2024-05-25T06:11:29.980 INFO:tasks.workunit.client.0.smithi160.stdout:9/48: readlink l1 0 2024-05-25T06:11:29.980 INFO:tasks.workunit.client.0.smithi160.stdout:6/86: getdents d5/de/d16/d1b 0 2024-05-25T06:11:29.982 INFO:tasks.workunit.client.0.smithi160.stdout:3/33: getdents . 0 2024-05-25T06:11:29.982 INFO:tasks.workunit.client.1.smithi203.stdout:3/586: link d10/d16/l43 d10/lc1 0 2024-05-25T06:11:29.983 INFO:tasks.workunit.client.1.smithi203.stdout:2/535: mknod d2/d1a/d33/cb4 0 2024-05-25T06:11:29.984 INFO:tasks.workunit.client.1.smithi203.stdout:4/478: mkdir d0/d8/d3a/d81/d93/d98 0 2024-05-25T06:11:29.984 INFO:tasks.workunit.client.0.smithi160.stdout:8/132: rmdir d5/d7/d8 39 2024-05-25T06:11:29.985 INFO:tasks.workunit.client.1.smithi203.stdout:4/479: dread - d0/d9/d50/d79/f40 zero size 2024-05-25T06:11:29.986 INFO:tasks.workunit.client.1.smithi203.stdout:3/587: creat d10/d16/d50/fc2 x:0 0 0 2024-05-25T06:11:29.987 INFO:tasks.workunit.client.0.smithi160.stdout:5/78: creat db/f1a x:0 0 0 2024-05-25T06:11:29.989 INFO:tasks.workunit.client.0.smithi160.stdout:9/49: symlink d4/lb 0 2024-05-25T06:11:29.989 INFO:tasks.workunit.client.0.smithi160.stdout:9/50: dread - d4/fa zero size 2024-05-25T06:11:29.990 INFO:tasks.workunit.client.1.smithi203.stdout:4/480: chown d0/d8/d1f/d26/l1d 379 1 2024-05-25T06:11:29.990 INFO:tasks.workunit.client.1.smithi203.stdout:3/588: rename d10/d22/l83 to d10/d16/d71/lc3 0 2024-05-25T06:11:29.991 INFO:tasks.workunit.client.1.smithi203.stdout:3/589: fdatasync f0 0 2024-05-25T06:11:29.991 INFO:tasks.workunit.client.1.smithi203.stdout:1/439: dread d14/d37/f7a [0,4194304] 0 2024-05-25T06:11:29.992 INFO:tasks.workunit.client.0.smithi160.stdout:3/34: symlink ld 0 2024-05-25T06:11:29.993 INFO:tasks.workunit.client.0.smithi160.stdout:9/51: dread d4/f8 [0,4194304] 0 2024-05-25T06:11:29.993 INFO:tasks.workunit.client.1.smithi203.stdout:1/440: mkdir d14/d7c/d9d 0 2024-05-25T06:11:29.994 INFO:tasks.workunit.client.1.smithi203.stdout:4/481: mkdir d0/d8/d3a/d81/d93/d98/d99 0 2024-05-25T06:11:29.994 INFO:tasks.workunit.client.0.smithi160.stdout:8/133: unlink d5/c6 0 2024-05-25T06:11:29.995 INFO:tasks.workunit.client.1.smithi203.stdout:3/590: mknod d10/d16/d23/d46/cc4 0 2024-05-25T06:11:29.995 INFO:tasks.workunit.client.1.smithi203.stdout:1/441: read - d14/d57/d29/f80 zero size 2024-05-25T06:11:29.996 INFO:tasks.workunit.client.1.smithi203.stdout:3/591: readlink d10/d22/lba 0 2024-05-25T06:11:29.996 INFO:tasks.workunit.client.1.smithi203.stdout:4/482: write d0/d9/d11/f66 [410764,70278] 0 2024-05-25T06:11:29.997 INFO:tasks.workunit.client.0.smithi160.stdout:6/87: mkdir d5/de/d16/d1b/d28 0 2024-05-25T06:11:29.998 INFO:tasks.workunit.client.1.smithi203.stdout:1/442: mknod d14/d7c/c9e 0 2024-05-25T06:11:29.999 INFO:tasks.workunit.client.0.smithi160.stdout:3/35: creat fe x:0 0 0 2024-05-25T06:11:29.999 INFO:tasks.workunit.client.1.smithi203.stdout:4/483: fdatasync d0/d9/d50/d6b/f51 0 2024-05-25T06:11:30.000 INFO:tasks.workunit.client.0.smithi160.stdout:3/36: dread - fe zero size 2024-05-25T06:11:30.000 INFO:tasks.workunit.client.1.smithi203.stdout:1/443: fsync d14/d57/d1d/d3e/f9a 0 2024-05-25T06:11:30.000 INFO:tasks.workunit.client.1.smithi203.stdout:3/592: write d10/d16/d23/d67/f6f [446105,124668] 0 2024-05-25T06:11:30.001 INFO:tasks.workunit.client.0.smithi160.stdout:8/134: fsync d5/d9/f1c 0 2024-05-25T06:11:30.002 INFO:tasks.workunit.client.1.smithi203.stdout:3/593: chown d10/f51 317 1 2024-05-25T06:11:30.004 INFO:tasks.workunit.client.0.smithi160.stdout:7/65: sync 2024-05-25T06:11:30.004 INFO:tasks.workunit.client.0.smithi160.stdout:0/35: sync 2024-05-25T06:11:30.004 INFO:tasks.workunit.client.0.smithi160.stdout:1/51: sync 2024-05-25T06:11:30.005 INFO:tasks.workunit.client.0.smithi160.stdout:9/52: creat d4/d9/fc x:0 0 0 2024-05-25T06:11:30.005 INFO:tasks.workunit.client.0.smithi160.stdout:2/44: dwrite f5 [0,4194304] 0 2024-05-25T06:11:30.005 INFO:tasks.workunit.client.1.smithi203.stdout:2/536: dread d2/f8 [0,4194304] 0 2024-05-25T06:11:30.006 INFO:tasks.workunit.client.1.smithi203.stdout:1/444: rename d14/d5f to d14/d57/d1d/d9f 0 2024-05-25T06:11:30.007 INFO:tasks.workunit.client.0.smithi160.stdout:6/88: dread d5/de/f17 [0,4194304] 0 2024-05-25T06:11:30.008 INFO:tasks.workunit.client.0.smithi160.stdout:3/37: creat ff x:0 0 0 2024-05-25T06:11:30.008 INFO:tasks.workunit.client.1.smithi203.stdout:4/484: dread d0/f3 [0,4194304] 0 2024-05-25T06:11:30.008 INFO:tasks.workunit.client.1.smithi203.stdout:3/594: truncate d10/d16/d23/d4a/d88/fc0 591013 0 2024-05-25T06:11:30.008 INFO:tasks.workunit.client.1.smithi203.stdout:2/537: write d2/d4/d10/d5f/fa2 [247198,14957] 0 2024-05-25T06:11:30.009 INFO:tasks.workunit.client.0.smithi160.stdout:2/45: chown f8 2889 1 2024-05-25T06:11:30.009 INFO:tasks.workunit.client.0.smithi160.stdout:6/89: read d5/fa [1775910,9826] 0 2024-05-25T06:11:30.010 INFO:tasks.workunit.client.0.smithi160.stdout:2/46: stat ca 0 2024-05-25T06:11:30.011 INFO:tasks.workunit.client.1.smithi203.stdout:4/485: fdatasync d0/d9/d50/d6b/f30 0 2024-05-25T06:11:30.012 INFO:tasks.workunit.client.1.smithi203.stdout:3/595: fsync d10/d16/d23/d46/f4b 0 2024-05-25T06:11:30.012 INFO:tasks.workunit.client.1.smithi203.stdout:1/445: dread d14/d57/d1d/d3e/f90 [0,4194304] 0 2024-05-25T06:11:30.013 INFO:tasks.workunit.client.1.smithi203.stdout:3/596: chown le 9 1 2024-05-25T06:11:30.014 INFO:tasks.workunit.client.0.smithi160.stdout:1/52: creat fb x:0 0 0 2024-05-25T06:11:30.014 INFO:tasks.workunit.client.0.smithi160.stdout:8/135: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:30.015 INFO:tasks.workunit.client.1.smithi203.stdout:2/538: creat d2/d1a/d33/d80/fb5 x:0 0 0 2024-05-25T06:11:30.015 INFO:tasks.workunit.client.1.smithi203.stdout:4/486: read d0/d8/d3a/f49 [37448,94743] 0 2024-05-25T06:11:30.017 INFO:tasks.workunit.client.0.smithi160.stdout:1/53: stat fb 0 2024-05-25T06:11:30.017 INFO:tasks.workunit.client.0.smithi160.stdout:0/36: dread f2 [0,4194304] 0 2024-05-25T06:11:30.018 INFO:tasks.workunit.client.1.smithi203.stdout:4/487: chown d0/d8/f44 116 1 2024-05-25T06:11:30.020 INFO:tasks.workunit.client.0.smithi160.stdout:9/53: dread d4/f8 [0,4194304] 0 2024-05-25T06:11:30.020 INFO:tasks.workunit.client.0.smithi160.stdout:7/66: dread d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:30.021 INFO:tasks.workunit.client.0.smithi160.stdout:0/37: truncate f5 1361814 0 2024-05-25T06:11:30.022 INFO:tasks.workunit.client.0.smithi160.stdout:3/38: mknod c10 0 2024-05-25T06:11:30.022 INFO:tasks.workunit.client.0.smithi160.stdout:5/79: getdents db/dc 0 2024-05-25T06:11:30.023 INFO:tasks.workunit.client.1.smithi203.stdout:2/539: creat d2/d4/d10/d32/fb6 x:0 0 0 2024-05-25T06:11:30.024 INFO:tasks.workunit.client.1.smithi203.stdout:1/446: dwrite d14/f54 [0,4194304] 0 2024-05-25T06:11:30.025 INFO:tasks.workunit.client.1.smithi203.stdout:3/597: dwrite d10/f51 [0,4194304] 0 2024-05-25T06:11:30.026 INFO:tasks.workunit.client.1.smithi203.stdout:4/488: mkdir d0/d9/d50/d79/d9a 0 2024-05-25T06:11:30.026 INFO:tasks.workunit.client.0.smithi160.stdout:6/90: rename d5/c20 to d5/de/c29 0 2024-05-25T06:11:30.026 INFO:tasks.workunit.client.0.smithi160.stdout:3/39: chown fe 15267 1 2024-05-25T06:11:30.028 INFO:tasks.workunit.client.0.smithi160.stdout:8/136: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:30.031 INFO:tasks.workunit.client.1.smithi203.stdout:3/598: symlink d10/d16/d23/d38/lc5 0 2024-05-25T06:11:30.032 INFO:tasks.workunit.client.0.smithi160.stdout:2/47: link f8 fb 0 2024-05-25T06:11:30.032 INFO:tasks.workunit.client.0.smithi160.stdout:3/40: creat f11 x:0 0 0 2024-05-25T06:11:30.038 INFO:tasks.workunit.client.1.smithi203.stdout:4/489: mknod d0/d9/d50/c9b 0 2024-05-25T06:11:30.039 INFO:tasks.workunit.client.1.smithi203.stdout:4/490: chown d0/d9/l63 1146 1 2024-05-25T06:11:30.041 INFO:tasks.workunit.client.0.smithi160.stdout:6/91: write d5/de/f17 [1956802,10406] 0 2024-05-25T06:11:30.042 INFO:tasks.workunit.client.0.smithi160.stdout:0/38: dread f2 [0,4194304] 0 2024-05-25T06:11:30.043 INFO:tasks.workunit.client.0.smithi160.stdout:2/48: creat fc x:0 0 0 2024-05-25T06:11:30.044 INFO:tasks.workunit.client.0.smithi160.stdout:3/41: symlink l12 0 2024-05-25T06:11:30.044 INFO:tasks.workunit.client.0.smithi160.stdout:5/80: dwrite db/f11 [0,4194304] 0 2024-05-25T06:11:30.044 INFO:tasks.workunit.client.0.smithi160.stdout:7/67: dread d6/dc/ff [0,4194304] 0 2024-05-25T06:11:30.046 INFO:tasks.workunit.client.0.smithi160.stdout:6/92: write d5/de/f22 [894062,111121] 0 2024-05-25T06:11:30.046 INFO:tasks.workunit.client.0.smithi160.stdout:0/39: symlink l7 0 2024-05-25T06:11:30.046 INFO:tasks.workunit.client.1.smithi203.stdout:4/491: dread d0/d9/f64 [0,4194304] 0 2024-05-25T06:11:30.048 INFO:tasks.workunit.client.0.smithi160.stdout:8/137: symlink d5/d7/d8/d1f/d1e/d28/l2b 0 2024-05-25T06:11:30.049 INFO:tasks.workunit.client.0.smithi160.stdout:3/42: creat f13 x:0 0 0 2024-05-25T06:11:30.049 INFO:tasks.workunit.client.1.smithi203.stdout:4/492: mkdir d0/d8/d1f/d26/d3b/d9c 0 2024-05-25T06:11:30.049 INFO:tasks.workunit.client.0.smithi160.stdout:3/43: readlink l12 0 2024-05-25T06:11:30.050 INFO:tasks.workunit.client.0.smithi160.stdout:5/81: symlink db/dc/l1b 0 2024-05-25T06:11:30.051 INFO:tasks.workunit.client.0.smithi160.stdout:7/68: mknod d6/dc/c1b 0 2024-05-25T06:11:30.052 INFO:tasks.workunit.client.0.smithi160.stdout:7/69: dread - d6/f12 zero size 2024-05-25T06:11:30.053 INFO:tasks.workunit.client.0.smithi160.stdout:3/44: mknod c14 0 2024-05-25T06:11:30.054 INFO:tasks.workunit.client.0.smithi160.stdout:8/138: mknod d5/d7/d8/d1f/d1e/d28/c2c 0 2024-05-25T06:11:30.054 INFO:tasks.workunit.client.0.smithi160.stdout:7/70: write d6/f12 [407977,13141] 0 2024-05-25T06:11:30.055 INFO:tasks.workunit.client.1.smithi203.stdout:4/493: mkdir d0/d9/d9d 0 2024-05-25T06:11:30.055 INFO:tasks.workunit.client.0.smithi160.stdout:3/45: read - f8 zero size 2024-05-25T06:11:30.057 INFO:tasks.workunit.client.0.smithi160.stdout:5/82: rename db/dc/d13/d19 to db/d1c 0 2024-05-25T06:11:30.058 INFO:tasks.workunit.client.0.smithi160.stdout:2/49: dwrite fb [0,4194304] 0 2024-05-25T06:11:30.058 INFO:tasks.workunit.client.0.smithi160.stdout:7/71: fdatasync d6/fb 0 2024-05-25T06:11:30.061 INFO:tasks.workunit.client.0.smithi160.stdout:2/50: dread - fc zero size 2024-05-25T06:11:30.061 INFO:tasks.workunit.client.1.smithi203.stdout:4/494: rename d0/d9/d11/f5b to d0/d9/f9e 0 2024-05-25T06:11:30.062 INFO:tasks.workunit.client.1.smithi203.stdout:4/495: chown d0/d8/d1f/d26/l1d 6 1 2024-05-25T06:11:30.066 INFO:tasks.workunit.client.0.smithi160.stdout:6/93: dread d5/fa [0,4194304] 0 2024-05-25T06:11:30.067 INFO:tasks.workunit.client.0.smithi160.stdout:2/51: symlink ld 0 2024-05-25T06:11:30.069 INFO:tasks.workunit.client.0.smithi160.stdout:8/139: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:30.070 INFO:tasks.workunit.client.0.smithi160.stdout:2/52: dread - f9 zero size 2024-05-25T06:11:30.070 INFO:tasks.workunit.client.1.smithi203.stdout:4/496: creat d0/f9f x:0 0 0 2024-05-25T06:11:30.072 INFO:tasks.workunit.client.0.smithi160.stdout:0/40: getdents . 0 2024-05-25T06:11:30.072 INFO:tasks.workunit.client.0.smithi160.stdout:5/83: dread f1 [0,4194304] 0 2024-05-25T06:11:30.072 INFO:tasks.workunit.client.0.smithi160.stdout:6/94: creat d5/de/f2a x:0 0 0 2024-05-25T06:11:30.075 INFO:tasks.workunit.client.0.smithi160.stdout:5/84: write db/f11 [4342417,79050] 0 2024-05-25T06:11:30.078 INFO:tasks.workunit.client.0.smithi160.stdout:8/140: mknod d5/d7/d8/d1f/d1e/d28/c2d 0 2024-05-25T06:11:30.080 INFO:tasks.workunit.client.0.smithi160.stdout:0/41: write f2 [872297,58386] 0 2024-05-25T06:11:30.080 INFO:tasks.workunit.client.0.smithi160.stdout:5/85: mkdir db/dc/d13/d1d 0 2024-05-25T06:11:30.082 INFO:tasks.workunit.client.0.smithi160.stdout:8/141: symlink d5/d9/l2e 0 2024-05-25T06:11:30.085 INFO:tasks.workunit.client.0.smithi160.stdout:0/42: write f2 [409459,38728] 0 2024-05-25T06:11:30.088 INFO:tasks.workunit.client.1.smithi203.stdout:1/447: truncate d14/d57/d6b/f50 146969 0 2024-05-25T06:11:30.090 INFO:tasks.workunit.client.0.smithi160.stdout:2/53: getdents . 0 2024-05-25T06:11:30.090 INFO:tasks.workunit.client.0.smithi160.stdout:5/86: mknod db/dc/d13/c1e 0 2024-05-25T06:11:30.090 INFO:tasks.workunit.client.0.smithi160.stdout:0/43: chown l4 19039 1 2024-05-25T06:11:30.091 INFO:tasks.workunit.client.1.smithi203.stdout:1/448: mknod d14/d57/d1d/d3e/ca0 0 2024-05-25T06:11:30.091 INFO:tasks.workunit.client.0.smithi160.stdout:7/72: dwrite d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:30.092 INFO:tasks.workunit.client.0.smithi160.stdout:0/44: fsync f5 0 2024-05-25T06:11:30.092 INFO:tasks.workunit.client.1.smithi203.stdout:4/497: fsync d0/f9f 0 2024-05-25T06:11:30.092 INFO:tasks.workunit.client.0.smithi160.stdout:6/95: dwrite d5/f11 [0,4194304] 0 2024-05-25T06:11:30.094 INFO:tasks.workunit.client.1.smithi203.stdout:3/599: dwrite d10/d16/d23/d45/f4e [0,4194304] 0 2024-05-25T06:11:30.095 INFO:tasks.workunit.client.1.smithi203.stdout:6/705: sync 2024-05-25T06:11:30.095 INFO:tasks.workunit.client.0.smithi160.stdout:2/54: unlink fc 0 2024-05-25T06:11:30.097 INFO:tasks.workunit.client.1.smithi203.stdout:4/498: write d0/d8/d52/f71 [130426,68166] 0 2024-05-25T06:11:30.099 INFO:tasks.workunit.client.0.smithi160.stdout:0/45: symlink l8 0 2024-05-25T06:11:30.100 INFO:tasks.workunit.client.0.smithi160.stdout:5/87: dread f9 [4194304,4194304] 0 2024-05-25T06:11:30.102 INFO:tasks.workunit.client.0.smithi160.stdout:7/73: symlink d6/l1c 0 2024-05-25T06:11:30.103 INFO:tasks.workunit.client.1.smithi203.stdout:1/449: dread d14/d57/d1d/f67 [0,4194304] 0 2024-05-25T06:11:30.106 INFO:tasks.workunit.client.0.smithi160.stdout:4/180: sync 2024-05-25T06:11:30.106 INFO:tasks.workunit.client.0.smithi160.stdout:2/55: unlink ca 0 2024-05-25T06:11:30.106 INFO:tasks.workunit.client.0.smithi160.stdout:2/56: rmdir - no directory 2024-05-25T06:11:30.108 INFO:tasks.workunit.client.1.smithi203.stdout:1/450: write d14/d37/f85 [924061,83253] 0 2024-05-25T06:11:30.108 INFO:tasks.workunit.client.0.smithi160.stdout:6/96: mkdir d5/de/d16/d1b/d2b 0 2024-05-25T06:11:30.109 INFO:tasks.workunit.client.1.smithi203.stdout:4/499: chown d0/d8/d3a/c87 91609 1 2024-05-25T06:11:30.110 INFO:tasks.workunit.client.0.smithi160.stdout:0/46: mknod c9 0 2024-05-25T06:11:30.110 INFO:tasks.workunit.client.0.smithi160.stdout:0/47: rmdir - no directory 2024-05-25T06:11:30.110 INFO:tasks.workunit.client.0.smithi160.stdout:0/48: rmdir - no directory 2024-05-25T06:11:30.111 INFO:tasks.workunit.client.1.smithi203.stdout:3/600: dwrite d10/d16/d23/fb9 [0,4194304] 0 2024-05-25T06:11:30.111 INFO:tasks.workunit.client.1.smithi203.stdout:6/706: link d8/d11/d15/f37 d8/d11/d1b/d75/fdd 0 2024-05-25T06:11:30.111 INFO:tasks.workunit.client.1.smithi203.stdout:4/500: truncate d0/d9/f61 4810100 0 2024-05-25T06:11:30.112 INFO:tasks.workunit.client.1.smithi203.stdout:3/601: chown d10/fa2 57 1 2024-05-25T06:11:30.112 INFO:tasks.workunit.client.0.smithi160.stdout:5/88: creat db/dc/d13/f1f x:0 0 0 2024-05-25T06:11:30.114 INFO:tasks.workunit.client.0.smithi160.stdout:5/89: read - db/dc/fd zero size 2024-05-25T06:11:30.114 INFO:tasks.workunit.client.1.smithi203.stdout:6/707: stat d8/d11/d15/l33 0 2024-05-25T06:11:30.114 INFO:tasks.workunit.client.0.smithi160.stdout:7/74: mknod d6/c1d 0 2024-05-25T06:11:30.115 INFO:tasks.workunit.client.1.smithi203.stdout:3/602: dread - d10/d16/d23/d67/fae zero size 2024-05-25T06:11:30.116 INFO:tasks.workunit.client.0.smithi160.stdout:5/90: write db/f1a [688836,123012] 0 2024-05-25T06:11:30.117 INFO:tasks.workunit.client.1.smithi203.stdout:1/451: mkdir d14/d57/d6b/da1 0 2024-05-25T06:11:30.118 INFO:tasks.workunit.client.0.smithi160.stdout:0/49: dwrite f5 [0,4194304] 0 2024-05-25T06:11:30.119 INFO:tasks.workunit.client.1.smithi203.stdout:6/708: read d8/d3d/f9a [1522227,2886] 0 2024-05-25T06:11:30.122 INFO:tasks.workunit.client.1.smithi203.stdout:1/452: unlink d14/f99 0 2024-05-25T06:11:30.124 INFO:tasks.workunit.client.1.smithi203.stdout:3/603: rmdir d10/d16/d23/d6c 39 2024-05-25T06:11:30.125 INFO:tasks.workunit.client.0.smithi160.stdout:2/57: creat fe x:0 0 0 2024-05-25T06:11:30.125 INFO:tasks.workunit.client.0.smithi160.stdout:8/142: link d5/d7/c1d d5/d7/d8/d27/c2f 0 2024-05-25T06:11:30.126 INFO:tasks.workunit.client.0.smithi160.stdout:2/58: stat ld 0 2024-05-25T06:11:30.127 INFO:tasks.workunit.client.1.smithi203.stdout:6/709: dwrite d8/d11/d15/d28/d52/d43/f92 [0,4194304] 0 2024-05-25T06:11:30.127 INFO:tasks.workunit.client.0.smithi160.stdout:6/97: creat d5/f2c x:0 0 0 2024-05-25T06:11:30.128 INFO:tasks.workunit.client.1.smithi203.stdout:3/604: chown d10/d22/d2a/d99/ca8 250 1 2024-05-25T06:11:30.128 INFO:tasks.workunit.client.1.smithi203.stdout:1/453: mknod d14/d57/d29/d94/ca2 0 2024-05-25T06:11:30.129 INFO:tasks.workunit.client.0.smithi160.stdout:5/91: rename f5 to db/dc/d13/d1d/f20 0 2024-05-25T06:11:30.130 INFO:tasks.workunit.client.0.smithi160.stdout:8/143: mkdir d5/d30 0 2024-05-25T06:11:30.130 INFO:tasks.workunit.client.0.smithi160.stdout:2/59: write f8 [3517634,20678] 0 2024-05-25T06:11:30.131 INFO:tasks.workunit.client.1.smithi203.stdout:6/710: mkdir d8/d2a/d6c/dde 0 2024-05-25T06:11:30.133 INFO:tasks.workunit.client.1.smithi203.stdout:6/711: chown d8/d2a/d6c/d87 63556706 1 2024-05-25T06:11:30.136 INFO:tasks.workunit.client.0.smithi160.stdout:6/98: write d5/f26 [2993366,41674] 0 2024-05-25T06:11:30.136 INFO:tasks.workunit.client.0.smithi160.stdout:4/181: getdents d4/d1a 0 2024-05-25T06:11:30.136 INFO:tasks.workunit.client.1.smithi203.stdout:4/501: write d0/d8/d3a/f49 [179054,100148] 0 2024-05-25T06:11:30.137 INFO:tasks.workunit.client.0.smithi160.stdout:2/60: chown f9 342 1 2024-05-25T06:11:30.138 INFO:tasks.workunit.client.1.smithi203.stdout:6/712: creat d8/d11/d1b/d75/dcb/fdf x:0 0 0 2024-05-25T06:11:30.139 INFO:tasks.workunit.client.1.smithi203.stdout:3/605: dwrite d10/d16/d23/d4a/d88/f8d [0,4194304] 0 2024-05-25T06:11:30.139 INFO:tasks.workunit.client.0.smithi160.stdout:2/61: readlink l6 0 2024-05-25T06:11:30.140 INFO:tasks.workunit.client.0.smithi160.stdout:0/50: dwrite f5 [0,4194304] 0 2024-05-25T06:11:30.140 INFO:tasks.workunit.client.0.smithi160.stdout:6/99: write d5/f2c [672133,80565] 0 2024-05-25T06:11:30.144 INFO:tasks.workunit.client.1.smithi203.stdout:3/606: dread - d10/d16/d50/f87 zero size 2024-05-25T06:11:30.146 INFO:tasks.workunit.client.0.smithi160.stdout:7/75: dwrite d6/dc/ff [0,4194304] 0 2024-05-25T06:11:30.150 INFO:tasks.workunit.client.0.smithi160.stdout:4/182: dread d4/d6/d28/f25 [0,4194304] 0 2024-05-25T06:11:30.150 INFO:tasks.workunit.client.1.smithi203.stdout:1/454: dwrite d14/d57/d1d/f67 [0,4194304] 0 2024-05-25T06:11:30.152 INFO:tasks.workunit.client.1.smithi203.stdout:3/607: chown d10/d16/d23/d4a/f58 721650 1 2024-05-25T06:11:30.152 INFO:tasks.workunit.client.1.smithi203.stdout:6/713: creat d8/fe0 x:0 0 0 2024-05-25T06:11:30.154 INFO:tasks.workunit.client.0.smithi160.stdout:4/183: chown d4/f2a 0 1 2024-05-25T06:11:30.154 INFO:tasks.workunit.client.0.smithi160.stdout:7/76: write d6/dc/f18 [208557,126478] 0 2024-05-25T06:11:30.154 INFO:tasks.workunit.client.0.smithi160.stdout:0/51: creat fa x:0 0 0 2024-05-25T06:11:30.156 INFO:tasks.workunit.client.0.smithi160.stdout:7/77: chown d6/dc 170076287 1 2024-05-25T06:11:30.157 INFO:tasks.workunit.client.0.smithi160.stdout:2/62: dread fb [0,4194304] 0 2024-05-25T06:11:30.157 INFO:tasks.workunit.client.0.smithi160.stdout:7/78: chown d6/dc/l13 14 1 2024-05-25T06:11:30.159 INFO:tasks.workunit.client.1.smithi203.stdout:4/502: dwrite d0/d8/d3a/d81/f96 [0,4194304] 0 2024-05-25T06:11:30.160 INFO:tasks.workunit.client.1.smithi203.stdout:3/608: mkdir d10/d16/d23/d3a/dc6 0 2024-05-25T06:11:30.161 INFO:tasks.workunit.client.1.smithi203.stdout:3/609: stat d10/d22/f5b 0 2024-05-25T06:11:30.163 INFO:tasks.workunit.client.1.smithi203.stdout:3/610: stat d10/d22/d2a/d99 0 2024-05-25T06:11:30.164 INFO:tasks.workunit.client.0.smithi160.stdout:0/52: dread f2 [0,4194304] 0 2024-05-25T06:11:30.164 INFO:tasks.workunit.client.0.smithi160.stdout:2/63: creat ff x:0 0 0 2024-05-25T06:11:30.164 INFO:tasks.workunit.client.1.smithi203.stdout:6/714: mknod d8/d2a/d6c/dc1/dd8/ce1 0 2024-05-25T06:11:30.165 INFO:tasks.workunit.client.0.smithi160.stdout:2/64: stat fe 0 2024-05-25T06:11:30.165 INFO:tasks.workunit.client.1.smithi203.stdout:1/455: dwrite d14/d57/f58 [0,4194304] 0 2024-05-25T06:11:30.166 INFO:tasks.workunit.client.1.smithi203.stdout:4/503: unlink d0/d8/d1f/c39 0 2024-05-25T06:11:30.167 INFO:tasks.workunit.client.0.smithi160.stdout:7/79: creat d6/dc/f1e x:0 0 0 2024-05-25T06:11:30.168 INFO:tasks.workunit.client.1.smithi203.stdout:1/456: fdatasync d14/d57/d1d/d3e/d36/f59 0 2024-05-25T06:11:30.171 INFO:tasks.workunit.client.0.smithi160.stdout:0/53: dread f2 [0,4194304] 0 2024-05-25T06:11:30.171 INFO:tasks.workunit.client.1.smithi203.stdout:4/504: symlink d0/d9/d11/la0 0 2024-05-25T06:11:30.171 INFO:tasks.workunit.client.1.smithi203.stdout:1/457: dread - d14/d57/d1d/d3e/d36/f8d zero size 2024-05-25T06:11:30.173 INFO:tasks.workunit.client.0.smithi160.stdout:2/65: link ff f10 0 2024-05-25T06:11:30.173 INFO:tasks.workunit.client.1.smithi203.stdout:3/611: creat d10/d16/d50/d8b/db3/fc7 x:0 0 0 2024-05-25T06:11:30.174 INFO:tasks.workunit.client.0.smithi160.stdout:7/80: unlink d6/l1a 0 2024-05-25T06:11:30.175 INFO:tasks.workunit.client.0.smithi160.stdout:0/54: creat fb x:0 0 0 2024-05-25T06:11:30.176 INFO:tasks.workunit.client.1.smithi203.stdout:6/715: getdents d8/d2a/d6c 0 2024-05-25T06:11:30.176 INFO:tasks.workunit.client.0.smithi160.stdout:5/92: truncate db/f11 3765555 0 2024-05-25T06:11:30.177 INFO:tasks.workunit.client.1.smithi203.stdout:4/505: creat d0/d8/d1f/d26/d43/fa1 x:0 0 0 2024-05-25T06:11:30.178 INFO:tasks.workunit.client.0.smithi160.stdout:2/66: creat f11 x:0 0 0 2024-05-25T06:11:30.178 INFO:tasks.workunit.client.1.smithi203.stdout:3/612: creat d10/d16/d71/fc8 x:0 0 0 2024-05-25T06:11:30.179 INFO:tasks.workunit.client.0.smithi160.stdout:7/81: unlink d6/dc/f18 0 2024-05-25T06:11:30.179 INFO:tasks.workunit.client.0.smithi160.stdout:0/55: mknod cc 0 2024-05-25T06:11:30.180 INFO:tasks.workunit.client.1.smithi203.stdout:1/458: link d14/d37/f7a d14/d57/d97/fa3 0 2024-05-25T06:11:30.180 INFO:tasks.workunit.client.0.smithi160.stdout:5/93: creat db/dc/d13/f21 x:0 0 0 2024-05-25T06:11:30.180 INFO:tasks.workunit.client.1.smithi203.stdout:1/459: fsync d14/f1a 0 2024-05-25T06:11:30.181 INFO:tasks.workunit.client.1.smithi203.stdout:1/460: stat d14/d57/c60 0 2024-05-25T06:11:30.181 INFO:tasks.workunit.client.1.smithi203.stdout:4/506: read d0/f16 [33478,105268] 0 2024-05-25T06:11:30.183 INFO:tasks.workunit.client.0.smithi160.stdout:0/56: link cc cd 0 2024-05-25T06:11:30.185 INFO:tasks.workunit.client.1.smithi203.stdout:3/613: creat d10/d16/fc9 x:0 0 0 2024-05-25T06:11:30.186 INFO:tasks.workunit.client.1.smithi203.stdout:1/461: dread d14/d57/f58 [0,4194304] 0 2024-05-25T06:11:30.187 INFO:tasks.workunit.client.1.smithi203.stdout:3/614: write d10/d16/d23/d46/fbb [341610,62057] 0 2024-05-25T06:11:30.187 INFO:tasks.workunit.client.1.smithi203.stdout:4/507: unlink d0/d9/l2c 0 2024-05-25T06:11:30.188 INFO:tasks.workunit.client.0.smithi160.stdout:8/144: truncate d5/fe 474084 0 2024-05-25T06:11:30.189 INFO:tasks.workunit.client.1.smithi203.stdout:6/716: getdents d8/d2a 0 2024-05-25T06:11:30.189 INFO:tasks.workunit.client.0.smithi160.stdout:7/82: dwrite d6/dc/f17 [0,4194304] 0 2024-05-25T06:11:30.190 INFO:tasks.workunit.client.1.smithi203.stdout:6/717: read - d8/d11/d15/fbc zero size 2024-05-25T06:11:30.191 INFO:tasks.workunit.client.1.smithi203.stdout:6/718: dread - d8/d11/d1b/d75/dcb/fcd zero size 2024-05-25T06:11:30.192 INFO:tasks.workunit.client.0.smithi160.stdout:7/83: write d6/dc/f16 [539944,50423] 0 2024-05-25T06:11:30.192 INFO:tasks.workunit.client.0.smithi160.stdout:0/57: unlink cd 0 2024-05-25T06:11:30.193 INFO:tasks.workunit.client.0.smithi160.stdout:5/94: dwrite f1 [0,4194304] 0 2024-05-25T06:11:30.196 INFO:tasks.workunit.client.0.smithi160.stdout:7/84: dread - d6/dc/f19 zero size 2024-05-25T06:11:30.200 INFO:tasks.workunit.client.1.smithi203.stdout:4/508: unlink d0/d9/d11/l85 0 2024-05-25T06:11:30.200 INFO:tasks.workunit.client.0.smithi160.stdout:0/58: mknod ce 0 2024-05-25T06:11:30.202 INFO:tasks.workunit.client.0.smithi160.stdout:4/184: truncate d4/d6/fe 508163 0 2024-05-25T06:11:30.204 INFO:tasks.workunit.client.1.smithi203.stdout:1/462: dwrite d14/d57/d97/fa3 [0,4194304] 0 2024-05-25T06:11:30.205 INFO:tasks.workunit.client.0.smithi160.stdout:5/95: creat db/f22 x:0 0 0 2024-05-25T06:11:30.205 INFO:tasks.workunit.client.0.smithi160.stdout:8/145: rename d5/d7/d8/d27/c2f to d5/d7/d8/d1f/d1e/c31 0 2024-05-25T06:11:30.206 INFO:tasks.workunit.client.1.smithi203.stdout:6/719: dwrite d8/d11/d15/fbc [0,4194304] 0 2024-05-25T06:11:30.207 INFO:tasks.workunit.client.0.smithi160.stdout:6/100: dwrite d5/fa [0,4194304] 0 2024-05-25T06:11:30.208 INFO:tasks.workunit.client.0.smithi160.stdout:8/146: chown d5/d9/f26 1891299 1 2024-05-25T06:11:30.209 INFO:tasks.workunit.client.0.smithi160.stdout:0/59: symlink lf 0 2024-05-25T06:11:30.209 INFO:tasks.workunit.client.0.smithi160.stdout:8/147: write d5/d7/d8/d1f/f22 [294242,23435] 0 2024-05-25T06:11:30.214 INFO:tasks.workunit.client.1.smithi203.stdout:1/463: unlink d14/d7c/f91 0 2024-05-25T06:11:30.215 INFO:tasks.workunit.client.0.smithi160.stdout:7/85: symlink d6/dc/l1f 0 2024-05-25T06:11:30.215 INFO:tasks.workunit.client.1.smithi203.stdout:6/720: creat d8/d11/d1b/d75/dcb/dcf/d46/d88/da5/fe2 x:0 0 0 2024-05-25T06:11:30.216 INFO:tasks.workunit.client.1.smithi203.stdout:1/464: chown d14/d57/d29/f9b 1772 1 2024-05-25T06:11:30.216 INFO:tasks.workunit.client.1.smithi203.stdout:4/509: mkdir d0/d8/d10/d91/da2 0 2024-05-25T06:11:30.217 INFO:tasks.workunit.client.0.smithi160.stdout:4/185: dwrite d4/d6/d21/f32 [0,4194304] 0 2024-05-25T06:11:30.219 INFO:tasks.workunit.client.0.smithi160.stdout:6/101: dwrite d5/f14 [0,4194304] 0 2024-05-25T06:11:30.220 INFO:tasks.workunit.client.0.smithi160.stdout:4/186: chown d4/f33 248658 1 2024-05-25T06:11:30.221 INFO:tasks.workunit.client.0.smithi160.stdout:6/102: read d5/f7 [692872,88627] 0 2024-05-25T06:11:30.221 INFO:tasks.workunit.client.1.smithi203.stdout:4/510: chown d0/d9/d11/d54/l82 3892 1 2024-05-25T06:11:30.222 INFO:tasks.workunit.client.1.smithi203.stdout:4/511: chown d0/d9/d50/d6b/f88 30421690 1 2024-05-25T06:11:30.225 INFO:tasks.workunit.client.0.smithi160.stdout:9/54: sync 2024-05-25T06:11:30.225 INFO:tasks.workunit.client.0.smithi160.stdout:6/103: chown d5/de/d18/c1d 13 1 2024-05-25T06:11:30.225 INFO:tasks.workunit.client.0.smithi160.stdout:2/67: getdents . 0 2024-05-25T06:11:30.227 INFO:tasks.workunit.client.0.smithi160.stdout:1/54: sync 2024-05-25T06:11:30.228 INFO:tasks.workunit.client.1.smithi203.stdout:3/615: truncate d10/d16/d23/d45/f9b 791069 0 2024-05-25T06:11:30.230 INFO:tasks.workunit.client.0.smithi160.stdout:4/187: dread d4/d6/d28/f25 [0,4194304] 0 2024-05-25T06:11:30.233 INFO:tasks.workunit.client.0.smithi160.stdout:0/60: creat f10 x:0 0 0 2024-05-25T06:11:30.235 INFO:tasks.workunit.client.0.smithi160.stdout:8/148: mknod d5/c32 0 2024-05-25T06:11:30.236 INFO:tasks.workunit.client.1.smithi203.stdout:4/512: dwrite d0/d8/d3a/f56 [0,4194304] 0 2024-05-25T06:11:30.236 INFO:tasks.workunit.client.1.smithi203.stdout:6/721: getdents d8/d11/d15/d28/d52/d43/d73 0 2024-05-25T06:11:30.237 INFO:tasks.workunit.client.0.smithi160.stdout:4/188: chown d4/d6/l1f 1297536 1 2024-05-25T06:11:30.240 INFO:tasks.workunit.client.0.smithi160.stdout:9/55: dread d4/f8 [0,4194304] 0 2024-05-25T06:11:30.240 INFO:tasks.workunit.client.0.smithi160.stdout:8/149: read d5/d7/d8/d1f/f22 [59294,110035] 0 2024-05-25T06:11:30.241 INFO:tasks.workunit.client.1.smithi203.stdout:3/616: dwrite d10/d16/d23/d3a/f53 [0,4194304] 0 2024-05-25T06:11:30.241 INFO:tasks.workunit.client.0.smithi160.stdout:4/189: readlink d4/d6/d21/l36 0 2024-05-25T06:11:30.241 INFO:tasks.workunit.client.0.smithi160.stdout:5/96: truncate f7 2483499 0 2024-05-25T06:11:30.243 INFO:tasks.workunit.client.0.smithi160.stdout:6/104: chown d5/de/d16/l24 12606 1 2024-05-25T06:11:30.243 INFO:tasks.workunit.client.0.smithi160.stdout:2/68: getdents . 0 2024-05-25T06:11:30.244 INFO:tasks.workunit.client.0.smithi160.stdout:9/56: readlink d4/lb 0 2024-05-25T06:11:30.244 INFO:tasks.workunit.client.1.smithi203.stdout:4/513: mknod d0/d8/d10/d91/da2/ca3 0 2024-05-25T06:11:30.245 INFO:tasks.workunit.client.0.smithi160.stdout:5/97: write f8 [828370,61191] 0 2024-05-25T06:11:30.245 INFO:tasks.workunit.client.1.smithi203.stdout:3/617: mkdir d10/d16/d23/d4a/d88/dca 0 2024-05-25T06:11:30.246 INFO:tasks.workunit.client.0.smithi160.stdout:2/69: truncate f10 324051 0 2024-05-25T06:11:30.246 INFO:tasks.workunit.client.0.smithi160.stdout:0/61: dwrite f10 [0,4194304] 0 2024-05-25T06:11:30.246 INFO:tasks.workunit.client.1.smithi203.stdout:3/618: fdatasync f0 0 2024-05-25T06:11:30.247 INFO:tasks.workunit.client.1.smithi203.stdout:3/619: dread - d10/d16/d23/d46/f4b zero size 2024-05-25T06:11:30.248 INFO:tasks.workunit.client.0.smithi160.stdout:2/70: fsync fe 0 2024-05-25T06:11:30.248 INFO:tasks.workunit.client.0.smithi160.stdout:0/62: write f5 [4344620,65633] 0 2024-05-25T06:11:30.248 INFO:tasks.workunit.client.0.smithi160.stdout:1/55: dwrite fa [0,4194304] 0 2024-05-25T06:11:30.249 INFO:tasks.workunit.client.1.smithi203.stdout:4/514: fsync d0/d9/f9e 0 2024-05-25T06:11:30.250 INFO:tasks.workunit.client.0.smithi160.stdout:0/63: stat f2 0 2024-05-25T06:11:30.251 INFO:tasks.workunit.client.1.smithi203.stdout:4/515: readlink d0/d8/d3a/l67 0 2024-05-25T06:11:30.251 INFO:tasks.workunit.client.0.smithi160.stdout:2/71: chown ff 11429 1 2024-05-25T06:11:30.252 INFO:tasks.workunit.client.1.smithi203.stdout:3/620: mknod d10/d16/d50/d8b/db3/ccb 0 2024-05-25T06:11:30.255 INFO:tasks.workunit.client.0.smithi160.stdout:4/190: symlink d4/d6/d28/l39 0 2024-05-25T06:11:30.256 INFO:tasks.workunit.client.1.smithi203.stdout:1/465: fdatasync d14/d37/f7a 0 2024-05-25T06:11:30.257 INFO:tasks.workunit.client.0.smithi160.stdout:6/105: dwrite d5/de/ff [0,4194304] 0 2024-05-25T06:11:30.261 INFO:tasks.workunit.client.1.smithi203.stdout:4/516: link d0/f16 d0/d9/fa4 0 2024-05-25T06:11:30.264 INFO:tasks.workunit.client.1.smithi203.stdout:4/517: write d0/d8/d3a/f6d [564568,7978] 0 2024-05-25T06:11:30.273 INFO:tasks.workunit.client.1.smithi203.stdout:3/621: dread d10/d22/f86 [0,4194304] 0 2024-05-25T06:11:30.275 INFO:tasks.workunit.client.1.smithi203.stdout:4/518: dwrite d0/d8/d52/f5e [0,4194304] 0 2024-05-25T06:11:30.276 INFO:tasks.workunit.client.1.smithi203.stdout:3/622: write d10/d16/d23/d67/f94 [2060986,75006] 0 2024-05-25T06:11:30.280 INFO:tasks.workunit.client.0.smithi160.stdout:3/46: sync 2024-05-25T06:11:30.282 INFO:tasks.workunit.client.1.smithi203.stdout:4/519: chown d0/d9/d50/d79/c69 93811 1 2024-05-25T06:11:30.283 INFO:tasks.workunit.client.0.smithi160.stdout:7/86: rmdir d6/dc 39 2024-05-25T06:11:30.283 INFO:tasks.workunit.client.0.smithi160.stdout:7/87: fsync f1 0 2024-05-25T06:11:30.285 INFO:tasks.workunit.client.1.smithi203.stdout:3/623: write d10/d16/d23/d67/f6f [1464544,42626] 0 2024-05-25T06:11:30.286 INFO:tasks.workunit.client.1.smithi203.stdout:1/466: getdents d14/d57/d1d 0 2024-05-25T06:11:30.287 INFO:tasks.workunit.client.1.smithi203.stdout:3/624: dread - d10/d22/d2a/d99/fb6 zero size 2024-05-25T06:11:30.287 INFO:tasks.workunit.client.1.smithi203.stdout:6/722: dwrite d8/d5a/d8b/f96 [0,4194304] 0 2024-05-25T06:11:30.288 INFO:tasks.workunit.client.0.smithi160.stdout:0/64: mknod c11 0 2024-05-25T06:11:30.288 INFO:tasks.workunit.client.0.smithi160.stdout:0/65: chown fb 17789 1 2024-05-25T06:11:30.289 INFO:tasks.workunit.client.0.smithi160.stdout:2/72: fsync f8 0 2024-05-25T06:11:30.289 INFO:tasks.workunit.client.0.smithi160.stdout:2/73: chown f7 2852 1 2024-05-25T06:11:30.295 INFO:tasks.workunit.client.0.smithi160.stdout:4/191: symlink d4/d6/l3a 0 2024-05-25T06:11:30.296 INFO:tasks.workunit.client.1.smithi203.stdout:1/467: mkdir d14/d57/d97/da4 0 2024-05-25T06:11:30.297 INFO:tasks.workunit.client.1.smithi203.stdout:1/468: readlink d14/d57/l88 0 2024-05-25T06:11:30.297 INFO:tasks.workunit.client.1.smithi203.stdout:1/469: stat d14/d57/d97 0 2024-05-25T06:11:30.298 INFO:tasks.workunit.client.0.smithi160.stdout:2/74: dwrite f9 [0,4194304] 0 2024-05-25T06:11:30.298 INFO:tasks.workunit.client.1.smithi203.stdout:1/470: write d14/d57/d1d/f66 [1682957,111985] 0 2024-05-25T06:11:30.299 INFO:tasks.workunit.client.1.smithi203.stdout:1/471: stat d14/c1f 0 2024-05-25T06:11:30.299 INFO:tasks.workunit.client.1.smithi203.stdout:1/472: dread - d14/d57/d1d/d3e/f72 zero size 2024-05-25T06:11:30.299 INFO:tasks.workunit.client.0.smithi160.stdout:2/75: dread - fe zero size 2024-05-25T06:11:30.300 INFO:tasks.workunit.client.1.smithi203.stdout:1/473: dread - d14/d57/d6b/f8e zero size 2024-05-25T06:11:30.300 INFO:tasks.workunit.client.1.smithi203.stdout:6/723: unlink d8/d11/d1b/d75/dcb/dcf/cb2 0 2024-05-25T06:11:30.301 INFO:tasks.workunit.client.0.smithi160.stdout:2/76: write f7 [1920693,11466] 0 2024-05-25T06:11:30.301 INFO:tasks.workunit.client.1.smithi203.stdout:4/520: symlink d0/d8/d1f/d26/d3b/d9c/la5 0 2024-05-25T06:11:30.302 INFO:tasks.workunit.client.1.smithi203.stdout:3/625: creat d10/d16/d23/d3a/dc6/fcc x:0 0 0 2024-05-25T06:11:30.303 INFO:tasks.workunit.client.0.smithi160.stdout:2/77: chown f7 16588 1 2024-05-25T06:11:30.303 INFO:tasks.workunit.client.0.smithi160.stdout:2/78: rmdir - no directory 2024-05-25T06:11:30.304 INFO:tasks.workunit.client.0.smithi160.stdout:9/57: link f2 d4/fd 0 2024-05-25T06:11:30.305 INFO:tasks.workunit.client.0.smithi160.stdout:9/58: truncate d4/d9/fc 607841 0 2024-05-25T06:11:30.305 INFO:tasks.workunit.client.0.smithi160.stdout:9/59: chown d4 42 1 2024-05-25T06:11:30.307 INFO:tasks.workunit.client.1.smithi203.stdout:4/521: symlink d0/d8/d3a/d81/d93/la6 0 2024-05-25T06:11:30.308 INFO:tasks.workunit.client.1.smithi203.stdout:3/626: creat d10/d16/d23/d45/fcd x:0 0 0 2024-05-25T06:11:30.309 INFO:tasks.workunit.client.1.smithi203.stdout:8/614: sync 2024-05-25T06:11:30.309 INFO:tasks.workunit.client.0.smithi160.stdout:3/47: write fb [1584864,120488] 0 2024-05-25T06:11:30.310 INFO:tasks.workunit.client.0.smithi160.stdout:8/150: write d5/fe [506211,80479] 0 2024-05-25T06:11:30.310 INFO:tasks.workunit.client.0.smithi160.stdout:3/48: dread - f11 zero size 2024-05-25T06:11:30.311 INFO:tasks.workunit.client.1.smithi203.stdout:1/474: dread d14/d57/f58 [0,4194304] 0 2024-05-25T06:11:30.311 INFO:tasks.workunit.client.0.smithi160.stdout:3/49: truncate fe 236388 0 2024-05-25T06:11:30.312 INFO:tasks.workunit.client.0.smithi160.stdout:0/66: creat f12 x:0 0 0 2024-05-25T06:11:30.312 INFO:tasks.workunit.client.0.smithi160.stdout:3/50: readlink l12 0 2024-05-25T06:11:30.312 INFO:tasks.workunit.client.0.smithi160.stdout:1/56: write f2 [1318810,93905] 0 2024-05-25T06:11:30.315 INFO:tasks.workunit.client.1.smithi203.stdout:6/724: dwrite d8/d5a/d9b/fd4 [0,4194304] 0 2024-05-25T06:11:30.316 INFO:tasks.workunit.client.0.smithi160.stdout:1/57: write fa [2255881,100716] 0 2024-05-25T06:11:30.316 INFO:tasks.workunit.client.0.smithi160.stdout:2/79: mkdir d12 0 2024-05-25T06:11:30.318 INFO:tasks.workunit.client.1.smithi203.stdout:8/615: mkdir d2/d5/d10/d45/d5d/d76/dc7 0 2024-05-25T06:11:30.319 INFO:tasks.workunit.client.0.smithi160.stdout:4/192: fsync d4/f33 0 2024-05-25T06:11:30.319 INFO:tasks.workunit.client.0.smithi160.stdout:9/60: creat d4/d9/fe x:0 0 0 2024-05-25T06:11:30.320 INFO:tasks.workunit.client.1.smithi203.stdout:1/475: mknod d14/ca5 0 2024-05-25T06:11:30.320 INFO:tasks.workunit.client.1.smithi203.stdout:6/725: dread - d8/d11/d1b/d75/dcb/dcf/d46/d88/da5/fc7 zero size 2024-05-25T06:11:30.320 INFO:tasks.workunit.client.1.smithi203.stdout:1/476: chown d14 96550 1 2024-05-25T06:11:30.322 INFO:tasks.workunit.client.1.smithi203.stdout:8/616: write d2/d5/d10/d45/d5d/d68/f87 [1468969,120267] 0 2024-05-25T06:11:30.322 INFO:tasks.workunit.client.1.smithi203.stdout:4/522: link d0/d8/d3a/f77 d0/d8/d3a/d81/d93/d98/d99/fa7 0 2024-05-25T06:11:30.323 INFO:tasks.workunit.client.1.smithi203.stdout:6/726: dread - d8/d11/d15/d28/d52/d43/d73/fb5 zero size 2024-05-25T06:11:30.324 INFO:tasks.workunit.client.1.smithi203.stdout:1/477: readlink d14/l27 0 2024-05-25T06:11:30.324 INFO:tasks.workunit.client.1.smithi203.stdout:8/617: write d2/d5/d10/d11/de/f1a [79520,44072] 0 2024-05-25T06:11:30.325 INFO:tasks.workunit.client.0.smithi160.stdout:8/151: mkdir d5/d9/d33 0 2024-05-25T06:11:30.328 INFO:tasks.workunit.client.1.smithi203.stdout:6/727: mknod d8/d2a/d6c/ce3 0 2024-05-25T06:11:30.329 INFO:tasks.workunit.client.0.smithi160.stdout:5/98: getdents db/dc/d13/d1d 0 2024-05-25T06:11:30.330 INFO:tasks.workunit.client.1.smithi203.stdout:1/478: mkdir d14/d57/d1d/d9f/da6 0 2024-05-25T06:11:30.330 INFO:tasks.workunit.client.1.smithi203.stdout:1/479: readlink d14/d57/d1d/d9f/l70 0 2024-05-25T06:11:30.331 INFO:tasks.workunit.client.1.smithi203.stdout:1/480: chown d14/d57/d1d/d3e/d36 144050 1 2024-05-25T06:11:30.331 INFO:tasks.workunit.client.1.smithi203.stdout:8/618: dread d2/d5/d10/d11/d59/f63 [0,4194304] 0 2024-05-25T06:11:30.332 INFO:tasks.workunit.client.1.smithi203.stdout:1/481: readlink d14/d57/d1d/l48 0 2024-05-25T06:11:30.332 INFO:tasks.workunit.client.1.smithi203.stdout:4/523: rename d0/d9/f64 to d0/fa8 0 2024-05-25T06:11:30.332 INFO:tasks.workunit.client.0.smithi160.stdout:1/58: mknod cc 0 2024-05-25T06:11:30.332 INFO:tasks.workunit.client.1.smithi203.stdout:8/619: stat d2/ca9 0 2024-05-25T06:11:30.334 INFO:tasks.workunit.client.1.smithi203.stdout:6/728: unlink f1 0 2024-05-25T06:11:30.335 INFO:tasks.workunit.client.0.smithi160.stdout:4/193: readlink d4/d1a/l20 0 2024-05-25T06:11:30.336 INFO:tasks.workunit.client.0.smithi160.stdout:9/61: mkdir d4/d9/df 0 2024-05-25T06:11:30.336 INFO:tasks.workunit.client.1.smithi203.stdout:4/524: chown d0/d8/d1f/c21 254 1 2024-05-25T06:11:30.337 INFO:tasks.workunit.client.0.smithi160.stdout:6/106: truncate d5/de/ff 2640235 0 2024-05-25T06:11:30.337 INFO:tasks.workunit.client.0.smithi160.stdout:3/51: dwrite f11 [0,4194304] 0 2024-05-25T06:11:30.339 INFO:tasks.workunit.client.0.smithi160.stdout:9/62: read - d4/fa zero size 2024-05-25T06:11:30.339 INFO:tasks.workunit.client.0.smithi160.stdout:4/194: chown d4/d6/l1f 7 1 2024-05-25T06:11:30.339 INFO:tasks.workunit.client.0.smithi160.stdout:0/67: dwrite f10 [0,4194304] 0 2024-05-25T06:11:30.340 INFO:tasks.workunit.client.1.smithi203.stdout:6/729: read d8/d3d/f9a [1708935,95406] 0 2024-05-25T06:11:30.341 INFO:tasks.workunit.client.0.smithi160.stdout:5/99: unlink db/dc/fd 0 2024-05-25T06:11:30.342 INFO:tasks.workunit.client.0.smithi160.stdout:7/88: link d6/l7 d6/l20 0 2024-05-25T06:11:30.342 INFO:tasks.workunit.client.0.smithi160.stdout:4/195: dread - d4/d1a/f38 zero size 2024-05-25T06:11:30.343 INFO:tasks.workunit.client.0.smithi160.stdout:2/80: mkdir d12/d13 0 2024-05-25T06:11:30.344 INFO:tasks.workunit.client.1.smithi203.stdout:6/730: dread d8/d11/d1b/d75/dcb/dcf/f4b [4194304,4194304] 0 2024-05-25T06:11:30.344 INFO:tasks.workunit.client.0.smithi160.stdout:5/100: chown db/dc/d13/d1d 403337 1 2024-05-25T06:11:30.345 INFO:tasks.workunit.client.0.smithi160.stdout:8/152: creat d5/d30/f34 x:0 0 0 2024-05-25T06:11:30.345 INFO:tasks.workunit.client.1.smithi203.stdout:4/525: mkdir d0/d8/d10/da9 0 2024-05-25T06:11:30.345 INFO:tasks.workunit.client.0.smithi160.stdout:2/81: chown f10 137002 1 2024-05-25T06:11:30.347 INFO:tasks.workunit.client.1.smithi203.stdout:4/526: write d0/f5 [3404598,52341] 0 2024-05-25T06:11:30.349 INFO:tasks.workunit.client.1.smithi203.stdout:4/527: fsync d0/d9/f9e 0 2024-05-25T06:11:30.352 INFO:tasks.workunit.client.1.smithi203.stdout:6/731: write d8/d11/d1b/d75/dcb/dcf/d46/fd2 [10993377,12545] 0 2024-05-25T06:11:30.354 INFO:tasks.workunit.client.1.smithi203.stdout:3/627: dread d10/d16/d23/d46/faf [0,4194304] 0 2024-05-25T06:11:30.356 INFO:tasks.workunit.client.1.smithi203.stdout:8/620: dread d2/f14 [4194304,4194304] 0 2024-05-25T06:11:30.356 INFO:tasks.workunit.client.0.smithi160.stdout:3/52: creat f15 x:0 0 0 2024-05-25T06:11:30.357 INFO:tasks.workunit.client.1.smithi203.stdout:1/482: link d14/d57/d29/f9b d14/d57/d7e/fa7 0 2024-05-25T06:11:30.357 INFO:tasks.workunit.client.0.smithi160.stdout:0/68: dwrite f5 [0,4194304] 0 2024-05-25T06:11:30.358 INFO:tasks.workunit.client.0.smithi160.stdout:2/82: dread fb [0,4194304] 0 2024-05-25T06:11:30.359 INFO:tasks.workunit.client.1.smithi203.stdout:1/483: stat d14/d57/d1d/f30 0 2024-05-25T06:11:30.359 INFO:tasks.workunit.client.0.smithi160.stdout:6/107: dread d5/f26 [0,4194304] 0 2024-05-25T06:11:30.359 INFO:tasks.workunit.client.0.smithi160.stdout:3/53: dread - f8 zero size 2024-05-25T06:11:30.359 INFO:tasks.workunit.client.0.smithi160.stdout:5/101: rmdir db/dc/d13 39 2024-05-25T06:11:30.359 INFO:tasks.workunit.client.1.smithi203.stdout:4/528: rmdir d0/d8/d52/d59/d7a/d83 39 2024-05-25T06:11:30.360 INFO:tasks.workunit.client.0.smithi160.stdout:6/108: stat d5/f14 0 2024-05-25T06:11:30.360 INFO:tasks.workunit.client.0.smithi160.stdout:3/54: fsync f13 0 2024-05-25T06:11:30.361 INFO:tasks.workunit.client.0.smithi160.stdout:6/109: dread - d5/de/f12 zero size 2024-05-25T06:11:30.363 INFO:tasks.workunit.client.0.smithi160.stdout:8/153: creat d5/f35 x:0 0 0 2024-05-25T06:11:30.363 INFO:tasks.workunit.client.0.smithi160.stdout:2/83: symlink d12/l14 0 2024-05-25T06:11:30.367 INFO:tasks.workunit.client.1.smithi203.stdout:1/484: mkdir d14/d57/d1d/da8 0 2024-05-25T06:11:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:7/89: mkdir d6/dc/d21 0 2024-05-25T06:11:30.370 INFO:tasks.workunit.client.0.smithi160.stdout:0/69: rename cc to c13 0 2024-05-25T06:11:30.371 INFO:tasks.workunit.client.1.smithi203.stdout:3/628: dwrite f1 [4194304,4194304] 0 2024-05-25T06:11:30.372 INFO:tasks.workunit.client.0.smithi160.stdout:2/84: fsync f8 0 2024-05-25T06:11:30.373 INFO:tasks.workunit.client.0.smithi160.stdout:5/102: mkdir db/dc/d13/d1d/d23 0 2024-05-25T06:11:30.373 INFO:tasks.workunit.client.0.smithi160.stdout:3/55: dread f11 [0,4194304] 0 2024-05-25T06:11:30.374 INFO:tasks.workunit.client.0.smithi160.stdout:9/63: write d4/fd [4870808,75934] 0 2024-05-25T06:11:30.375 INFO:tasks.workunit.client.0.smithi160.stdout:5/103: fsync fa 0 2024-05-25T06:11:30.375 INFO:tasks.workunit.client.0.smithi160.stdout:4/196: link d4/d6/d21/f27 d4/f3b 0 2024-05-25T06:11:30.376 INFO:tasks.workunit.client.1.smithi203.stdout:6/732: link d8/d11/d47/fbf d8/d5a/d8b/fe4 0 2024-05-25T06:11:30.376 INFO:tasks.workunit.client.0.smithi160.stdout:8/154: dwrite d5/f29 [0,4194304] 0 2024-05-25T06:11:30.378 INFO:tasks.workunit.client.1.smithi203.stdout:3/629: creat d10/d16/d23/d46/fce x:0 0 0 2024-05-25T06:11:30.379 INFO:tasks.workunit.client.1.smithi203.stdout:8/621: creat d2/d5/d10/d11/fc8 x:0 0 0 2024-05-25T06:11:30.382 INFO:tasks.workunit.client.1.smithi203.stdout:3/630: write d10/fb5 [960696,12441] 0 2024-05-25T06:11:30.383 INFO:tasks.workunit.client.1.smithi203.stdout:6/733: mknod d8/ce5 0 2024-05-25T06:11:30.385 INFO:tasks.workunit.client.0.smithi160.stdout:4/197: fdatasync d4/f3b 0 2024-05-25T06:11:30.386 INFO:tasks.workunit.client.1.smithi203.stdout:8/622: creat d2/d5/d10/fc9 x:0 0 0 2024-05-25T06:11:30.387 INFO:tasks.workunit.client.0.smithi160.stdout:8/155: creat d5/f36 x:0 0 0 2024-05-25T06:11:30.388 INFO:tasks.workunit.client.1.smithi203.stdout:4/529: getdents d0/d8/d1f/d26/d3b/d9c 0 2024-05-25T06:11:30.391 INFO:tasks.workunit.client.0.smithi160.stdout:7/90: symlink d6/l22 0 2024-05-25T06:11:30.391 INFO:tasks.workunit.client.0.smithi160.stdout:2/85: dwrite f7 [0,4194304] 0 2024-05-25T06:11:30.391 INFO:tasks.workunit.client.0.smithi160.stdout:3/56: dread fb [0,4194304] 0 2024-05-25T06:11:30.392 INFO:tasks.workunit.client.0.smithi160.stdout:7/91: readlink d6/le 0 2024-05-25T06:11:30.395 INFO:tasks.workunit.client.0.smithi160.stdout:9/64: dwrite d4/f8 [0,4194304] 0 2024-05-25T06:11:30.395 INFO:tasks.workunit.client.1.smithi203.stdout:8/623: symlink d2/d5/d10/d45/lca 0 2024-05-25T06:11:30.396 INFO:tasks.workunit.client.0.smithi160.stdout:8/156: unlink d5/f36 0 2024-05-25T06:11:30.397 INFO:tasks.workunit.client.1.smithi203.stdout:8/624: dread - d2/d5/d28/d38/fa6 zero size 2024-05-25T06:11:30.397 INFO:tasks.workunit.client.1.smithi203.stdout:4/530: dwrite d0/d9/f9e [0,4194304] 0 2024-05-25T06:11:30.398 INFO:tasks.workunit.client.0.smithi160.stdout:3/57: stat c7 0 2024-05-25T06:11:30.398 INFO:tasks.workunit.client.0.smithi160.stdout:5/104: mkdir db/dc/d13/d1d/d23/d24 0 2024-05-25T06:11:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:30 smithi203 ceph-mon[36030]: pgmap v15: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 128 MiB/s wr, 328 op/s 2024-05-25T06:11:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:30 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:30 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:30.401 INFO:tasks.workunit.client.1.smithi203.stdout:6/734: dread d8/d70/f94 [0,4194304] 0 2024-05-25T06:11:30.401 INFO:tasks.workunit.client.1.smithi203.stdout:8/625: chown d2/d5/d28/d38/d4e/d97/c55 25310852 1 2024-05-25T06:11:30.401 INFO:tasks.workunit.client.1.smithi203.stdout:6/735: fsync d8/d2a/f67 0 2024-05-25T06:11:30.403 INFO:tasks.workunit.client.1.smithi203.stdout:4/531: symlink d0/d8/d1f/d26/d43/laa 0 2024-05-25T06:11:30.404 INFO:tasks.workunit.client.0.smithi160.stdout:6/110: getdents d5/de/d16 0 2024-05-25T06:11:30.405 INFO:tasks.workunit.client.1.smithi203.stdout:4/532: dread - d0/d9/d50/d6b/f8a zero size 2024-05-25T06:11:30.405 INFO:tasks.workunit.client.0.smithi160.stdout:0/70: getdents . 0 2024-05-25T06:11:30.407 INFO:tasks.workunit.client.0.smithi160.stdout:4/198: link d4/d6/d28/f17 d4/d6/d35/f3c 0 2024-05-25T06:11:30.408 INFO:tasks.workunit.client.0.smithi160.stdout:2/86: creat d12/d13/f15 x:0 0 0 2024-05-25T06:11:30.409 INFO:tasks.workunit.client.0.smithi160.stdout:6/111: truncate d5/f2c 1288813 0 2024-05-25T06:11:30.411 INFO:tasks.workunit.client.1.smithi203.stdout:4/533: chown d0/d8/d1f/d26/c37 7201 1 2024-05-25T06:11:30.414 INFO:tasks.workunit.client.0.smithi160.stdout:5/105: mknod db/dc/d13/c25 0 2024-05-25T06:11:30.414 INFO:tasks.workunit.client.0.smithi160.stdout:2/87: dread - d12/d13/f15 zero size 2024-05-25T06:11:30.415 INFO:tasks.workunit.client.0.smithi160.stdout:8/157: dwrite d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:30.418 INFO:tasks.workunit.client.0.smithi160.stdout:2/88: readlink d12/l14 0 2024-05-25T06:11:30.420 INFO:tasks.workunit.client.1.smithi203.stdout:6/736: getdents d8/d11/d1b/d75/dcb/dcf/d80 0 2024-05-25T06:11:30.422 INFO:tasks.workunit.client.0.smithi160.stdout:2/89: chown f8 1757483 1 2024-05-25T06:11:30.422 INFO:tasks.workunit.client.0.smithi160.stdout:5/106: creat db/f26 x:0 0 0 2024-05-25T06:11:30.423 INFO:tasks.workunit.client.1.smithi203.stdout:4/534: dwrite d0/d9/f17 [0,4194304] 0 2024-05-25T06:11:30.423 INFO:tasks.workunit.client.0.smithi160.stdout:8/158: creat d5/d7/d8/f37 x:0 0 0 2024-05-25T06:11:30.424 INFO:tasks.workunit.client.1.smithi203.stdout:4/535: write d0/d8/d3a/f49 [4027739,98477] 0 2024-05-25T06:11:30.425 INFO:tasks.workunit.client.0.smithi160.stdout:3/58: getdents . 0 2024-05-25T06:11:30.426 INFO:tasks.workunit.client.1.smithi203.stdout:6/737: chown d8/d11/d15/l86 3286 1 2024-05-25T06:11:30.428 INFO:tasks.workunit.client.1.smithi203.stdout:4/536: unlink d0/d9/f61 0 2024-05-25T06:11:30.429 INFO:tasks.workunit.client.0.smithi160.stdout:2/90: symlink d12/l16 0 2024-05-25T06:11:30.429 INFO:tasks.workunit.client.0.smithi160.stdout:0/71: dwrite f12 [0,4194304] 0 2024-05-25T06:11:30.430 INFO:tasks.workunit.client.1.smithi203.stdout:4/537: chown d0/d9/d50/c9b 6724 1 2024-05-25T06:11:30.430 INFO:tasks.workunit.client.0.smithi160.stdout:2/91: rename d12 to d12/d17 22 2024-05-25T06:11:30.434 INFO:tasks.workunit.client.1.smithi203.stdout:4/538: mknod d0/d8/d3a/d81/d93/d98/cab 0 2024-05-25T06:11:30.434 INFO:tasks.workunit.client.1.smithi203.stdout:3/631: write d10/d16/d23/d45/f9b [1280343,21823] 0 2024-05-25T06:11:30.435 INFO:tasks.workunit.client.0.smithi160.stdout:8/159: unlink d5/f29 0 2024-05-25T06:11:30.435 INFO:tasks.workunit.client.0.smithi160.stdout:6/112: dwrite d5/de/f22 [0,4194304] 0 2024-05-25T06:11:30.435 INFO:tasks.workunit.client.0.smithi160.stdout:8/160: read - d5/f35 zero size 2024-05-25T06:11:30.435 INFO:tasks.workunit.client.1.smithi203.stdout:4/539: fdatasync d0/d8/f23 0 2024-05-25T06:11:30.437 INFO:tasks.workunit.client.1.smithi203.stdout:3/632: write d10/f1a [4522275,128130] 0 2024-05-25T06:11:30.438 INFO:tasks.workunit.client.0.smithi160.stdout:3/59: creat f16 x:0 0 0 2024-05-25T06:11:30.438 INFO:tasks.workunit.client.0.smithi160.stdout:7/92: truncate d6/dc/f15 1758191 0 2024-05-25T06:11:30.438 INFO:tasks.workunit.client.0.smithi160.stdout:0/72: mknod c14 0 2024-05-25T06:11:30.439 INFO:tasks.workunit.client.1.smithi203.stdout:3/633: stat d10/d16/d23/f6e 0 2024-05-25T06:11:30.439 INFO:tasks.workunit.client.0.smithi160.stdout:5/107: dread f7 [0,4194304] 0 2024-05-25T06:11:30.440 INFO:tasks.workunit.client.0.smithi160.stdout:2/92: mknod d12/d13/c18 0 2024-05-25T06:11:30.440 INFO:tasks.workunit.client.0.smithi160.stdout:0/73: write f12 [3299204,25097] 0 2024-05-25T06:11:30.442 INFO:tasks.workunit.client.0.smithi160.stdout:6/113: rmdir d5 39 2024-05-25T06:11:30.442 INFO:tasks.workunit.client.0.smithi160.stdout:2/93: dread - fe zero size 2024-05-25T06:11:30.443 INFO:tasks.workunit.client.0.smithi160.stdout:4/199: getdents d4/d6/d21 0 2024-05-25T06:11:30.444 INFO:tasks.workunit.client.0.smithi160.stdout:5/108: write db/dc/d13/f21 [243246,106402] 0 2024-05-25T06:11:30.444 INFO:tasks.workunit.client.0.smithi160.stdout:3/60: truncate fe 305993 0 2024-05-25T06:11:30.445 INFO:tasks.workunit.client.1.smithi203.stdout:4/540: dwrite d0/d9/d50/d6b/f88 [0,4194304] 0 2024-05-25T06:11:30.446 INFO:tasks.workunit.client.0.smithi160.stdout:8/161: rename d5/d9/f1c to d5/d7/d8/d1f/d1e/d28/f38 0 2024-05-25T06:11:30.446 INFO:tasks.workunit.client.1.smithi203.stdout:3/634: read d10/d16/d23/d46/faf [1139791,110900] 0 2024-05-25T06:11:30.447 INFO:tasks.workunit.client.0.smithi160.stdout:2/94: dread ff [0,4194304] 0 2024-05-25T06:11:30.447 INFO:tasks.workunit.client.1.smithi203.stdout:6/738: dread d8/d70/d38/fab [0,4194304] 0 2024-05-25T06:11:30.450 INFO:tasks.workunit.client.1.smithi203.stdout:3/635: symlink d10/d22/lcf 0 2024-05-25T06:11:30.450 INFO:tasks.workunit.client.0.smithi160.stdout:5/109: dread f4 [0,4194304] 0 2024-05-25T06:11:30.450 INFO:tasks.workunit.client.1.smithi203.stdout:4/541: write d0/d8/d3a/f46 [1350888,24541] 0 2024-05-25T06:11:30.452 INFO:tasks.workunit.client.0.smithi160.stdout:0/74: rename fb to f15 0 2024-05-25T06:11:30.456 INFO:tasks.workunit.client.1.smithi203.stdout:6/739: getdents d8/d5a/d9b/daf 0 2024-05-25T06:11:30.456 INFO:tasks.workunit.client.0.smithi160.stdout:3/61: mknod c17 0 2024-05-25T06:11:30.458 INFO:tasks.workunit.client.0.smithi160.stdout:7/93: truncate f1 964895 0 2024-05-25T06:11:30.459 INFO:tasks.workunit.client.0.smithi160.stdout:5/110: dread db/dc/fe [0,4194304] 0 2024-05-25T06:11:30.459 INFO:tasks.workunit.client.0.smithi160.stdout:6/114: dread d5/f7 [0,4194304] 0 2024-05-25T06:11:30.461 INFO:tasks.workunit.client.0.smithi160.stdout:5/111: chown db/lf 954244 1 2024-05-25T06:11:30.461 INFO:tasks.workunit.client.0.smithi160.stdout:3/62: truncate ff 233172 0 2024-05-25T06:11:30.462 INFO:tasks.workunit.client.1.smithi203.stdout:6/740: write d8/d11/d1b/d75/fda [993121,113109] 0 2024-05-25T06:11:30.463 INFO:tasks.workunit.client.1.smithi203.stdout:3/636: dwrite d10/d16/d23/d3a/f80 [0,4194304] 0 2024-05-25T06:11:30.464 INFO:tasks.workunit.client.0.smithi160.stdout:8/162: mkdir d5/d39 0 2024-05-25T06:11:30.464 INFO:tasks.workunit.client.0.smithi160.stdout:2/95: symlink d12/l19 0 2024-05-25T06:11:30.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/75: dwrite f5 [4194304,4194304] 0 2024-05-25T06:11:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:30 smithi160 ceph-mon[29041]: pgmap v15: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 128 MiB/s wr, 328 op/s 2024-05-25T06:11:30.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:30 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:30.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:30 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:30.471 INFO:tasks.workunit.client.1.smithi203.stdout:4/542: dwrite d0/d8/d52/f6f [0,4194304] 0 2024-05-25T06:11:30.475 INFO:tasks.workunit.client.1.smithi203.stdout:6/741: link d8/l16 d8/d2a/da9/le6 0 2024-05-25T06:11:30.477 INFO:tasks.workunit.client.1.smithi203.stdout:3/637: rename d10/d16/d71/l6b to d10/d16/d23/d4a/d88/dca/ld0 0 2024-05-25T06:11:30.477 INFO:tasks.workunit.client.0.smithi160.stdout:8/163: dwrite d5/f35 [0,4194304] 0 2024-05-25T06:11:30.478 INFO:tasks.workunit.client.0.smithi160.stdout:6/115: creat d5/de/d16/d1b/f2d x:0 0 0 2024-05-25T06:11:30.480 INFO:tasks.workunit.client.0.smithi160.stdout:3/63: unlink l12 0 2024-05-25T06:11:30.481 INFO:tasks.workunit.client.0.smithi160.stdout:3/64: chown cc 253 1 2024-05-25T06:11:30.481 INFO:tasks.workunit.client.0.smithi160.stdout:3/65: chown ld 0 1 2024-05-25T06:11:30.482 INFO:tasks.workunit.client.1.smithi203.stdout:3/638: readlink l3 0 2024-05-25T06:11:30.483 INFO:tasks.workunit.client.0.smithi160.stdout:2/96: mknod d12/c1a 0 2024-05-25T06:11:30.484 INFO:tasks.workunit.client.1.smithi203.stdout:6/742: creat d8/d5a/d9b/dcc/fe7 x:0 0 0 2024-05-25T06:11:30.486 INFO:tasks.workunit.client.1.smithi203.stdout:4/543: link d0/d8/f23 d0/d8/d1f/d26/d3b/fac 0 2024-05-25T06:11:30.486 INFO:tasks.workunit.client.0.smithi160.stdout:0/76: creat f16 x:0 0 0 2024-05-25T06:11:30.488 INFO:tasks.workunit.client.1.smithi203.stdout:4/544: fdatasync d0/d8/d10/f7f 0 2024-05-25T06:11:30.489 INFO:tasks.workunit.client.0.smithi160.stdout:2/97: dwrite f8 [0,4194304] 0 2024-05-25T06:11:30.490 INFO:tasks.workunit.client.0.smithi160.stdout:7/94: symlink d6/l23 0 2024-05-25T06:11:30.490 INFO:tasks.workunit.client.1.smithi203.stdout:3/639: link d10/d16/d23/d38/f3e d10/d16/d23/d4a/fd1 0 2024-05-25T06:11:30.491 INFO:tasks.workunit.client.0.smithi160.stdout:7/95: readlink d6/l1c 0 2024-05-25T06:11:30.493 INFO:tasks.workunit.client.1.smithi203.stdout:4/545: rename d0/d8/d1f/d26/d43/f68 to d0/d8/d3a/d81/d93/fad 0 2024-05-25T06:11:30.494 INFO:tasks.workunit.client.1.smithi203.stdout:6/743: getdents d8/d11 0 2024-05-25T06:11:30.496 INFO:tasks.workunit.client.1.smithi203.stdout:3/640: dread d10/d16/d23/d4a/f58 [0,4194304] 0 2024-05-25T06:11:30.498 INFO:tasks.workunit.client.1.smithi203.stdout:3/641: write d10/d16/d23/f2b [2188030,45922] 0 2024-05-25T06:11:30.498 INFO:tasks.workunit.client.1.smithi203.stdout:6/744: mknod d8/d5a/d9b/ce8 0 2024-05-25T06:11:30.498 INFO:tasks.workunit.client.0.smithi160.stdout:3/66: unlink c7 0 2024-05-25T06:11:30.499 INFO:tasks.workunit.client.1.smithi203.stdout:4/546: rename d0/d9/d50/d79/c69 to d0/d8/d1f/d26/d3b/d9c/cae 0 2024-05-25T06:11:30.502 INFO:tasks.workunit.client.1.smithi203.stdout:6/745: creat d8/d2a/d63/fe9 x:0 0 0 2024-05-25T06:11:30.502 INFO:tasks.workunit.client.0.smithi160.stdout:0/77: mknod c17 0 2024-05-25T06:11:30.503 INFO:tasks.workunit.client.0.smithi160.stdout:0/78: chown f5 7768011 1 2024-05-25T06:11:30.503 INFO:tasks.workunit.client.0.smithi160.stdout:0/79: dread - f15 zero size 2024-05-25T06:11:30.505 INFO:tasks.workunit.client.1.smithi203.stdout:6/746: creat d8/d11/d15/d28/d52/d43/fea x:0 0 0 2024-05-25T06:11:30.505 INFO:tasks.workunit.client.0.smithi160.stdout:6/116: mknod d5/de/d16/d1b/d28/c2e 0 2024-05-25T06:11:30.507 INFO:tasks.workunit.client.1.smithi203.stdout:6/747: read d8/f12 [2610743,46362] 0 2024-05-25T06:11:30.508 INFO:tasks.workunit.client.0.smithi160.stdout:8/164: link d5/d7/d8/f16 d5/d30/f3a 0 2024-05-25T06:11:30.509 INFO:tasks.workunit.client.1.smithi203.stdout:6/748: rmdir d8/d2a 39 2024-05-25T06:11:30.510 INFO:tasks.workunit.client.0.smithi160.stdout:3/67: creat f18 x:0 0 0 2024-05-25T06:11:30.510 INFO:tasks.workunit.client.1.smithi203.stdout:3/642: dwrite d10/d16/d23/d38/f3e [0,4194304] 0 2024-05-25T06:11:30.511 INFO:tasks.workunit.client.1.smithi203.stdout:3/643: chown d10/d16/d23/d46/faf 1 1 2024-05-25T06:11:30.511 INFO:tasks.workunit.client.0.smithi160.stdout:4/200: dwrite d4/d6/fe [0,4194304] 0 2024-05-25T06:11:30.512 INFO:tasks.workunit.client.1.smithi203.stdout:3/644: write d10/d16/f19 [3287198,25480] 0 2024-05-25T06:11:30.516 INFO:tasks.workunit.client.1.smithi203.stdout:6/749: creat d8/d2a/d6c/feb x:0 0 0 2024-05-25T06:11:30.517 INFO:tasks.workunit.client.1.smithi203.stdout:3/645: mknod d10/d16/d23/d67/db8/cd2 0 2024-05-25T06:11:30.517 INFO:tasks.workunit.client.0.smithi160.stdout:3/68: dwrite f15 [0,4194304] 0 2024-05-25T06:11:30.518 INFO:tasks.workunit.client.1.smithi203.stdout:3/646: dread - d10/d16/d50/d7d/f9d zero size 2024-05-25T06:11:30.520 INFO:tasks.workunit.client.1.smithi203.stdout:4/547: truncate d0/d8/d3a/f53 2110899 0 2024-05-25T06:11:30.522 INFO:tasks.workunit.client.1.smithi203.stdout:4/548: chown d0/d8/d1f/d26/d3b/f8c 11 1 2024-05-25T06:11:30.528 INFO:tasks.workunit.client.1.smithi203.stdout:4/549: creat d0/d9/d50/faf x:0 0 0 2024-05-25T06:11:30.530 INFO:tasks.workunit.client.1.smithi203.stdout:6/750: dwrite d8/d11/d47/fb8 [0,4194304] 0 2024-05-25T06:11:30.531 INFO:tasks.workunit.client.1.smithi203.stdout:3/647: dwrite d10/f1a [0,4194304] 0 2024-05-25T06:11:30.532 INFO:tasks.workunit.client.1.smithi203.stdout:4/550: symlink d0/d8/d1f/d26/lb0 0 2024-05-25T06:11:30.533 INFO:tasks.workunit.client.1.smithi203.stdout:3/648: mkdir d10/d16/d23/d46/dd3 0 2024-05-25T06:11:30.535 INFO:tasks.workunit.client.1.smithi203.stdout:6/751: rename d8/d11/d47/d60/f55 to d8/d11/d15/d28/d52/d43/fec 0 2024-05-25T06:11:30.536 INFO:tasks.workunit.client.1.smithi203.stdout:4/551: symlink d0/d8/d10/lb1 0 2024-05-25T06:11:30.538 INFO:tasks.workunit.client.1.smithi203.stdout:6/752: creat d8/d5a/d9b/fed x:0 0 0 2024-05-25T06:11:30.539 INFO:tasks.workunit.client.1.smithi203.stdout:4/552: creat d0/d8/d3a/d81/fb2 x:0 0 0 2024-05-25T06:11:30.543 INFO:tasks.workunit.client.1.smithi203.stdout:6/753: read d8/d11/d15/d28/d52/d43/f92 [445767,92691] 0 2024-05-25T06:11:30.543 INFO:tasks.workunit.client.1.smithi203.stdout:3/649: dwrite d10/d16/d23/d45/f4e [0,4194304] 0 2024-05-25T06:11:30.546 INFO:tasks.workunit.client.1.smithi203.stdout:3/650: truncate d10/d16/d50/f84 4726587 0 2024-05-25T06:11:30.549 INFO:tasks.workunit.client.1.smithi203.stdout:3/651: write d10/d22/f35 [2874803,19020] 0 2024-05-25T06:11:30.549 INFO:tasks.workunit.client.1.smithi203.stdout:4/553: rename d0/d9/d11/f2f to d0/d8/d1f/d26/d3b/d9c/fb3 0 2024-05-25T06:11:30.554 INFO:tasks.workunit.client.1.smithi203.stdout:4/554: dread d0/d8/d52/f5e [0,4194304] 0 2024-05-25T06:11:30.568 INFO:tasks.workunit.client.1.smithi203.stdout:4/555: rename d0/d9/l63 to d0/d9/d11/d54/lb4 0 2024-05-25T06:11:30.568 INFO:tasks.workunit.client.1.smithi203.stdout:6/754: link d8/d70/f94 d8/d11/d47/fee 0 2024-05-25T06:11:30.569 INFO:tasks.workunit.client.1.smithi203.stdout:4/556: write d0/d9/f9e [4330146,34058] 0 2024-05-25T06:11:30.573 INFO:tasks.workunit.client.1.smithi203.stdout:4/557: rename d0/d9/d50/c9b to d0/d8/d3a/d81/d93/cb5 0 2024-05-25T06:11:30.603 INFO:tasks.workunit.client.1.smithi203.stdout:9/474: sync 2024-05-25T06:11:30.605 INFO:tasks.workunit.client.1.smithi203.stdout:9/475: chown d0/d57/d8f 134 1 2024-05-25T06:11:30.605 INFO:tasks.workunit.client.1.smithi203.stdout:4/558: mkdir d0/d8/d3a/db6 0 2024-05-25T06:11:30.606 INFO:tasks.workunit.client.1.smithi203.stdout:6/755: dwrite d8/d11/d1b/d75/dcb/dcf/d46/d88/da5/fe2 [0,4194304] 0 2024-05-25T06:11:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:0/80: mknod c18 0 2024-05-25T06:11:30.608 INFO:tasks.workunit.client.1.smithi203.stdout:4/559: truncate d0/d8/d1f/f2b 8700108 0 2024-05-25T06:11:30.609 INFO:tasks.workunit.client.1.smithi203.stdout:9/476: mknod d0/d17/d20/d3c/d5c/ca2 0 2024-05-25T06:11:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:0/81: write fa [912347,6946] 0 2024-05-25T06:11:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:0/82: stat l4 0 2024-05-25T06:11:30.609 INFO:tasks.workunit.client.1.smithi203.stdout:4/560: dread - d0/d9/f94 zero size 2024-05-25T06:11:30.610 INFO:tasks.workunit.client.1.smithi203.stdout:6/756: symlink d8/d2a/d6c/lef 0 2024-05-25T06:11:30.612 INFO:tasks.workunit.client.1.smithi203.stdout:9/477: fsync d0/d57/d8f/f9e 0 2024-05-25T06:11:30.612 INFO:tasks.workunit.client.1.smithi203.stdout:4/561: chown d0/d8/d1f/d26/f8e 2613 1 2024-05-25T06:11:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:1/59: sync 2024-05-25T06:11:30.615 INFO:tasks.workunit.client.1.smithi203.stdout:4/562: fsync d0/d8/d10/f5c 0 2024-05-25T06:11:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:8/165: fsync d5/d7/d8/d1f/d1e/d28/f38 0 2024-05-25T06:11:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:5/112: write db/f11 [2903916,383] 0 2024-05-25T06:11:30.616 INFO:tasks.workunit.client.1.smithi203.stdout:9/478: symlink d0/d17/d20/d6e/d75/d78/la3 0 2024-05-25T06:11:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:8/166: readlink d5/d9/l2e 0 2024-05-25T06:11:30.618 INFO:tasks.workunit.client.1.smithi203.stdout:9/479: write d0/d17/d23/f90 [2510067,85451] 0 2024-05-25T06:11:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:3/69: mkdir d19 0 2024-05-25T06:11:30.622 INFO:tasks.workunit.client.1.smithi203.stdout:4/563: link d0/d8/d1f/d26/d3b/f7b d0/d8/d1f/fb7 0 2024-05-25T06:11:30.623 INFO:tasks.workunit.client.1.smithi203.stdout:9/480: symlink d0/d17/d20/d6e/d75/la4 0 2024-05-25T06:11:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:1/60: mknod cd 0 2024-05-25T06:11:30.625 INFO:tasks.workunit.client.1.smithi203.stdout:4/564: creat d0/d9/d50/d79/fb8 x:0 0 0 2024-05-25T06:11:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:4/201: mkdir d4/d6/d35/d3d 0 2024-05-25T06:11:30.626 INFO:tasks.workunit.client.1.smithi203.stdout:3/652: write d10/d16/d23/d4a/f61 [944244,24326] 0 2024-05-25T06:11:30.627 INFO:tasks.workunit.client.1.smithi203.stdout:4/565: chown d0/d8/d1f/fb7 513069877 1 2024-05-25T06:11:30.628 INFO:tasks.workunit.client.1.smithi203.stdout:9/481: mknod d0/df/d37/ca5 0 2024-05-25T06:11:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:1/61: mkdir de 0 2024-05-25T06:11:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:0/83: dwrite f15 [0,4194304] 0 2024-05-25T06:11:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:6/117: rmdir d5/de/d16/d1b 39 2024-05-25T06:11:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:5/113: mkdir db/dc/d27 0 2024-05-25T06:11:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:4/202: creat d4/d1c/f3e x:0 0 0 2024-05-25T06:11:30.634 INFO:tasks.workunit.client.1.smithi203.stdout:9/482: dread d0/df/d2d/f72 [0,4194304] 0 2024-05-25T06:11:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:6/118: read d5/de/fd [657677,6806] 0 2024-05-25T06:11:30.637 INFO:tasks.workunit.client.1.smithi203.stdout:9/483: fsync d0/d57/d8f/f9b 0 2024-05-25T06:11:30.638 INFO:tasks.workunit.client.1.smithi203.stdout:3/653: getdents d10/d16/d23/d46 0 2024-05-25T06:11:30.638 INFO:tasks.workunit.client.1.smithi203.stdout:9/484: chown d0/f24 211547802 1 2024-05-25T06:11:30.638 INFO:tasks.workunit.client.1.smithi203.stdout:6/757: write d8/d11/d1b/f4d [658836,105741] 0 2024-05-25T06:11:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:3/70: dwrite f11 [0,4194304] 0 2024-05-25T06:11:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:0/84: symlink l19 0 2024-05-25T06:11:30.642 INFO:tasks.workunit.client.1.smithi203.stdout:4/566: link d0/d8/f23 d0/d8/d10/d91/da2/fb9 0 2024-05-25T06:11:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:1/62: dwrite fb [0,4194304] 0 2024-05-25T06:11:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:4/203: creat d4/d6/d21/f3f x:0 0 0 2024-05-25T06:11:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:4/204: stat d4/f11 0 2024-05-25T06:11:30.649 INFO:tasks.workunit.client.1.smithi203.stdout:6/758: rmdir d8/d11/d15/d28/d52/d43 39 2024-05-25T06:11:30.650 INFO:tasks.workunit.client.1.smithi203.stdout:6/759: stat d8/d70/d38/fab 0 2024-05-25T06:11:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:7/96: link d6/dc/f15 d6/dc/f24 0 2024-05-25T06:11:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:6/119: dwrite d5/f11 [0,4194304] 0 2024-05-25T06:11:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:4/205: dread - d4/d6/d21/f3f zero size 2024-05-25T06:11:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:4/206: fdatasync d4/d1c/f3e 0 2024-05-25T06:11:30.653 INFO:tasks.workunit.client.1.smithi203.stdout:9/485: creat d0/fa6 x:0 0 0 2024-05-25T06:11:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:7/97: truncate d6/dc/f1e 842231 0 2024-05-25T06:11:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:1/63: dread f5 [0,4194304] 0 2024-05-25T06:11:30.655 INFO:tasks.workunit.client.1.smithi203.stdout:4/567: dread d0/f38 [0,4194304] 0 2024-05-25T06:11:30.659 INFO:tasks.workunit.client.1.smithi203.stdout:6/760: symlink d8/d11/d15/d28/d52/lf0 0 2024-05-25T06:11:30.660 INFO:tasks.workunit.client.1.smithi203.stdout:4/568: truncate d0/d8/d3a/d81/fb2 58203 0 2024-05-25T06:11:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:3/71: readlink ld 0 2024-05-25T06:11:30.662 INFO:tasks.workunit.client.1.smithi203.stdout:9/486: link d0/d17/d23/f5a d0/df/d73/fa7 0 2024-05-25T06:11:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:7/98: dwrite d6/dc/f17 [0,4194304] 0 2024-05-25T06:11:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:1/64: dread fa [0,4194304] 0 2024-05-25T06:11:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:8/167: getdents d5/d7/d8/d1f/d1e/d28 0 2024-05-25T06:11:30.664 INFO:tasks.workunit.client.1.smithi203.stdout:6/761: symlink d8/d2a/lf1 0 2024-05-25T06:11:30.664 INFO:tasks.workunit.client.1.smithi203.stdout:4/569: truncate d0/d9/d50/d79/fb8 183952 0 2024-05-25T06:11:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:7/99: readlink d6/l1c 0 2024-05-25T06:11:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:1/65: truncate f5 4219729 0 2024-05-25T06:11:30.665 INFO:tasks.workunit.client.1.smithi203.stdout:4/570: chown d0/d9/d50 197 1 2024-05-25T06:11:30.666 INFO:tasks.workunit.client.1.smithi203.stdout:9/487: unlink d0/df/d73/fa1 0 2024-05-25T06:11:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:7/100: dread d6/f12 [0,4194304] 0 2024-05-25T06:11:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:6/120: mknod d5/de/c2f 0 2024-05-25T06:11:30.671 INFO:tasks.workunit.client.1.smithi203.stdout:4/571: creat d0/d8/d10/d91/fba x:0 0 0 2024-05-25T06:11:30.673 INFO:tasks.workunit.client.1.smithi203.stdout:9/488: write d0/d17/d20/d6e/d75/d78/f94 [783253,31009] 0 2024-05-25T06:11:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:4/207: rename d4/d6/d28/f1e to d4/d1a/f40 0 2024-05-25T06:11:30.677 INFO:tasks.workunit.client.1.smithi203.stdout:3/654: dread d10/d16/d23/f2b [0,4194304] 0 2024-05-25T06:11:30.680 INFO:tasks.workunit.client.1.smithi203.stdout:9/489: mkdir d0/df/d37/da8 0 2024-05-25T06:11:30.680 INFO:tasks.workunit.client.1.smithi203.stdout:6/762: dwrite d8/f12 [4194304,4194304] 0 2024-05-25T06:11:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:5/114: link f6 db/dc/d13/f28 0 2024-05-25T06:11:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:6/121: rmdir d5 39 2024-05-25T06:11:30.684 INFO:tasks.workunit.client.1.smithi203.stdout:9/490: symlink d0/df/d2b/la9 0 2024-05-25T06:11:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:8/168: rename d5/f1a to d5/d39/f3b 0 2024-05-25T06:11:30.688 INFO:tasks.workunit.client.1.smithi203.stdout:9/491: dread - d0/df/f5f zero size 2024-05-25T06:11:30.689 INFO:tasks.workunit.client.1.smithi203.stdout:4/572: dwrite d0/d8/d10/f7e [0,4194304] 0 2024-05-25T06:11:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:1/66: creat de/ff x:0 0 0 2024-05-25T06:11:30.695 INFO:tasks.workunit.client.1.smithi203.stdout:3/655: dwrite d10/d16/d50/d8b/fad [0,4194304] 0 2024-05-25T06:11:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:5/115: mknod db/d1c/c29 0 2024-05-25T06:11:30.696 INFO:tasks.workunit.client.1.smithi203.stdout:9/492: chown d0/f52 34 1 2024-05-25T06:11:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:7/101: fsync d6/dc/f15 0 2024-05-25T06:11:30.699 INFO:tasks.workunit.client.1.smithi203.stdout:4/573: rename d0/d9/d11/d54/l72 to d0/d9/lbb 0 2024-05-25T06:11:30.701 INFO:tasks.workunit.client.1.smithi203.stdout:9/493: creat d0/df/faa x:0 0 0 2024-05-25T06:11:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:4/208: getdents d4/d6/d35/d3d 0 2024-05-25T06:11:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:8/169: mknod d5/d7/d8/d1f/d1e/c3c 0 2024-05-25T06:11:30.704 INFO:tasks.workunit.client.1.smithi203.stdout:4/574: dread - d0/d8/f8d zero size 2024-05-25T06:11:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:1/67: creat de/f10 x:0 0 0 2024-05-25T06:11:30.705 INFO:tasks.workunit.client.1.smithi203.stdout:3/656: dwrite d10/d16/d50/d8b/fad [0,4194304] 0 2024-05-25T06:11:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:5/116: chown db/dc/d13/f28 994 1 2024-05-25T06:11:30.707 INFO:tasks.workunit.client.1.smithi203.stdout:3/657: chown d10/l15 57360 1 2024-05-25T06:11:30.707 INFO:tasks.workunit.client.1.smithi203.stdout:9/494: symlink d0/df/d37/d4f/lab 0 2024-05-25T06:11:30.709 INFO:tasks.workunit.client.1.smithi203.stdout:4/575: symlink d0/d9/d50/d6b/lbc 0 2024-05-25T06:11:30.709 INFO:tasks.workunit.client.1.smithi203.stdout:3/658: write d10/d16/d23/d67/f94 [1523757,113030] 0 2024-05-25T06:11:30.710 INFO:tasks.workunit.client.1.smithi203.stdout:3/659: readlink d10/d16/l60 0 2024-05-25T06:11:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:8/170: dwrite d5/d7/d8/f25 [0,4194304] 0 2024-05-25T06:11:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:6/122: mkdir d5/d30 0 2024-05-25T06:11:30.715 INFO:tasks.workunit.client.1.smithi203.stdout:4/576: mknod d0/d8/d10/d91/cbd 0 2024-05-25T06:11:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:4/209: write d4/f26 [8162317,96842] 0 2024-05-25T06:11:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:6/123: stat d5/de/d16 0 2024-05-25T06:11:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:3/72: link c17 d19/c1a 0 2024-05-25T06:11:30.717 INFO:tasks.workunit.client.1.smithi203.stdout:6/763: dread d8/d11/d15/d28/d52/d43/f32 [0,4194304] 0 2024-05-25T06:11:30.719 INFO:tasks.workunit.client.1.smithi203.stdout:9/495: rename d0/df/d2d/c69 to d0/d57/cac 0 2024-05-25T06:11:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:6/124: mknod d5/c31 0 2024-05-25T06:11:30.725 INFO:tasks.workunit.client.1.smithi203.stdout:6/764: mknod d8/d2a/d6c/dc1/dd8/cf2 0 2024-05-25T06:11:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:4/210: rename d4/d6/d21/l37 to d4/d6/d35/l41 0 2024-05-25T06:11:30.728 INFO:tasks.workunit.client.1.smithi203.stdout:9/496: link d0/df/d73/c9f d0/df/d2d/d92/cad 0 2024-05-25T06:11:30.731 INFO:tasks.workunit.client.1.smithi203.stdout:9/497: mkdir d0/d57/dae 0 2024-05-25T06:11:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:7/102: dwrite d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:8/171: dwrite d5/fe [0,4194304] 0 2024-05-25T06:11:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:7/103: readlink d6/l1c 0 2024-05-25T06:11:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:5/117: dwrite db/dc/f15 [0,4194304] 0 2024-05-25T06:11:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:9/65: sync 2024-05-25T06:11:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:2/98: sync 2024-05-25T06:11:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:2/99: stat f5 0 2024-05-25T06:11:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:3/73: dwrite f0 [0,4194304] 0 2024-05-25T06:11:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:6/125: dread d5/de/ff [0,4194304] 0 2024-05-25T06:11:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:4/211: dwrite d4/d1a/f38 [0,4194304] 0 2024-05-25T06:11:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:0/85: sync 2024-05-25T06:11:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:0/86: dread f2 [0,4194304] 0 2024-05-25T06:11:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:0/87: write f16 [689719,71109] 0 2024-05-25T06:11:30.776 INFO:tasks.workunit.client.1.smithi203.stdout:2/540: sync 2024-05-25T06:11:30.778 INFO:tasks.workunit.client.1.smithi203.stdout:7/512: sync 2024-05-25T06:11:30.778 INFO:tasks.workunit.client.1.smithi203.stdout:0/391: sync 2024-05-25T06:11:30.778 INFO:tasks.workunit.client.1.smithi203.stdout:5/786: sync 2024-05-25T06:11:30.783 INFO:tasks.workunit.client.1.smithi203.stdout:9/498: dwrite d0/df/faa [0,4194304] 0 2024-05-25T06:11:30.783 INFO:tasks.workunit.client.1.smithi203.stdout:7/513: chown d1/d9/d2c 699031 1 2024-05-25T06:11:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:7/104: rmdir d6 39 2024-05-25T06:11:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:1/68: rmdir de 39 2024-05-25T06:11:30.785 INFO:tasks.workunit.client.1.smithi203.stdout:6/765: dread f2 [0,4194304] 0 2024-05-25T06:11:30.785 INFO:tasks.workunit.client.1.smithi203.stdout:7/514: stat f0 0 2024-05-25T06:11:30.785 INFO:tasks.workunit.client.1.smithi203.stdout:2/541: rmdir d2/d1a 39 2024-05-25T06:11:30.787 INFO:tasks.workunit.client.1.smithi203.stdout:0/392: creat dd/d8c/f8e x:0 0 0 2024-05-25T06:11:30.788 INFO:tasks.workunit.client.1.smithi203.stdout:9/499: read d0/f52 [1568531,35749] 0 2024-05-25T06:11:30.791 INFO:tasks.workunit.client.0.smithi160.stdout:2/100: symlink d12/l1b 0 2024-05-25T06:11:30.792 INFO:tasks.workunit.client.0.smithi160.stdout:2/101: write f8 [5079017,72103] 0 2024-05-25T06:11:30.794 INFO:tasks.workunit.client.1.smithi203.stdout:2/542: write d2/d38/d2f/f61 [536113,78835] 0 2024-05-25T06:11:30.794 INFO:tasks.workunit.client.1.smithi203.stdout:0/393: fdatasync dd/d10/f29 0 2024-05-25T06:11:30.795 INFO:tasks.workunit.client.0.smithi160.stdout:8/172: dwrite d5/d9/f13 [0,4194304] 0 2024-05-25T06:11:30.795 INFO:tasks.workunit.client.1.smithi203.stdout:0/394: write dd/f64 [176159,20258] 0 2024-05-25T06:11:30.796 INFO:tasks.workunit.client.0.smithi160.stdout:8/173: dread - d5/d9/f26 zero size 2024-05-25T06:11:30.796 INFO:tasks.workunit.client.0.smithi160.stdout:6/126: rename d5/de/d16 to d5/de/d32 0 2024-05-25T06:11:30.796 INFO:tasks.workunit.client.1.smithi203.stdout:4/577: dread d0/d8/d3a/f46 [0,4194304] 0 2024-05-25T06:11:30.800 INFO:tasks.workunit.client.1.smithi203.stdout:5/787: mkdir dc/d11/d17/d1a/d2d/d4d/d64/d114 0 2024-05-25T06:11:30.800 INFO:tasks.workunit.client.1.smithi203.stdout:4/578: chown d0/d8/d10/d91/da2/ca3 44 1 2024-05-25T06:11:30.801 INFO:tasks.workunit.client.0.smithi160.stdout:0/88: rename ce to c1a 0 2024-05-25T06:11:30.802 INFO:tasks.workunit.client.0.smithi160.stdout:0/89: write f15 [4753561,2387] 0 2024-05-25T06:11:30.802 INFO:tasks.workunit.client.1.smithi203.stdout:9/500: rename d0/d17/d20/d6e/d75/c7e to d0/df/d2d/d92/caf 0 2024-05-25T06:11:30.804 INFO:tasks.workunit.client.0.smithi160.stdout:2/102: mkdir d12/d1c 0 2024-05-25T06:11:30.805 INFO:tasks.workunit.client.1.smithi203.stdout:6/766: mknod d8/d11/d1b/d75/dcb/dcf/d46/cf3 0 2024-05-25T06:11:30.805 INFO:tasks.workunit.client.0.smithi160.stdout:8/174: dread d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:30.805 INFO:tasks.workunit.client.1.smithi203.stdout:6/767: fsync d8/d11/d47/fb8 0 2024-05-25T06:11:30.806 INFO:tasks.workunit.client.0.smithi160.stdout:3/74: creat d19/f1b x:0 0 0 2024-05-25T06:11:30.807 INFO:tasks.workunit.client.1.smithi203.stdout:6/768: chown d8/d2a/l3a 25608214 1 2024-05-25T06:11:30.808 INFO:tasks.workunit.client.0.smithi160.stdout:6/127: fdatasync d5/de/d32/d1b/f2d 0 2024-05-25T06:11:30.809 INFO:tasks.workunit.client.1.smithi203.stdout:2/543: mknod d2/cb7 0 2024-05-25T06:11:30.809 INFO:tasks.workunit.client.0.smithi160.stdout:8/175: fsync d5/d7/d8/f37 0 2024-05-25T06:11:30.810 INFO:tasks.workunit.client.1.smithi203.stdout:5/788: getdents dc/d11/d17/d1a/dd2 0 2024-05-25T06:11:30.811 INFO:tasks.workunit.client.1.smithi203.stdout:4/579: mkdir d0/d9/d50/d79/d9a/dbe 0 2024-05-25T06:11:30.814 INFO:tasks.workunit.client.1.smithi203.stdout:0/395: getdents dd/d3d 0 2024-05-25T06:11:30.815 INFO:tasks.workunit.client.0.smithi160.stdout:3/75: dread - f16 zero size 2024-05-25T06:11:30.815 INFO:tasks.workunit.client.0.smithi160.stdout:6/128: write d5/de/f22 [3408831,80452] 0 2024-05-25T06:11:30.815 INFO:tasks.workunit.client.1.smithi203.stdout:4/580: fsync d0/d9/f94 0 2024-05-25T06:11:30.817 INFO:tasks.workunit.client.1.smithi203.stdout:6/769: dread d8/d11/d47/f8a [0,4194304] 0 2024-05-25T06:11:30.817 INFO:tasks.workunit.client.1.smithi203.stdout:5/789: symlink dc/d11/d17/d1a/d103/d98/l115 0 2024-05-25T06:11:30.817 INFO:tasks.workunit.client.1.smithi203.stdout:9/501: link d0/d17/d20/d6e/d75/d78/la3 d0/lb0 0 2024-05-25T06:11:30.819 INFO:tasks.workunit.client.1.smithi203.stdout:2/544: dwrite d2/d4/d10/f21 [0,4194304] 0 2024-05-25T06:11:30.819 INFO:tasks.workunit.client.1.smithi203.stdout:0/396: mknod dd/d32/d37/c8f 0 2024-05-25T06:11:30.822 INFO:tasks.workunit.client.1.smithi203.stdout:2/545: write d2/d42/f83 [1097539,82779] 0 2024-05-25T06:11:30.822 INFO:tasks.workunit.client.0.smithi160.stdout:0/90: dwrite f5 [0,4194304] 0 2024-05-25T06:11:30.823 INFO:tasks.workunit.client.0.smithi160.stdout:3/76: mknod d19/c1c 0 2024-05-25T06:11:30.823 INFO:tasks.workunit.client.0.smithi160.stdout:0/91: write f16 [284402,64368] 0 2024-05-25T06:11:30.823 INFO:tasks.workunit.client.1.smithi203.stdout:2/546: stat d2/d42 0 2024-05-25T06:11:30.825 INFO:tasks.workunit.client.0.smithi160.stdout:6/129: symlink d5/de/d32/d1b/d28/l33 0 2024-05-25T06:11:30.825 INFO:tasks.workunit.client.1.smithi203.stdout:6/770: unlink d8/d11/f23 0 2024-05-25T06:11:30.826 INFO:tasks.workunit.client.0.smithi160.stdout:1/69: dwrite de/ff [0,4194304] 0 2024-05-25T06:11:30.826 INFO:tasks.workunit.client.0.smithi160.stdout:2/103: dwrite f8 [0,4194304] 0 2024-05-25T06:11:30.829 INFO:tasks.workunit.client.0.smithi160.stdout:4/212: rmdir d4/d6 39 2024-05-25T06:11:30.831 INFO:tasks.workunit.client.1.smithi203.stdout:7/515: dread d1/d11/d20/f28 [0,4194304] 0 2024-05-25T06:11:30.831 INFO:tasks.workunit.client.0.smithi160.stdout:3/77: dread - f13 zero size 2024-05-25T06:11:30.832 INFO:tasks.workunit.client.1.smithi203.stdout:9/502: rmdir d0/d17/d20/d3c 39 2024-05-25T06:11:30.835 INFO:tasks.workunit.client.1.smithi203.stdout:0/397: creat dd/d5c/f90 x:0 0 0 2024-05-25T06:11:30.837 INFO:tasks.workunit.client.1.smithi203.stdout:0/398: write dd/d3a/f3b [2028127,57657] 0 2024-05-25T06:11:30.838 INFO:tasks.workunit.client.0.smithi160.stdout:9/66: truncate d4/f8 4643977 0 2024-05-25T06:11:30.839 INFO:tasks.workunit.client.1.smithi203.stdout:5/790: mkdir dc/d11/d17/d1a/d2d/d53/da2/d116 0 2024-05-25T06:11:30.840 INFO:tasks.workunit.client.0.smithi160.stdout:4/213: unlink d4/d1c/c31 0 2024-05-25T06:11:30.841 INFO:tasks.workunit.client.0.smithi160.stdout:6/130: creat d5/f34 x:0 0 0 2024-05-25T06:11:30.841 INFO:tasks.workunit.client.0.smithi160.stdout:3/78: readlink l4 0 2024-05-25T06:11:30.842 INFO:tasks.workunit.client.0.smithi160.stdout:5/118: dwrite f6 [0,4194304] 0 2024-05-25T06:11:30.842 INFO:tasks.workunit.client.0.smithi160.stdout:2/104: link f7 d12/f1d 0 2024-05-25T06:11:30.844 INFO:tasks.workunit.client.0.smithi160.stdout:0/92: stat c1a 0 2024-05-25T06:11:30.844 INFO:tasks.workunit.client.0.smithi160.stdout:5/119: write db/dc/f15 [7531372,113385] 0 2024-05-25T06:11:30.845 INFO:tasks.workunit.client.1.smithi203.stdout:4/581: rename d0/d8/d3a/f5d to d0/d9/d9d/fbf 0 2024-05-25T06:11:30.846 INFO:tasks.workunit.client.1.smithi203.stdout:4/582: chown d0/f38 466948753 1 2024-05-25T06:11:30.848 INFO:tasks.workunit.client.0.smithi160.stdout:5/120: write db/f26 [862524,17365] 0 2024-05-25T06:11:30.850 INFO:tasks.workunit.client.0.smithi160.stdout:4/214: read d4/d6/d35/f3c [2993985,80106] 0 2024-05-25T06:11:30.853 INFO:tasks.workunit.client.0.smithi160.stdout:8/176: write d5/d7/d8/d1f/f22 [390101,112384] 0 2024-05-25T06:11:30.855 INFO:tasks.workunit.client.1.smithi203.stdout:7/516: mknod d1/d9/d2c/d3a/d4e/d50/cb0 0 2024-05-25T06:11:30.855 INFO:tasks.workunit.client.0.smithi160.stdout:0/93: dread fa [0,4194304] 0 2024-05-25T06:11:30.856 INFO:tasks.workunit.client.0.smithi160.stdout:3/79: unlink c9 0 2024-05-25T06:11:30.856 INFO:tasks.workunit.client.1.smithi203.stdout:9/503: creat d0/df/d2b/d96/fb1 x:0 0 0 2024-05-25T06:11:30.857 INFO:tasks.workunit.client.0.smithi160.stdout:6/131: mkdir d5/de/d32/d35 0 2024-05-25T06:11:30.859 INFO:tasks.workunit.client.1.smithi203.stdout:5/791: rmdir dc/d11/d17/d1a/d2d/d4d/ded 39 2024-05-25T06:11:30.860 INFO:tasks.workunit.client.0.smithi160.stdout:7/105: dwrite f1 [0,4194304] 0 2024-05-25T06:11:30.860 INFO:tasks.workunit.client.0.smithi160.stdout:2/105: creat d12/d1c/f1e x:0 0 0 2024-05-25T06:11:30.862 INFO:tasks.workunit.client.1.smithi203.stdout:4/583: rename d0/d8/l35 to d0/d9/d50/d6b/lc0 0 2024-05-25T06:11:30.862 INFO:tasks.workunit.client.0.smithi160.stdout:0/94: creat f1b x:0 0 0 2024-05-25T06:11:30.863 INFO:tasks.workunit.client.1.smithi203.stdout:9/504: rmdir d0/df/d37/d4f 39 2024-05-25T06:11:30.863 INFO:tasks.workunit.client.1.smithi203.stdout:4/584: write d0/d9/d50/d6b/f30 [1513140,57654] 0 2024-05-25T06:11:30.865 INFO:tasks.workunit.client.0.smithi160.stdout:3/80: rmdir d19 39 2024-05-25T06:11:30.866 INFO:tasks.workunit.client.0.smithi160.stdout:5/121: dwrite f8 [0,4194304] 0 2024-05-25T06:11:30.866 INFO:tasks.workunit.client.1.smithi203.stdout:2/547: creat d2/fb8 x:0 0 0 2024-05-25T06:11:30.867 INFO:tasks.workunit.client.0.smithi160.stdout:0/95: write f12 [4979399,91282] 0 2024-05-25T06:11:30.867 INFO:tasks.workunit.client.1.smithi203.stdout:2/548: read - d2/fb8 zero size 2024-05-25T06:11:30.870 INFO:tasks.workunit.client.1.smithi203.stdout:2/549: write d2/d4/d10/f9b [954065,22238] 0 2024-05-25T06:11:30.873 INFO:tasks.workunit.client.0.smithi160.stdout:0/96: read f5 [3440022,102530] 0 2024-05-25T06:11:30.873 INFO:tasks.workunit.client.1.smithi203.stdout:4/585: dwrite d0/d8/f44 [0,4194304] 0 2024-05-25T06:11:30.874 INFO:tasks.workunit.client.0.smithi160.stdout:5/122: chown db/dc/d13/l16 886 1 2024-05-25T06:11:30.877 INFO:tasks.workunit.client.1.smithi203.stdout:0/399: getdents dd/d3d/d59 0 2024-05-25T06:11:30.878 INFO:tasks.workunit.client.0.smithi160.stdout:9/67: unlink d4/f8 0 2024-05-25T06:11:30.880 INFO:tasks.workunit.client.0.smithi160.stdout:7/106: unlink d6/dc/f1e 0 2024-05-25T06:11:30.883 INFO:tasks.workunit.client.1.smithi203.stdout:0/400: mkdir dd/d3d/d48/d91 0 2024-05-25T06:11:30.883 INFO:tasks.workunit.client.0.smithi160.stdout:4/215: dwrite d4/d1a/f40 [4194304,4194304] 0 2024-05-25T06:11:30.884 INFO:tasks.workunit.client.0.smithi160.stdout:2/106: mknod d12/d1c/c1f 0 2024-05-25T06:11:30.885 INFO:tasks.workunit.client.0.smithi160.stdout:1/70: dwrite f5 [0,4194304] 0 2024-05-25T06:11:30.885 INFO:tasks.workunit.client.0.smithi160.stdout:8/177: dread d5/d7/d8/f16 [4194304,4194304] 0 2024-05-25T06:11:30.887 INFO:tasks.workunit.client.1.smithi203.stdout:2/550: dwrite d2/d6/f26 [0,4194304] 0 2024-05-25T06:11:30.888 INFO:tasks.workunit.client.0.smithi160.stdout:4/216: read d4/d6/f7 [1657245,33240] 0 2024-05-25T06:11:30.891 INFO:tasks.workunit.client.0.smithi160.stdout:9/68: rename l1 to d4/l10 0 2024-05-25T06:11:30.892 INFO:tasks.workunit.client.1.smithi203.stdout:4/586: link d0/d8/d3a/f56 d0/d9/d50/d6b/fc1 0 2024-05-25T06:11:30.897 INFO:tasks.workunit.client.0.smithi160.stdout:3/81: dread f15 [0,4194304] 0 2024-05-25T06:11:30.900 INFO:tasks.workunit.client.1.smithi203.stdout:4/587: dread d0/d8/d3a/f46 [0,4194304] 0 2024-05-25T06:11:30.901 INFO:tasks.workunit.client.0.smithi160.stdout:3/82: dread - f16 zero size 2024-05-25T06:11:30.901 INFO:tasks.workunit.client.1.smithi203.stdout:0/401: rename dd/d3d/d4f/l53 to dd/d3a/d66/d6e/d70/d84/l92 0 2024-05-25T06:11:30.904 INFO:tasks.workunit.client.0.smithi160.stdout:7/107: creat d6/dc/f25 x:0 0 0 2024-05-25T06:11:30.904 INFO:tasks.workunit.client.0.smithi160.stdout:8/178: creat d5/d30/f3d x:0 0 0 2024-05-25T06:11:30.905 INFO:tasks.workunit.client.1.smithi203.stdout:2/551: getdents d2/d1a/d89/daa 0 2024-05-25T06:11:30.907 INFO:tasks.workunit.client.1.smithi203.stdout:0/402: truncate f5 2351970 0 2024-05-25T06:11:30.908 INFO:tasks.workunit.client.1.smithi203.stdout:6/771: truncate d8/d11/d15/f2e 889590 0 2024-05-25T06:11:30.910 INFO:tasks.workunit.client.0.smithi160.stdout:5/123: dwrite db/f22 [0,4194304] 0 2024-05-25T06:11:30.910 INFO:tasks.workunit.client.0.smithi160.stdout:1/71: rename f1 to de/f11 0 2024-05-25T06:11:30.910 INFO:tasks.workunit.client.0.smithi160.stdout:2/107: dwrite f9 [4194304,4194304] 0 2024-05-25T06:11:30.911 INFO:tasks.workunit.client.1.smithi203.stdout:7/517: truncate d1/d9/f29 2617790 0 2024-05-25T06:11:30.911 INFO:tasks.workunit.client.0.smithi160.stdout:8/179: fdatasync d5/fb 0 2024-05-25T06:11:30.912 INFO:tasks.workunit.client.0.smithi160.stdout:0/97: dwrite f2 [0,4194304] 0 2024-05-25T06:11:30.916 INFO:tasks.workunit.client.0.smithi160.stdout:8/180: read d5/fe [1078685,105554] 0 2024-05-25T06:11:30.917 INFO:tasks.workunit.client.0.smithi160.stdout:3/83: chown f11 371079004 1 2024-05-25T06:11:30.919 INFO:tasks.workunit.client.0.smithi160.stdout:2/108: chown d12/d1c/f1e 110 1 2024-05-25T06:11:30.920 INFO:tasks.workunit.client.0.smithi160.stdout:7/108: read d6/f12 [34192,89546] 0 2024-05-25T06:11:30.920 INFO:tasks.workunit.client.1.smithi203.stdout:5/792: dwrite dc/d11/d17/d1a/f61 [0,4194304] 0 2024-05-25T06:11:30.921 INFO:tasks.workunit.client.0.smithi160.stdout:7/109: read d6/dc/f17 [1135578,122433] 0 2024-05-25T06:11:30.921 INFO:tasks.workunit.client.0.smithi160.stdout:7/110: write d6/dc/ff [689630,30171] 0 2024-05-25T06:11:30.925 INFO:tasks.workunit.client.0.smithi160.stdout:8/181: truncate d5/d30/f34 1033019 0 2024-05-25T06:11:30.925 INFO:tasks.workunit.client.1.smithi203.stdout:7/518: getdents d1/d9/d2c/d36/d3b 0 2024-05-25T06:11:30.925 INFO:tasks.workunit.client.1.smithi203.stdout:2/552: link d2/d1a/d33/l76 d2/d38/d43/lb9 0 2024-05-25T06:11:30.930 INFO:tasks.workunit.client.0.smithi160.stdout:6/132: truncate d5/de/f22 2273737 0 2024-05-25T06:11:30.931 INFO:tasks.workunit.client.0.smithi160.stdout:1/72: mknod de/c12 0 2024-05-25T06:11:30.931 INFO:tasks.workunit.client.0.smithi160.stdout:0/98: rename c6 to c1c 0 2024-05-25T06:11:30.932 INFO:tasks.workunit.client.1.smithi203.stdout:5/793: dwrite dc/d11/d17/f31 [0,4194304] 0 2024-05-25T06:11:30.933 INFO:tasks.workunit.client.1.smithi203.stdout:2/553: symlink d2/d4/d10/d58/da0/lba 0 2024-05-25T06:11:30.934 INFO:tasks.workunit.client.1.smithi203.stdout:7/519: mknod d1/d9/d60/cb1 0 2024-05-25T06:11:30.935 INFO:tasks.workunit.client.0.smithi160.stdout:5/124: creat db/f2a x:0 0 0 2024-05-25T06:11:30.936 INFO:tasks.workunit.client.1.smithi203.stdout:5/794: rmdir dc/d11/d17/d1a/d103/d98 39 2024-05-25T06:11:30.936 INFO:tasks.workunit.client.0.smithi160.stdout:2/109: creat d12/f20 x:0 0 0 2024-05-25T06:11:30.937 INFO:tasks.workunit.client.0.smithi160.stdout:5/125: readlink db/dc/d13/l18 0 2024-05-25T06:11:30.938 INFO:tasks.workunit.client.0.smithi160.stdout:4/217: write d4/d6/d35/f3c [2461795,20919] 0 2024-05-25T06:11:30.938 INFO:tasks.workunit.client.0.smithi160.stdout:5/126: readlink db/dc/d13/l18 0 2024-05-25T06:11:30.940 INFO:tasks.workunit.client.1.smithi203.stdout:2/554: mknod d2/d4/d10/d58/cbb 0 2024-05-25T06:11:30.941 INFO:tasks.workunit.client.0.smithi160.stdout:8/182: fdatasync d5/d7/f19 0 2024-05-25T06:11:30.942 INFO:tasks.workunit.client.0.smithi160.stdout:4/218: truncate d4/d1c/f3e 662105 0 2024-05-25T06:11:30.942 INFO:tasks.workunit.client.1.smithi203.stdout:2/555: write d2/d4/f2e [6008073,126221] 0 2024-05-25T06:11:30.943 INFO:tasks.workunit.client.1.smithi203.stdout:5/795: creat dc/d11/d17/d1a/d72/dc8/f117 x:0 0 0 2024-05-25T06:11:30.944 INFO:tasks.workunit.client.1.smithi203.stdout:2/556: chown d2/d1a/db0 918702375 1 2024-05-25T06:11:30.945 INFO:tasks.workunit.client.0.smithi160.stdout:0/99: rename l19 to l1d 0 2024-05-25T06:11:30.945 INFO:tasks.workunit.client.0.smithi160.stdout:4/219: chown d4/d6/l16 207021939 1 2024-05-25T06:11:30.946 INFO:tasks.workunit.client.1.smithi203.stdout:6/772: dread d8/d70/d38/f39 [0,4194304] 0 2024-05-25T06:11:30.946 INFO:tasks.workunit.client.0.smithi160.stdout:9/69: dwrite f2 [4194304,4194304] 0 2024-05-25T06:11:30.948 INFO:tasks.workunit.client.0.smithi160.stdout:7/111: mknod d6/c26 0 2024-05-25T06:11:30.949 INFO:tasks.workunit.client.1.smithi203.stdout:7/520: symlink d1/d11/d3d/d4c/lb2 0 2024-05-25T06:11:30.951 INFO:tasks.workunit.client.0.smithi160.stdout:5/127: creat db/d1c/f2b x:0 0 0 2024-05-25T06:11:30.954 INFO:tasks.workunit.client.0.smithi160.stdout:8/183: dread d5/fb [0,4194304] 0 2024-05-25T06:11:30.956 INFO:tasks.workunit.client.0.smithi160.stdout:0/100: creat f1e x:0 0 0 2024-05-25T06:11:30.956 INFO:tasks.workunit.client.0.smithi160.stdout:2/110: dwrite fb [4194304,4194304] 0 2024-05-25T06:11:30.956 INFO:tasks.workunit.client.1.smithi203.stdout:2/557: dwrite d2/d38/d43/d60/d6b/dae/f51 [4194304,4194304] 0 2024-05-25T06:11:30.958 INFO:tasks.workunit.client.0.smithi160.stdout:4/220: mkdir d4/d6/d21/d42 0 2024-05-25T06:11:30.959 INFO:tasks.workunit.client.1.smithi203.stdout:4/588: truncate d0/d9/d11/f66 138267 0 2024-05-25T06:11:30.960 INFO:tasks.workunit.client.0.smithi160.stdout:5/128: unlink db/f1a 0 2024-05-25T06:11:30.960 INFO:tasks.workunit.client.0.smithi160.stdout:3/84: link ld d19/l1d 0 2024-05-25T06:11:30.960 INFO:tasks.workunit.client.0.smithi160.stdout:4/221: write d4/f33 [5925837,96324] 0 2024-05-25T06:11:30.961 INFO:tasks.workunit.client.1.smithi203.stdout:0/403: write f9 [2343070,70878] 0 2024-05-25T06:11:30.963 INFO:tasks.workunit.client.1.smithi203.stdout:7/521: getdents d1/d11/d3d/d9a/da4 0 2024-05-25T06:11:30.963 INFO:tasks.workunit.client.0.smithi160.stdout:8/184: creat d5/f3e x:0 0 0 2024-05-25T06:11:30.964 INFO:tasks.workunit.client.1.smithi203.stdout:4/589: chown d0/d9 2859 1 2024-05-25T06:11:30.966 INFO:tasks.workunit.client.0.smithi160.stdout:9/70: getdents d4/d9/df 0 2024-05-25T06:11:30.968 INFO:tasks.workunit.client.1.smithi203.stdout:7/522: write d1/f1c [3611844,85767] 0 2024-05-25T06:11:30.970 INFO:tasks.workunit.client.1.smithi203.stdout:0/404: rmdir dd/d3d/d48 39 2024-05-25T06:11:30.971 INFO:tasks.workunit.client.0.smithi160.stdout:0/101: dread f15 [4194304,4194304] 0 2024-05-25T06:11:30.971 INFO:tasks.workunit.client.0.smithi160.stdout:7/112: symlink d6/l27 0 2024-05-25T06:11:30.972 INFO:tasks.workunit.client.1.smithi203.stdout:2/558: dwrite d2/d42/d4e/d68/faf [0,4194304] 0 2024-05-25T06:11:30.973 INFO:tasks.workunit.client.0.smithi160.stdout:5/129: creat db/dc/d13/f2c x:0 0 0 2024-05-25T06:11:30.973 INFO:tasks.workunit.client.0.smithi160.stdout:4/222: mknod d4/d6/d28/c43 0 2024-05-25T06:11:30.974 INFO:tasks.workunit.client.1.smithi203.stdout:0/405: chown dd/d3a/d66/d6e/d70 815186 1 2024-05-25T06:11:30.976 INFO:tasks.workunit.client.0.smithi160.stdout:9/71: mkdir d4/d11 0 2024-05-25T06:11:30.977 INFO:tasks.workunit.client.1.smithi203.stdout:2/559: creat d2/d38/d43/d60/fbc x:0 0 0 2024-05-25T06:11:30.978 INFO:tasks.workunit.client.1.smithi203.stdout:2/560: dread - d2/d38/d43/d60/fbc zero size 2024-05-25T06:11:30.979 INFO:tasks.workunit.client.1.smithi203.stdout:0/406: mkdir dd/d3d/d75/d93 0 2024-05-25T06:11:30.979 INFO:tasks.workunit.client.0.smithi160.stdout:9/72: write d4/fd [4685573,37685] 0 2024-05-25T06:11:30.981 INFO:tasks.workunit.client.1.smithi203.stdout:4/590: creat d0/d9/d11/d76/fc2 x:0 0 0 2024-05-25T06:11:30.983 INFO:tasks.workunit.client.0.smithi160.stdout:0/102: creat f1f x:0 0 0 2024-05-25T06:11:30.983 INFO:tasks.workunit.client.0.smithi160.stdout:7/113: dread d6/dc/f24 [0,4194304] 0 2024-05-25T06:11:30.983 INFO:tasks.workunit.client.0.smithi160.stdout:9/73: read d4/fd [6032797,83651] 0 2024-05-25T06:11:30.983 INFO:tasks.workunit.client.1.smithi203.stdout:5/796: dread dc/d11/f29 [0,4194304] 0 2024-05-25T06:11:30.984 INFO:tasks.workunit.client.1.smithi203.stdout:4/591: read - d0/d8/f8d zero size 2024-05-25T06:11:30.984 INFO:tasks.workunit.client.0.smithi160.stdout:0/103: truncate f1e 66485 0 2024-05-25T06:11:30.984 INFO:tasks.workunit.client.0.smithi160.stdout:4/223: unlink d4/d1a/c2b 0 2024-05-25T06:11:30.985 INFO:tasks.workunit.client.1.smithi203.stdout:0/407: readlink dd/d3a/d66/d6e/d70/d84/l92 0 2024-05-25T06:11:30.985 INFO:tasks.workunit.client.0.smithi160.stdout:5/130: truncate f4 5748142 0 2024-05-25T06:11:30.986 INFO:tasks.workunit.client.0.smithi160.stdout:7/114: write d6/dc/f19 [599127,122616] 0 2024-05-25T06:11:30.988 INFO:tasks.workunit.client.0.smithi160.stdout:9/74: write d4/fd [4126185,62165] 0 2024-05-25T06:11:30.989 INFO:tasks.workunit.client.0.smithi160.stdout:3/85: dwrite f8 [0,4194304] 0 2024-05-25T06:11:30.989 INFO:tasks.workunit.client.0.smithi160.stdout:4/224: write d4/d1c/f3e [1158894,18111] 0 2024-05-25T06:11:30.992 INFO:tasks.workunit.client.1.smithi203.stdout:0/408: fsync dd/d10/d34/f58 0 2024-05-25T06:11:30.994 INFO:tasks.workunit.client.1.smithi203.stdout:5/797: dread - dc/d11/d17/d1a/d2d/d53/fec zero size 2024-05-25T06:11:30.994 INFO:tasks.workunit.client.0.smithi160.stdout:6/133: truncate d5/f2c 761047 0 2024-05-25T06:11:30.996 INFO:tasks.workunit.client.1.smithi203.stdout:6/773: dwrite d8/fa0 [0,4194304] 0 2024-05-25T06:11:30.997 INFO:tasks.workunit.client.1.smithi203.stdout:2/561: dwrite d2/d4/d10/d32/f52 [0,4194304] 0 2024-05-25T06:11:30.997 INFO:tasks.workunit.client.0.smithi160.stdout:0/104: dwrite f1e [0,4194304] 0 2024-05-25T06:11:30.998 INFO:tasks.workunit.client.1.smithi203.stdout:5/798: stat f1 0 2024-05-25T06:11:30.998 INFO:tasks.workunit.client.1.smithi203.stdout:0/409: chown dd/d3d/d59/f83 6441688 1 2024-05-25T06:11:30.999 INFO:tasks.workunit.client.0.smithi160.stdout:0/105: chown l8 13400 1 2024-05-25T06:11:30.999 INFO:tasks.workunit.client.0.smithi160.stdout:7/115: dread d6/dc/f16 [0,4194304] 0 2024-05-25T06:11:31.000 INFO:tasks.workunit.client.0.smithi160.stdout:3/86: stat cc 0 2024-05-25T06:11:31.002 INFO:tasks.workunit.client.1.smithi203.stdout:0/410: write dd/d45/f5b [822927,88562] 0 2024-05-25T06:11:31.006 INFO:tasks.workunit.client.0.smithi160.stdout:8/185: dread d5/d7/d8/f16 [0,4194304] 0 2024-05-25T06:11:31.007 INFO:tasks.workunit.client.1.smithi203.stdout:1/485: sync 2024-05-25T06:11:31.007 INFO:tasks.workunit.client.1.smithi203.stdout:6/774: write d8/d11/d15/d28/d52/d43/f32 [2479889,38214] 0 2024-05-25T06:11:31.007 INFO:tasks.workunit.client.1.smithi203.stdout:2/562: rmdir d2/d1a/d89 39 2024-05-25T06:11:31.007 INFO:tasks.workunit.client.1.smithi203.stdout:8/626: sync 2024-05-25T06:11:31.009 INFO:tasks.workunit.client.0.smithi160.stdout:6/134: rename d5/de/d32/d1b/d28 to d5/d30/d36 0 2024-05-25T06:11:31.010 INFO:tasks.workunit.client.1.smithi203.stdout:5/799: creat dc/d11/d17/d1a/d2d/d53/da2/f118 x:0 0 0 2024-05-25T06:11:31.010 INFO:tasks.workunit.client.1.smithi203.stdout:8/627: chown d2/d5/d10/f53 1322608 1 2024-05-25T06:11:31.012 INFO:tasks.workunit.client.0.smithi160.stdout:5/131: getdents db/dc/d27 0 2024-05-25T06:11:31.012 INFO:tasks.workunit.client.1.smithi203.stdout:4/592: rename d0/d8/d3a/d81/d93/la6 to d0/d9/d50/lc3 0 2024-05-25T06:11:31.013 INFO:tasks.workunit.client.0.smithi160.stdout:5/132: chown db/dc/d13/d1d 3102145 1 2024-05-25T06:11:31.014 INFO:tasks.workunit.client.1.smithi203.stdout:6/775: creat d8/d5a/d9b/daf/ff4 x:0 0 0 2024-05-25T06:11:31.015 INFO:tasks.workunit.client.1.smithi203.stdout:2/563: mknod d2/d38/cbd 0 2024-05-25T06:11:31.015 INFO:tasks.workunit.client.0.smithi160.stdout:0/106: creat f20 x:0 0 0 2024-05-25T06:11:31.015 INFO:tasks.workunit.client.0.smithi160.stdout:0/107: chown c1 124987 1 2024-05-25T06:11:31.016 INFO:tasks.workunit.client.1.smithi203.stdout:2/564: write d2/f6f [8283077,34782] 0 2024-05-25T06:11:31.018 INFO:tasks.workunit.client.0.smithi160.stdout:7/116: unlink d6/le 0 2024-05-25T06:11:31.019 INFO:tasks.workunit.client.1.smithi203.stdout:8/628: symlink d2/d5/d10/d4a/db6/lcb 0 2024-05-25T06:11:31.021 INFO:tasks.workunit.client.0.smithi160.stdout:4/225: mknod d4/d6/d21/d42/c44 0 2024-05-25T06:11:31.021 INFO:tasks.workunit.client.1.smithi203.stdout:6/776: rename d8/fd to d8/d11/d15/d28/d52/d43/ff5 0 2024-05-25T06:11:31.021 INFO:tasks.workunit.client.1.smithi203.stdout:8/629: read d2/d5/d18/f1b [8087663,76045] 0 2024-05-25T06:11:31.022 INFO:tasks.workunit.client.0.smithi160.stdout:4/226: write d4/d6/d28/f17 [8645946,66824] 0 2024-05-25T06:11:31.024 INFO:tasks.workunit.client.1.smithi203.stdout:0/411: getdents dd/d10/d34/d4c 0 2024-05-25T06:11:31.025 INFO:tasks.workunit.client.0.smithi160.stdout:8/186: creat d5/d7/d8/d1f/d1e/d28/f3f x:0 0 0 2024-05-25T06:11:31.025 INFO:tasks.workunit.client.0.smithi160.stdout:6/135: unlink d5/de/c2f 0 2024-05-25T06:11:31.026 INFO:tasks.workunit.client.0.smithi160.stdout:6/136: write d5/f1e [500702,56858] 0 2024-05-25T06:11:31.027 INFO:tasks.workunit.client.1.smithi203.stdout:2/565: creat d2/d1a/d87/fbe x:0 0 0 2024-05-25T06:11:31.028 INFO:tasks.workunit.client.1.smithi203.stdout:1/486: dread d14/d57/d1d/d3e/d36/f39 [0,4194304] 0 2024-05-25T06:11:31.028 INFO:tasks.workunit.client.1.smithi203.stdout:6/777: mknod d8/d70/d38/cf6 0 2024-05-25T06:11:31.028 INFO:tasks.workunit.client.1.smithi203.stdout:8/630: write d2/d5/d28/d38/d4e/f51 [1069851,77043] 0 2024-05-25T06:11:31.030 INFO:tasks.workunit.client.0.smithi160.stdout:0/108: creat f21 x:0 0 0 2024-05-25T06:11:31.031 INFO:tasks.workunit.client.0.smithi160.stdout:0/109: fdatasync f1f 0 2024-05-25T06:11:31.031 INFO:tasks.workunit.client.0.smithi160.stdout:0/110: chown f16 2478664 1 2024-05-25T06:11:31.032 INFO:tasks.workunit.client.0.smithi160.stdout:4/227: rename d4/d6/f2c to d4/d6/f45 0 2024-05-25T06:11:31.033 INFO:tasks.workunit.client.1.smithi203.stdout:8/631: mknod d2/d5/d18/d31/ccc 0 2024-05-25T06:11:31.034 INFO:tasks.workunit.client.1.smithi203.stdout:6/778: mknod d8/d11/d1b/d75/cf7 0 2024-05-25T06:11:31.034 INFO:tasks.workunit.client.0.smithi160.stdout:3/87: rename f8 to d19/f1e 0 2024-05-25T06:11:31.038 INFO:tasks.workunit.client.0.smithi160.stdout:7/117: symlink d6/dc/d21/l28 0 2024-05-25T06:11:31.040 INFO:tasks.workunit.client.0.smithi160.stdout:0/111: symlink l22 0 2024-05-25T06:11:31.041 INFO:tasks.workunit.client.0.smithi160.stdout:4/228: mknod d4/d6/d21/d42/c46 0 2024-05-25T06:11:31.042 INFO:tasks.workunit.client.0.smithi160.stdout:6/137: dwrite d5/de/f17 [4194304,4194304] 0 2024-05-25T06:11:31.043 INFO:tasks.workunit.client.1.smithi203.stdout:8/632: symlink d2/d5/d18/d31/dac/dc2/d8a/lcd 0 2024-05-25T06:11:31.044 INFO:tasks.workunit.client.1.smithi203.stdout:1/487: dwrite d14/d57/d1d/d3e/f61 [0,4194304] 0 2024-05-25T06:11:31.045 INFO:tasks.workunit.client.0.smithi160.stdout:3/88: chown f18 0 1 2024-05-25T06:11:31.045 INFO:tasks.workunit.client.0.smithi160.stdout:5/133: rmdir db/dc/d27 0 2024-05-25T06:11:31.046 INFO:tasks.workunit.client.1.smithi203.stdout:6/779: rename d8/d11/d15/d28/d52/d43 to d8/d11/d15/d28/d52/d43/df8 22 2024-05-25T06:11:31.046 INFO:tasks.workunit.client.1.smithi203.stdout:0/412: dread dd/d32/d37/f42 [0,4194304] 0 2024-05-25T06:11:31.047 INFO:tasks.workunit.client.1.smithi203.stdout:4/593: dwrite d0/d9/d9d/fbf [0,4194304] 0 2024-05-25T06:11:31.047 INFO:tasks.workunit.client.0.smithi160.stdout:4/229: dread d4/d6/fe [0,4194304] 0 2024-05-25T06:11:31.049 INFO:tasks.workunit.client.1.smithi203.stdout:1/488: chown d14/d81 509655 1 2024-05-25T06:11:31.049 INFO:tasks.workunit.client.0.smithi160.stdout:4/230: write d4/f2a [844191,94871] 0 2024-05-25T06:11:31.049 INFO:tasks.workunit.client.0.smithi160.stdout:4/231: stat d4/f11 0 2024-05-25T06:11:31.050 INFO:tasks.workunit.client.1.smithi203.stdout:8/633: truncate d2/d5/d10/d11/de/fa2 520491 0 2024-05-25T06:11:31.050 INFO:tasks.workunit.client.1.smithi203.stdout:0/413: dread - dd/d5c/f90 zero size 2024-05-25T06:11:31.051 INFO:tasks.workunit.client.1.smithi203.stdout:4/594: readlink d0/d9/d11/la0 0 2024-05-25T06:11:31.053 INFO:tasks.workunit.client.0.smithi160.stdout:7/118: write d6/dc/f24 [2976437,33514] 0 2024-05-25T06:11:31.054 INFO:tasks.workunit.client.0.smithi160.stdout:0/112: creat f23 x:0 0 0 2024-05-25T06:11:31.055 INFO:tasks.workunit.client.1.smithi203.stdout:5/800: dread dc/d11/d17/d1a/d103/d98/f9f [0,4194304] 0 2024-05-25T06:11:31.055 INFO:tasks.workunit.client.1.smithi203.stdout:0/414: symlink dd/d3d/d59/l94 0 2024-05-25T06:11:31.057 INFO:tasks.workunit.client.0.smithi160.stdout:0/113: read f1e [2074057,44500] 0 2024-05-25T06:11:31.059 INFO:tasks.workunit.client.1.smithi203.stdout:4/595: write d0/d8/d3a/d81/fb2 [416287,67357] 0 2024-05-25T06:11:31.059 INFO:tasks.workunit.client.1.smithi203.stdout:8/634: symlink d2/d5/d10/lce 0 2024-05-25T06:11:31.059 INFO:tasks.workunit.client.0.smithi160.stdout:0/114: stat f15 0 2024-05-25T06:11:31.062 INFO:tasks.workunit.client.1.smithi203.stdout:0/415: symlink dd/d3d/l95 0 2024-05-25T06:11:31.062 INFO:tasks.workunit.client.0.smithi160.stdout:5/134: dwrite f6 [0,4194304] 0 2024-05-25T06:11:31.063 INFO:tasks.workunit.client.0.smithi160.stdout:4/232: creat d4/d6/d21/f47 x:0 0 0 2024-05-25T06:11:31.063 INFO:tasks.workunit.client.1.smithi203.stdout:4/596: rmdir d0/d8/d3a/d81/d93/d98 39 2024-05-25T06:11:31.065 INFO:tasks.workunit.client.0.smithi160.stdout:5/135: read - db/dc/d13/f2c zero size 2024-05-25T06:11:31.066 INFO:tasks.workunit.client.0.smithi160.stdout:4/233: write d4/f11 [4676565,7300] 0 2024-05-25T06:11:31.067 INFO:tasks.workunit.client.0.smithi160.stdout:0/115: creat f24 x:0 0 0 2024-05-25T06:11:31.068 INFO:tasks.workunit.client.0.smithi160.stdout:7/119: dwrite d6/dc/f19 [0,4194304] 0 2024-05-25T06:11:31.070 INFO:tasks.workunit.client.0.smithi160.stdout:3/89: rename ld to d19/l1f 0 2024-05-25T06:11:31.072 INFO:tasks.workunit.client.0.smithi160.stdout:0/116: write f1b [172391,59833] 0 2024-05-25T06:11:31.073 INFO:tasks.workunit.client.1.smithi203.stdout:8/635: dwrite d2/d5/d10/d11/d1f/f72 [0,4194304] 0 2024-05-25T06:11:31.073 INFO:tasks.workunit.client.1.smithi203.stdout:0/416: dwrite dd/f5a [0,4194304] 0 2024-05-25T06:11:31.074 INFO:tasks.workunit.client.0.smithi160.stdout:8/187: getdents d5/d9 0 2024-05-25T06:11:31.076 INFO:tasks.workunit.client.0.smithi160.stdout:6/138: creat d5/de/d32/f37 x:0 0 0 2024-05-25T06:11:31.077 INFO:tasks.workunit.client.1.smithi203.stdout:4/597: rename d0/d8/d3a/f49 to d0/d9/d50/fc4 0 2024-05-25T06:11:31.078 INFO:tasks.workunit.client.0.smithi160.stdout:0/117: fsync f1e 0 2024-05-25T06:11:31.079 INFO:tasks.workunit.client.1.smithi203.stdout:4/598: fsync d0/d9/d50/d79/f40 0 2024-05-25T06:11:31.079 INFO:tasks.workunit.client.1.smithi203.stdout:8/636: fdatasync d2/d5/d2d/fbe 0 2024-05-25T06:11:31.079 INFO:tasks.workunit.client.0.smithi160.stdout:3/90: mknod d19/c20 0 2024-05-25T06:11:31.080 INFO:tasks.workunit.client.0.smithi160.stdout:0/118: stat c11 0 2024-05-25T06:11:31.080 INFO:tasks.workunit.client.1.smithi203.stdout:0/417: rmdir dd/d3a/d66/d88 39 2024-05-25T06:11:31.081 INFO:tasks.workunit.client.0.smithi160.stdout:0/119: stat c9 0 2024-05-25T06:11:31.081 INFO:tasks.workunit.client.0.smithi160.stdout:7/120: mknod d6/c29 0 2024-05-25T06:11:31.082 INFO:tasks.workunit.client.0.smithi160.stdout:3/91: chown d19/c1c 556770005 1 2024-05-25T06:11:31.084 INFO:tasks.workunit.client.0.smithi160.stdout:4/234: dwrite d4/f3b [0,4194304] 0 2024-05-25T06:11:31.085 INFO:tasks.workunit.client.0.smithi160.stdout:0/120: chown l8 1751064 1 2024-05-25T06:11:31.085 INFO:tasks.workunit.client.1.smithi203.stdout:0/418: mkdir dd/d45/d96 0 2024-05-25T06:11:31.088 INFO:tasks.workunit.client.1.smithi203.stdout:0/419: truncate dd/d10/d34/d4c/f7e 65859 0 2024-05-25T06:11:31.089 INFO:tasks.workunit.client.0.smithi160.stdout:0/121: write f21 [806319,77438] 0 2024-05-25T06:11:31.089 INFO:tasks.workunit.client.1.smithi203.stdout:8/637: creat d2/d5/d18/d31/fcf x:0 0 0 2024-05-25T06:11:31.091 INFO:tasks.workunit.client.0.smithi160.stdout:3/92: rmdir d19 39 2024-05-25T06:11:31.093 INFO:tasks.workunit.client.0.smithi160.stdout:8/188: dread d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:31.093 INFO:tasks.workunit.client.0.smithi160.stdout:7/121: dread d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:31.095 INFO:tasks.workunit.client.1.smithi203.stdout:8/638: unlink d2/d5/d28/d38/f47 0 2024-05-25T06:11:31.095 INFO:tasks.workunit.client.0.smithi160.stdout:8/189: chown d5/d39/f3b 12066 1 2024-05-25T06:11:31.095 INFO:tasks.workunit.client.1.smithi203.stdout:1/489: write d14/d57/d29/f2b [6981509,32001] 0 2024-05-25T06:11:31.096 INFO:tasks.workunit.client.0.smithi160.stdout:4/235: rename d4/d6/f45 to d4/d6/d21/f48 0 2024-05-25T06:11:31.099 INFO:tasks.workunit.client.0.smithi160.stdout:0/122: readlink l1d 0 2024-05-25T06:11:31.101 INFO:tasks.workunit.client.1.smithi203.stdout:1/490: rmdir d14/d57/d1d/d9f 39 2024-05-25T06:11:31.105 INFO:tasks.workunit.client.1.smithi203.stdout:8/639: symlink d2/d5/d10/d45/d5d/ld0 0 2024-05-25T06:11:31.105 INFO:tasks.workunit.client.0.smithi160.stdout:7/122: creat d6/dc/d21/f2a x:0 0 0 2024-05-25T06:11:31.105 INFO:tasks.workunit.client.0.smithi160.stdout:3/93: dwrite f0 [0,4194304] 0 2024-05-25T06:11:31.106 INFO:tasks.workunit.client.0.smithi160.stdout:8/190: dread d5/d7/d8/f16 [0,4194304] 0 2024-05-25T06:11:31.107 INFO:tasks.workunit.client.0.smithi160.stdout:4/236: mkdir d4/d6/d21/d42/d49 0 2024-05-25T06:11:31.109 INFO:tasks.workunit.client.0.smithi160.stdout:3/94: truncate f13 217748 0 2024-05-25T06:11:31.111 INFO:tasks.workunit.client.1.smithi203.stdout:1/491: creat d14/d57/fa9 x:0 0 0 2024-05-25T06:11:31.112 INFO:tasks.workunit.client.0.smithi160.stdout:0/123: dwrite f20 [0,4194304] 0 2024-05-25T06:11:31.113 INFO:tasks.workunit.client.1.smithi203.stdout:8/640: rename d2/l56 to d2/d5/d10/d11/de/ld1 0 2024-05-25T06:11:31.114 INFO:tasks.workunit.client.0.smithi160.stdout:8/191: unlink d5/d7/c12 0 2024-05-25T06:11:31.114 INFO:tasks.workunit.client.1.smithi203.stdout:1/492: readlink d14/d37/d5e/l8c 0 2024-05-25T06:11:31.115 INFO:tasks.workunit.client.1.smithi203.stdout:0/420: truncate dd/f44 2446418 0 2024-05-25T06:11:31.115 INFO:tasks.workunit.client.1.smithi203.stdout:5/801: dwrite dc/d11/d17/d1a/d22/f39 [0,4194304] 0 2024-05-25T06:11:31.117 INFO:tasks.workunit.client.1.smithi203.stdout:1/493: readlink d14/d37/d5e/l8c 0 2024-05-25T06:11:31.119 INFO:tasks.workunit.client.1.smithi203.stdout:6/780: dread d8/d11/d1b/d75/dcb/dcf/d46/d88/f9c [0,4194304] 0 2024-05-25T06:11:31.120 INFO:tasks.workunit.client.0.smithi160.stdout:0/124: rename fa to f25 0 2024-05-25T06:11:31.121 INFO:tasks.workunit.client.1.smithi203.stdout:0/421: rmdir dd/d5c 39 2024-05-25T06:11:31.123 INFO:tasks.workunit.client.1.smithi203.stdout:4/599: dread d0/d9/d50/fc4 [0,4194304] 0 2024-05-25T06:11:31.123 INFO:tasks.workunit.client.0.smithi160.stdout:6/139: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:31.123 INFO:tasks.workunit.client.0.smithi160.stdout:5/136: dread db/dc/f15 [4194304,4194304] 0 2024-05-25T06:11:31.124 INFO:tasks.workunit.client.0.smithi160.stdout:8/192: creat d5/d9/f40 x:0 0 0 2024-05-25T06:11:31.124 INFO:tasks.workunit.client.0.smithi160.stdout:3/95: symlink d19/l21 0 2024-05-25T06:11:31.126 INFO:tasks.workunit.client.1.smithi203.stdout:1/494: chown d14/d57/d1d/d3e/d36/l92 195 1 2024-05-25T06:11:31.127 INFO:tasks.workunit.client.1.smithi203.stdout:0/422: mknod dd/d3a/d66/d6e/c97 0 2024-05-25T06:11:31.127 INFO:tasks.workunit.client.1.smithi203.stdout:5/802: creat dc/d3c/d3f/f119 x:0 0 0 2024-05-25T06:11:31.130 INFO:tasks.workunit.client.1.smithi203.stdout:1/495: dread - d14/d57/d29/f7b zero size 2024-05-25T06:11:31.133 INFO:tasks.workunit.client.0.smithi160.stdout:8/193: symlink d5/d7/d8/d1f/l41 0 2024-05-25T06:11:31.133 INFO:tasks.workunit.client.1.smithi203.stdout:8/641: rename d2/f99 to d2/d5/d10/d11/d1f/fd2 0 2024-05-25T06:11:31.135 INFO:tasks.workunit.client.0.smithi160.stdout:6/140: link d5/de/d32/c25 d5/c38 0 2024-05-25T06:11:31.136 INFO:tasks.workunit.client.1.smithi203.stdout:0/423: chown dd/d5c/f73 1574319 1 2024-05-25T06:11:31.137 INFO:tasks.workunit.client.1.smithi203.stdout:6/781: link d8/l16 d8/d11/d15/d28/d52/d43/d73/lf9 0 2024-05-25T06:11:31.138 INFO:tasks.workunit.client.0.smithi160.stdout:8/194: creat d5/d7/d8/d1f/f42 x:0 0 0 2024-05-25T06:11:31.138 INFO:tasks.workunit.client.0.smithi160.stdout:5/137: dwrite f1 [0,4194304] 0 2024-05-25T06:11:31.140 INFO:tasks.workunit.client.1.smithi203.stdout:5/803: creat dc/d11/f11a x:0 0 0 2024-05-25T06:11:31.140 INFO:tasks.workunit.client.0.smithi160.stdout:6/141: mkdir d5/d39 0 2024-05-25T06:11:31.140 INFO:tasks.workunit.client.0.smithi160.stdout:5/138: stat db/f22 0 2024-05-25T06:11:31.140 INFO:tasks.workunit.client.0.smithi160.stdout:8/195: chown d5/fb 170867472 1 2024-05-25T06:11:31.140 INFO:tasks.workunit.client.0.smithi160.stdout:3/96: dwrite f15 [0,4194304] 0 2024-05-25T06:11:31.141 INFO:tasks.workunit.client.1.smithi203.stdout:8/642: mknod d2/d5/d18/dbf/cd3 0 2024-05-25T06:11:31.143 INFO:tasks.workunit.client.1.smithi203.stdout:6/782: mkdir d8/d11/dfa 0 2024-05-25T06:11:31.145 INFO:tasks.workunit.client.1.smithi203.stdout:6/783: readlink d8/d2a/da9/lba 0 2024-05-25T06:11:31.146 INFO:tasks.workunit.client.0.smithi160.stdout:8/196: write d5/d9/f40 [617047,94831] 0 2024-05-25T06:11:31.146 INFO:tasks.workunit.client.0.smithi160.stdout:6/142: mknod d5/d30/c3a 0 2024-05-25T06:11:31.146 INFO:tasks.workunit.client.0.smithi160.stdout:0/125: fsync f20 0 2024-05-25T06:11:31.147 INFO:tasks.workunit.client.1.smithi203.stdout:8/643: creat d2/d5/d10/d11/d59/fd4 x:0 0 0 2024-05-25T06:11:31.148 INFO:tasks.workunit.client.1.smithi203.stdout:0/424: getdents dd/d3a/d66/d6e/d70 0 2024-05-25T06:11:31.148 INFO:tasks.workunit.client.0.smithi160.stdout:6/143: chown d5/de/d18/c1d 1990708 1 2024-05-25T06:11:31.149 INFO:tasks.workunit.client.0.smithi160.stdout:5/139: creat db/dc/d13/d1d/d23/f2d x:0 0 0 2024-05-25T06:11:31.149 INFO:tasks.workunit.client.0.smithi160.stdout:5/140: read f8 [2366122,22352] 0 2024-05-25T06:11:31.149 INFO:tasks.workunit.client.1.smithi203.stdout:8/644: dread - d2/d5/d10/fc9 zero size 2024-05-25T06:11:31.150 INFO:tasks.workunit.client.0.smithi160.stdout:8/197: write d5/d7/d8/d1f/f42 [12327,116090] 0 2024-05-25T06:11:31.150 INFO:tasks.workunit.client.0.smithi160.stdout:0/126: rename c14 to c26 0 2024-05-25T06:11:31.150 INFO:tasks.workunit.client.0.smithi160.stdout:7/123: dread d6/fb [0,4194304] 0 2024-05-25T06:11:31.151 INFO:tasks.workunit.client.1.smithi203.stdout:5/804: dwrite dc/d11/d17/f5d [0,4194304] 0 2024-05-25T06:11:31.157 INFO:tasks.workunit.client.1.smithi203.stdout:6/784: truncate d8/d11/f90 1525864 0 2024-05-25T06:11:31.157 INFO:tasks.workunit.client.1.smithi203.stdout:0/425: fsync dd/d3a/d66/f69 0 2024-05-25T06:11:31.159 INFO:tasks.workunit.client.0.smithi160.stdout:4/237: dread d4/d6/d35/f3c [4194304,4194304] 0 2024-05-25T06:11:31.160 INFO:tasks.workunit.client.0.smithi160.stdout:5/141: dread db/dc/f15 [0,4194304] 0 2024-05-25T06:11:31.161 INFO:tasks.workunit.client.1.smithi203.stdout:6/785: write d8/d11/d47/fbe [901972,92944] 0 2024-05-25T06:11:31.163 INFO:tasks.workunit.client.0.smithi160.stdout:8/198: dwrite d5/d9/f40 [0,4194304] 0 2024-05-25T06:11:31.165 INFO:tasks.workunit.client.1.smithi203.stdout:4/600: dread d0/d8/d1f/fb7 [0,4194304] 0 2024-05-25T06:11:31.166 INFO:tasks.workunit.client.0.smithi160.stdout:5/142: fdatasync db/f22 0 2024-05-25T06:11:31.166 INFO:tasks.workunit.client.1.smithi203.stdout:5/805: mknod dc/d11/d17/d1a/d2d/d4d/d101/c11b 0 2024-05-25T06:11:31.167 INFO:tasks.workunit.client.1.smithi203.stdout:5/806: chown f6 111601062 1 2024-05-25T06:11:31.168 INFO:tasks.workunit.client.0.smithi160.stdout:3/97: getdents d19 0 2024-05-25T06:11:31.169 INFO:tasks.workunit.client.0.smithi160.stdout:7/124: mkdir d6/d2b 0 2024-05-25T06:11:31.170 INFO:tasks.workunit.client.1.smithi203.stdout:6/786: write d8/d11/d1b/d75/dcb/dcf/d46/fd2 [10525601,98808] 0 2024-05-25T06:11:31.172 INFO:tasks.workunit.client.0.smithi160.stdout:6/144: rmdir d5/de/d32 39 2024-05-25T06:11:31.174 INFO:tasks.workunit.client.0.smithi160.stdout:8/199: symlink d5/d39/l43 0 2024-05-25T06:11:31.175 INFO:tasks.workunit.client.1.smithi203.stdout:0/426: link dd/d10/d34/f3c dd/d5c/f98 0 2024-05-25T06:11:31.176 INFO:tasks.workunit.client.1.smithi203.stdout:1/496: write d14/d81/f8a [551011,89877] 0 2024-05-25T06:11:31.177 INFO:tasks.workunit.client.1.smithi203.stdout:5/807: creat dc/d11/d17/d44/f11c x:0 0 0 2024-05-25T06:11:31.179 INFO:tasks.workunit.client.1.smithi203.stdout:8/645: rmdir d2/d5/d10/d11/de/da8 0 2024-05-25T06:11:31.181 INFO:tasks.workunit.client.1.smithi203.stdout:6/787: mkdir d8/d11/d1b/d75/dcb/dcf/d80/dfb 0 2024-05-25T06:11:31.181 INFO:tasks.workunit.client.1.smithi203.stdout:6/788: readlink d8/d11/d47/d60/l9d 0 2024-05-25T06:11:31.182 INFO:tasks.workunit.client.0.smithi160.stdout:5/143: fsync f4 0 2024-05-25T06:11:31.183 INFO:tasks.workunit.client.1.smithi203.stdout:6/789: write d8/d5a/d8b/f96 [2283834,77801] 0 2024-05-25T06:11:31.184 INFO:tasks.workunit.client.0.smithi160.stdout:8/200: dwrite d5/d7/d8/d1f/d1e/d28/f3f [0,4194304] 0 2024-05-25T06:11:31.184 INFO:tasks.workunit.client.0.smithi160.stdout:7/125: mkdir d6/d2c 0 2024-05-25T06:11:31.184 INFO:tasks.workunit.client.0.smithi160.stdout:4/238: write d4/d6/f7 [287433,4441] 0 2024-05-25T06:11:31.185 INFO:tasks.workunit.client.0.smithi160.stdout:4/239: chown d4/d6/fe 1196356 1 2024-05-25T06:11:31.186 INFO:tasks.workunit.client.1.smithi203.stdout:8/646: unlink d2/d5/d18/d31/f39 0 2024-05-25T06:11:31.188 INFO:tasks.workunit.client.0.smithi160.stdout:8/201: fsync d5/d7/d8/f16 0 2024-05-25T06:11:31.188 INFO:tasks.workunit.client.0.smithi160.stdout:5/144: mkdir db/dc/d13/d1d/d23/d24/d2e 0 2024-05-25T06:11:31.189 INFO:tasks.workunit.client.0.smithi160.stdout:4/240: mknod d4/d6/d35/c4a 0 2024-05-25T06:11:31.190 INFO:tasks.workunit.client.0.smithi160.stdout:1/73: sync 2024-05-25T06:11:31.190 INFO:tasks.workunit.client.1.smithi203.stdout:1/497: creat d14/d57/d6b/da1/faa x:0 0 0 2024-05-25T06:11:31.192 INFO:tasks.workunit.client.0.smithi160.stdout:8/202: truncate d5/d7/f19 4037501 0 2024-05-25T06:11:31.196 INFO:tasks.workunit.client.0.smithi160.stdout:0/127: truncate f2 3749509 0 2024-05-25T06:11:31.196 INFO:tasks.workunit.client.0.smithi160.stdout:5/145: dwrite db/f22 [0,4194304] 0 2024-05-25T06:11:31.197 INFO:tasks.workunit.client.0.smithi160.stdout:1/74: truncate f7 1080762 0 2024-05-25T06:11:31.199 INFO:tasks.workunit.client.0.smithi160.stdout:4/241: creat d4/d6/d21/d42/d49/f4b x:0 0 0 2024-05-25T06:11:31.201 INFO:tasks.workunit.client.0.smithi160.stdout:4/242: write d4/d6/f22 [569131,96792] 0 2024-05-25T06:11:31.202 INFO:tasks.workunit.client.0.smithi160.stdout:1/75: creat de/f13 x:0 0 0 2024-05-25T06:11:31.203 INFO:tasks.workunit.client.0.smithi160.stdout:5/146: rename db/f26 to db/dc/d13/d1d/d23/d24/d2e/f2f 0 2024-05-25T06:11:31.204 INFO:tasks.workunit.client.0.smithi160.stdout:6/145: write d5/f2c [756816,79585] 0 2024-05-25T06:11:31.206 INFO:tasks.workunit.client.0.smithi160.stdout:7/126: write d6/f12 [164866,111178] 0 2024-05-25T06:11:31.207 INFO:tasks.workunit.client.0.smithi160.stdout:6/146: write d5/f7 [2983612,12185] 0 2024-05-25T06:11:31.208 INFO:tasks.workunit.client.1.smithi203.stdout:3/660: sync 2024-05-25T06:11:31.208 INFO:tasks.workunit.client.0.smithi160.stdout:7/127: readlink d6/dc/d21/l28 0 2024-05-25T06:11:31.209 INFO:tasks.workunit.client.1.smithi203.stdout:8/647: symlink d2/d5/d18/d31/dac/dc2/d88/ld5 0 2024-05-25T06:11:31.210 INFO:tasks.workunit.client.0.smithi160.stdout:8/203: rmdir d5/d7/d8/d27 0 2024-05-25T06:11:31.211 INFO:tasks.workunit.client.1.smithi203.stdout:1/498: fdatasync d14/d57/d29/f2b 0 2024-05-25T06:11:31.211 INFO:tasks.workunit.client.0.smithi160.stdout:5/147: dread f8 [0,4194304] 0 2024-05-25T06:11:31.212 INFO:tasks.workunit.client.1.smithi203.stdout:6/790: mkdir d8/d11/d15/d28/d52/d43/d73/dfc 0 2024-05-25T06:11:31.212 INFO:tasks.workunit.client.0.smithi160.stdout:1/76: creat de/f14 x:0 0 0 2024-05-25T06:11:31.216 INFO:tasks.workunit.client.1.smithi203.stdout:4/601: dread d0/d8/d1f/d26/f1e [8388608,4194304] 0 2024-05-25T06:11:31.216 INFO:tasks.workunit.client.1.smithi203.stdout:1/499: mknod d14/d57/d7e/cab 0 2024-05-25T06:11:31.217 INFO:tasks.workunit.client.1.smithi203.stdout:6/791: chown d8/l16 0 1 2024-05-25T06:11:31.220 INFO:tasks.workunit.client.0.smithi160.stdout:0/128: dread f2 [0,4194304] 0 2024-05-25T06:11:31.220 INFO:tasks.workunit.client.1.smithi203.stdout:3/661: mknod d10/d16/d23/d45/cd4 0 2024-05-25T06:11:31.220 INFO:tasks.workunit.client.0.smithi160.stdout:0/129: chown c11 370 1 2024-05-25T06:11:31.221 INFO:tasks.workunit.client.1.smithi203.stdout:0/427: dwrite dd/d32/f54 [0,4194304] 0 2024-05-25T06:11:31.225 INFO:tasks.workunit.client.1.smithi203.stdout:6/792: chown d8/d11/d47/d60/l58 5679516 1 2024-05-25T06:11:31.225 INFO:tasks.workunit.client.1.smithi203.stdout:3/662: chown d10/d16/d50/d8b/f52 0 1 2024-05-25T06:11:31.225 INFO:tasks.workunit.client.0.smithi160.stdout:5/148: write f9 [1196571,77673] 0 2024-05-25T06:11:31.226 INFO:tasks.workunit.client.0.smithi160.stdout:5/149: chown db/dc 1908437948 1 2024-05-25T06:11:31.226 INFO:tasks.workunit.client.1.smithi203.stdout:6/793: dread - d8/d11/d47/f99 zero size 2024-05-25T06:11:31.226 INFO:tasks.workunit.client.1.smithi203.stdout:3/663: fdatasync d10/d22/f35 0 2024-05-25T06:11:31.226 INFO:tasks.workunit.client.1.smithi203.stdout:0/428: rename dd/d3a/d66/d6e to dd/d10/d34/d4c/d99 0 2024-05-25T06:11:31.228 INFO:tasks.workunit.client.1.smithi203.stdout:4/602: dread d0/d9/f17 [0,4194304] 0 2024-05-25T06:11:31.229 INFO:tasks.workunit.client.1.smithi203.stdout:3/664: dread - d10/d16/d23/d45/f93 zero size 2024-05-25T06:11:31.229 INFO:tasks.workunit.client.0.smithi160.stdout:6/147: mknod d5/de/c3b 0 2024-05-25T06:11:31.229 INFO:tasks.workunit.client.1.smithi203.stdout:8/648: link d2/d5/d10/f53 d2/d5/d18/d31/dac/fd6 0 2024-05-25T06:11:31.230 INFO:tasks.workunit.client.0.smithi160.stdout:4/243: dread d4/d6/d21/f32 [0,4194304] 0 2024-05-25T06:11:31.231 INFO:tasks.workunit.client.0.smithi160.stdout:7/128: mkdir d6/d2b/d2d 0 2024-05-25T06:11:31.232 INFO:tasks.workunit.client.0.smithi160.stdout:4/244: dread - d4/d6/d21/f48 zero size 2024-05-25T06:11:31.233 INFO:tasks.workunit.client.0.smithi160.stdout:1/77: dread fa [0,4194304] 0 2024-05-25T06:11:31.233 INFO:tasks.workunit.client.1.smithi203.stdout:4/603: read d0/d8/d10/d91/da2/fb9 [2451209,4660] 0 2024-05-25T06:11:31.235 INFO:tasks.workunit.client.0.smithi160.stdout:8/204: rename d5/d7/d8 to d5/d44 0 2024-05-25T06:11:31.236 INFO:tasks.workunit.client.1.smithi203.stdout:8/649: mknod d2/d5/d18/d31/dac/dc2/d8a/d8f/cd7 0 2024-05-25T06:11:31.236 INFO:tasks.workunit.client.0.smithi160.stdout:0/130: mknod c27 0 2024-05-25T06:11:31.237 INFO:tasks.workunit.client.1.smithi203.stdout:6/794: link d8/d2a/d6c/lef d8/d11/d15/d28/d52/d43/lfd 0 2024-05-25T06:11:31.239 INFO:tasks.workunit.client.1.smithi203.stdout:3/665: dwrite d10/d16/d23/d46/f79 [0,4194304] 0 2024-05-25T06:11:31.239 INFO:tasks.workunit.client.0.smithi160.stdout:0/131: write f1e [552495,126961] 0 2024-05-25T06:11:31.240 INFO:tasks.workunit.client.0.smithi160.stdout:7/129: dread d6/fb [0,4194304] 0 2024-05-25T06:11:31.241 INFO:tasks.workunit.client.1.smithi203.stdout:4/604: creat d0/d9/d50/fc5 x:0 0 0 2024-05-25T06:11:31.242 INFO:tasks.workunit.client.1.smithi203.stdout:8/650: symlink d2/d5/d18/d31/dac/dc2/ld8 0 2024-05-25T06:11:31.245 INFO:tasks.workunit.client.1.smithi203.stdout:4/605: dread - d0/d9/f94 zero size 2024-05-25T06:11:31.245 INFO:tasks.workunit.client.0.smithi160.stdout:4/245: dwrite d4/d6/d21/f27 [0,4194304] 0 2024-05-25T06:11:31.246 INFO:tasks.workunit.client.0.smithi160.stdout:1/78: dwrite de/ff [4194304,4194304] 0 2024-05-25T06:11:31.248 INFO:tasks.workunit.client.1.smithi203.stdout:8/651: symlink d2/d5/d2d/ld9 0 2024-05-25T06:11:31.251 INFO:tasks.workunit.client.0.smithi160.stdout:8/205: mknod d5/d44/d1f/d1e/c45 0 2024-05-25T06:11:31.252 INFO:tasks.workunit.client.1.smithi203.stdout:3/666: read d10/d16/f19 [3246626,103624] 0 2024-05-25T06:11:31.253 INFO:tasks.workunit.client.1.smithi203.stdout:4/606: creat d0/d9/d9d/fc6 x:0 0 0 2024-05-25T06:11:31.255 INFO:tasks.workunit.client.0.smithi160.stdout:0/132: dwrite f10 [4194304,4194304] 0 2024-05-25T06:11:31.258 INFO:tasks.workunit.client.0.smithi160.stdout:5/150: creat db/dc/f30 x:0 0 0 2024-05-25T06:11:31.260 INFO:tasks.workunit.client.0.smithi160.stdout:4/246: getdents d4/d6/d35/d3d 0 2024-05-25T06:11:31.261 INFO:tasks.workunit.client.0.smithi160.stdout:6/148: getdents d5/de/d18 0 2024-05-25T06:11:31.263 INFO:tasks.workunit.client.0.smithi160.stdout:8/206: mknod d5/d44/d1f/d1e/d28/c46 0 2024-05-25T06:11:31.263 INFO:tasks.workunit.client.0.smithi160.stdout:4/247: chown d4/d6/d21/f3f 30 1 2024-05-25T06:11:31.266 INFO:tasks.workunit.client.0.smithi160.stdout:6/149: rmdir d5 39 2024-05-25T06:11:31.267 INFO:tasks.workunit.client.1.smithi203.stdout:0/429: dread dd/d3d/d4f/f50 [0,4194304] 0 2024-05-25T06:11:31.268 INFO:tasks.workunit.client.0.smithi160.stdout:5/151: dwrite db/f2a [0,4194304] 0 2024-05-25T06:11:31.270 INFO:tasks.workunit.client.1.smithi203.stdout:0/430: write dd/d3d/d4f/d62/d78/f7d [807134,78144] 0 2024-05-25T06:11:31.272 INFO:tasks.workunit.client.1.smithi203.stdout:0/431: write dd/d3d/d4f/d62/d78/f7d [1776694,53079] 0 2024-05-25T06:11:31.273 INFO:tasks.workunit.client.0.smithi160.stdout:8/207: dwrite d5/d44/f37 [0,4194304] 0 2024-05-25T06:11:31.275 INFO:tasks.workunit.client.0.smithi160.stdout:5/152: fdatasync db/dc/d13/d1d/d23/d24/d2e/f2f 0 2024-05-25T06:11:31.275 INFO:tasks.workunit.client.0.smithi160.stdout:4/248: getdents d4/d6/d21/d42/d49 0 2024-05-25T06:11:31.277 INFO:tasks.workunit.client.1.smithi203.stdout:0/432: mknod dd/d3d/d75/d93/c9a 0 2024-05-25T06:11:31.278 INFO:tasks.workunit.client.1.smithi203.stdout:0/433: readlink dd/l43 0 2024-05-25T06:11:31.281 INFO:tasks.workunit.client.1.smithi203.stdout:0/434: creat dd/d3d/d59/f9b x:0 0 0 2024-05-25T06:11:31.284 INFO:tasks.workunit.client.0.smithi160.stdout:8/208: dwrite d5/d44/d1f/d1e/d28/f38 [0,4194304] 0 2024-05-25T06:11:31.286 INFO:tasks.workunit.client.0.smithi160.stdout:5/153: dwrite db/d1c/f2b [0,4194304] 0 2024-05-25T06:11:31.287 INFO:tasks.workunit.client.1.smithi203.stdout:0/435: creat dd/d3d/d4f/d62/f9c x:0 0 0 2024-05-25T06:11:31.290 INFO:tasks.workunit.client.1.smithi203.stdout:0/436: truncate dd/d3d/d4f/d62/f74 1808166 0 2024-05-25T06:11:31.291 INFO:tasks.workunit.client.0.smithi160.stdout:5/154: write db/dc/d13/d1d/f20 [2874486,35254] 0 2024-05-25T06:11:31.292 INFO:tasks.workunit.client.1.smithi203.stdout:6/795: dread f0 [0,4194304] 0 2024-05-25T06:11:31.296 INFO:tasks.workunit.client.0.smithi160.stdout:8/209: dwrite d5/d9/f26 [0,4194304] 0 2024-05-25T06:11:31.296 INFO:tasks.workunit.client.0.smithi160.stdout:7/130: truncate d6/dc/f15 953097 0 2024-05-25T06:11:31.297 INFO:tasks.workunit.client.1.smithi203.stdout:6/796: fdatasync d8/d5a/d9b/fd4 0 2024-05-25T06:11:31.300 INFO:tasks.workunit.client.1.smithi203.stdout:8/652: truncate d2/d5/f44 1253543 0 2024-05-25T06:11:31.300 INFO:tasks.workunit.client.1.smithi203.stdout:4/607: write d0/f16 [5833614,129887] 0 2024-05-25T06:11:31.301 INFO:tasks.workunit.client.1.smithi203.stdout:6/797: mknod d8/d11/d47/d60/cfe 0 2024-05-25T06:11:31.302 INFO:tasks.workunit.client.0.smithi160.stdout:5/155: mkdir db/dc/d13/d1d/d31 0 2024-05-25T06:11:31.304 INFO:tasks.workunit.client.0.smithi160.stdout:5/156: write db/f2a [2286389,40869] 0 2024-05-25T06:11:31.306 INFO:tasks.workunit.client.0.smithi160.stdout:0/133: truncate f25 315941 0 2024-05-25T06:11:31.308 INFO:tasks.workunit.client.1.smithi203.stdout:8/653: rename d2/d5/d10/d4a/c6a to d2/d5/d28/d38/d4e/d97/da1/cda 0 2024-05-25T06:11:31.308 INFO:tasks.workunit.client.1.smithi203.stdout:0/437: dread dd/d5c/f98 [0,4194304] 0 2024-05-25T06:11:31.309 INFO:tasks.workunit.client.1.smithi203.stdout:6/798: creat d8/d11/d15/d28/d52/d43/d73/db7/fff x:0 0 0 2024-05-25T06:11:31.311 INFO:tasks.workunit.client.1.smithi203.stdout:4/608: link d0/d8/f44 d0/d9/d11/d54/fc7 0 2024-05-25T06:11:31.311 INFO:tasks.workunit.client.0.smithi160.stdout:6/150: write d5/f14 [4922668,85644] 0 2024-05-25T06:11:31.311 INFO:tasks.workunit.client.0.smithi160.stdout:1/79: dwrite f7 [0,4194304] 0 2024-05-25T06:11:31.312 INFO:tasks.workunit.client.1.smithi203.stdout:4/609: dread - d0/d8/d10/f7f zero size 2024-05-25T06:11:31.314 INFO:tasks.workunit.client.0.smithi160.stdout:0/134: rename c1a to c28 0 2024-05-25T06:11:31.314 INFO:tasks.workunit.client.1.smithi203.stdout:6/799: creat d8/d5a/d9b/dcc/f100 x:0 0 0 2024-05-25T06:11:31.317 INFO:tasks.workunit.client.0.smithi160.stdout:8/210: dwrite d5/d39/f3b [0,4194304] 0 2024-05-25T06:11:31.318 INFO:tasks.workunit.client.1.smithi203.stdout:6/800: fdatasync d8/d11/f8e 0 2024-05-25T06:11:31.319 INFO:tasks.workunit.client.0.smithi160.stdout:5/157: dread db/d1c/f2b [0,4194304] 0 2024-05-25T06:11:31.321 INFO:tasks.workunit.client.0.smithi160.stdout:5/158: chown db/d1c 41 1 2024-05-25T06:11:31.321 INFO:tasks.workunit.client.0.smithi160.stdout:6/151: unlink d5/fb 0 2024-05-25T06:11:31.322 INFO:tasks.workunit.client.1.smithi203.stdout:4/610: dwrite d0/f9f [0,4194304] 0 2024-05-25T06:11:31.323 INFO:tasks.workunit.client.0.smithi160.stdout:5/159: write db/dc/d13/f2c [310014,30853] 0 2024-05-25T06:11:31.325 INFO:tasks.workunit.client.0.smithi160.stdout:0/135: mknod c29 0 2024-05-25T06:11:31.325 INFO:tasks.workunit.client.1.smithi203.stdout:3/667: truncate d10/d16/d50/f9e 3968814 0 2024-05-25T06:11:31.326 INFO:tasks.workunit.client.0.smithi160.stdout:8/211: symlink d5/l47 0 2024-05-25T06:11:31.326 INFO:tasks.workunit.client.0.smithi160.stdout:0/136: chown c18 31360738 1 2024-05-25T06:11:31.327 INFO:tasks.workunit.client.1.smithi203.stdout:3/668: dread - d10/d16/d50/d8b/db3/fc7 zero size 2024-05-25T06:11:31.328 INFO:tasks.workunit.client.0.smithi160.stdout:1/80: dwrite f2 [0,4194304] 0 2024-05-25T06:11:31.329 INFO:tasks.workunit.client.0.smithi160.stdout:1/81: chown de/ff 325936257 1 2024-05-25T06:11:31.329 INFO:tasks.workunit.client.1.smithi203.stdout:4/611: write d0/d8/d1f/d26/f2a [183454,93087] 0 2024-05-25T06:11:31.332 INFO:tasks.workunit.client.0.smithi160.stdout:0/137: symlink l2a 0 2024-05-25T06:11:31.333 INFO:tasks.workunit.client.0.smithi160.stdout:8/212: creat d5/f48 x:0 0 0 2024-05-25T06:11:31.334 INFO:tasks.workunit.client.0.smithi160.stdout:8/213: fdatasync d5/d9/fc 0 2024-05-25T06:11:31.335 INFO:tasks.workunit.client.0.smithi160.stdout:6/152: symlink d5/de/d32/d1b/l3c 0 2024-05-25T06:11:31.337 INFO:tasks.workunit.client.0.smithi160.stdout:0/138: mkdir d2b 0 2024-05-25T06:11:31.338 INFO:tasks.workunit.client.1.smithi203.stdout:3/669: dwrite d10/d16/d23/d46/faf [0,4194304] 0 2024-05-25T06:11:31.339 INFO:tasks.workunit.client.1.smithi203.stdout:4/612: rename d0/d8/d1f/d26/d3b/d9c/cae to d0/d8/d3a/d81/d93/d98/cc8 0 2024-05-25T06:11:31.340 INFO:tasks.workunit.client.1.smithi203.stdout:3/670: write d10/d16/d23/d38/f3e [3098859,41792] 0 2024-05-25T06:11:31.341 INFO:tasks.workunit.client.0.smithi160.stdout:8/214: mkdir d5/d44/d1f/d1e/d28/d49 0 2024-05-25T06:11:31.341 INFO:tasks.workunit.client.1.smithi203.stdout:3/671: chown f8 11062692 1 2024-05-25T06:11:31.344 INFO:tasks.workunit.client.1.smithi203.stdout:4/613: write d0/d8/d1f/d26/d3b/f5a [266778,59160] 0 2024-05-25T06:11:31.344 INFO:tasks.workunit.client.0.smithi160.stdout:1/82: dwrite f9 [0,4194304] 0 2024-05-25T06:11:31.344 INFO:tasks.workunit.client.0.smithi160.stdout:5/160: dwrite db/d1c/f2b [4194304,4194304] 0 2024-05-25T06:11:31.344 INFO:tasks.workunit.client.0.smithi160.stdout:8/215: stat d5/d44/d1f/d1e/c3c 0 2024-05-25T06:11:31.344 INFO:tasks.workunit.client.0.smithi160.stdout:6/153: unlink d5/de/l23 0 2024-05-25T06:11:31.346 INFO:tasks.workunit.client.1.smithi203.stdout:4/614: write d0/d8/d1f/d26/d3b/f97 [789594,90415] 0 2024-05-25T06:11:31.347 INFO:tasks.workunit.client.0.smithi160.stdout:5/161: write f1 [1621409,84593] 0 2024-05-25T06:11:31.348 INFO:tasks.workunit.client.1.smithi203.stdout:3/672: unlink d10/d16/d71/la0 0 2024-05-25T06:11:31.348 INFO:tasks.workunit.client.1.smithi203.stdout:4/615: chown d0/d9/f9e 3212 1 2024-05-25T06:11:31.349 INFO:tasks.workunit.client.1.smithi203.stdout:4/616: write d0/d9/d50/d6b/f88 [4122065,28998] 0 2024-05-25T06:11:31.351 INFO:tasks.workunit.client.0.smithi160.stdout:0/139: unlink c18 0 2024-05-25T06:11:31.351 INFO:tasks.workunit.client.0.smithi160.stdout:6/154: write d5/de/d32/d1b/f2d [552407,13018] 0 2024-05-25T06:11:31.354 INFO:tasks.workunit.client.1.smithi203.stdout:8/654: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:31.355 INFO:tasks.workunit.client.0.smithi160.stdout:6/155: write d5/de/f12 [270234,18367] 0 2024-05-25T06:11:31.356 INFO:tasks.workunit.client.1.smithi203.stdout:3/673: rename d10/f9a to d10/d16/d23/d46/fd5 0 2024-05-25T06:11:31.362 INFO:tasks.workunit.client.0.smithi160.stdout:5/162: dwrite db/dc/f30 [0,4194304] 0 2024-05-25T06:11:31.367 INFO:tasks.workunit.client.0.smithi160.stdout:0/140: mknod d2b/c2c 0 2024-05-25T06:11:31.367 INFO:tasks.workunit.client.0.smithi160.stdout:1/83: dwrite f7 [0,4194304] 0 2024-05-25T06:11:31.368 INFO:tasks.workunit.client.0.smithi160.stdout:0/141: write f20 [3494049,62093] 0 2024-05-25T06:11:31.374 INFO:tasks.workunit.client.1.smithi203.stdout:8/655: write d2/d5/d28/d38/d4e/d97/fbd [4322202,59998] 0 2024-05-25T06:11:31.375 INFO:tasks.workunit.client.0.smithi160.stdout:6/156: truncate d5/de/f22 1333624 0 2024-05-25T06:11:31.379 INFO:tasks.workunit.client.0.smithi160.stdout:6/157: creat d5/d30/f3d x:0 0 0 2024-05-25T06:11:31.379 INFO:tasks.workunit.client.0.smithi160.stdout:1/84: link fa de/f15 0 2024-05-25T06:11:31.379 INFO:tasks.workunit.client.0.smithi160.stdout:6/158: stat d5/l13 0 2024-05-25T06:11:31.380 INFO:tasks.workunit.client.1.smithi203.stdout:0/438: dread dd/d10/f31 [0,4194304] 0 2024-05-25T06:11:31.382 INFO:tasks.workunit.client.1.smithi203.stdout:0/439: write dd/d10/f29 [8212203,58150] 0 2024-05-25T06:11:31.385 INFO:tasks.workunit.client.0.smithi160.stdout:1/85: dwrite f7 [0,4194304] 0 2024-05-25T06:11:31.388 INFO:tasks.workunit.client.0.smithi160.stdout:1/86: creat de/f16 x:0 0 0 2024-05-25T06:11:31.390 INFO:tasks.workunit.client.1.smithi203.stdout:4/617: creat d0/d8/d1f/d26/fc9 x:0 0 0 2024-05-25T06:11:31.391 INFO:tasks.workunit.client.0.smithi160.stdout:1/87: dread f5 [0,4194304] 0 2024-05-25T06:11:31.391 INFO:tasks.workunit.client.0.smithi160.stdout:1/88: dread - de/f13 zero size 2024-05-25T06:11:31.392 INFO:tasks.workunit.client.0.smithi160.stdout:2/111: sync 2024-05-25T06:11:31.392 INFO:tasks.workunit.client.0.smithi160.stdout:9/75: sync 2024-05-25T06:11:31.393 INFO:tasks.workunit.client.1.smithi203.stdout:0/440: fsync dd/d10/f33 0 2024-05-25T06:11:31.396 INFO:tasks.workunit.client.1.smithi203.stdout:8/656: link d2/d5/c23 d2/d5/d28/d38/d81/cdb 0 2024-05-25T06:11:31.399 INFO:tasks.workunit.client.0.smithi160.stdout:4/249: sync 2024-05-25T06:11:31.399 INFO:tasks.workunit.client.0.smithi160.stdout:3/98: sync 2024-05-25T06:11:31.399 INFO:tasks.workunit.client.1.smithi203.stdout:0/441: mkdir dd/d3a/d66/d9d 0 2024-05-25T06:11:31.402 INFO:tasks.workunit.client.0.smithi160.stdout:9/76: rename d4/d9/df to d4/d11/d12 0 2024-05-25T06:11:31.403 INFO:tasks.workunit.client.1.smithi203.stdout:8/657: rename d2/d5/d10/d11/de/f13 to d2/d5/d28/d38/d4e/d97/da1/fdc 0 2024-05-25T06:11:31.405 INFO:tasks.workunit.client.1.smithi203.stdout:4/618: getdents d0/d8/d52 0 2024-05-25T06:11:31.408 INFO:tasks.workunit.client.1.smithi203.stdout:0/442: dwrite dd/d3d/d4f/d62/f74 [0,4194304] 0 2024-05-25T06:11:31.410 INFO:tasks.workunit.client.0.smithi160.stdout:4/250: unlink d4/d6/d21/l36 0 2024-05-25T06:11:31.413 INFO:tasks.workunit.client.0.smithi160.stdout:7/131: write d6/dc/f24 [870973,120237] 0 2024-05-25T06:11:31.415 INFO:tasks.workunit.client.1.smithi203.stdout:4/619: write d0/d9/f17 [2886367,1648] 0 2024-05-25T06:11:31.415 INFO:tasks.workunit.client.1.smithi203.stdout:9/505: sync 2024-05-25T06:11:31.416 INFO:tasks.workunit.client.1.smithi203.stdout:7/523: sync 2024-05-25T06:11:31.416 INFO:tasks.workunit.client.1.smithi203.stdout:2/566: sync 2024-05-25T06:11:31.418 INFO:tasks.workunit.client.0.smithi160.stdout:3/99: mkdir d19/d22 0 2024-05-25T06:11:31.418 INFO:tasks.workunit.client.1.smithi203.stdout:4/620: stat d0/d8/d3a 0 2024-05-25T06:11:31.419 INFO:tasks.workunit.client.0.smithi160.stdout:9/77: write f2 [8882031,81983] 0 2024-05-25T06:11:31.421 INFO:tasks.workunit.client.1.smithi203.stdout:2/567: stat d2/d4/d10/la8 0 2024-05-25T06:11:31.422 INFO:tasks.workunit.client.0.smithi160.stdout:9/78: write d4/d9/fc [764761,83393] 0 2024-05-25T06:11:31.422 INFO:tasks.workunit.client.0.smithi160.stdout:7/132: unlink d6/dc/f16 0 2024-05-25T06:11:31.422 INFO:tasks.workunit.client.1.smithi203.stdout:0/443: dwrite dd/d5c/f90 [0,4194304] 0 2024-05-25T06:11:31.422 INFO:tasks.workunit.client.1.smithi203.stdout:7/524: mkdir d1/d9/d2c/d62/db3 0 2024-05-25T06:11:31.423 INFO:tasks.workunit.client.0.smithi160.stdout:7/133: stat d6/c26 0 2024-05-25T06:11:31.425 INFO:tasks.workunit.client.1.smithi203.stdout:2/568: write d2/d1a/d33/d80/fb5 [332338,97697] 0 2024-05-25T06:11:31.426 INFO:tasks.workunit.client.0.smithi160.stdout:5/163: truncate db/d1c/f2b 1314489 0 2024-05-25T06:11:31.429 INFO:tasks.workunit.client.0.smithi160.stdout:4/251: dread d4/d6/d28/f25 [0,4194304] 0 2024-05-25T06:11:31.430 INFO:tasks.workunit.client.0.smithi160.stdout:7/134: truncate d6/dc/f15 1280979 0 2024-05-25T06:11:31.431 INFO:tasks.workunit.client.0.smithi160.stdout:3/100: dwrite f11 [0,4194304] 0 2024-05-25T06:11:31.433 INFO:tasks.workunit.client.1.smithi203.stdout:7/525: fsync d1/d11/d20/f23 0 2024-05-25T06:11:31.434 INFO:tasks.workunit.client.0.smithi160.stdout:7/135: stat d6/dc/l1f 0 2024-05-25T06:11:31.436 INFO:tasks.workunit.client.0.smithi160.stdout:6/159: truncate d5/f1e 136305 0 2024-05-25T06:11:31.437 INFO:tasks.workunit.client.0.smithi160.stdout:4/252: write d4/d6/d21/f47 [156369,27085] 0 2024-05-25T06:11:31.437 INFO:tasks.workunit.client.0.smithi160.stdout:3/101: mknod d19/c23 0 2024-05-25T06:11:31.437 INFO:tasks.workunit.client.0.smithi160.stdout:8/216: dwrite d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:31.440 INFO:tasks.workunit.client.0.smithi160.stdout:8/217: readlink d5/d44/d1f/l41 0 2024-05-25T06:11:31.441 INFO:tasks.workunit.client.1.smithi203.stdout:3/674: dwrite d10/d16/d23/d6c/f63 [0,4194304] 0 2024-05-25T06:11:31.442 INFO:tasks.workunit.client.1.smithi203.stdout:9/506: rename d0/d17/d20/d6e/d75/d78 to d0/d17/d20/d3c/db2 0 2024-05-25T06:11:31.447 INFO:tasks.workunit.client.0.smithi160.stdout:7/136: mknod d6/c2e 0 2024-05-25T06:11:31.447 INFO:tasks.workunit.client.0.smithi160.stdout:2/112: dwrite ff [0,4194304] 0 2024-05-25T06:11:31.452 INFO:tasks.workunit.client.0.smithi160.stdout:5/164: dwrite db/dc/d13/f21 [0,4194304] 0 2024-05-25T06:11:31.456 INFO:tasks.workunit.client.0.smithi160.stdout:4/253: dread d4/d6/d28/f25 [0,4194304] 0 2024-05-25T06:11:31.457 INFO:tasks.workunit.client.1.smithi203.stdout:8/658: dread d2/d5/d28/d38/d4e/d97/fbd [0,4194304] 0 2024-05-25T06:11:31.459 INFO:tasks.workunit.client.0.smithi160.stdout:8/218: truncate d5/d9/f24 125987 0 2024-05-25T06:11:31.459 INFO:tasks.workunit.client.1.smithi203.stdout:4/621: write d0/d8/d3a/f77 [1100533,32341] 0 2024-05-25T06:11:31.462 INFO:tasks.workunit.client.1.smithi203.stdout:7/526: creat d1/d9/fb4 x:0 0 0 2024-05-25T06:11:31.463 INFO:tasks.workunit.client.1.smithi203.stdout:8/659: symlink d2/d5/d28/d38/d4e/d97/da1/ldd 0 2024-05-25T06:11:31.464 INFO:tasks.workunit.client.0.smithi160.stdout:4/254: write d4/f11 [4562029,17817] 0 2024-05-25T06:11:31.468 INFO:tasks.workunit.client.1.smithi203.stdout:8/660: read d2/d5/d10/d11/d1f/f72 [1268467,122331] 0 2024-05-25T06:11:31.470 INFO:tasks.workunit.client.0.smithi160.stdout:7/137: dwrite d6/f12 [0,4194304] 0 2024-05-25T06:11:31.472 INFO:tasks.workunit.client.0.smithi160.stdout:3/102: creat d19/d22/f24 x:0 0 0 2024-05-25T06:11:31.473 INFO:tasks.workunit.client.0.smithi160.stdout:5/165: creat db/dc/d13/d1d/d23/d24/d2e/f32 x:0 0 0 2024-05-25T06:11:31.477 INFO:tasks.workunit.client.0.smithi160.stdout:8/219: rename d5/d30 to d5/d44/d1f/d1e/d28/d4a 0 2024-05-25T06:11:31.477 INFO:tasks.workunit.client.1.smithi203.stdout:2/569: dread d2/f6f [0,4194304] 0 2024-05-25T06:11:31.480 INFO:tasks.workunit.client.0.smithi160.stdout:2/113: dwrite f11 [0,4194304] 0 2024-05-25T06:11:31.481 INFO:tasks.workunit.client.0.smithi160.stdout:8/220: write d5/f3e [892620,14659] 0 2024-05-25T06:11:31.487 INFO:tasks.workunit.client.0.smithi160.stdout:5/166: dread f8 [0,4194304] 0 2024-05-25T06:11:31.488 INFO:tasks.workunit.client.0.smithi160.stdout:5/167: read db/dc/f15 [1450389,64909] 0 2024-05-25T06:11:31.490 INFO:tasks.workunit.client.0.smithi160.stdout:9/79: truncate f2 3066586 0 2024-05-25T06:11:31.493 INFO:tasks.workunit.client.0.smithi160.stdout:8/221: dwrite d5/d9/fc [0,4194304] 0 2024-05-25T06:11:31.495 INFO:tasks.workunit.client.0.smithi160.stdout:8/222: fdatasync d5/d44/d1f/d1e/d28/d4a/f34 0 2024-05-25T06:11:31.498 INFO:tasks.workunit.client.0.smithi160.stdout:5/168: creat db/dc/d13/d1d/d23/f33 x:0 0 0 2024-05-25T06:11:31.499 INFO:tasks.workunit.client.0.smithi160.stdout:7/138: link d6/dc/l1f d6/l2f 0 2024-05-25T06:11:31.502 INFO:tasks.workunit.client.0.smithi160.stdout:2/114: link ld d12/l21 0 2024-05-25T06:11:31.507 INFO:tasks.workunit.client.0.smithi160.stdout:8/223: rmdir d5/d9 39 2024-05-25T06:11:31.508 INFO:tasks.workunit.client.0.smithi160.stdout:5/169: dread fa [0,4194304] 0 2024-05-25T06:11:31.509 INFO:tasks.workunit.client.0.smithi160.stdout:0/142: dwrite f25 [0,4194304] 0 2024-05-25T06:11:31.509 INFO:tasks.workunit.client.0.smithi160.stdout:7/139: symlink d6/dc/d21/l30 0 2024-05-25T06:11:31.510 INFO:tasks.workunit.client.0.smithi160.stdout:6/160: truncate d5/de/ff 2076285 0 2024-05-25T06:11:31.510 INFO:tasks.workunit.client.1.smithi203.stdout:4/622: stat d0/d9/d11/f66 0 2024-05-25T06:11:31.511 INFO:tasks.workunit.client.0.smithi160.stdout:5/170: chown db/dc/f30 4814 1 2024-05-25T06:11:31.513 INFO:tasks.workunit.client.0.smithi160.stdout:7/140: dread - d6/dc/f25 zero size 2024-05-25T06:11:31.513 INFO:tasks.workunit.client.0.smithi160.stdout:0/143: write f20 [81717,20463] 0 2024-05-25T06:11:31.515 INFO:tasks.workunit.client.1.smithi203.stdout:0/444: dwrite dd/f44 [0,4194304] 0 2024-05-25T06:11:31.516 INFO:tasks.workunit.client.0.smithi160.stdout:7/141: write d6/dc/f24 [823421,23247] 0 2024-05-25T06:11:31.516 INFO:tasks.workunit.client.1.smithi203.stdout:0/445: stat dd/d45/d96 0 2024-05-25T06:11:31.521 INFO:tasks.workunit.client.0.smithi160.stdout:4/255: truncate d4/d6/d21/f32 1649438 0 2024-05-25T06:11:31.521 INFO:tasks.workunit.client.0.smithi160.stdout:5/171: dread db/dc/d13/f21 [0,4194304] 0 2024-05-25T06:11:31.521 INFO:tasks.workunit.client.0.smithi160.stdout:0/144: fdatasync f2 0 2024-05-25T06:11:31.522 INFO:tasks.workunit.client.0.smithi160.stdout:4/256: stat d4/d1a/f40 0 2024-05-25T06:11:31.527 INFO:tasks.workunit.client.0.smithi160.stdout:8/224: creat d5/d9/f4b x:0 0 0 2024-05-25T06:11:31.529 INFO:tasks.workunit.client.0.smithi160.stdout:6/161: write d5/de/d32/f37 [476557,27007] 0 2024-05-25T06:11:31.531 INFO:tasks.workunit.client.1.smithi203.stdout:3/675: getdents d10/d16 0 2024-05-25T06:11:31.532 INFO:tasks.workunit.client.0.smithi160.stdout:4/257: dwrite d4/f11 [4194304,4194304] 0 2024-05-25T06:11:31.536 INFO:tasks.workunit.client.0.smithi160.stdout:4/258: mkdir d4/d6/d21/d4c 0 2024-05-25T06:11:31.536 INFO:tasks.workunit.client.0.smithi160.stdout:8/225: mknod d5/d44/d1f/d1e/d28/d49/c4c 0 2024-05-25T06:11:31.537 INFO:tasks.workunit.client.0.smithi160.stdout:6/162: mknod d5/d39/c3e 0 2024-05-25T06:11:31.539 INFO:tasks.workunit.client.1.smithi203.stdout:9/507: creat d0/df/d37/fb3 x:0 0 0 2024-05-25T06:11:31.540 INFO:tasks.workunit.client.0.smithi160.stdout:4/259: unlink d4/d6/d21/f47 0 2024-05-25T06:11:31.542 INFO:tasks.workunit.client.1.smithi203.stdout:0/446: unlink dd/d10/f41 0 2024-05-25T06:11:31.543 INFO:tasks.workunit.client.1.smithi203.stdout:4/623: mknod d0/d8/d3a/d81/d93/d98/d99/cca 0 2024-05-25T06:11:31.544 INFO:tasks.workunit.client.0.smithi160.stdout:6/163: unlink d5/de/f2a 0 2024-05-25T06:11:31.546 INFO:tasks.workunit.client.1.smithi203.stdout:7/527: dread d1/d9/f29 [0,4194304] 0 2024-05-25T06:11:31.547 INFO:tasks.workunit.client.0.smithi160.stdout:4/260: symlink d4/d6/d21/l4d 0 2024-05-25T06:11:31.547 INFO:tasks.workunit.client.0.smithi160.stdout:9/80: dread f2 [0,4194304] 0 2024-05-25T06:11:31.547 INFO:tasks.workunit.client.0.smithi160.stdout:8/226: dwrite d5/d9/f40 [0,4194304] 0 2024-05-25T06:11:31.548 INFO:tasks.workunit.client.0.smithi160.stdout:9/81: chown d4/d11/d12 0 1 2024-05-25T06:11:31.549 INFO:tasks.workunit.client.1.smithi203.stdout:3/676: symlink d10/d22/ld6 0 2024-05-25T06:11:31.550 INFO:tasks.workunit.client.1.smithi203.stdout:3/677: stat d10/d16/d50/f64 0 2024-05-25T06:11:31.550 INFO:tasks.workunit.client.0.smithi160.stdout:9/82: chown d4/d9/fc 213 1 2024-05-25T06:11:31.551 INFO:tasks.workunit.client.1.smithi203.stdout:3/678: dread - d10/d16/d23/d46/fbf zero size 2024-05-25T06:11:31.551 INFO:tasks.workunit.client.0.smithi160.stdout:0/145: dread f15 [0,4194304] 0 2024-05-25T06:11:31.554 INFO:tasks.workunit.client.0.smithi160.stdout:0/146: write f25 [1487503,39957] 0 2024-05-25T06:11:31.554 INFO:tasks.workunit.client.0.smithi160.stdout:8/227: creat d5/d44/d1f/d1e/d28/f4d x:0 0 0 2024-05-25T06:11:31.554 INFO:tasks.workunit.client.1.smithi203.stdout:9/508: creat d0/d17/d20/d3c/db2/d7a/fb4 x:0 0 0 2024-05-25T06:11:31.556 INFO:tasks.workunit.client.0.smithi160.stdout:4/261: creat d4/d6/d28/f4e x:0 0 0 2024-05-25T06:11:31.558 INFO:tasks.workunit.client.0.smithi160.stdout:7/142: truncate f1 774087 0 2024-05-25T06:11:31.558 INFO:tasks.workunit.client.1.smithi203.stdout:4/624: chown d0/fa8 5 1 2024-05-25T06:11:31.559 INFO:tasks.workunit.client.1.smithi203.stdout:8/661: creat d2/d5/d28/d38/fde x:0 0 0 2024-05-25T06:11:31.560 INFO:tasks.workunit.client.1.smithi203.stdout:2/570: link d2/d38/d2f/l47 d2/lbf 0 2024-05-25T06:11:31.562 INFO:tasks.workunit.client.1.smithi203.stdout:5/808: sync 2024-05-25T06:11:31.562 INFO:tasks.workunit.client.1.smithi203.stdout:8/662: stat d2/d5/d10/d45/d5d/d76 0 2024-05-25T06:11:31.562 INFO:tasks.workunit.client.0.smithi160.stdout:0/147: dread f1b [0,4194304] 0 2024-05-25T06:11:31.563 INFO:tasks.workunit.client.0.smithi160.stdout:7/143: write d6/dc/f25 [191575,129625] 0 2024-05-25T06:11:31.567 INFO:tasks.workunit.client.0.smithi160.stdout:9/83: dwrite d4/fa [0,4194304] 0 2024-05-25T06:11:31.568 INFO:tasks.workunit.client.1.smithi203.stdout:4/625: dwrite d0/d8/d10/d91/fba [0,4194304] 0 2024-05-25T06:11:31.570 INFO:tasks.workunit.client.0.smithi160.stdout:0/148: rename c1 to d2b/c2d 0 2024-05-25T06:11:31.571 INFO:tasks.workunit.client.1.smithi203.stdout:8/663: dread d2/d5/f8 [0,4194304] 0 2024-05-25T06:11:31.571 INFO:tasks.workunit.client.1.smithi203.stdout:4/626: readlink d0/d9/d11/l74 0 2024-05-25T06:11:31.572 INFO:tasks.workunit.client.1.smithi203.stdout:4/627: fdatasync d0/d9/d50/d79/f5f 0 2024-05-25T06:11:31.573 INFO:tasks.workunit.client.0.smithi160.stdout:0/149: stat f16 0 2024-05-25T06:11:31.577 INFO:tasks.workunit.client.1.smithi203.stdout:2/571: write d2/d4/d10/d5f/f25 [523753,7659] 0 2024-05-25T06:11:31.578 INFO:tasks.workunit.client.0.smithi160.stdout:0/150: dread f1b [0,4194304] 0 2024-05-25T06:11:31.580 INFO:tasks.workunit.client.0.smithi160.stdout:8/228: dwrite d5/d44/d1f/d1e/d28/f4d [0,4194304] 0 2024-05-25T06:11:31.580 INFO:tasks.workunit.client.0.smithi160.stdout:4/262: dwrite d4/d1c/f3e [0,4194304] 0 2024-05-25T06:11:31.582 INFO:tasks.workunit.client.0.smithi160.stdout:7/144: truncate d6/fb 1651787 0 2024-05-25T06:11:31.585 INFO:tasks.workunit.client.1.smithi203.stdout:9/509: dread d0/f21 [0,4194304] 0 2024-05-25T06:11:31.586 INFO:tasks.workunit.client.0.smithi160.stdout:9/84: dwrite d4/d9/fc [0,4194304] 0 2024-05-25T06:11:31.588 INFO:tasks.workunit.client.0.smithi160.stdout:8/229: mkdir d5/d44/d1f/d1e/d28/d4a/d4e 0 2024-05-25T06:11:31.592 INFO:tasks.workunit.client.1.smithi203.stdout:8/664: mkdir d2/d5/d28/d38/d4e/d97/ddf 0 2024-05-25T06:11:31.593 INFO:tasks.workunit.client.1.smithi203.stdout:2/572: fsync d2/d42/fa9 0 2024-05-25T06:11:31.594 INFO:tasks.workunit.client.0.smithi160.stdout:9/85: creat d4/d9/f13 x:0 0 0 2024-05-25T06:11:31.595 INFO:tasks.workunit.client.1.smithi203.stdout:0/447: getdents dd/d32 0 2024-05-25T06:11:31.596 INFO:tasks.workunit.client.0.smithi160.stdout:8/230: dread d5/d9/f26 [0,4194304] 0 2024-05-25T06:11:31.597 INFO:tasks.workunit.client.0.smithi160.stdout:0/151: getdents d2b 0 2024-05-25T06:11:31.597 INFO:tasks.workunit.client.1.smithi203.stdout:9/510: creat d0/d57/d8f/fb5 x:0 0 0 2024-05-25T06:11:31.597 INFO:tasks.workunit.client.0.smithi160.stdout:7/145: creat d6/d2c/f31 x:0 0 0 2024-05-25T06:11:31.598 INFO:tasks.workunit.client.1.smithi203.stdout:0/448: write dd/d3d/d4f/f87 [966660,122296] 0 2024-05-25T06:11:31.600 INFO:tasks.workunit.client.1.smithi203.stdout:0/449: dread - dd/d3d/d4f/f8a zero size 2024-05-25T06:11:31.600 INFO:tasks.workunit.client.0.smithi160.stdout:0/152: dread f1b [0,4194304] 0 2024-05-25T06:11:31.601 INFO:tasks.workunit.client.1.smithi203.stdout:3/679: rmdir d10 39 2024-05-25T06:11:31.603 INFO:tasks.workunit.client.1.smithi203.stdout:7/528: rmdir d1/d11/d3d/dad 0 2024-05-25T06:11:31.603 INFO:tasks.workunit.client.1.smithi203.stdout:0/450: dread dd/d10/f33 [0,4194304] 0 2024-05-25T06:11:31.604 INFO:tasks.workunit.client.1.smithi203.stdout:0/451: readlink dd/d10/l3f 0 2024-05-25T06:11:31.605 INFO:tasks.workunit.client.1.smithi203.stdout:2/573: creat d2/d4/d10/d5f/fc0 x:0 0 0 2024-05-25T06:11:31.606 INFO:tasks.workunit.client.1.smithi203.stdout:8/665: rmdir d2/d5/d18/d31/dac/dc2/d88/d93 39 2024-05-25T06:11:31.606 INFO:tasks.workunit.client.0.smithi160.stdout:9/86: getdents d4/d11/d12 0 2024-05-25T06:11:31.607 INFO:tasks.workunit.client.0.smithi160.stdout:0/153: dwrite f10 [4194304,4194304] 0 2024-05-25T06:11:31.607 INFO:tasks.workunit.client.1.smithi203.stdout:9/511: mkdir d0/d17/db6 0 2024-05-25T06:11:31.608 INFO:tasks.workunit.client.1.smithi203.stdout:0/452: dread dd/d10/f4a [0,4194304] 0 2024-05-25T06:11:31.608 INFO:tasks.workunit.client.1.smithi203.stdout:8/666: write d2/d5/d28/d38/f49 [963622,44648] 0 2024-05-25T06:11:31.609 INFO:tasks.workunit.client.0.smithi160.stdout:0/154: write f10 [255747,91014] 0 2024-05-25T06:11:31.613 INFO:tasks.workunit.client.0.smithi160.stdout:0/155: creat d2b/f2e x:0 0 0 2024-05-25T06:11:31.613 INFO:tasks.workunit.client.0.smithi160.stdout:0/156: write f21 [1378850,44293] 0 2024-05-25T06:11:31.614 INFO:tasks.workunit.client.1.smithi203.stdout:3/680: fdatasync d10/d16/d50/d8b/fb0 0 2024-05-25T06:11:31.618 INFO:tasks.workunit.client.0.smithi160.stdout:0/157: dread f20 [0,4194304] 0 2024-05-25T06:11:31.618 INFO:tasks.workunit.client.1.smithi203.stdout:2/574: dwrite d2/d4/d10/d5f/fc0 [0,4194304] 0 2024-05-25T06:11:31.618 INFO:tasks.workunit.client.1.smithi203.stdout:8/667: mknod d2/d5/d28/d38/d4e/ce0 0 2024-05-25T06:11:31.618 INFO:tasks.workunit.client.1.smithi203.stdout:0/453: fsync dd/d10/f6d 0 2024-05-25T06:11:31.620 INFO:tasks.workunit.client.1.smithi203.stdout:7/529: mkdir d1/d9/d2c/d3a/d4e/db5 0 2024-05-25T06:11:31.623 INFO:tasks.workunit.client.1.smithi203.stdout:3/681: rename d10/d16/d50/f87 to d10/d16/d23/d45/fd7 0 2024-05-25T06:11:31.624 INFO:tasks.workunit.client.1.smithi203.stdout:9/512: link d0/d17/d23/f8e d0/df/d37/fb7 0 2024-05-25T06:11:31.625 INFO:tasks.workunit.client.1.smithi203.stdout:6/801: sync 2024-05-25T06:11:31.625 INFO:tasks.workunit.client.1.smithi203.stdout:1/500: sync 2024-05-25T06:11:31.626 INFO:tasks.workunit.client.0.smithi160.stdout:0/158: mknod d2b/c2f 0 2024-05-25T06:11:31.626 INFO:tasks.workunit.client.0.smithi160.stdout:0/159: readlink l22 0 2024-05-25T06:11:31.627 INFO:tasks.workunit.client.0.smithi160.stdout:0/160: dread - f24 zero size 2024-05-25T06:11:31.628 INFO:tasks.workunit.client.0.smithi160.stdout:7/146: dread f1 [0,4194304] 0 2024-05-25T06:11:31.629 INFO:tasks.workunit.client.0.smithi160.stdout:9/87: write f2 [1503877,126409] 0 2024-05-25T06:11:31.631 INFO:tasks.workunit.client.1.smithi203.stdout:0/454: symlink dd/d10/d34/d4c/l9e 0 2024-05-25T06:11:31.632 INFO:tasks.workunit.client.0.smithi160.stdout:9/88: mkdir d4/d11/d14 0 2024-05-25T06:11:31.634 INFO:tasks.workunit.client.1.smithi203.stdout:4/628: dwrite d0/d9/d50/d6b/fc1 [0,4194304] 0 2024-05-25T06:11:31.635 INFO:tasks.workunit.client.1.smithi203.stdout:2/575: rename d2/d4/d10/d32/l65 to d2/d1a/db0/lc1 0 2024-05-25T06:11:31.637 INFO:tasks.workunit.client.1.smithi203.stdout:4/629: fdatasync d0/d9/d9d/fc6 0 2024-05-25T06:11:31.637 INFO:tasks.workunit.client.1.smithi203.stdout:5/809: dwrite dc/d11/f105 [0,4194304] 0 2024-05-25T06:11:31.637 INFO:tasks.workunit.client.0.smithi160.stdout:0/161: dwrite f20 [0,4194304] 0 2024-05-25T06:11:31.640 INFO:tasks.workunit.client.1.smithi203.stdout:6/802: fsync f2 0 2024-05-25T06:11:31.641 INFO:tasks.workunit.client.0.smithi160.stdout:0/162: creat d2b/f30 x:0 0 0 2024-05-25T06:11:31.642 INFO:tasks.workunit.client.0.smithi160.stdout:0/163: stat d2b/c2f 0 2024-05-25T06:11:31.642 INFO:tasks.workunit.client.1.smithi203.stdout:3/682: creat d10/d16/d23/d4a/d88/fd8 x:0 0 0 2024-05-25T06:11:31.642 INFO:tasks.workunit.client.1.smithi203.stdout:0/455: rmdir dd/d3a 39 2024-05-25T06:11:31.644 INFO:tasks.workunit.client.1.smithi203.stdout:1/501: mkdir d14/dac 0 2024-05-25T06:11:31.645 INFO:tasks.workunit.client.1.smithi203.stdout:8/668: symlink d2/d5/d28/d38/d4e/d97/ddf/le1 0 2024-05-25T06:11:31.646 INFO:tasks.workunit.client.0.smithi160.stdout:4/263: dread d4/d1a/f40 [0,4194304] 0 2024-05-25T06:11:31.646 INFO:tasks.workunit.client.1.smithi203.stdout:5/810: mknod dc/d11/d17/c11d 0 2024-05-25T06:11:31.647 INFO:tasks.workunit.client.0.smithi160.stdout:0/164: mkdir d2b/d31 0 2024-05-25T06:11:31.647 INFO:tasks.workunit.client.1.smithi203.stdout:4/630: mknod d0/d8/d1f/d26/d3b/ccb 0 2024-05-25T06:11:31.650 INFO:tasks.workunit.client.0.smithi160.stdout:7/147: write d6/dc/f17 [1023832,56157] 0 2024-05-25T06:11:31.653 INFO:tasks.workunit.client.0.smithi160.stdout:0/165: unlink l8 0 2024-05-25T06:11:31.653 INFO:tasks.workunit.client.1.smithi203.stdout:7/530: getdents d1/d9/d2c 0 2024-05-25T06:11:31.654 INFO:tasks.workunit.client.1.smithi203.stdout:6/803: read d8/d11/d15/f2e [797694,41289] 0 2024-05-25T06:11:31.654 INFO:tasks.workunit.client.1.smithi203.stdout:1/502: write d14/d57/d29/f80 [793150,58804] 0 2024-05-25T06:11:31.656 INFO:tasks.workunit.client.1.smithi203.stdout:8/669: mknod d2/d5/d18/d31/dac/dc2/d8a/d8f/ce2 0 2024-05-25T06:11:31.659 INFO:tasks.workunit.client.0.smithi160.stdout:4/264: dwrite d4/d6/d28/f17 [4194304,4194304] 0 2024-05-25T06:11:31.659 INFO:tasks.workunit.client.1.smithi203.stdout:3/683: creat d10/d16/d23/d46/dd3/fd9 x:0 0 0 2024-05-25T06:11:31.663 INFO:tasks.workunit.client.1.smithi203.stdout:7/531: symlink d1/d11/lb6 0 2024-05-25T06:11:31.664 INFO:tasks.workunit.client.1.smithi203.stdout:0/456: dwrite dd/d3d/d59/f9b [0,4194304] 0 2024-05-25T06:11:31.665 INFO:tasks.workunit.client.1.smithi203.stdout:6/804: write d8/d11/d15/d28/d52/d43/f84 [2077961,7905] 0 2024-05-25T06:11:31.665 INFO:tasks.workunit.client.1.smithi203.stdout:9/513: dwrite d0/f6f [0,4194304] 0 2024-05-25T06:11:31.666 INFO:tasks.workunit.client.1.smithi203.stdout:8/670: creat d2/d5/d18/dbf/fe3 x:0 0 0 2024-05-25T06:11:31.671 INFO:tasks.workunit.client.1.smithi203.stdout:4/631: dwrite d0/d9/d50/d79/f65 [0,4194304] 0 2024-05-25T06:11:31.672 INFO:tasks.workunit.client.1.smithi203.stdout:8/671: fdatasync d2/d5/d18/d31/f34 0 2024-05-25T06:11:31.674 INFO:tasks.workunit.client.1.smithi203.stdout:3/684: truncate f0 3228390 0 2024-05-25T06:11:31.676 INFO:tasks.workunit.client.1.smithi203.stdout:8/672: read - d2/d5/d2d/fa3 zero size 2024-05-25T06:11:31.677 INFO:tasks.workunit.client.1.smithi203.stdout:7/532: mkdir d1/db7 0 2024-05-25T06:11:31.677 INFO:tasks.workunit.client.1.smithi203.stdout:5/811: creat dc/d11/d17/d1a/d2d/d4d/d64/dd4/f11e x:0 0 0 2024-05-25T06:11:31.680 INFO:tasks.workunit.client.1.smithi203.stdout:6/805: dwrite d8/d11/f90 [0,4194304] 0 2024-05-25T06:11:31.683 INFO:tasks.workunit.client.1.smithi203.stdout:3/685: chown d10/lc1 88896 1 2024-05-25T06:11:31.683 INFO:tasks.workunit.client.1.smithi203.stdout:0/457: creat dd/d45/d96/f9f x:0 0 0 2024-05-25T06:11:31.686 INFO:tasks.workunit.client.1.smithi203.stdout:4/632: write d0/d8/d3a/d81/d93/fad [97843,114044] 0 2024-05-25T06:11:31.688 INFO:tasks.workunit.client.1.smithi203.stdout:7/533: creat d1/d9/d2c/d36/d54/fb8 x:0 0 0 2024-05-25T06:11:31.689 INFO:tasks.workunit.client.1.smithi203.stdout:3/686: mkdir d10/d16/d23/d4a/d88/dca/dda 0 2024-05-25T06:11:31.691 INFO:tasks.workunit.client.1.smithi203.stdout:0/458: mkdir dd/d3d/da0 0 2024-05-25T06:11:31.693 INFO:tasks.workunit.client.1.smithi203.stdout:4/633: creat d0/d8/d3a/d81/d93/d98/d99/fcc x:0 0 0 2024-05-25T06:11:31.695 INFO:tasks.workunit.client.1.smithi203.stdout:6/806: mknod d8/d11/d15/d28/c101 0 2024-05-25T06:11:31.698 INFO:tasks.workunit.client.1.smithi203.stdout:3/687: rename d10/d16/d50/d8b/f52 to d10/d16/d23/d4a/d88/fdb 0 2024-05-25T06:11:31.700 INFO:tasks.workunit.client.1.smithi203.stdout:8/673: dwrite d2/d5/d28/d38/d4e/f74 [0,4194304] 0 2024-05-25T06:11:31.700 INFO:tasks.workunit.client.1.smithi203.stdout:0/459: write dd/d3d/d4f/d62/f9c [604054,15322] 0 2024-05-25T06:11:31.701 INFO:tasks.workunit.client.1.smithi203.stdout:4/634: creat d0/d8/d1f/d26/d43/fcd x:0 0 0 2024-05-25T06:11:31.703 INFO:tasks.workunit.client.1.smithi203.stdout:5/812: dwrite dc/d11/d17/d1a/d2d/d53/fec [0,4194304] 0 2024-05-25T06:11:31.707 INFO:tasks.workunit.client.1.smithi203.stdout:4/635: chown d0/d8/d10/d91/fba 2 1 2024-05-25T06:11:31.708 INFO:tasks.workunit.client.1.smithi203.stdout:6/807: mknod d8/d11/d15/d28/d52/d43/d73/c102 0 2024-05-25T06:11:31.713 INFO:tasks.workunit.client.1.smithi203.stdout:4/636: stat d0/d8/d3a/d81/d93/d98/d99 0 2024-05-25T06:11:31.713 INFO:tasks.workunit.client.1.smithi203.stdout:7/534: dwrite d1/d9/d2c/d3a/f4b [0,4194304] 0 2024-05-25T06:11:31.714 INFO:tasks.workunit.client.1.smithi203.stdout:8/674: chown d2/d5/d18/d31/dac/dc2/d8a/d8f/cd7 0 1 2024-05-25T06:11:31.715 INFO:tasks.workunit.client.1.smithi203.stdout:5/813: symlink dc/d3c/df5/l11f 0 2024-05-25T06:11:31.715 INFO:tasks.workunit.client.1.smithi203.stdout:4/637: fsync d0/d9/d50/d79/f40 0 2024-05-25T06:11:31.717 INFO:tasks.workunit.client.1.smithi203.stdout:9/514: write d0/df/d73/f53 [661239,105775] 0 2024-05-25T06:11:31.718 INFO:tasks.workunit.client.1.smithi203.stdout:3/688: getdents d10/d16/d23/d45 0 2024-05-25T06:11:31.720 INFO:tasks.workunit.client.1.smithi203.stdout:6/808: symlink d8/d11/d1b/d75/dcb/dcf/d46/d88/l103 0 2024-05-25T06:11:31.723 INFO:tasks.workunit.client.1.smithi203.stdout:7/535: creat d1/d11/d3d/fb9 x:0 0 0 2024-05-25T06:11:31.725 INFO:tasks.workunit.client.1.smithi203.stdout:3/689: link d10/d16/fc9 d10/d16/d23/d45/fdc 0 2024-05-25T06:11:31.728 INFO:tasks.workunit.client.1.smithi203.stdout:4/638: rename d0/d8/d1f/d26/d43 to d0/d9/d50/dce 0 2024-05-25T06:11:31.731 INFO:tasks.workunit.client.1.smithi203.stdout:7/536: dwrite d1/d11/f30 [0,4194304] 0 2024-05-25T06:11:31.733 INFO:tasks.workunit.client.0.smithi160.stdout:7/148: mkdir d6/d32 0 2024-05-25T06:11:31.734 INFO:tasks.workunit.client.0.smithi160.stdout:7/149: write d6/dc/f17 [586525,84730] 0 2024-05-25T06:11:31.734 INFO:tasks.workunit.client.1.smithi203.stdout:7/537: chown d1/d9/d2c/d3a/d4e/c71 0 1 2024-05-25T06:11:31.738 INFO:tasks.workunit.client.0.smithi160.stdout:0/166: link c28 d2b/d31/c32 0 2024-05-25T06:11:31.739 INFO:tasks.workunit.client.1.smithi203.stdout:0/460: dwrite dd/fe [0,4194304] 0 2024-05-25T06:11:31.739 INFO:tasks.workunit.client.0.smithi160.stdout:4/265: creat d4/f4f x:0 0 0 2024-05-25T06:11:31.744 INFO:tasks.workunit.client.0.smithi160.stdout:3/103: sync 2024-05-25T06:11:31.744 INFO:tasks.workunit.client.0.smithi160.stdout:1/89: sync 2024-05-25T06:11:31.745 INFO:tasks.workunit.client.1.smithi203.stdout:7/538: dwrite d1/d9/d2c/d36/f9d [0,4194304] 0 2024-05-25T06:11:31.746 INFO:tasks.workunit.client.0.smithi160.stdout:7/150: dwrite d6/dc/d21/f2a [0,4194304] 0 2024-05-25T06:11:31.747 INFO:tasks.workunit.client.0.smithi160.stdout:3/104: fsync f15 0 2024-05-25T06:11:31.750 INFO:tasks.workunit.client.0.smithi160.stdout:4/266: creat d4/d1a/f50 x:0 0 0 2024-05-25T06:11:31.751 INFO:tasks.workunit.client.1.smithi203.stdout:8/675: rename d2/d5/d10/d11/d59/cb8 to d2/d5/d28/d38/d4e/d97/ddf/ce4 0 2024-05-25T06:11:31.753 INFO:tasks.workunit.client.0.smithi160.stdout:7/151: readlink d6/dc/l1f 0 2024-05-25T06:11:31.754 INFO:tasks.workunit.client.0.smithi160.stdout:0/167: dread f1e [0,4194304] 0 2024-05-25T06:11:31.754 INFO:tasks.workunit.client.1.smithi203.stdout:4/639: mknod d0/d8/d3a/ccf 0 2024-05-25T06:11:31.756 INFO:tasks.workunit.client.0.smithi160.stdout:1/90: dwrite f7 [0,4194304] 0 2024-05-25T06:11:31.757 INFO:tasks.workunit.client.0.smithi160.stdout:7/152: truncate d6/dc/f25 766240 0 2024-05-25T06:11:31.757 INFO:tasks.workunit.client.0.smithi160.stdout:4/267: mknod d4/d6/d35/d3d/c51 0 2024-05-25T06:11:31.758 INFO:tasks.workunit.client.1.smithi203.stdout:0/461: write dd/d3a/f4b [4299100,285] 0 2024-05-25T06:11:31.759 INFO:tasks.workunit.client.0.smithi160.stdout:0/168: symlink d2b/l33 0 2024-05-25T06:11:31.760 INFO:tasks.workunit.client.0.smithi160.stdout:7/153: fdatasync d6/dc/f25 0 2024-05-25T06:11:31.760 INFO:tasks.workunit.client.1.smithi203.stdout:6/809: rename d8/d11/d1b/d75/fdd to d8/d11/d15/d28/d52/d43/f104 0 2024-05-25T06:11:31.762 INFO:tasks.workunit.client.0.smithi160.stdout:1/91: creat de/f17 x:0 0 0 2024-05-25T06:11:31.764 INFO:tasks.workunit.client.0.smithi160.stdout:3/105: dwrite fe [0,4194304] 0 2024-05-25T06:11:31.765 INFO:tasks.workunit.client.1.smithi203.stdout:4/640: dwrite d0/d8/d1f/d26/d3b/f97 [0,4194304] 0 2024-05-25T06:11:31.766 INFO:tasks.workunit.client.0.smithi160.stdout:4/268: dread d4/f2a [0,4194304] 0 2024-05-25T06:11:31.770 INFO:tasks.workunit.client.1.smithi203.stdout:0/462: symlink dd/d45/la1 0 2024-05-25T06:11:31.771 INFO:tasks.workunit.client.1.smithi203.stdout:8/676: dwrite d2/d5/d10/d45/d5d/d68/f7d [0,4194304] 0 2024-05-25T06:11:31.773 INFO:tasks.workunit.client.0.smithi160.stdout:1/92: symlink de/l18 0 2024-05-25T06:11:31.778 INFO:tasks.workunit.client.0.smithi160.stdout:7/154: dwrite d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:31.778 INFO:tasks.workunit.client.0.smithi160.stdout:3/106: dwrite f16 [0,4194304] 0 2024-05-25T06:11:31.778 INFO:tasks.workunit.client.0.smithi160.stdout:4/269: write d4/d6/d28/f25 [34532,16782] 0 2024-05-25T06:11:31.779 INFO:tasks.workunit.client.1.smithi203.stdout:6/810: creat d8/d2a/d6c/dc1/f105 x:0 0 0 2024-05-25T06:11:31.781 INFO:tasks.workunit.client.1.smithi203.stdout:0/463: dwrite dd/d3d/d4f/d62/f74 [0,4194304] 0 2024-05-25T06:11:31.781 INFO:tasks.workunit.client.0.smithi160.stdout:1/93: creat de/f19 x:0 0 0 2024-05-25T06:11:31.783 INFO:tasks.workunit.client.1.smithi203.stdout:0/464: write dd/f64 [423276,102205] 0 2024-05-25T06:11:31.786 INFO:tasks.workunit.client.0.smithi160.stdout:1/94: mknod de/c1a 0 2024-05-25T06:11:31.788 INFO:tasks.workunit.client.0.smithi160.stdout:1/95: chown f5 490 1 2024-05-25T06:11:31.788 INFO:tasks.workunit.client.0.smithi160.stdout:4/270: fsync d4/d6/d21/f24 0 2024-05-25T06:11:31.788 INFO:tasks.workunit.client.0.smithi160.stdout:3/107: symlink d19/d22/l25 0 2024-05-25T06:11:31.789 INFO:tasks.workunit.client.1.smithi203.stdout:8/677: creat d2/d5/d2d/fe5 x:0 0 0 2024-05-25T06:11:31.790 INFO:tasks.workunit.client.0.smithi160.stdout:7/155: mknod d6/c33 0 2024-05-25T06:11:31.794 INFO:tasks.workunit.client.1.smithi203.stdout:8/678: read d2/d5/d28/d38/d4e/d97/fbd [3358227,65272] 0 2024-05-25T06:11:31.795 INFO:tasks.workunit.client.1.smithi203.stdout:6/811: unlink d8/d2a/f4a 0 2024-05-25T06:11:31.796 INFO:tasks.workunit.client.1.smithi203.stdout:6/812: chown d8/d11/d15/d28/d52/d43/ff5 8 1 2024-05-25T06:11:31.797 INFO:tasks.workunit.client.0.smithi160.stdout:0/169: write f2 [1684081,74551] 0 2024-05-25T06:11:31.797 INFO:tasks.workunit.client.1.smithi203.stdout:7/539: truncate d1/f38 1832595 0 2024-05-25T06:11:31.798 INFO:tasks.workunit.client.1.smithi203.stdout:0/465: chown dd/d3a/d66/d88/c89 5141337 1 2024-05-25T06:11:31.798 INFO:tasks.workunit.client.1.smithi203.stdout:0/466: fdatasync f9 0 2024-05-25T06:11:31.799 INFO:tasks.workunit.client.0.smithi160.stdout:4/271: dwrite d4/d6/d21/d42/d49/f4b [0,4194304] 0 2024-05-25T06:11:31.799 INFO:tasks.workunit.client.1.smithi203.stdout:4/641: creat d0/d8/d52/d59/d7a/d83/fd0 x:0 0 0 2024-05-25T06:11:31.799 INFO:tasks.workunit.client.0.smithi160.stdout:4/272: chown d4/d1a 32 1 2024-05-25T06:11:31.801 INFO:tasks.workunit.client.1.smithi203.stdout:3/690: rename d10/d16/fc9 to d10/d16/d23/fdd 0 2024-05-25T06:11:31.805 INFO:tasks.workunit.client.1.smithi203.stdout:7/540: truncate d1/d9/f29 2784296 0 2024-05-25T06:11:31.807 INFO:tasks.workunit.client.0.smithi160.stdout:4/273: dwrite d4/f26 [0,4194304] 0 2024-05-25T06:11:31.807 INFO:tasks.workunit.client.1.smithi203.stdout:6/813: symlink d8/d11/d47/d60/l106 0 2024-05-25T06:11:31.807 INFO:tasks.workunit.client.1.smithi203.stdout:7/541: write d1/d9/d2c/f5d [1468121,118723] 0 2024-05-25T06:11:31.812 INFO:tasks.workunit.client.0.smithi160.stdout:4/274: dread d4/d1a/f40 [4194304,4194304] 0 2024-05-25T06:11:31.812 INFO:tasks.workunit.client.0.smithi160.stdout:1/96: truncate fa 2162009 0 2024-05-25T06:11:31.812 INFO:tasks.workunit.client.1.smithi203.stdout:7/542: creat d1/d9/d2c/d36/d66/fba x:0 0 0 2024-05-25T06:11:31.813 INFO:tasks.workunit.client.0.smithi160.stdout:4/275: truncate d4/d6/d28/f17 9189443 0 2024-05-25T06:11:31.816 INFO:tasks.workunit.client.1.smithi203.stdout:8/679: mkdir d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6 0 2024-05-25T06:11:31.816 INFO:tasks.workunit.client.1.smithi203.stdout:0/467: link dd/d3d/d48/f7a dd/d3d/d4f/fa2 0 2024-05-25T06:11:31.817 INFO:tasks.workunit.client.1.smithi203.stdout:8/680: chown d2/d5/d10/d11/fc8 1833599 1 2024-05-25T06:11:31.818 INFO:tasks.workunit.client.0.smithi160.stdout:4/276: fdatasync d4/d6/d21/f48 0 2024-05-25T06:11:31.818 INFO:tasks.workunit.client.1.smithi203.stdout:6/814: link d8/d11/d15/f79 d8/d2a/f107 0 2024-05-25T06:11:31.818 INFO:tasks.workunit.client.0.smithi160.stdout:3/108: link c10 d19/d22/c26 0 2024-05-25T06:11:31.819 INFO:tasks.workunit.client.0.smithi160.stdout:3/109: write f15 [1260862,44856] 0 2024-05-25T06:11:31.820 INFO:tasks.workunit.client.1.smithi203.stdout:0/468: dread - dd/d3d/d4f/f56 zero size 2024-05-25T06:11:31.820 INFO:tasks.workunit.client.1.smithi203.stdout:8/681: dread - d2/d5/d10/d4a/fba zero size 2024-05-25T06:11:31.820 INFO:tasks.workunit.client.0.smithi160.stdout:0/170: creat d2b/d31/f34 x:0 0 0 2024-05-25T06:11:31.822 INFO:tasks.workunit.client.0.smithi160.stdout:3/110: creat d19/f27 x:0 0 0 2024-05-25T06:11:31.823 INFO:tasks.workunit.client.1.smithi203.stdout:0/469: write dd/d10/d34/f58 [612334,60846] 0 2024-05-25T06:11:31.823 INFO:tasks.workunit.client.0.smithi160.stdout:3/111: read - f18 zero size 2024-05-25T06:11:31.823 INFO:tasks.workunit.client.0.smithi160.stdout:0/171: mkdir d2b/d31/d35 0 2024-05-25T06:11:31.825 INFO:tasks.workunit.client.0.smithi160.stdout:3/112: creat d19/d22/f28 x:0 0 0 2024-05-25T06:11:31.825 INFO:tasks.workunit.client.1.smithi203.stdout:6/815: getdents d8/d11/d15 0 2024-05-25T06:11:31.826 INFO:tasks.workunit.client.1.smithi203.stdout:0/470: mknod dd/d32/ca3 0 2024-05-25T06:11:31.829 INFO:tasks.workunit.client.1.smithi203.stdout:0/471: chown dd/d10/d34/d4c/d99/l77 74334671 1 2024-05-25T06:11:31.831 INFO:tasks.workunit.client.1.smithi203.stdout:8/682: dwrite d2/d5/d28/d38/d4e/d97/f66 [4194304,4194304] 0 2024-05-25T06:11:31.832 INFO:tasks.workunit.client.0.smithi160.stdout:0/172: dwrite f24 [0,4194304] 0 2024-05-25T06:11:31.833 INFO:tasks.workunit.client.0.smithi160.stdout:0/173: dread - d2b/f30 zero size 2024-05-25T06:11:31.836 INFO:tasks.workunit.client.1.smithi203.stdout:0/472: symlink dd/d45/la4 0 2024-05-25T06:11:31.839 INFO:tasks.workunit.client.0.smithi160.stdout:3/113: dwrite fb [0,4194304] 0 2024-05-25T06:11:31.842 INFO:tasks.workunit.client.1.smithi203.stdout:8/683: mkdir d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7 0 2024-05-25T06:11:31.848 INFO:tasks.workunit.client.0.smithi160.stdout:0/174: mkdir d2b/d31/d35/d36 0 2024-05-25T06:11:31.849 INFO:tasks.workunit.client.1.smithi203.stdout:4/642: write d0/d8/d1f/d26/d3b/f97 [4700927,70004] 0 2024-05-25T06:11:31.849 INFO:tasks.workunit.client.0.smithi160.stdout:3/114: dwrite f13 [0,4194304] 0 2024-05-25T06:11:31.850 INFO:tasks.workunit.client.1.smithi203.stdout:4/643: dread - d0/d8/d10/f58 zero size 2024-05-25T06:11:31.850 INFO:tasks.workunit.client.0.smithi160.stdout:3/115: stat l4 0 2024-05-25T06:11:31.851 INFO:tasks.workunit.client.1.smithi203.stdout:0/473: getdents dd/d10/d34/d4c/d99/d70 0 2024-05-25T06:11:31.857 INFO:tasks.workunit.client.1.smithi203.stdout:4/644: write d0/d8/d1f/f2b [3676779,77689] 0 2024-05-25T06:11:31.860 INFO:tasks.workunit.client.1.smithi203.stdout:7/543: write d1/d9/f18 [1135309,64557] 0 2024-05-25T06:11:31.860 INFO:tasks.workunit.client.0.smithi160.stdout:3/116: dwrite ff [0,4194304] 0 2024-05-25T06:11:31.861 INFO:tasks.workunit.client.0.smithi160.stdout:3/117: chown f18 8736 1 2024-05-25T06:11:31.862 INFO:tasks.workunit.client.1.smithi203.stdout:4/645: unlink d0/d9/d11/d54/c70 0 2024-05-25T06:11:31.863 INFO:tasks.workunit.client.1.smithi203.stdout:3/691: getdents d10/d16/d50/d8b/db3 0 2024-05-25T06:11:31.864 INFO:tasks.workunit.client.1.smithi203.stdout:4/646: truncate d0/d8/d1f/d26/fc9 213958 0 2024-05-25T06:11:31.865 INFO:tasks.workunit.client.1.smithi203.stdout:4/647: fsync d0/d9/d50/fc5 0 2024-05-25T06:11:31.867 INFO:tasks.workunit.client.1.smithi203.stdout:3/692: mknod d10/d16/d23/d4a/cde 0 2024-05-25T06:11:31.869 INFO:tasks.workunit.client.1.smithi203.stdout:4/648: creat d0/d9/d50/fd1 x:0 0 0 2024-05-25T06:11:31.872 INFO:tasks.workunit.client.1.smithi203.stdout:4/649: symlink d0/d8/d10/d91/ld2 0 2024-05-25T06:11:31.872 INFO:tasks.workunit.client.0.smithi160.stdout:3/118: fdatasync f13 0 2024-05-25T06:11:31.874 INFO:tasks.workunit.client.0.smithi160.stdout:3/119: write f0 [2296305,55801] 0 2024-05-25T06:11:31.874 INFO:tasks.workunit.client.0.smithi160.stdout:3/120: dread - f18 zero size 2024-05-25T06:11:31.875 INFO:tasks.workunit.client.1.smithi203.stdout:6/816: truncate d8/d11/d15/d28/d52/d43/f32 11335701 0 2024-05-25T06:11:31.875 INFO:tasks.workunit.client.0.smithi160.stdout:3/121: write f13 [3005445,13196] 0 2024-05-25T06:11:31.879 INFO:tasks.workunit.client.1.smithi203.stdout:8/684: truncate d2/d5/d28/d38/d4e/fbb 251857 0 2024-05-25T06:11:31.881 INFO:tasks.workunit.client.1.smithi203.stdout:8/685: chown d2/d5/d10/d45/d5d/f5c 221403321 1 2024-05-25T06:11:31.882 INFO:tasks.workunit.client.0.smithi160.stdout:3/122: dwrite d19/d22/f28 [0,4194304] 0 2024-05-25T06:11:31.882 INFO:tasks.workunit.client.1.smithi203.stdout:6/817: dwrite d8/d11/d15/f59 [0,4194304] 0 2024-05-25T06:11:31.884 INFO:tasks.workunit.client.1.smithi203.stdout:4/650: link d0/d9/d50/d79/l73 d0/d8/d3a/d81/d93/d98/d99/ld3 0 2024-05-25T06:11:31.884 INFO:tasks.workunit.client.1.smithi203.stdout:7/544: truncate d1/d9/f76 1161837 0 2024-05-25T06:11:31.884 INFO:tasks.workunit.client.0.smithi160.stdout:2/115: sync 2024-05-25T06:11:31.888 INFO:tasks.workunit.client.0.smithi160.stdout:2/116: rmdir d12 39 2024-05-25T06:11:31.889 INFO:tasks.workunit.client.0.smithi160.stdout:3/123: dwrite d19/d22/f28 [0,4194304] 0 2024-05-25T06:11:31.891 INFO:tasks.workunit.client.0.smithi160.stdout:3/124: readlink l4 0 2024-05-25T06:11:31.891 INFO:tasks.workunit.client.0.smithi160.stdout:2/117: fsync fb 0 2024-05-25T06:11:31.892 INFO:tasks.workunit.client.1.smithi203.stdout:7/545: symlink d1/d11/d20/lbb 0 2024-05-25T06:11:31.897 INFO:tasks.workunit.client.1.smithi203.stdout:4/651: mkdir d0/d8/d3a/d81/dd4 0 2024-05-25T06:11:31.900 INFO:tasks.workunit.client.0.smithi160.stdout:2/118: symlink d12/d13/l22 0 2024-05-25T06:11:31.903 INFO:tasks.workunit.client.1.smithi203.stdout:7/546: dwrite d1/d11/f21 [0,4194304] 0 2024-05-25T06:11:31.903 INFO:tasks.workunit.client.1.smithi203.stdout:4/652: mknod d0/d9/d50/d79/d9a/cd5 0 2024-05-25T06:11:31.906 INFO:tasks.workunit.client.0.smithi160.stdout:2/119: read d12/f1d [1308068,20970] 0 2024-05-25T06:11:31.907 INFO:tasks.workunit.client.0.smithi160.stdout:2/120: write f9 [4401971,69396] 0 2024-05-25T06:11:31.907 INFO:tasks.workunit.client.1.smithi203.stdout:4/653: fdatasync d0/f16 0 2024-05-25T06:11:31.911 INFO:tasks.workunit.client.0.smithi160.stdout:2/121: rename l6 to d12/d1c/l23 0 2024-05-25T06:11:31.911 INFO:tasks.workunit.client.0.smithi160.stdout:2/122: dread - fe zero size 2024-05-25T06:11:31.912 INFO:tasks.workunit.client.0.smithi160.stdout:2/123: fdatasync f5 0 2024-05-25T06:11:31.914 INFO:tasks.workunit.client.0.smithi160.stdout:2/124: creat d12/d13/f24 x:0 0 0 2024-05-25T06:11:31.914 INFO:tasks.workunit.client.1.smithi203.stdout:4/654: rename d0/fa8 to d0/d8/d52/d59/d7a/d83/fd6 0 2024-05-25T06:11:31.915 INFO:tasks.workunit.client.1.smithi203.stdout:7/547: dwrite d1/d9/d2c/d62/d9b/fa0 [0,4194304] 0 2024-05-25T06:11:31.916 INFO:tasks.workunit.client.0.smithi160.stdout:2/125: creat d12/f25 x:0 0 0 2024-05-25T06:11:31.917 INFO:tasks.workunit.client.0.smithi160.stdout:2/126: read - d12/f20 zero size 2024-05-25T06:11:31.917 INFO:tasks.workunit.client.0.smithi160.stdout:2/127: dread - d12/d1c/f1e zero size 2024-05-25T06:11:31.918 INFO:tasks.workunit.client.0.smithi160.stdout:2/128: chown f10 433409 1 2024-05-25T06:11:31.918 INFO:tasks.workunit.client.1.smithi203.stdout:0/474: dread dd/d32/d37/f52 [0,4194304] 0 2024-05-25T06:11:31.920 INFO:tasks.workunit.client.1.smithi203.stdout:7/548: link d1/f32 d1/d9/d2c/d36/fbc 0 2024-05-25T06:11:31.920 INFO:tasks.workunit.client.0.smithi160.stdout:2/129: mknod d12/d1c/c26 0 2024-05-25T06:11:31.923 INFO:tasks.workunit.client.1.smithi203.stdout:0/475: rename dd/d45/la1 to dd/d3d/d75/d93/la5 0 2024-05-25T06:11:31.925 INFO:tasks.workunit.client.1.smithi203.stdout:7/549: unlink d1/d9/d2c/d36/d66/l9e 0 2024-05-25T06:11:31.926 INFO:tasks.workunit.client.1.smithi203.stdout:0/476: symlink dd/d32/la6 0 2024-05-25T06:11:31.928 INFO:tasks.workunit.client.0.smithi160.stdout:2/130: dwrite f7 [0,4194304] 0 2024-05-25T06:11:31.929 INFO:tasks.workunit.client.0.smithi160.stdout:2/131: readlink d12/l19 0 2024-05-25T06:11:31.932 INFO:tasks.workunit.client.1.smithi203.stdout:0/477: truncate dd/d32/d37/f4e 374979 0 2024-05-25T06:11:31.935 INFO:tasks.workunit.client.1.smithi203.stdout:0/478: mknod dd/d3a/d66/ca7 0 2024-05-25T06:11:31.937 INFO:tasks.workunit.client.0.smithi160.stdout:2/132: dwrite f11 [0,4194304] 0 2024-05-25T06:11:31.939 INFO:tasks.workunit.client.0.smithi160.stdout:3/125: dread f15 [0,4194304] 0 2024-05-25T06:11:31.939 INFO:tasks.workunit.client.1.smithi203.stdout:0/479: link dd/d10/d34/d4c/d99/f82 dd/d10/d34/fa8 0 2024-05-25T06:11:31.941 INFO:tasks.workunit.client.1.smithi203.stdout:0/480: symlink dd/d3a/d66/d9d/la9 0 2024-05-25T06:11:31.941 INFO:tasks.workunit.client.0.smithi160.stdout:3/126: rename f16 to d19/f29 0 2024-05-25T06:11:31.943 INFO:tasks.workunit.client.1.smithi203.stdout:0/481: mknod dd/d45/caa 0 2024-05-25T06:11:31.945 INFO:tasks.workunit.client.1.smithi203.stdout:0/482: creat dd/d3d/d4f/d62/d78/fab x:0 0 0 2024-05-25T06:11:31.946 INFO:tasks.workunit.client.1.smithi203.stdout:0/483: creat dd/d3d/d59/fac x:0 0 0 2024-05-25T06:11:31.948 INFO:tasks.workunit.client.1.smithi203.stdout:0/484: mkdir dd/d3d/d48/d91/dad 0 2024-05-25T06:11:31.956 INFO:tasks.workunit.client.0.smithi160.stdout:2/133: dread f8 [4194304,4194304] 0 2024-05-25T06:11:31.959 INFO:tasks.workunit.client.0.smithi160.stdout:2/134: unlink d12/f20 0 2024-05-25T06:11:31.959 INFO:tasks.workunit.client.0.smithi160.stdout:2/135: stat d12/f1d 0 2024-05-25T06:11:31.959 INFO:tasks.workunit.client.0.smithi160.stdout:2/136: chown f7 38137797 1 2024-05-25T06:11:32.006 INFO:tasks.workunit.client.1.smithi203.stdout:2/576: sync 2024-05-25T06:11:32.008 INFO:tasks.workunit.client.1.smithi203.stdout:2/577: read d2/d4/d10/d5f/fa2 [127577,1369] 0 2024-05-25T06:11:32.010 INFO:tasks.workunit.client.1.smithi203.stdout:2/578: symlink d2/d38/d43/lc2 0 2024-05-25T06:11:32.012 INFO:tasks.workunit.client.1.smithi203.stdout:2/579: rename d2/d1a/f7f to d2/d42/fc3 0 2024-05-25T06:11:32.020 INFO:tasks.workunit.client.0.smithi160.stdout:5/172: sync 2024-05-25T06:11:32.021 INFO:tasks.workunit.client.0.smithi160.stdout:5/173: readlink db/dc/d13/l18 0 2024-05-25T06:11:32.029 INFO:tasks.workunit.client.1.smithi203.stdout:2/580: dread d2/d4/d10/f21 [0,4194304] 0 2024-05-25T06:11:32.030 INFO:tasks.workunit.client.1.smithi203.stdout:2/581: fsync d2/d38/d2f/f4f 0 2024-05-25T06:11:32.031 INFO:tasks.workunit.client.0.smithi160.stdout:5/174: dwrite db/dc/d13/f1f [0,4194304] 0 2024-05-25T06:11:32.036 INFO:tasks.workunit.client.0.smithi160.stdout:5/175: dread f8 [0,4194304] 0 2024-05-25T06:11:32.037 INFO:tasks.workunit.client.1.smithi203.stdout:2/582: getdents d2/d4/d10/d8c 0 2024-05-25T06:11:32.038 INFO:tasks.workunit.client.0.smithi160.stdout:5/176: readlink db/dc/d13/l16 0 2024-05-25T06:11:32.038 INFO:tasks.workunit.client.1.smithi203.stdout:6/818: write d8/d11/d47/fee [860529,20417] 0 2024-05-25T06:11:32.039 INFO:tasks.workunit.client.1.smithi203.stdout:8/686: truncate d2/d5/d10/d45/d5d/d68/f87 2448425 0 2024-05-25T06:11:32.039 INFO:tasks.workunit.client.1.smithi203.stdout:6/819: write d8/fe0 [541845,14548] 0 2024-05-25T06:11:32.039 INFO:tasks.workunit.client.1.smithi203.stdout:2/583: chown d2/d4/d10/d32/f1f 432204319 1 2024-05-25T06:11:32.041 INFO:tasks.workunit.client.0.smithi160.stdout:5/177: creat db/dc/d13/d1d/d23/d24/d2e/f34 x:0 0 0 2024-05-25T06:11:32.042 INFO:tasks.workunit.client.0.smithi160.stdout:5/178: write f1 [1231550,13182] 0 2024-05-25T06:11:32.042 INFO:tasks.workunit.client.1.smithi203.stdout:8/687: chown d2/d5/d10/d11/de/f1a 10524 1 2024-05-25T06:11:32.045 INFO:tasks.workunit.client.1.smithi203.stdout:0/485: chown dd/d3d/d75/d93/la5 20007547 1 2024-05-25T06:11:32.046 INFO:tasks.workunit.client.1.smithi203.stdout:7/550: rmdir d1/d9/d2c/d36/d66 39 2024-05-25T06:11:32.046 INFO:tasks.workunit.client.0.smithi160.stdout:5/179: rename fa to db/dc/d13/d1d/d23/d24/d2e/f35 0 2024-05-25T06:11:32.047 INFO:tasks.workunit.client.1.smithi203.stdout:7/551: stat d1/d9/fe 0 2024-05-25T06:11:32.047 INFO:tasks.workunit.client.0.smithi160.stdout:5/180: read db/dc/f15 [3425732,95124] 0 2024-05-25T06:11:32.048 INFO:tasks.workunit.client.1.smithi203.stdout:2/584: link d2/d4/d10/f9b d2/d42/d8e/fc4 0 2024-05-25T06:11:32.049 INFO:tasks.workunit.client.1.smithi203.stdout:6/820: dwrite d8/fc [0,4194304] 0 2024-05-25T06:11:32.051 INFO:tasks.workunit.client.1.smithi203.stdout:0/486: write dd/d32/f7c [967908,13690] 0 2024-05-25T06:11:32.052 INFO:tasks.workunit.client.0.smithi160.stdout:5/181: symlink db/dc/d13/d1d/d23/d24/d2e/l36 0 2024-05-25T06:11:32.052 INFO:tasks.workunit.client.0.smithi160.stdout:5/182: chown db/dc/l1b 156 1 2024-05-25T06:11:32.053 INFO:tasks.workunit.client.1.smithi203.stdout:2/585: write d2/d1a/d33/d80/fb5 [1295436,11700] 0 2024-05-25T06:11:32.055 INFO:tasks.workunit.client.1.smithi203.stdout:0/487: truncate dd/d3a/f7f 858738 0 2024-05-25T06:11:32.057 INFO:tasks.workunit.client.0.smithi160.stdout:5/183: unlink db/dc/f30 0 2024-05-25T06:11:32.057 INFO:tasks.workunit.client.0.smithi160.stdout:2/137: fdatasync f7 0 2024-05-25T06:11:32.058 INFO:tasks.workunit.client.1.smithi203.stdout:8/688: dwrite d2/d5/d10/d11/de/ff [0,4194304] 0 2024-05-25T06:11:32.058 INFO:tasks.workunit.client.1.smithi203.stdout:6/821: unlink d8/d11/d15/f2b 0 2024-05-25T06:11:32.061 INFO:tasks.workunit.client.1.smithi203.stdout:0/488: rename dd/d8c/f8e to dd/d3a/fae 0 2024-05-25T06:11:32.065 INFO:tasks.workunit.client.1.smithi203.stdout:2/586: dread d2/d6/f46 [0,4194304] 0 2024-05-25T06:11:32.068 INFO:tasks.workunit.client.1.smithi203.stdout:2/587: readlink d2/d38/d43/l78 0 2024-05-25T06:11:32.068 INFO:tasks.workunit.client.1.smithi203.stdout:0/489: rmdir dd/d3d/d59 39 2024-05-25T06:11:32.068 INFO:tasks.workunit.client.1.smithi203.stdout:6/822: dread d8/fe0 [0,4194304] 0 2024-05-25T06:11:32.068 INFO:tasks.workunit.client.0.smithi160.stdout:5/184: getdents db 0 2024-05-25T06:11:32.069 INFO:tasks.workunit.client.0.smithi160.stdout:2/138: getdents d12/d13 0 2024-05-25T06:11:32.073 INFO:tasks.workunit.client.0.smithi160.stdout:2/139: creat d12/d1c/f27 x:0 0 0 2024-05-25T06:11:32.073 INFO:tasks.workunit.client.0.smithi160.stdout:2/140: dread - d12/d1c/f27 zero size 2024-05-25T06:11:32.074 INFO:tasks.workunit.client.1.smithi203.stdout:2/588: rename d2/d38/d2f/f34 to d2/d1a/d33/d80/fc5 0 2024-05-25T06:11:32.076 INFO:tasks.workunit.client.0.smithi160.stdout:2/141: mknod d12/d13/c28 0 2024-05-25T06:11:32.076 INFO:tasks.workunit.client.1.smithi203.stdout:0/490: dread dd/d3d/d4f/f87 [0,4194304] 0 2024-05-25T06:11:32.078 INFO:tasks.workunit.client.0.smithi160.stdout:2/142: mknod d12/c29 0 2024-05-25T06:11:32.081 INFO:tasks.workunit.client.1.smithi203.stdout:2/589: truncate d2/d38/d2f/fab 460389 0 2024-05-25T06:11:32.082 INFO:tasks.workunit.client.1.smithi203.stdout:2/590: chown d2/fb8 1186 1 2024-05-25T06:11:32.084 INFO:tasks.workunit.client.1.smithi203.stdout:2/591: stat d2/d4/d10/d58/da0 0 2024-05-25T06:11:32.084 INFO:tasks.workunit.client.1.smithi203.stdout:0/491: dwrite dd/d3d/d75/f79 [0,4194304] 0 2024-05-25T06:11:32.086 INFO:tasks.workunit.client.0.smithi160.stdout:2/143: dwrite d12/d13/f15 [0,4194304] 0 2024-05-25T06:11:32.086 INFO:tasks.workunit.client.1.smithi203.stdout:0/492: fdatasync dd/d5c/f90 0 2024-05-25T06:11:32.091 INFO:tasks.workunit.client.0.smithi160.stdout:5/185: dread db/dc/d13/f28 [0,4194304] 0 2024-05-25T06:11:32.093 INFO:tasks.workunit.client.1.smithi203.stdout:7/552: dwrite d1/d9/d2c/d3a/d4e/f37 [0,4194304] 0 2024-05-25T06:11:32.093 INFO:tasks.workunit.client.0.smithi160.stdout:2/144: unlink ld 0 2024-05-25T06:11:32.097 INFO:tasks.workunit.client.0.smithi160.stdout:5/186: mkdir db/d37 0 2024-05-25T06:11:32.098 INFO:tasks.workunit.client.1.smithi203.stdout:8/689: dread d2/d5/d28/d38/d4e/fbb [0,4194304] 0 2024-05-25T06:11:32.099 INFO:tasks.workunit.client.0.smithi160.stdout:2/145: mkdir d12/d1c/d2a 0 2024-05-25T06:11:32.101 INFO:tasks.workunit.client.0.smithi160.stdout:6/164: sync 2024-05-25T06:11:32.101 INFO:tasks.workunit.client.0.smithi160.stdout:8/231: sync 2024-05-25T06:11:32.101 INFO:tasks.workunit.client.0.smithi160.stdout:9/89: sync 2024-05-25T06:11:32.101 INFO:tasks.workunit.client.1.smithi203.stdout:0/493: creat dd/d5c/d63/faf x:0 0 0 2024-05-25T06:11:32.101 INFO:tasks.workunit.client.1.smithi203.stdout:5/814: sync 2024-05-25T06:11:32.102 INFO:tasks.workunit.client.1.smithi203.stdout:1/503: sync 2024-05-25T06:11:32.102 INFO:tasks.workunit.client.1.smithi203.stdout:9/515: sync 2024-05-25T06:11:32.102 INFO:tasks.workunit.client.0.smithi160.stdout:9/90: stat d4/d11/d12 0 2024-05-25T06:11:32.102 INFO:tasks.workunit.client.0.smithi160.stdout:8/232: write d5/f3e [587158,104575] 0 2024-05-25T06:11:32.103 INFO:tasks.workunit.client.1.smithi203.stdout:6/823: dread d8/d11/d1b/d75/dcb/dcf/f7e [0,4194304] 0 2024-05-25T06:11:32.103 INFO:tasks.workunit.client.0.smithi160.stdout:5/187: rename db/dc/d13/d1d/d23/d24/d2e/f34 to db/d1c/f38 0 2024-05-25T06:11:32.104 INFO:tasks.workunit.client.0.smithi160.stdout:2/146: creat d12/d1c/f2b x:0 0 0 2024-05-25T06:11:32.105 INFO:tasks.workunit.client.0.smithi160.stdout:2/147: write f7 [6102354,24707] 0 2024-05-25T06:11:32.106 INFO:tasks.workunit.client.1.smithi203.stdout:7/553: mkdir d1/dbd 0 2024-05-25T06:11:32.106 INFO:tasks.workunit.client.0.smithi160.stdout:2/148: write f10 [433154,33275] 0 2024-05-25T06:11:32.107 INFO:tasks.workunit.client.0.smithi160.stdout:6/165: creat d5/f3f x:0 0 0 2024-05-25T06:11:32.107 INFO:tasks.workunit.client.1.smithi203.stdout:2/592: getdents d2/d4/d10/d32 0 2024-05-25T06:11:32.108 INFO:tasks.workunit.client.0.smithi160.stdout:2/149: chown d12/d13/c28 32254507 1 2024-05-25T06:11:32.108 INFO:tasks.workunit.client.1.smithi203.stdout:9/516: truncate d0/d17/d20/d3c/db2/d7a/fb4 994237 0 2024-05-25T06:11:32.108 INFO:tasks.workunit.client.0.smithi160.stdout:9/91: symlink d4/d9/l15 0 2024-05-25T06:11:32.108 INFO:tasks.workunit.client.1.smithi203.stdout:5/815: chown dc/f80 11078 1 2024-05-25T06:11:32.110 INFO:tasks.workunit.client.1.smithi203.stdout:6/824: mkdir d8/d70/d38/d108 0 2024-05-25T06:11:32.110 INFO:tasks.workunit.client.0.smithi160.stdout:8/233: unlink d5/d9/l2e 0 2024-05-25T06:11:32.112 INFO:tasks.workunit.client.1.smithi203.stdout:2/593: mknod d2/d6/cc6 0 2024-05-25T06:11:32.112 INFO:tasks.workunit.client.1.smithi203.stdout:6/825: fsync d8/d11/d1b/d75/fda 0 2024-05-25T06:11:32.114 INFO:tasks.workunit.client.1.smithi203.stdout:8/690: rename d2/d5/d10/d11/d59/f63 to d2/d5/d18/d31/dac/dc2/d8a/fe8 0 2024-05-25T06:11:32.114 INFO:tasks.workunit.client.0.smithi160.stdout:2/150: creat d12/d1c/f2c x:0 0 0 2024-05-25T06:11:32.114 INFO:tasks.workunit.client.1.smithi203.stdout:9/517: mkdir d0/df/d37/da8/db8 0 2024-05-25T06:11:32.116 INFO:tasks.workunit.client.0.smithi160.stdout:8/234: dread d5/d44/d1f/f42 [0,4194304] 0 2024-05-25T06:11:32.116 INFO:tasks.workunit.client.0.smithi160.stdout:9/92: dwrite f2 [0,4194304] 0 2024-05-25T06:11:32.118 INFO:tasks.workunit.client.0.smithi160.stdout:5/188: link db/dc/d13/l18 db/l39 0 2024-05-25T06:11:32.118 INFO:tasks.workunit.client.1.smithi203.stdout:5/816: symlink dc/d11/d18/dc0/dd0/l120 0 2024-05-25T06:11:32.118 INFO:tasks.workunit.client.0.smithi160.stdout:6/166: mkdir d5/de/d32/d35/d40 0 2024-05-25T06:11:32.121 INFO:tasks.workunit.client.0.smithi160.stdout:9/93: creat d4/d11/d14/f16 x:0 0 0 2024-05-25T06:11:32.121 INFO:tasks.workunit.client.1.smithi203.stdout:6/826: symlink d8/d70/d38/d108/l109 0 2024-05-25T06:11:32.121 INFO:tasks.workunit.client.1.smithi203.stdout:3/693: sync 2024-05-25T06:11:32.121 INFO:tasks.workunit.client.1.smithi203.stdout:4/655: sync 2024-05-25T06:11:32.122 INFO:tasks.workunit.client.0.smithi160.stdout:5/189: rmdir db/dc/d13/d1d/d23/d24 39 2024-05-25T06:11:32.122 INFO:tasks.workunit.client.0.smithi160.stdout:9/94: chown d4 8097 1 2024-05-25T06:11:32.123 INFO:tasks.workunit.client.1.smithi203.stdout:9/518: creat d0/df/d37/da8/db8/fb9 x:0 0 0 2024-05-25T06:11:32.124 INFO:tasks.workunit.client.0.smithi160.stdout:2/151: fdatasync d12/d13/f15 0 2024-05-25T06:11:32.125 INFO:tasks.workunit.client.1.smithi203.stdout:2/594: creat d2/d1a/fc7 x:0 0 0 2024-05-25T06:11:32.126 INFO:tasks.workunit.client.0.smithi160.stdout:8/235: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:32.127 INFO:tasks.workunit.client.1.smithi203.stdout:3/694: write d10/d22/f86 [1450063,99389] 0 2024-05-25T06:11:32.128 INFO:tasks.workunit.client.1.smithi203.stdout:4/656: write d0/d8/d1f/fb7 [4166939,17423] 0 2024-05-25T06:11:32.128 INFO:tasks.workunit.client.0.smithi160.stdout:1/97: sync 2024-05-25T06:11:32.128 INFO:tasks.workunit.client.0.smithi160.stdout:7/156: sync 2024-05-25T06:11:32.129 INFO:tasks.workunit.client.0.smithi160.stdout:3/127: sync 2024-05-25T06:11:32.129 INFO:tasks.workunit.client.0.smithi160.stdout:0/175: sync 2024-05-25T06:11:32.129 INFO:tasks.workunit.client.0.smithi160.stdout:4/277: sync 2024-05-25T06:11:32.131 INFO:tasks.workunit.client.1.smithi203.stdout:1/504: dread d14/d57/d1d/d3e/f61 [0,4194304] 0 2024-05-25T06:11:32.132 INFO:tasks.workunit.client.1.smithi203.stdout:5/817: creat dc/d11/d17/d1a/d72/de4/f121 x:0 0 0 2024-05-25T06:11:32.133 INFO:tasks.workunit.client.0.smithi160.stdout:3/128: write d19/d22/f28 [538655,106896] 0 2024-05-25T06:11:32.135 INFO:tasks.workunit.client.0.smithi160.stdout:7/157: readlink d6/l27 0 2024-05-25T06:11:32.136 INFO:tasks.workunit.client.1.smithi203.stdout:0/494: write dd/d10/d34/f3c [547760,105621] 0 2024-05-25T06:11:32.137 INFO:tasks.workunit.client.1.smithi203.stdout:5/818: chown dc/d11/d17/d1a/d72/f109 473 1 2024-05-25T06:11:32.137 INFO:tasks.workunit.client.0.smithi160.stdout:5/190: write db/dc/d13/d1d/d23/d24/d2e/f32 [26957,112816] 0 2024-05-25T06:11:32.138 INFO:tasks.workunit.client.0.smithi160.stdout:8/236: dread d5/d44/d1f/f22 [0,4194304] 0 2024-05-25T06:11:32.139 INFO:tasks.workunit.client.0.smithi160.stdout:1/98: mkdir de/d1b 0 2024-05-25T06:11:32.140 INFO:tasks.workunit.client.1.smithi203.stdout:0/495: fdatasync dd/d5c/f68 0 2024-05-25T06:11:32.140 INFO:tasks.workunit.client.1.smithi203.stdout:1/505: read d14/d57/d1d/d3e/d36/f39 [1015216,56185] 0 2024-05-25T06:11:32.140 INFO:tasks.workunit.client.1.smithi203.stdout:2/595: mkdir d2/d4/d10/dc8 0 2024-05-25T06:11:32.140 INFO:tasks.workunit.client.0.smithi160.stdout:9/95: dwrite d4/fa [0,4194304] 0 2024-05-25T06:11:32.141 INFO:tasks.workunit.client.1.smithi203.stdout:0/496: stat dd/d10/d34/d4c 0 2024-05-25T06:11:32.142 INFO:tasks.workunit.client.1.smithi203.stdout:3/695: rename d10/d16/d23/d45/fd7 to d10/d16/d23/d4a/d88/fdf 0 2024-05-25T06:11:32.142 INFO:tasks.workunit.client.1.smithi203.stdout:4/657: dwrite d0/d9/f17 [0,4194304] 0 2024-05-25T06:11:32.143 INFO:tasks.workunit.client.0.smithi160.stdout:1/99: write de/f10 [145994,23063] 0 2024-05-25T06:11:32.146 INFO:tasks.workunit.client.1.smithi203.stdout:3/696: fdatasync d10/d16/d50/f84 0 2024-05-25T06:11:32.146 INFO:tasks.workunit.client.0.smithi160.stdout:1/100: fsync de/f14 0 2024-05-25T06:11:32.147 INFO:tasks.workunit.client.0.smithi160.stdout:2/152: dwrite d12/f25 [0,4194304] 0 2024-05-25T06:11:32.148 INFO:tasks.workunit.client.1.smithi203.stdout:6/827: creat d8/d2a/d6c/d87/f10a x:0 0 0 2024-05-25T06:11:32.152 INFO:tasks.workunit.client.1.smithi203.stdout:6/828: write d8/d2a/d63/fe9 [827341,19783] 0 2024-05-25T06:11:32.153 INFO:tasks.workunit.client.0.smithi160.stdout:4/278: mknod d4/c52 0 2024-05-25T06:11:32.154 INFO:tasks.workunit.client.0.smithi160.stdout:5/191: write db/dc/d13/d1d/d23/f33 [802149,39631] 0 2024-05-25T06:11:32.155 INFO:tasks.workunit.client.1.smithi203.stdout:2/596: creat d2/d1a/d33/d80/fc9 x:0 0 0 2024-05-25T06:11:32.157 INFO:tasks.workunit.client.1.smithi203.stdout:8/691: write d2/d5/d10/f53 [1093475,127587] 0 2024-05-25T06:11:32.157 INFO:tasks.workunit.client.1.smithi203.stdout:4/658: dwrite d0/d8/d1f/d26/d3b/f97 [4194304,4194304] 0 2024-05-25T06:11:32.158 INFO:tasks.workunit.client.0.smithi160.stdout:6/167: dwrite d5/f1e [0,4194304] 0 2024-05-25T06:11:32.159 INFO:tasks.workunit.client.0.smithi160.stdout:6/168: chown d5/de/d32/f37 4650409 1 2024-05-25T06:11:32.161 INFO:tasks.workunit.client.1.smithi203.stdout:6/829: chown d8/d11/d15/d28/f1d 792993 1 2024-05-25T06:11:32.161 INFO:tasks.workunit.client.1.smithi203.stdout:1/506: getdents d14/d57/d1d/da8 0 2024-05-25T06:11:32.161 INFO:tasks.workunit.client.1.smithi203.stdout:0/497: getdents dd/d3d/d48/d91/dad 0 2024-05-25T06:11:32.161 INFO:tasks.workunit.client.0.smithi160.stdout:3/129: rename d19/f1b to d19/d22/f2a 0 2024-05-25T06:11:32.162 INFO:tasks.workunit.client.1.smithi203.stdout:3/697: creat d10/d16/d23/d6c/fe0 x:0 0 0 2024-05-25T06:11:32.163 INFO:tasks.workunit.client.0.smithi160.stdout:9/96: mknod d4/c17 0 2024-05-25T06:11:32.163 INFO:tasks.workunit.client.0.smithi160.stdout:3/130: truncate d19/d22/f24 664006 0 2024-05-25T06:11:32.163 INFO:tasks.workunit.client.1.smithi203.stdout:6/830: write d8/d11/d15/f59 [703440,4282] 0 2024-05-25T06:11:32.165 INFO:tasks.workunit.client.1.smithi203.stdout:3/698: dread - d10/d16/d23/d45/f93 zero size 2024-05-25T06:11:32.168 INFO:tasks.workunit.client.1.smithi203.stdout:4/659: chown d0/d8/d10/d91/da2/ca3 57 1 2024-05-25T06:11:32.168 INFO:tasks.workunit.client.1.smithi203.stdout:0/498: fdatasync dd/d3d/d4f/f50 0 2024-05-25T06:11:32.168 INFO:tasks.workunit.client.0.smithi160.stdout:6/169: dwrite d5/f7 [0,4194304] 0 2024-05-25T06:11:32.170 INFO:tasks.workunit.client.0.smithi160.stdout:5/192: symlink db/dc/d13/d1d/l3a 0 2024-05-25T06:11:32.171 INFO:tasks.workunit.client.1.smithi203.stdout:7/554: dread d1/d9/d2c/d36/d66/f8d [0,4194304] 0 2024-05-25T06:11:32.173 INFO:tasks.workunit.client.0.smithi160.stdout:4/279: rename d4/d6/d35/f3c to d4/d6/d35/f53 0 2024-05-25T06:11:32.174 INFO:tasks.workunit.client.0.smithi160.stdout:4/280: read d4/f11 [5722386,58119] 0 2024-05-25T06:11:32.176 INFO:tasks.workunit.client.1.smithi203.stdout:1/507: mkdir d14/d57/d97/da4/dad 0 2024-05-25T06:11:32.177 INFO:tasks.workunit.client.1.smithi203.stdout:6/831: mkdir d8/d2a/d6c/dc1/d10b 0 2024-05-25T06:11:32.179 INFO:tasks.workunit.client.1.smithi203.stdout:5/819: write dc/d11/fab [506987,20562] 0 2024-05-25T06:11:32.181 INFO:tasks.workunit.client.0.smithi160.stdout:9/97: unlink d4/d11/d14/f16 0 2024-05-25T06:11:32.182 INFO:tasks.workunit.client.0.smithi160.stdout:1/101: mkdir de/d1b/d1c 0 2024-05-25T06:11:32.183 INFO:tasks.workunit.client.0.smithi160.stdout:9/98: write d4/fa [3676611,107418] 0 2024-05-25T06:11:32.185 INFO:tasks.workunit.client.0.smithi160.stdout:9/99: chown d4/d9/l15 1788 1 2024-05-25T06:11:32.186 INFO:tasks.workunit.client.0.smithi160.stdout:9/100: stat d4/d9/f13 0 2024-05-25T06:11:32.186 INFO:tasks.workunit.client.0.smithi160.stdout:4/281: dwrite d4/f3b [4194304,4194304] 0 2024-05-25T06:11:32.187 INFO:tasks.workunit.client.0.smithi160.stdout:9/101: read d4/fd [630344,97883] 0 2024-05-25T06:11:32.188 INFO:tasks.workunit.client.0.smithi160.stdout:4/282: read d4/d1c/f3e [3533213,45096] 0 2024-05-25T06:11:32.188 INFO:tasks.workunit.client.1.smithi203.stdout:9/519: dwrite d0/d17/d20/f31 [0,4194304] 0 2024-05-25T06:11:32.192 INFO:tasks.workunit.client.1.smithi203.stdout:7/555: fdatasync d1/d11/d3d/d4c/f93 0 2024-05-25T06:11:32.192 INFO:tasks.workunit.client.1.smithi203.stdout:6/832: mkdir d8/d2a/d6c/d10c 0 2024-05-25T06:11:32.194 INFO:tasks.workunit.client.0.smithi160.stdout:5/193: rename db/d1c to db/dc/d13/d1d/d31/d3b 0 2024-05-25T06:11:32.196 INFO:tasks.workunit.client.1.smithi203.stdout:6/833: chown d8/d11/d1b/d75/dcb/dcf/d80/ca3 79 1 2024-05-25T06:11:32.197 INFO:tasks.workunit.client.1.smithi203.stdout:0/499: creat dd/d10/d34/d4c/d99/d70/fb0 x:0 0 0 2024-05-25T06:11:32.202 INFO:tasks.workunit.client.1.smithi203.stdout:9/520: rmdir d0/d17/d20/d29 39 2024-05-25T06:11:32.205 INFO:tasks.workunit.client.0.smithi160.stdout:2/153: mknod d12/d13/c2d 0 2024-05-25T06:11:32.205 INFO:tasks.workunit.client.0.smithi160.stdout:3/131: link d19/d22/f28 d19/f2b 0 2024-05-25T06:11:32.205 INFO:tasks.workunit.client.1.smithi203.stdout:1/508: dread d14/d57/d1d/d3e/d36/f39 [0,4194304] 0 2024-05-25T06:11:32.206 INFO:tasks.workunit.client.0.smithi160.stdout:7/158: getdents d6/dc 0 2024-05-25T06:11:32.210 INFO:tasks.workunit.client.0.smithi160.stdout:9/102: mknod d4/d9/c18 0 2024-05-25T06:11:32.210 INFO:tasks.workunit.client.0.smithi160.stdout:9/103: chown f2 94 1 2024-05-25T06:11:32.211 INFO:tasks.workunit.client.0.smithi160.stdout:9/104: chown d4/d9/c18 72669123 1 2024-05-25T06:11:32.212 INFO:tasks.workunit.client.1.smithi203.stdout:7/556: unlink d1/d11/d3d/l88 0 2024-05-25T06:11:32.214 INFO:tasks.workunit.client.1.smithi203.stdout:5/820: creat dc/d11/d17/d1a/d2d/d4d/f122 x:0 0 0 2024-05-25T06:11:32.215 INFO:tasks.workunit.client.0.smithi160.stdout:5/194: mknod db/dc/d13/c3c 0 2024-05-25T06:11:32.216 INFO:tasks.workunit.client.1.smithi203.stdout:2/597: dwrite d2/d38/d2f/fab [0,4194304] 0 2024-05-25T06:11:32.217 INFO:tasks.workunit.client.1.smithi203.stdout:0/500: mkdir dd/d3d/d48/db1 0 2024-05-25T06:11:32.218 INFO:tasks.workunit.client.1.smithi203.stdout:1/509: mknod d14/d57/d29/d94/cae 0 2024-05-25T06:11:32.219 INFO:tasks.workunit.client.1.smithi203.stdout:3/699: getdents d10/d16/d50/d8b 0 2024-05-25T06:11:32.221 INFO:tasks.workunit.client.0.smithi160.stdout:8/237: truncate d5/d44/d1f/d1e/d28/f3f 2846488 0 2024-05-25T06:11:32.223 INFO:tasks.workunit.client.0.smithi160.stdout:3/132: creat d19/d22/f2c x:0 0 0 2024-05-25T06:11:32.223 INFO:tasks.workunit.client.0.smithi160.stdout:3/133: dread - d19/d22/f2c zero size 2024-05-25T06:11:32.224 INFO:tasks.workunit.client.0.smithi160.stdout:7/159: mknod d6/dc/c34 0 2024-05-25T06:11:32.229 INFO:tasks.workunit.client.0.smithi160.stdout:9/105: mknod d4/d11/c19 0 2024-05-25T06:11:32.230 INFO:tasks.workunit.client.1.smithi203.stdout:4/660: dwrite d0/d8/d1f/d26/d3b/fac [0,4194304] 0 2024-05-25T06:11:32.231 INFO:tasks.workunit.client.0.smithi160.stdout:2/154: mkdir d12/d1c/d2e 0 2024-05-25T06:11:32.232 INFO:tasks.workunit.client.1.smithi203.stdout:8/692: dread d2/f14 [0,4194304] 0 2024-05-25T06:11:32.233 INFO:tasks.workunit.client.1.smithi203.stdout:2/598: mkdir d2/d4/d10/d32/dca 0 2024-05-25T06:11:32.233 INFO:tasks.workunit.client.0.smithi160.stdout:8/238: symlink d5/d39/l4f 0 2024-05-25T06:11:32.233 INFO:tasks.workunit.client.0.smithi160.stdout:1/102: dread fb [0,4194304] 0 2024-05-25T06:11:32.234 INFO:tasks.workunit.client.1.smithi203.stdout:5/821: write dc/f80 [2459822,13676] 0 2024-05-25T06:11:32.234 INFO:tasks.workunit.client.0.smithi160.stdout:7/160: dwrite d6/dc/f17 [0,4194304] 0 2024-05-25T06:11:32.235 INFO:tasks.workunit.client.0.smithi160.stdout:3/134: creat d19/d22/f2d x:0 0 0 2024-05-25T06:11:32.240 INFO:tasks.workunit.client.0.smithi160.stdout:5/195: mknod db/dc/c3d 0 2024-05-25T06:11:32.241 INFO:tasks.workunit.client.0.smithi160.stdout:9/106: symlink d4/d11/d14/l1a 0 2024-05-25T06:11:32.241 INFO:tasks.workunit.client.0.smithi160.stdout:5/196: readlink db/l14 0 2024-05-25T06:11:32.242 INFO:tasks.workunit.client.0.smithi160.stdout:7/161: dwrite d6/dc/f24 [0,4194304] 0 2024-05-25T06:11:32.244 INFO:tasks.workunit.client.0.smithi160.stdout:7/162: fsync d6/f12 0 2024-05-25T06:11:32.244 INFO:tasks.workunit.client.0.smithi160.stdout:4/283: dread d4/d6/d35/f53 [0,4194304] 0 2024-05-25T06:11:32.246 INFO:tasks.workunit.client.1.smithi203.stdout:8/693: dread - d2/d5/d10/d11/d1f/d75/f8d zero size 2024-05-25T06:11:32.246 INFO:tasks.workunit.client.1.smithi203.stdout:3/700: dwrite d10/d22/f86 [0,4194304] 0 2024-05-25T06:11:32.247 INFO:tasks.workunit.client.1.smithi203.stdout:4/661: write d0/d8/d3a/f46 [2353142,109797] 0 2024-05-25T06:11:32.247 INFO:tasks.workunit.client.0.smithi160.stdout:8/239: mknod d5/d44/d1f/d1e/d28/c50 0 2024-05-25T06:11:32.248 INFO:tasks.workunit.client.1.smithi203.stdout:1/510: getdents d14/d57/d7e 0 2024-05-25T06:11:32.249 INFO:tasks.workunit.client.1.smithi203.stdout:2/599: chown d2/fe 10120805 1 2024-05-25T06:11:32.250 INFO:tasks.workunit.client.0.smithi160.stdout:8/240: write d5/d7/f19 [2635870,123896] 0 2024-05-25T06:11:32.253 INFO:tasks.workunit.client.1.smithi203.stdout:5/822: stat dc/d11/d17/d1a/d22/f39 0 2024-05-25T06:11:32.253 INFO:tasks.workunit.client.0.smithi160.stdout:7/163: dwrite d6/dc/ff [4194304,4194304] 0 2024-05-25T06:11:32.253 INFO:tasks.workunit.client.0.smithi160.stdout:2/155: dread d12/f1d [0,4194304] 0 2024-05-25T06:11:32.254 INFO:tasks.workunit.client.1.smithi203.stdout:6/834: dread d8/d11/d15/fbc [0,4194304] 0 2024-05-25T06:11:32.255 INFO:tasks.workunit.client.0.smithi160.stdout:5/197: write db/dc/d13/f28 [5016603,109371] 0 2024-05-25T06:11:32.255 INFO:tasks.workunit.client.0.smithi160.stdout:8/241: chown d5/d44/d1f/d1e/d28/f38 3700 1 2024-05-25T06:11:32.256 INFO:tasks.workunit.client.0.smithi160.stdout:2/156: write d12/f1d [5477257,71501] 0 2024-05-25T06:11:32.257 INFO:tasks.workunit.client.1.smithi203.stdout:0/501: dwrite dd/d10/d34/f3c [0,4194304] 0 2024-05-25T06:11:32.258 INFO:tasks.workunit.client.0.smithi160.stdout:2/157: write d12/d1c/f27 [695322,6279] 0 2024-05-25T06:11:32.258 INFO:tasks.workunit.client.1.smithi203.stdout:2/600: stat d2/d38/d2f/f4f 0 2024-05-25T06:11:32.260 INFO:tasks.workunit.client.0.smithi160.stdout:4/284: mkdir d4/d1c/d54 0 2024-05-25T06:11:32.264 INFO:tasks.workunit.client.1.smithi203.stdout:4/662: creat d0/d8/d3a/d81/d93/d98/d99/fd7 x:0 0 0 2024-05-25T06:11:32.267 INFO:tasks.workunit.client.1.smithi203.stdout:4/663: write d0/d9/d50/fd1 [370594,93874] 0 2024-05-25T06:11:32.267 INFO:tasks.workunit.client.1.smithi203.stdout:8/694: creat d2/d5/d10/d45/d5d/fe9 x:0 0 0 2024-05-25T06:11:32.269 INFO:tasks.workunit.client.1.smithi203.stdout:7/557: dwrite d1/d9/d2c/d3a/f4f [0,4194304] 0 2024-05-25T06:11:32.270 INFO:tasks.workunit.client.1.smithi203.stdout:2/601: readlink d2/d38/l73 0 2024-05-25T06:11:32.271 INFO:tasks.workunit.client.0.smithi160.stdout:7/164: unlink d6/d2c/f31 0 2024-05-25T06:11:32.271 INFO:tasks.workunit.client.0.smithi160.stdout:8/242: unlink d5/d44/d1f/d1e/d28/c46 0 2024-05-25T06:11:32.272 INFO:tasks.workunit.client.0.smithi160.stdout:2/158: creat d12/d1c/f2f x:0 0 0 2024-05-25T06:11:32.272 INFO:tasks.workunit.client.0.smithi160.stdout:8/243: write d5/d9/f24 [683194,43572] 0 2024-05-25T06:11:32.273 INFO:tasks.workunit.client.0.smithi160.stdout:8/244: fsync d5/f48 0 2024-05-25T06:11:32.274 INFO:tasks.workunit.client.1.smithi203.stdout:9/521: dread d0/d17/f4b [0,4194304] 0 2024-05-25T06:11:32.274 INFO:tasks.workunit.client.1.smithi203.stdout:3/701: symlink d10/d16/d23/d4a/d88/le1 0 2024-05-25T06:11:32.275 INFO:tasks.workunit.client.0.smithi160.stdout:4/285: rmdir d4/d6/d35 39 2024-05-25T06:11:32.275 INFO:tasks.workunit.client.1.smithi203.stdout:5/823: chown dc/d11/d17/d1a/d2d/d4d/dcc/f102 6 1 2024-05-25T06:11:32.277 INFO:tasks.workunit.client.0.smithi160.stdout:4/286: write d4/d6/d21/f24 [4316106,126527] 0 2024-05-25T06:11:32.278 INFO:tasks.workunit.client.1.smithi203.stdout:3/702: fdatasync d10/d16/d23/d3a/f55 0 2024-05-25T06:11:32.278 INFO:tasks.workunit.client.0.smithi160.stdout:7/165: stat d6/l20 0 2024-05-25T06:11:32.280 INFO:tasks.workunit.client.0.smithi160.stdout:9/107: getdents d4 0 2024-05-25T06:11:32.280 INFO:tasks.workunit.client.1.smithi203.stdout:4/664: unlink d0/d9/d11/l74 0 2024-05-25T06:11:32.281 INFO:tasks.workunit.client.1.smithi203.stdout:2/602: unlink d2/d4/d10/d32/f3d 0 2024-05-25T06:11:32.281 INFO:tasks.workunit.client.1.smithi203.stdout:0/502: creat dd/d10/d34/d4c/d5d/fb2 x:0 0 0 2024-05-25T06:11:32.284 INFO:tasks.workunit.client.0.smithi160.stdout:7/166: mknod d6/d2b/c35 0 2024-05-25T06:11:32.286 INFO:tasks.workunit.client.0.smithi160.stdout:9/108: write d4/d9/fe [790725,16569] 0 2024-05-25T06:11:32.287 INFO:tasks.workunit.client.0.smithi160.stdout:2/159: dwrite d12/d1c/f2b [0,4194304] 0 2024-05-25T06:11:32.287 INFO:tasks.workunit.client.1.smithi203.stdout:7/558: readlink d1/d11/d3d/l86 0 2024-05-25T06:11:32.287 INFO:tasks.workunit.client.1.smithi203.stdout:1/511: link d14/d37/f85 d14/d57/d1d/faf 0 2024-05-25T06:11:32.288 INFO:tasks.workunit.client.1.smithi203.stdout:9/522: read d0/d17/d20/d29/f32 [608613,88716] 0 2024-05-25T06:11:32.288 INFO:tasks.workunit.client.0.smithi160.stdout:8/245: rename l4 to d5/d7/l51 0 2024-05-25T06:11:32.290 INFO:tasks.workunit.client.0.smithi160.stdout:4/287: write d4/d6/d35/f53 [4511243,109579] 0 2024-05-25T06:11:32.290 INFO:tasks.workunit.client.1.smithi203.stdout:4/665: write d0/d9/d11/d76/fc2 [596336,23725] 0 2024-05-25T06:11:32.293 INFO:tasks.workunit.client.0.smithi160.stdout:4/288: write d4/f33 [5136401,95174] 0 2024-05-25T06:11:32.293 INFO:tasks.workunit.client.1.smithi203.stdout:7/559: creat d1/d9/d2c/d3a/d4e/d50/fbe x:0 0 0 2024-05-25T06:11:32.296 INFO:tasks.workunit.client.0.smithi160.stdout:7/167: dwrite d6/dc/f17 [4194304,4194304] 0 2024-05-25T06:11:32.297 INFO:tasks.workunit.client.0.smithi160.stdout:3/135: getdents d19/d22 0 2024-05-25T06:11:32.298 INFO:tasks.workunit.client.0.smithi160.stdout:9/109: creat d4/d9/f1b x:0 0 0 2024-05-25T06:11:32.299 INFO:tasks.workunit.client.0.smithi160.stdout:5/198: dread f1 [0,4194304] 0 2024-05-25T06:11:32.300 INFO:tasks.workunit.client.1.smithi203.stdout:8/695: creat d2/d5/d18/d31/dac/dc2/d88/fea x:0 0 0 2024-05-25T06:11:32.302 INFO:tasks.workunit.client.0.smithi160.stdout:8/246: unlink d5/d44/d1f/d1e/d28/c2c 0 2024-05-25T06:11:32.307 INFO:tasks.workunit.client.1.smithi203.stdout:2/603: dwrite d2/d38/f6e [0,4194304] 0 2024-05-25T06:11:32.312 INFO:tasks.workunit.client.1.smithi203.stdout:1/512: dwrite d14/d57/d1d/f89 [0,4194304] 0 2024-05-25T06:11:32.313 INFO:tasks.workunit.client.0.smithi160.stdout:7/168: rename d6/l20 to d6/dc/l36 0 2024-05-25T06:11:32.313 INFO:tasks.workunit.client.0.smithi160.stdout:7/169: stat d6/d32 0 2024-05-25T06:11:32.313 INFO:tasks.workunit.client.0.smithi160.stdout:7/170: stat d6/dc/f24 0 2024-05-25T06:11:32.317 INFO:tasks.workunit.client.0.smithi160.stdout:3/136: creat d19/d22/f2e x:0 0 0 2024-05-25T06:11:32.317 INFO:tasks.workunit.client.1.smithi203.stdout:1/513: write d14/d57/d1d/d3e/d36/f8d [610768,122735] 0 2024-05-25T06:11:32.318 INFO:tasks.workunit.client.0.smithi160.stdout:9/110: creat d4/f1c x:0 0 0 2024-05-25T06:11:32.319 INFO:tasks.workunit.client.0.smithi160.stdout:5/199: fdatasync db/dc/f15 0 2024-05-25T06:11:32.319 INFO:tasks.workunit.client.1.smithi203.stdout:7/560: dwrite d1/d9/f18 [0,4194304] 0 2024-05-25T06:11:32.321 INFO:tasks.workunit.client.0.smithi160.stdout:1/103: write de/f15 [2006489,95147] 0 2024-05-25T06:11:32.321 INFO:tasks.workunit.client.0.smithi160.stdout:5/200: stat db/dc/d13/l16 0 2024-05-25T06:11:32.324 INFO:tasks.workunit.client.0.smithi160.stdout:7/171: dwrite d6/dc/f25 [0,4194304] 0 2024-05-25T06:11:32.325 INFO:tasks.workunit.client.0.smithi160.stdout:1/104: write de/f13 [49186,46758] 0 2024-05-25T06:11:32.326 INFO:tasks.workunit.client.1.smithi203.stdout:0/503: dwrite dd/d32/f54 [0,4194304] 0 2024-05-25T06:11:32.327 INFO:tasks.workunit.client.0.smithi160.stdout:0/176: sync 2024-05-25T06:11:32.327 INFO:tasks.workunit.client.1.smithi203.stdout:8/696: dread d2/f14 [4194304,4194304] 0 2024-05-25T06:11:32.328 INFO:tasks.workunit.client.0.smithi160.stdout:5/201: write db/dc/d13/f2c [966389,78435] 0 2024-05-25T06:11:32.330 INFO:tasks.workunit.client.1.smithi203.stdout:6/835: truncate d8/d70/f7c 1243843 0 2024-05-25T06:11:32.332 INFO:tasks.workunit.client.1.smithi203.stdout:0/504: chown dd/d10/d34/d4c/d99/c85 876 1 2024-05-25T06:11:32.337 INFO:tasks.workunit.client.0.smithi160.stdout:8/247: mkdir d5/d52 0 2024-05-25T06:11:32.337 INFO:tasks.workunit.client.0.smithi160.stdout:9/111: mknod d4/d11/d14/c1d 0 2024-05-25T06:11:32.339 INFO:tasks.workunit.client.0.smithi160.stdout:8/248: chown d5/d44/d1f/d1e/c45 55286 1 2024-05-25T06:11:32.340 INFO:tasks.workunit.client.1.smithi203.stdout:0/505: dread - dd/d32/f38 zero size 2024-05-25T06:11:32.343 INFO:tasks.workunit.client.0.smithi160.stdout:1/105: creat de/d1b/f1d x:0 0 0 2024-05-25T06:11:32.343 INFO:tasks.workunit.client.1.smithi203.stdout:6/836: mknod d8/d11/d15/d28/d52/d43/c10d 0 2024-05-25T06:11:32.343 INFO:tasks.workunit.client.1.smithi203.stdout:0/506: dread - dd/d3a/fae zero size 2024-05-25T06:11:32.343 INFO:tasks.workunit.client.0.smithi160.stdout:1/106: dread - de/f17 zero size 2024-05-25T06:11:32.346 INFO:tasks.workunit.client.0.smithi160.stdout:2/160: link d12/l21 d12/d1c/d2a/l30 0 2024-05-25T06:11:32.349 INFO:tasks.workunit.client.1.smithi203.stdout:1/514: dwrite d14/d57/d1d/d3e/d36/f8d [0,4194304] 0 2024-05-25T06:11:32.349 INFO:tasks.workunit.client.1.smithi203.stdout:6/837: creat d8/d5a/f10e x:0 0 0 2024-05-25T06:11:32.352 INFO:tasks.workunit.client.1.smithi203.stdout:1/515: chown d14/f54 1267746 1 2024-05-25T06:11:32.353 INFO:tasks.workunit.client.0.smithi160.stdout:9/112: mkdir d4/d11/d1e 0 2024-05-25T06:11:32.354 INFO:tasks.workunit.client.0.smithi160.stdout:8/249: mkdir d5/d9/d53 0 2024-05-25T06:11:32.356 INFO:tasks.workunit.client.0.smithi160.stdout:0/177: unlink d2b/d31/c32 0 2024-05-25T06:11:32.360 INFO:tasks.workunit.client.0.smithi160.stdout:1/107: symlink de/l1e 0 2024-05-25T06:11:32.361 INFO:tasks.workunit.client.0.smithi160.stdout:2/161: creat d12/d1c/f31 x:0 0 0 2024-05-25T06:11:32.362 INFO:tasks.workunit.client.1.smithi203.stdout:3/703: dread d10/d16/d23/d3a/f5d [0,4194304] 0 2024-05-25T06:11:32.363 INFO:tasks.workunit.client.1.smithi203.stdout:0/507: dwrite dd/d10/d34/d4c/d5d/fb2 [0,4194304] 0 2024-05-25T06:11:32.364 INFO:tasks.workunit.client.0.smithi160.stdout:5/202: rmdir db/d37 0 2024-05-25T06:11:32.366 INFO:tasks.workunit.client.0.smithi160.stdout:7/172: rmdir d6 39 2024-05-25T06:11:32.367 INFO:tasks.workunit.client.1.smithi203.stdout:1/516: creat d14/d57/d1d/d9f/fb0 x:0 0 0 2024-05-25T06:11:32.368 INFO:tasks.workunit.client.0.smithi160.stdout:3/137: rmdir d19/d22 39 2024-05-25T06:11:32.368 INFO:tasks.workunit.client.0.smithi160.stdout:4/289: dread d4/f3b [0,4194304] 0 2024-05-25T06:11:32.369 INFO:tasks.workunit.client.0.smithi160.stdout:2/162: dwrite ff [0,4194304] 0 2024-05-25T06:11:32.369 INFO:tasks.workunit.client.0.smithi160.stdout:9/113: creat d4/d11/d1e/f1f x:0 0 0 2024-05-25T06:11:32.371 INFO:tasks.workunit.client.0.smithi160.stdout:2/163: dread - fe zero size 2024-05-25T06:11:32.371 INFO:tasks.workunit.client.0.smithi160.stdout:9/114: write d4/f1c [688098,122279] 0 2024-05-25T06:11:32.372 INFO:tasks.workunit.client.0.smithi160.stdout:2/164: write f10 [218134,126565] 0 2024-05-25T06:11:32.373 INFO:tasks.workunit.client.1.smithi203.stdout:5/824: dread dc/d11/d42/d94/fc7 [0,4194304] 0 2024-05-25T06:11:32.378 INFO:tasks.workunit.client.1.smithi203.stdout:0/508: fdatasync dd/d3d/d48/f7a 0 2024-05-25T06:11:32.378 INFO:tasks.workunit.client.0.smithi160.stdout:0/178: mkdir d2b/d31/d35/d36/d37 0 2024-05-25T06:11:32.378 INFO:tasks.workunit.client.0.smithi160.stdout:0/179: truncate f1b 1071559 0 2024-05-25T06:11:32.378 INFO:tasks.workunit.client.0.smithi160.stdout:6/170: sync 2024-05-25T06:11:32.379 INFO:tasks.workunit.client.0.smithi160.stdout:5/203: creat db/dc/d13/d1d/d23/d24/f3e x:0 0 0 2024-05-25T06:11:32.380 INFO:tasks.workunit.client.1.smithi203.stdout:6/838: dwrite d8/f2c [0,4194304] 0 2024-05-25T06:11:32.380 INFO:tasks.workunit.client.1.smithi203.stdout:3/704: link d10/fb5 d10/d16/d23/d38/fe2 0 2024-05-25T06:11:32.381 INFO:tasks.workunit.client.1.smithi203.stdout:5/825: stat dc/d11/d18/dc0/d107 0 2024-05-25T06:11:32.383 INFO:tasks.workunit.client.1.smithi203.stdout:8/697: dread d2/d5/d10/d45/d5d/f5c [0,4194304] 0 2024-05-25T06:11:32.384 INFO:tasks.workunit.client.0.smithi160.stdout:9/115: mkdir d4/d20 0 2024-05-25T06:11:32.384 INFO:tasks.workunit.client.0.smithi160.stdout:5/204: chown db 54618 1 2024-05-25T06:11:32.384 INFO:tasks.workunit.client.1.smithi203.stdout:1/517: chown d14/d37/f85 4 1 2024-05-25T06:11:32.385 INFO:tasks.workunit.client.0.smithi160.stdout:8/250: rename d5/d9/d33 to d5/d54 0 2024-05-25T06:11:32.386 INFO:tasks.workunit.client.0.smithi160.stdout:2/165: dwrite fe [0,4194304] 0 2024-05-25T06:11:32.390 INFO:tasks.workunit.client.1.smithi203.stdout:6/839: write d8/d70/fa1 [258731,119382] 0 2024-05-25T06:11:32.390 INFO:tasks.workunit.client.0.smithi160.stdout:9/116: chown f2 15732418 1 2024-05-25T06:11:32.393 INFO:tasks.workunit.client.0.smithi160.stdout:3/138: mkdir d19/d2f 0 2024-05-25T06:11:32.393 INFO:tasks.workunit.client.1.smithi203.stdout:7/561: dwrite d1/d9/d60/f65 [0,4194304] 0 2024-05-25T06:11:32.397 INFO:tasks.workunit.client.0.smithi160.stdout:4/290: rename d4/d6/d21/d4c to d4/d6/d35/d3d/d55 0 2024-05-25T06:11:32.397 INFO:tasks.workunit.client.1.smithi203.stdout:3/705: truncate d10/d22/d2a/d99/fb6 14987 0 2024-05-25T06:11:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:32 smithi203 ceph-mon[36030]: pgmap v16: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 128 MiB/s wr, 328 op/s 2024-05-25T06:11:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:32 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:32 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:32.400 INFO:tasks.workunit.client.1.smithi203.stdout:2/604: dwrite d2/f8 [0,4194304] 0 2024-05-25T06:11:32.401 INFO:tasks.workunit.client.1.smithi203.stdout:4/666: dread d0/d9/d11/d54/fc7 [0,4194304] 0 2024-05-25T06:11:32.403 INFO:tasks.workunit.client.0.smithi160.stdout:5/205: write db/dc/d13/d1d/d23/f2d [848343,50380] 0 2024-05-25T06:11:32.404 INFO:tasks.workunit.client.0.smithi160.stdout:2/166: chown d12/d13/c28 0 1 2024-05-25T06:11:32.404 INFO:tasks.workunit.client.1.smithi203.stdout:3/706: fdatasync d10/d16/d23/d46/faf 0 2024-05-25T06:11:32.406 INFO:tasks.workunit.client.1.smithi203.stdout:1/518: dwrite d14/f54 [0,4194304] 0 2024-05-25T06:11:32.406 INFO:tasks.workunit.client.0.smithi160.stdout:0/180: dwrite d2b/d31/f34 [0,4194304] 0 2024-05-25T06:11:32.408 INFO:tasks.workunit.client.0.smithi160.stdout:1/108: getdents de 0 2024-05-25T06:11:32.410 INFO:tasks.workunit.client.1.smithi203.stdout:1/519: chown d14/d57/d97/l98 3014 1 2024-05-25T06:11:32.410 INFO:tasks.workunit.client.0.smithi160.stdout:7/173: dread d6/dc/f25 [0,4194304] 0 2024-05-25T06:11:32.411 INFO:tasks.workunit.client.0.smithi160.stdout:2/167: write ff [4958319,24527] 0 2024-05-25T06:11:32.411 INFO:tasks.workunit.client.1.smithi203.stdout:8/698: mkdir d2/d5/d18/d31/dac/dc2/d88/d93/deb 0 2024-05-25T06:11:32.411 INFO:tasks.workunit.client.0.smithi160.stdout:1/109: truncate f2 4597052 0 2024-05-25T06:11:32.412 INFO:tasks.workunit.client.1.smithi203.stdout:7/562: fsync d1/d9/fb 0 2024-05-25T06:11:32.412 INFO:tasks.workunit.client.0.smithi160.stdout:6/171: dwrite d5/f26 [4194304,4194304] 0 2024-05-25T06:11:32.413 INFO:tasks.workunit.client.1.smithi203.stdout:5/826: dread - dc/daa/fcf zero size 2024-05-25T06:11:32.414 INFO:tasks.workunit.client.0.smithi160.stdout:0/181: dread - d2b/f2e zero size 2024-05-25T06:11:32.414 INFO:tasks.workunit.client.1.smithi203.stdout:1/520: chown d14/dac 6120 1 2024-05-25T06:11:32.416 INFO:tasks.workunit.client.1.smithi203.stdout:6/840: creat d8/d11/d1b/d75/dcb/dcf/d80/f10f x:0 0 0 2024-05-25T06:11:32.418 INFO:tasks.workunit.client.1.smithi203.stdout:8/699: read - d2/d5/d28/d38/fde zero size 2024-05-25T06:11:32.423 INFO:tasks.workunit.client.0.smithi160.stdout:5/206: dread - db/dc/d13/d1d/d31/d3b/f38 zero size 2024-05-25T06:11:32.423 INFO:tasks.workunit.client.0.smithi160.stdout:9/117: getdents d4/d20 0 2024-05-25T06:11:32.424 INFO:tasks.workunit.client.0.smithi160.stdout:7/174: fsync d6/dc/ff 0 2024-05-25T06:11:32.424 INFO:tasks.workunit.client.0.smithi160.stdout:8/251: dwrite d5/f35 [4194304,4194304] 0 2024-05-25T06:11:32.424 INFO:tasks.workunit.client.0.smithi160.stdout:6/172: write d5/de/d32/f37 [92116,94857] 0 2024-05-25T06:11:32.426 INFO:tasks.workunit.client.1.smithi203.stdout:2/605: creat d2/d42/d8e/fcb x:0 0 0 2024-05-25T06:11:32.430 INFO:tasks.workunit.client.0.smithi160.stdout:2/168: fdatasync f5 0 2024-05-25T06:11:32.430 INFO:tasks.workunit.client.0.smithi160.stdout:4/291: rename d4/f33 to d4/d6/d35/d3d/d55/f56 0 2024-05-25T06:11:32.432 INFO:tasks.workunit.client.1.smithi203.stdout:6/841: chown d8/d11/d15/d28/d52/d43/d73/db7/fff 943354 1 2024-05-25T06:11:32.432 INFO:tasks.workunit.client.1.smithi203.stdout:7/563: dwrite d1/d9/d2c/d62/d9b/fa0 [0,4194304] 0 2024-05-25T06:11:32.434 INFO:tasks.workunit.client.1.smithi203.stdout:4/667: dwrite d0/d8/d1f/d26/d3b/f5a [0,4194304] 0 2024-05-25T06:11:32.434 INFO:tasks.workunit.client.0.smithi160.stdout:0/182: truncate f5 5842202 0 2024-05-25T06:11:32.435 INFO:tasks.workunit.client.1.smithi203.stdout:8/700: mknod d2/d5/d10/d45/d5d/d76/cec 0 2024-05-25T06:11:32.436 INFO:tasks.workunit.client.0.smithi160.stdout:8/252: write d5/d44/d1f/d1e/d28/f4d [2427503,32882] 0 2024-05-25T06:11:32.438 INFO:tasks.workunit.client.0.smithi160.stdout:1/110: symlink de/d1b/d1c/l1f 0 2024-05-25T06:11:32.439 INFO:tasks.workunit.client.0.smithi160.stdout:3/139: dread d19/f2b [0,4194304] 0 2024-05-25T06:11:32.440 INFO:tasks.workunit.client.0.smithi160.stdout:9/118: truncate d4/d9/f1b 858226 0 2024-05-25T06:11:32.444 INFO:tasks.workunit.client.1.smithi203.stdout:1/521: rmdir d14/d57/d1d/d9f/da6 0 2024-05-25T06:11:32.444 INFO:tasks.workunit.client.1.smithi203.stdout:4/668: dwrite d0/d9/fa4 [0,4194304] 0 2024-05-25T06:11:32.444 INFO:tasks.workunit.client.0.smithi160.stdout:1/111: symlink de/d1b/d1c/l20 0 2024-05-25T06:11:32.447 INFO:tasks.workunit.client.0.smithi160.stdout:6/173: dwrite d5/fa [0,4194304] 0 2024-05-25T06:11:32.450 INFO:tasks.workunit.client.0.smithi160.stdout:3/140: rename ff to d19/d2f/f30 0 2024-05-25T06:11:32.450 INFO:tasks.workunit.client.0.smithi160.stdout:5/207: dwrite db/f2a [0,4194304] 0 2024-05-25T06:11:32.450 INFO:tasks.workunit.client.1.smithi203.stdout:5/827: link dc/d11/d17/d1a/d103/fd9 dc/d11/d42/d94/f123 0 2024-05-25T06:11:32.454 INFO:tasks.workunit.client.0.smithi160.stdout:2/169: link d12/d13/f24 d12/d1c/f32 0 2024-05-25T06:11:32.454 INFO:tasks.workunit.client.1.smithi203.stdout:7/564: unlink d1/d9/d2c/d62/l9c 0 2024-05-25T06:11:32.457 INFO:tasks.workunit.client.0.smithi160.stdout:1/112: mkdir de/d21 0 2024-05-25T06:11:32.457 INFO:tasks.workunit.client.0.smithi160.stdout:5/208: write db/f2a [556188,80549] 0 2024-05-25T06:11:32.458 INFO:tasks.workunit.client.0.smithi160.stdout:0/183: dwrite f24 [0,4194304] 0 2024-05-25T06:11:32.460 INFO:tasks.workunit.client.1.smithi203.stdout:3/707: link d10/d16/f19 d10/d16/d23/d4a/fe3 0 2024-05-25T06:11:32.463 INFO:tasks.workunit.client.0.smithi160.stdout:8/253: dwrite d5/f3e [0,4194304] 0 2024-05-25T06:11:32.463 INFO:tasks.workunit.client.0.smithi160.stdout:6/174: creat d5/d39/f41 x:0 0 0 2024-05-25T06:11:32.464 INFO:tasks.workunit.client.1.smithi203.stdout:4/669: mkdir d0/d8/d1f/d26/d3b/d9c/dd8 0 2024-05-25T06:11:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:32 smithi160 ceph-mon[29041]: pgmap v16: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 128 MiB/s wr, 328 op/s 2024-05-25T06:11:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:32 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:32.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:32 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:32.471 INFO:tasks.workunit.client.0.smithi160.stdout:5/209: unlink db/dc/d13/f21 0 2024-05-25T06:11:32.471 INFO:tasks.workunit.client.1.smithi203.stdout:6/842: symlink d8/l110 0 2024-05-25T06:11:32.472 INFO:tasks.workunit.client.1.smithi203.stdout:5/828: mknod dc/d11/d17/d1a/d2d/d4d/d64/c124 0 2024-05-25T06:11:32.472 INFO:tasks.workunit.client.0.smithi160.stdout:1/113: mkdir de/d1b/d22 0 2024-05-25T06:11:32.473 INFO:tasks.workunit.client.1.smithi203.stdout:6/843: fsync d8/d2a/d63/fe9 0 2024-05-25T06:11:32.473 INFO:tasks.workunit.client.0.smithi160.stdout:5/210: write db/dc/d13/f2c [1722179,25668] 0 2024-05-25T06:11:32.473 INFO:tasks.workunit.client.0.smithi160.stdout:0/184: creat d2b/d31/f38 x:0 0 0 2024-05-25T06:11:32.475 INFO:tasks.workunit.client.1.smithi203.stdout:8/701: link d2/d5/d28/d38/d4e/d97/f77 d2/d5/d10/d4a/fed 0 2024-05-25T06:11:32.475 INFO:tasks.workunit.client.1.smithi203.stdout:3/708: stat d10/d16/d23/c82 0 2024-05-25T06:11:32.476 INFO:tasks.workunit.client.0.smithi160.stdout:2/170: truncate d12/f1d 1392531 0 2024-05-25T06:11:32.478 INFO:tasks.workunit.client.0.smithi160.stdout:0/185: chown f16 0 1 2024-05-25T06:11:32.479 INFO:tasks.workunit.client.0.smithi160.stdout:1/114: mkdir de/d1b/d22/d23 0 2024-05-25T06:11:32.479 INFO:tasks.workunit.client.0.smithi160.stdout:2/171: write d12/d1c/f2b [3260859,75056] 0 2024-05-25T06:11:32.479 INFO:tasks.workunit.client.0.smithi160.stdout:5/211: chown db/l14 107 1 2024-05-25T06:11:32.480 INFO:tasks.workunit.client.1.smithi203.stdout:7/565: symlink d1/d11/lbf 0 2024-05-25T06:11:32.481 INFO:tasks.workunit.client.0.smithi160.stdout:9/119: dread f2 [0,4194304] 0 2024-05-25T06:11:32.481 INFO:tasks.workunit.client.0.smithi160.stdout:3/141: readlink d19/d22/l25 0 2024-05-25T06:11:32.482 INFO:tasks.workunit.client.0.smithi160.stdout:6/175: creat d5/de/f42 x:0 0 0 2024-05-25T06:11:32.485 INFO:tasks.workunit.client.0.smithi160.stdout:1/115: write de/d1b/f1d [263096,210] 0 2024-05-25T06:11:32.486 INFO:tasks.workunit.client.1.smithi203.stdout:3/709: mkdir d10/d16/d23/d6c/de4 0 2024-05-25T06:11:32.486 INFO:tasks.workunit.client.1.smithi203.stdout:8/702: unlink d2/d5/d28/d38/d4e/d97/f9b 0 2024-05-25T06:11:32.487 INFO:tasks.workunit.client.1.smithi203.stdout:4/670: symlink d0/d8/ld9 0 2024-05-25T06:11:32.487 INFO:tasks.workunit.client.0.smithi160.stdout:5/212: write db/f11 [2079233,123894] 0 2024-05-25T06:11:32.487 INFO:tasks.workunit.client.0.smithi160.stdout:2/172: mknod d12/d1c/c33 0 2024-05-25T06:11:32.488 INFO:tasks.workunit.client.1.smithi203.stdout:5/829: creat dc/d11/d106/f125 x:0 0 0 2024-05-25T06:11:32.488 INFO:tasks.workunit.client.1.smithi203.stdout:6/844: mknod d8/d11/d1b/d75/dcb/dcf/d80/dfb/c111 0 2024-05-25T06:11:32.491 INFO:tasks.workunit.client.0.smithi160.stdout:4/292: dread d4/d6/d21/f32 [0,4194304] 0 2024-05-25T06:11:32.492 INFO:tasks.workunit.client.0.smithi160.stdout:8/254: dwrite d5/d44/d1f/d1e/d28/d4a/f3d [0,4194304] 0 2024-05-25T06:11:32.492 INFO:tasks.workunit.client.1.smithi203.stdout:1/522: getdents d14/d57/d6b/da1 0 2024-05-25T06:11:32.492 INFO:tasks.workunit.client.1.smithi203.stdout:6/845: stat d8/d5a/d9b/fed 0 2024-05-25T06:11:32.494 INFO:tasks.workunit.client.1.smithi203.stdout:8/703: write d2/d5/d10/d45/d5d/f82 [450585,24600] 0 2024-05-25T06:11:32.496 INFO:tasks.workunit.client.1.smithi203.stdout:3/710: rename d10/d22/f42 to d10/d16/d23/d4a/fe5 0 2024-05-25T06:11:32.496 INFO:tasks.workunit.client.0.smithi160.stdout:3/142: dwrite fb [0,4194304] 0 2024-05-25T06:11:32.497 INFO:tasks.workunit.client.1.smithi203.stdout:3/711: dread - d10/fa2 zero size 2024-05-25T06:11:32.498 INFO:tasks.workunit.client.0.smithi160.stdout:9/120: getdents d4/d9 0 2024-05-25T06:11:32.498 INFO:tasks.workunit.client.1.smithi203.stdout:4/671: unlink d0/d8/d10/d91/cbd 0 2024-05-25T06:11:32.498 INFO:tasks.workunit.client.0.smithi160.stdout:6/176: dread d5/fa [0,4194304] 0 2024-05-25T06:11:32.499 INFO:tasks.workunit.client.0.smithi160.stdout:9/121: chown d4/c17 1 1 2024-05-25T06:11:32.500 INFO:tasks.workunit.client.1.smithi203.stdout:3/712: fsync d10/d16/d23/d3a/f80 0 2024-05-25T06:11:32.501 INFO:tasks.workunit.client.1.smithi203.stdout:5/830: creat dc/d11/d17/d1a/d2d/d4d/dcc/f126 x:0 0 0 2024-05-25T06:11:32.501 INFO:tasks.workunit.client.1.smithi203.stdout:1/523: rmdir d14/d37/d5e 39 2024-05-25T06:11:32.502 INFO:tasks.workunit.client.0.smithi160.stdout:8/255: creat d5/d44/d1f/f55 x:0 0 0 2024-05-25T06:11:32.506 INFO:tasks.workunit.client.1.smithi203.stdout:1/524: truncate d14/d57/d1d/faf 1717080 0 2024-05-25T06:11:32.508 INFO:tasks.workunit.client.1.smithi203.stdout:7/566: creat d1/d11/fc0 x:0 0 0 2024-05-25T06:11:32.510 INFO:tasks.workunit.client.1.smithi203.stdout:8/704: dwrite d2/d5/d18/d31/f3d [0,4194304] 0 2024-05-25T06:11:32.511 INFO:tasks.workunit.client.0.smithi160.stdout:9/122: dwrite d4/d9/f1b [0,4194304] 0 2024-05-25T06:11:32.513 INFO:tasks.workunit.client.1.smithi203.stdout:8/705: write d2/d5/d10/d11/d59/fd4 [940734,75700] 0 2024-05-25T06:11:32.513 INFO:tasks.workunit.client.0.smithi160.stdout:3/143: dwrite d19/d22/f2c [0,4194304] 0 2024-05-25T06:11:32.514 INFO:tasks.workunit.client.1.smithi203.stdout:7/567: fdatasync d1/d11/f80 0 2024-05-25T06:11:32.514 INFO:tasks.workunit.client.1.smithi203.stdout:8/706: fdatasync d2/d5/d10/f4d 0 2024-05-25T06:11:32.518 INFO:tasks.workunit.client.0.smithi160.stdout:1/116: dread de/d1b/f1d [0,4194304] 0 2024-05-25T06:11:32.518 INFO:tasks.workunit.client.1.smithi203.stdout:6/846: rename d8/d11/d47/d60/cfe to d8/d11/d15/d28/d52/c112 0 2024-05-25T06:11:32.519 INFO:tasks.workunit.client.0.smithi160.stdout:4/293: rename d4/d1c/l1d to d4/d6/l57 0 2024-05-25T06:11:32.519 INFO:tasks.workunit.client.0.smithi160.stdout:9/123: symlink d4/d11/l21 0 2024-05-25T06:11:32.520 INFO:tasks.workunit.client.0.smithi160.stdout:3/144: symlink d19/d22/l31 0 2024-05-25T06:11:32.522 INFO:tasks.workunit.client.1.smithi203.stdout:3/713: mkdir d10/d16/da1/de6 0 2024-05-25T06:11:32.524 INFO:tasks.workunit.client.1.smithi203.stdout:4/672: read d0/d8/d10/d91/fba [633882,92077] 0 2024-05-25T06:11:32.527 INFO:tasks.workunit.client.1.smithi203.stdout:5/831: creat dc/d11/d42/d94/dad/f127 x:0 0 0 2024-05-25T06:11:32.527 INFO:tasks.workunit.client.1.smithi203.stdout:7/568: rename d1/db7 to d1/dbd/dc1 0 2024-05-25T06:11:32.529 INFO:tasks.workunit.client.0.smithi160.stdout:9/124: dwrite d4/fa [0,4194304] 0 2024-05-25T06:11:32.529 INFO:tasks.workunit.client.0.smithi160.stdout:3/145: dread f15 [0,4194304] 0 2024-05-25T06:11:32.530 INFO:tasks.workunit.client.1.smithi203.stdout:5/832: write dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdd [111755,104177] 0 2024-05-25T06:11:32.532 INFO:tasks.workunit.client.1.smithi203.stdout:3/714: symlink d10/d16/d23/d67/db8/le7 0 2024-05-25T06:11:32.533 INFO:tasks.workunit.client.0.smithi160.stdout:7/175: truncate d6/dc/d21/f2a 1891680 0 2024-05-25T06:11:32.534 INFO:tasks.workunit.client.1.smithi203.stdout:4/673: write d0/f3 [2420948,114174] 0 2024-05-25T06:11:32.534 INFO:tasks.workunit.client.1.smithi203.stdout:5/833: truncate dc/d11/d17/f54 1026145 0 2024-05-25T06:11:32.536 INFO:tasks.workunit.client.1.smithi203.stdout:3/715: write d10/d16/d23/d45/f93 [916405,30373] 0 2024-05-25T06:11:32.537 INFO:tasks.workunit.client.1.smithi203.stdout:8/707: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/fee x:0 0 0 2024-05-25T06:11:32.537 INFO:tasks.workunit.client.0.smithi160.stdout:4/294: chown d4/d1a/l20 113619 1 2024-05-25T06:11:32.538 INFO:tasks.workunit.client.1.smithi203.stdout:6/847: mkdir d8/d2a/d6c/d87/d113 0 2024-05-25T06:11:32.539 INFO:tasks.workunit.client.0.smithi160.stdout:8/256: link d5/d44/d1f/d1e/d28/l2b d5/d44/d1f/d1e/l56 0 2024-05-25T06:11:32.540 INFO:tasks.workunit.client.1.smithi203.stdout:7/569: mkdir d1/d9/dc2 0 2024-05-25T06:11:32.542 INFO:tasks.workunit.client.0.smithi160.stdout:9/125: dread d4/d9/fe [0,4194304] 0 2024-05-25T06:11:32.542 INFO:tasks.workunit.client.0.smithi160.stdout:3/146: mknod d19/d2f/c32 0 2024-05-25T06:11:32.543 INFO:tasks.workunit.client.1.smithi203.stdout:2/606: write d2/d4/d10/d32/f1b [6993562,94148] 0 2024-05-25T06:11:32.543 INFO:tasks.workunit.client.0.smithi160.stdout:8/257: dread - d5/f48 zero size 2024-05-25T06:11:32.546 INFO:tasks.workunit.client.1.smithi203.stdout:6/848: dread d8/fc [0,4194304] 0 2024-05-25T06:11:32.547 INFO:tasks.workunit.client.1.smithi203.stdout:8/708: symlink d2/d5/d10/d45/d90/lef 0 2024-05-25T06:11:32.548 INFO:tasks.workunit.client.0.smithi160.stdout:3/147: symlink d19/d2f/l33 0 2024-05-25T06:11:32.549 INFO:tasks.workunit.client.0.smithi160.stdout:0/186: rmdir d2b 39 2024-05-25T06:11:32.552 INFO:tasks.workunit.client.0.smithi160.stdout:8/258: dwrite d5/d9/f4b [0,4194304] 0 2024-05-25T06:11:32.553 INFO:tasks.workunit.client.1.smithi203.stdout:6/849: creat d8/d2a/da9/f114 x:0 0 0 2024-05-25T06:11:32.555 INFO:tasks.workunit.client.0.smithi160.stdout:3/148: unlink fb 0 2024-05-25T06:11:32.558 INFO:tasks.workunit.client.0.smithi160.stdout:3/149: write f0 [372934,27268] 0 2024-05-25T06:11:32.561 INFO:tasks.workunit.client.0.smithi160.stdout:2/173: dread f8 [0,4194304] 0 2024-05-25T06:11:32.562 INFO:tasks.workunit.client.1.smithi203.stdout:2/607: creat d2/d1a/d89/fcc x:0 0 0 2024-05-25T06:11:32.562 INFO:tasks.workunit.client.1.smithi203.stdout:5/834: rmdir dc/d11/d17/d1a/d2d/d4d/d64/d114 0 2024-05-25T06:11:32.562 INFO:tasks.workunit.client.1.smithi203.stdout:6/850: read f0 [10384949,127151] 0 2024-05-25T06:11:32.564 INFO:tasks.workunit.client.0.smithi160.stdout:6/177: dread f2 [0,4194304] 0 2024-05-25T06:11:32.565 INFO:tasks.workunit.client.0.smithi160.stdout:8/259: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:32.566 INFO:tasks.workunit.client.1.smithi203.stdout:2/608: creat d2/d42/d4e/d68/fcd x:0 0 0 2024-05-25T06:11:32.566 INFO:tasks.workunit.client.0.smithi160.stdout:5/213: truncate db/f11 3717672 0 2024-05-25T06:11:32.567 INFO:tasks.workunit.client.1.smithi203.stdout:5/835: symlink dc/d11/d106/l128 0 2024-05-25T06:11:32.573 INFO:tasks.workunit.client.1.smithi203.stdout:5/836: fsync dc/d11/d17/f56 0 2024-05-25T06:11:32.575 INFO:tasks.workunit.client.1.smithi203.stdout:2/609: mknod d2/d38/d43/d60/da3/cce 0 2024-05-25T06:11:32.575 INFO:tasks.workunit.client.0.smithi160.stdout:3/150: mkdir d19/d2f/d34 0 2024-05-25T06:11:32.578 INFO:tasks.workunit.client.0.smithi160.stdout:2/174: dwrite d12/d1c/f27 [0,4194304] 0 2024-05-25T06:11:32.580 INFO:tasks.workunit.client.1.smithi203.stdout:6/851: truncate d8/d11/d1b/d75/fda 515718 0 2024-05-25T06:11:32.581 INFO:tasks.workunit.client.0.smithi160.stdout:1/117: dread de/f11 [0,4194304] 0 2024-05-25T06:11:32.581 INFO:tasks.workunit.client.1.smithi203.stdout:6/852: dread - d8/d2a/d6c/dc1/f105 zero size 2024-05-25T06:11:32.581 INFO:tasks.workunit.client.1.smithi203.stdout:7/570: dread d1/f24 [0,4194304] 0 2024-05-25T06:11:32.582 INFO:tasks.workunit.client.0.smithi160.stdout:0/187: mknod d2b/d31/d35/d36/c39 0 2024-05-25T06:11:32.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/118: write de/ff [966480,63403] 0 2024-05-25T06:11:32.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/119: chown de 62 1 2024-05-25T06:11:32.584 INFO:tasks.workunit.client.0.smithi160.stdout:7/176: getdents d6/d2b 0 2024-05-25T06:11:32.585 INFO:tasks.workunit.client.1.smithi203.stdout:5/837: mknod dc/d11/d17/d1a/dd2/c129 0 2024-05-25T06:11:32.585 INFO:tasks.workunit.client.1.smithi203.stdout:3/716: dread d10/d16/d50/f84 [0,4194304] 0 2024-05-25T06:11:32.585 INFO:tasks.workunit.client.0.smithi160.stdout:1/120: chown de/f15 91333 1 2024-05-25T06:11:32.588 INFO:tasks.workunit.client.1.smithi203.stdout:7/571: creat d1/d9/d2c/d36/d54/fc3 x:0 0 0 2024-05-25T06:11:32.589 INFO:tasks.workunit.client.1.smithi203.stdout:2/610: truncate d2/d38/f1d 1649048 0 2024-05-25T06:11:32.590 INFO:tasks.workunit.client.0.smithi160.stdout:3/151: creat d19/d2f/f35 x:0 0 0 2024-05-25T06:11:32.591 INFO:tasks.workunit.client.0.smithi160.stdout:2/175: rmdir d12/d13 39 2024-05-25T06:11:32.593 INFO:tasks.workunit.client.0.smithi160.stdout:9/126: truncate d4/fa 2059453 0 2024-05-25T06:11:32.593 INFO:tasks.workunit.client.1.smithi203.stdout:3/717: fdatasync d10/d16/d50/f70 0 2024-05-25T06:11:32.594 INFO:tasks.workunit.client.1.smithi203.stdout:5/838: dwrite dc/fe6 [4194304,4194304] 0 2024-05-25T06:11:32.595 INFO:tasks.workunit.client.0.smithi160.stdout:9/127: fdatasync d4/d11/d1e/f1f 0 2024-05-25T06:11:32.595 INFO:tasks.workunit.client.0.smithi160.stdout:7/177: dwrite d6/dc/ff [0,4194304] 0 2024-05-25T06:11:32.596 INFO:tasks.workunit.client.0.smithi160.stdout:9/128: write d4/d9/f1b [4418103,8512] 0 2024-05-25T06:11:32.598 INFO:tasks.workunit.client.0.smithi160.stdout:0/188: creat d2b/f3a x:0 0 0 2024-05-25T06:11:32.600 INFO:tasks.workunit.client.1.smithi203.stdout:8/709: dwrite d2/f14 [0,4194304] 0 2024-05-25T06:11:32.602 INFO:tasks.workunit.client.0.smithi160.stdout:9/129: dread - d4/d9/f13 zero size 2024-05-25T06:11:32.602 INFO:tasks.workunit.client.1.smithi203.stdout:7/572: symlink d1/dbd/dc1/lc4 0 2024-05-25T06:11:32.603 INFO:tasks.workunit.client.0.smithi160.stdout:9/130: read f2 [280421,31922] 0 2024-05-25T06:11:32.604 INFO:tasks.workunit.client.0.smithi160.stdout:8/260: rmdir d5/d54 0 2024-05-25T06:11:32.606 INFO:tasks.workunit.client.0.smithi160.stdout:5/214: dread db/dc/fe [0,4194304] 0 2024-05-25T06:11:32.606 INFO:tasks.workunit.client.0.smithi160.stdout:3/152: symlink d19/l36 0 2024-05-25T06:11:32.607 INFO:tasks.workunit.client.1.smithi203.stdout:5/839: dwrite dc/d11/f11a [0,4194304] 0 2024-05-25T06:11:32.609 INFO:tasks.workunit.client.1.smithi203.stdout:6/853: link d8/d11/d47/d60/c64 d8/d11/d15/d28/d52/d43/d73/c115 0 2024-05-25T06:11:32.616 INFO:tasks.workunit.client.0.smithi160.stdout:2/176: rmdir d12/d1c/d2e 0 2024-05-25T06:11:32.617 INFO:tasks.workunit.client.0.smithi160.stdout:6/178: dread d5/f26 [0,4194304] 0 2024-05-25T06:11:32.618 INFO:tasks.workunit.client.1.smithi203.stdout:2/611: symlink d2/d1a/d89/daa/lcf 0 2024-05-25T06:11:32.619 INFO:tasks.workunit.client.0.smithi160.stdout:0/189: creat d2b/d31/d35/d36/d37/f3b x:0 0 0 2024-05-25T06:11:32.620 INFO:tasks.workunit.client.0.smithi160.stdout:9/131: mkdir d4/d20/d22 0 2024-05-25T06:11:32.620 INFO:tasks.workunit.client.1.smithi203.stdout:6/854: dwrite d8/d70/d38/f4c [0,4194304] 0 2024-05-25T06:11:32.620 INFO:tasks.workunit.client.0.smithi160.stdout:3/153: dwrite d19/d2f/f35 [0,4194304] 0 2024-05-25T06:11:32.622 INFO:tasks.workunit.client.1.smithi203.stdout:8/710: mkdir d2/d5/d28/d38/d4e/d97/ddf/df0 0 2024-05-25T06:11:32.625 INFO:tasks.workunit.client.1.smithi203.stdout:5/840: mknod dc/d11/d42/d94/d10a/c12a 0 2024-05-25T06:11:32.631 INFO:tasks.workunit.client.0.smithi160.stdout:2/177: rename d12 to d12/d13/d34 22 2024-05-25T06:11:32.631 INFO:tasks.workunit.client.1.smithi203.stdout:6/855: fsync d8/d11/d15/d28/f1d 0 2024-05-25T06:11:32.631 INFO:tasks.workunit.client.1.smithi203.stdout:7/573: dread d1/f1c [0,4194304] 0 2024-05-25T06:11:32.632 INFO:tasks.workunit.client.0.smithi160.stdout:9/132: rmdir d4/d11 39 2024-05-25T06:11:32.632 INFO:tasks.workunit.client.1.smithi203.stdout:3/718: unlink d10/d16/d71/lc3 0 2024-05-25T06:11:32.636 INFO:tasks.workunit.client.1.smithi203.stdout:5/841: unlink dc/d11/d17/d1a/d72/ce1 0 2024-05-25T06:11:32.636 INFO:tasks.workunit.client.0.smithi160.stdout:5/215: dwrite db/dc/d13/d1d/d23/d24/d2e/f35 [0,4194304] 0 2024-05-25T06:11:32.637 INFO:tasks.workunit.client.0.smithi160.stdout:2/178: creat d12/d13/f35 x:0 0 0 2024-05-25T06:11:32.637 INFO:tasks.workunit.client.1.smithi203.stdout:2/612: mknod d2/d4/d10/d32/dca/cd0 0 2024-05-25T06:11:32.640 INFO:tasks.workunit.client.1.smithi203.stdout:6/856: rmdir d8/d2a/d6c/dc1/dd8 39 2024-05-25T06:11:32.640 INFO:tasks.workunit.client.0.smithi160.stdout:5/216: mkdir db/dc/d13/d1d/d3f 0 2024-05-25T06:11:32.641 INFO:tasks.workunit.client.1.smithi203.stdout:6/857: fsync d8/d11/d47/f99 0 2024-05-25T06:11:32.642 INFO:tasks.workunit.client.1.smithi203.stdout:3/719: rmdir d10/d16/d23/d4a/d88 39 2024-05-25T06:11:32.646 INFO:tasks.workunit.client.0.smithi160.stdout:0/190: dwrite f1b [0,4194304] 0 2024-05-25T06:11:32.646 INFO:tasks.workunit.client.1.smithi203.stdout:7/574: rmdir d1/da9 0 2024-05-25T06:11:32.647 INFO:tasks.workunit.client.1.smithi203.stdout:2/613: dwrite d2/fb8 [0,4194304] 0 2024-05-25T06:11:32.650 INFO:tasks.workunit.client.1.smithi203.stdout:7/575: readlink d1/d11/d20/l64 0 2024-05-25T06:11:32.650 INFO:tasks.workunit.client.1.smithi203.stdout:3/720: readlink d10/d16/d23/d3a/la3 0 2024-05-25T06:11:32.650 INFO:tasks.workunit.client.1.smithi203.stdout:7/576: stat d1/d11/d20/lbb 0 2024-05-25T06:11:32.650 INFO:tasks.workunit.client.1.smithi203.stdout:6/858: mkdir d8/d5a/d9b/daf/d116 0 2024-05-25T06:11:32.651 INFO:tasks.workunit.client.1.smithi203.stdout:5/842: mknod dc/dcb/c12b 0 2024-05-25T06:11:32.651 INFO:tasks.workunit.client.0.smithi160.stdout:2/179: dwrite ff [0,4194304] 0 2024-05-25T06:11:32.651 INFO:tasks.workunit.client.0.smithi160.stdout:7/178: dread d6/dc/f19 [0,4194304] 0 2024-05-25T06:11:32.652 INFO:tasks.workunit.client.0.smithi160.stdout:2/180: write d12/d1c/f2f [294217,70265] 0 2024-05-25T06:11:32.653 INFO:tasks.workunit.client.1.smithi203.stdout:2/614: mkdir d2/d1a/d33/d80/dd1 0 2024-05-25T06:11:32.653 INFO:tasks.workunit.client.0.smithi160.stdout:0/191: read f21 [1034268,38118] 0 2024-05-25T06:11:32.655 INFO:tasks.workunit.client.1.smithi203.stdout:6/859: truncate d8/d11/d1b/d75/fd7 862950 0 2024-05-25T06:11:32.656 INFO:tasks.workunit.client.1.smithi203.stdout:6/860: chown d8/d70/c34 16 1 2024-05-25T06:11:32.657 INFO:tasks.workunit.client.0.smithi160.stdout:8/261: dread d5/d44/f37 [0,4194304] 0 2024-05-25T06:11:32.657 INFO:tasks.workunit.client.0.smithi160.stdout:5/217: rename db/dc/d13/d1d/d23/d24/d2e/f35 to db/dc/d13/d1d/d31/f40 0 2024-05-25T06:11:32.658 INFO:tasks.workunit.client.0.smithi160.stdout:5/218: chown db/dc/d13/c25 1136425 1 2024-05-25T06:11:32.659 INFO:tasks.workunit.client.1.smithi203.stdout:7/577: creat d1/d9/d2c/fc5 x:0 0 0 2024-05-25T06:11:32.659 INFO:tasks.workunit.client.0.smithi160.stdout:0/192: truncate d2b/d31/f38 253490 0 2024-05-25T06:11:32.661 INFO:tasks.workunit.client.0.smithi160.stdout:8/262: chown d5/fb 610730703 1 2024-05-25T06:11:32.663 INFO:tasks.workunit.client.1.smithi203.stdout:6/861: mknod d8/d5a/d9b/daf/d116/c117 0 2024-05-25T06:11:32.664 INFO:tasks.workunit.client.0.smithi160.stdout:3/154: getdents d19 0 2024-05-25T06:11:32.665 INFO:tasks.workunit.client.1.smithi203.stdout:7/578: symlink d1/lc6 0 2024-05-25T06:11:32.667 INFO:tasks.workunit.client.1.smithi203.stdout:2/615: dread d2/d4/d10/d32/f1b [0,4194304] 0 2024-05-25T06:11:32.668 INFO:tasks.workunit.client.0.smithi160.stdout:5/219: rename db/dc/d13/f28 to db/dc/d13/d1d/d31/d3b/f41 0 2024-05-25T06:11:32.669 INFO:tasks.workunit.client.1.smithi203.stdout:5/843: creat dc/f12c x:0 0 0 2024-05-25T06:11:32.669 INFO:tasks.workunit.client.0.smithi160.stdout:5/220: read f1 [1447649,5456] 0 2024-05-25T06:11:32.669 INFO:tasks.workunit.client.0.smithi160.stdout:7/179: dwrite d6/dc/f19 [0,4194304] 0 2024-05-25T06:11:32.672 INFO:tasks.workunit.client.0.smithi160.stdout:5/221: chown db/dc/d13/d1d/d31 48656 1 2024-05-25T06:11:32.672 INFO:tasks.workunit.client.0.smithi160.stdout:8/263: dwrite d5/fb [0,4194304] 0 2024-05-25T06:11:32.672 INFO:tasks.workunit.client.0.smithi160.stdout:0/193: link f10 d2b/d31/d35/d36/f3c 0 2024-05-25T06:11:32.676 INFO:tasks.workunit.client.0.smithi160.stdout:3/155: creat d19/d2f/f37 x:0 0 0 2024-05-25T06:11:32.677 INFO:tasks.workunit.client.0.smithi160.stdout:2/181: getdents d12/d1c/d2a 0 2024-05-25T06:11:32.679 INFO:tasks.workunit.client.0.smithi160.stdout:5/222: unlink db/dc/fe 0 2024-05-25T06:11:32.681 INFO:tasks.workunit.client.0.smithi160.stdout:8/264: unlink d5/d9/f13 0 2024-05-25T06:11:32.682 INFO:tasks.workunit.client.0.smithi160.stdout:5/223: write db/f22 [2021511,121598] 0 2024-05-25T06:11:32.682 INFO:tasks.workunit.client.0.smithi160.stdout:3/156: mkdir d19/d2f/d38 0 2024-05-25T06:11:32.684 INFO:tasks.workunit.client.0.smithi160.stdout:7/180: truncate d6/dc/d21/f2a 2607047 0 2024-05-25T06:11:32.686 INFO:tasks.workunit.client.0.smithi160.stdout:7/181: chown d6 6211 1 2024-05-25T06:11:32.687 INFO:tasks.workunit.client.0.smithi160.stdout:2/182: symlink d12/l36 0 2024-05-25T06:11:32.689 INFO:tasks.workunit.client.0.smithi160.stdout:8/265: fdatasync d5/d44/d1f/f42 0 2024-05-25T06:11:32.689 INFO:tasks.workunit.client.0.smithi160.stdout:9/133: dwrite d4/fd [4194304,4194304] 0 2024-05-25T06:11:32.690 INFO:tasks.workunit.client.0.smithi160.stdout:0/194: dwrite d2b/f30 [0,4194304] 0 2024-05-25T06:11:32.690 INFO:tasks.workunit.client.0.smithi160.stdout:8/266: stat d5/d39/l43 0 2024-05-25T06:11:32.691 INFO:tasks.workunit.client.0.smithi160.stdout:2/183: write d12/d13/f35 [852698,34231] 0 2024-05-25T06:11:32.691 INFO:tasks.workunit.client.0.smithi160.stdout:8/267: readlink d5/d7/l10 0 2024-05-25T06:11:32.692 INFO:tasks.workunit.client.0.smithi160.stdout:3/157: rename f15 to d19/d22/f39 0 2024-05-25T06:11:32.692 INFO:tasks.workunit.client.0.smithi160.stdout:2/184: stat fe 0 2024-05-25T06:11:32.692 INFO:tasks.workunit.client.0.smithi160.stdout:0/195: fsync f12 0 2024-05-25T06:11:32.693 INFO:tasks.workunit.client.0.smithi160.stdout:5/224: creat db/dc/d13/d1d/d31/d3b/f42 x:0 0 0 2024-05-25T06:11:32.695 INFO:tasks.workunit.client.0.smithi160.stdout:7/182: creat d6/d2c/f37 x:0 0 0 2024-05-25T06:11:32.697 INFO:tasks.workunit.client.0.smithi160.stdout:9/134: readlink d4/d11/l21 0 2024-05-25T06:11:32.697 INFO:tasks.workunit.client.0.smithi160.stdout:2/185: dread fb [0,4194304] 0 2024-05-25T06:11:32.698 INFO:tasks.workunit.client.0.smithi160.stdout:5/225: stat db/dc/d13/d1d/d23/d24/d2e/f2f 0 2024-05-25T06:11:32.699 INFO:tasks.workunit.client.0.smithi160.stdout:3/158: creat d19/f3a x:0 0 0 2024-05-25T06:11:32.699 INFO:tasks.workunit.client.0.smithi160.stdout:9/135: dread - d4/d11/d1e/f1f zero size 2024-05-25T06:11:32.700 INFO:tasks.workunit.client.0.smithi160.stdout:3/159: fdatasync f0 0 2024-05-25T06:11:32.700 INFO:tasks.workunit.client.0.smithi160.stdout:9/136: chown f2 111 1 2024-05-25T06:11:32.700 INFO:tasks.workunit.client.0.smithi160.stdout:0/196: rename d2b/l33 to d2b/d31/d35/d36/l3d 0 2024-05-25T06:11:32.701 INFO:tasks.workunit.client.0.smithi160.stdout:0/197: truncate f1f 864527 0 2024-05-25T06:11:32.704 INFO:tasks.workunit.client.0.smithi160.stdout:2/186: truncate f5 536243 0 2024-05-25T06:11:32.706 INFO:tasks.workunit.client.0.smithi160.stdout:9/137: dread f2 [4194304,4194304] 0 2024-05-25T06:11:32.708 INFO:tasks.workunit.client.0.smithi160.stdout:9/138: chown d4/d11 4 1 2024-05-25T06:11:32.709 INFO:tasks.workunit.client.0.smithi160.stdout:0/198: creat d2b/d31/d35/f3e x:0 0 0 2024-05-25T06:11:32.709 INFO:tasks.workunit.client.0.smithi160.stdout:3/160: dwrite f0 [0,4194304] 0 2024-05-25T06:11:32.711 INFO:tasks.workunit.client.0.smithi160.stdout:2/187: mkdir d12/d1c/d37 0 2024-05-25T06:11:32.712 INFO:tasks.workunit.client.0.smithi160.stdout:8/268: link d5/c32 d5/d44/d1f/d1e/d28/d49/c57 0 2024-05-25T06:11:32.713 INFO:tasks.workunit.client.0.smithi160.stdout:5/226: dwrite db/dc/d13/d1d/d31/d3b/f38 [0,4194304] 0 2024-05-25T06:11:32.717 INFO:tasks.workunit.client.0.smithi160.stdout:9/139: rename d4/d20 to d4/d11/d14/d23 0 2024-05-25T06:11:32.722 INFO:tasks.workunit.client.0.smithi160.stdout:9/140: dread f2 [4194304,4194304] 0 2024-05-25T06:11:32.722 INFO:tasks.workunit.client.0.smithi160.stdout:2/188: mknod d12/d1c/c38 0 2024-05-25T06:11:32.724 INFO:tasks.workunit.client.0.smithi160.stdout:8/269: mkdir d5/d44/d1f/d58 0 2024-05-25T06:11:32.725 INFO:tasks.workunit.client.0.smithi160.stdout:5/227: mknod db/dc/d13/d1d/d31/d3b/c43 0 2024-05-25T06:11:32.725 INFO:tasks.workunit.client.0.smithi160.stdout:2/189: symlink d12/d1c/d2a/l39 0 2024-05-25T06:11:32.726 INFO:tasks.workunit.client.0.smithi160.stdout:8/270: readlink d5/d7/l51 0 2024-05-25T06:11:32.727 INFO:tasks.workunit.client.0.smithi160.stdout:2/190: readlink d12/l14 0 2024-05-25T06:11:32.727 INFO:tasks.workunit.client.0.smithi160.stdout:8/271: dread - d5/f48 zero size 2024-05-25T06:11:32.727 INFO:tasks.workunit.client.0.smithi160.stdout:9/141: dread d4/fd [4194304,4194304] 0 2024-05-25T06:11:32.728 INFO:tasks.workunit.client.0.smithi160.stdout:5/228: rename db/f11 to db/dc/d13/f44 0 2024-05-25T06:11:32.729 INFO:tasks.workunit.client.0.smithi160.stdout:2/191: creat d12/d1c/d2a/f3a x:0 0 0 2024-05-25T06:11:32.730 INFO:tasks.workunit.client.0.smithi160.stdout:2/192: readlink d12/d13/l22 0 2024-05-25T06:11:32.730 INFO:tasks.workunit.client.0.smithi160.stdout:9/142: unlink d4/d11/d14/c1d 0 2024-05-25T06:11:32.731 INFO:tasks.workunit.client.0.smithi160.stdout:8/272: creat d5/d44/d1f/d1e/d28/d4a/f59 x:0 0 0 2024-05-25T06:11:32.733 INFO:tasks.workunit.client.0.smithi160.stdout:5/229: mkdir db/dc/d13/d1d/d23/d45 0 2024-05-25T06:11:32.733 INFO:tasks.workunit.client.1.smithi203.stdout:6/862: rename d8/d11/d15/d28/d52/d43/d73/c115 to d8/d5a/d9b/daf/c118 0 2024-05-25T06:11:32.734 INFO:tasks.workunit.client.0.smithi160.stdout:2/193: creat d12/d1c/d37/f3b x:0 0 0 2024-05-25T06:11:32.734 INFO:tasks.workunit.client.1.smithi203.stdout:2/616: creat d2/d1a/d89/fd2 x:0 0 0 2024-05-25T06:11:32.735 INFO:tasks.workunit.client.1.smithi203.stdout:6/863: read d8/d70/fa1 [1606573,55962] 0 2024-05-25T06:11:32.736 INFO:tasks.workunit.client.1.smithi203.stdout:6/864: write d8/d2a/d6c/d87/f10a [216079,6622] 0 2024-05-25T06:11:32.739 INFO:tasks.workunit.client.0.smithi160.stdout:3/161: dread d19/d2f/f30 [0,4194304] 0 2024-05-25T06:11:32.740 INFO:tasks.workunit.client.1.smithi203.stdout:5/844: mknod dc/d11/d17/d1a/d2d/d4d/ded/c12d 0 2024-05-25T06:11:32.752 INFO:tasks.workunit.client.1.smithi203.stdout:9/523: sync 2024-05-25T06:11:32.753 INFO:tasks.workunit.client.1.smithi203.stdout:5/845: write dc/d3c/d3f/fd8 [2025791,44146] 0 2024-05-25T06:11:32.755 INFO:tasks.workunit.client.1.smithi203.stdout:5/846: fdatasync dc/d11/f12 0 2024-05-25T06:11:32.758 INFO:tasks.workunit.client.1.smithi203.stdout:3/721: write d10/fb5 [1785448,49095] 0 2024-05-25T06:11:32.760 INFO:tasks.workunit.client.1.smithi203.stdout:8/711: dwrite d2/d5/d28/d38/d4e/f74 [0,4194304] 0 2024-05-25T06:11:32.760 INFO:tasks.workunit.client.1.smithi203.stdout:6/865: dread d8/d11/d47/fbe [0,4194304] 0 2024-05-25T06:11:32.761 INFO:tasks.workunit.client.1.smithi203.stdout:9/524: symlink d0/df/d2b/d96/lba 0 2024-05-25T06:11:32.762 INFO:tasks.workunit.client.1.smithi203.stdout:5/847: write dc/d11/d17/d1a/d2d/d4d/d64/f9e [574715,24670] 0 2024-05-25T06:11:32.763 INFO:tasks.workunit.client.1.smithi203.stdout:2/617: getdents d2/d38 0 2024-05-25T06:11:32.765 INFO:tasks.workunit.client.1.smithi203.stdout:5/848: stat dc/d11/d18/fc2 0 2024-05-25T06:11:32.766 INFO:tasks.workunit.client.1.smithi203.stdout:9/525: write d0/df/faa [1695460,42579] 0 2024-05-25T06:11:32.768 INFO:tasks.workunit.client.1.smithi203.stdout:3/722: truncate d10/d16/d23/d46/dd3/fd9 408316 0 2024-05-25T06:11:32.771 INFO:tasks.workunit.client.1.smithi203.stdout:7/579: getdents d1/d9/d2c/d62/d7a/d89 0 2024-05-25T06:11:32.772 INFO:tasks.workunit.client.0.smithi160.stdout:5/230: dread db/dc/d13/d1d/d23/d24/d2e/f32 [0,4194304] 0 2024-05-25T06:11:32.773 INFO:tasks.workunit.client.0.smithi160.stdout:2/194: read f8 [229819,79178] 0 2024-05-25T06:11:32.773 INFO:tasks.workunit.client.1.smithi203.stdout:9/526: symlink d0/d17/d20/d3c/db2/lbb 0 2024-05-25T06:11:32.774 INFO:tasks.workunit.client.0.smithi160.stdout:3/162: dread d19/d22/f39 [0,4194304] 0 2024-05-25T06:11:32.775 INFO:tasks.workunit.client.1.smithi203.stdout:2/618: dread d2/d4/d10/d5f/f81 [0,4194304] 0 2024-05-25T06:11:32.775 INFO:tasks.workunit.client.0.smithi160.stdout:9/143: dread d4/d9/fc [0,4194304] 0 2024-05-25T06:11:32.775 INFO:tasks.workunit.client.0.smithi160.stdout:8/273: dread d5/d44/d1f/d1e/d28/d4a/f3a [0,4194304] 0 2024-05-25T06:11:32.777 INFO:tasks.workunit.client.0.smithi160.stdout:3/163: fdatasync d19/d22/f24 0 2024-05-25T06:11:32.779 INFO:tasks.workunit.client.1.smithi203.stdout:5/849: symlink dc/daa/d10c/l12e 0 2024-05-25T06:11:32.779 INFO:tasks.workunit.client.1.smithi203.stdout:3/723: truncate d10/d16/d23/d4a/d88/f8d 4260811 0 2024-05-25T06:11:32.783 INFO:tasks.workunit.client.0.smithi160.stdout:2/195: mkdir d12/d3c 0 2024-05-25T06:11:32.783 INFO:tasks.workunit.client.1.smithi203.stdout:8/712: rmdir d2/d5/d10/d45/d5d/d76/dc7 0 2024-05-25T06:11:32.785 INFO:tasks.workunit.client.0.smithi160.stdout:7/183: dwrite f1 [0,4194304] 0 2024-05-25T06:11:32.785 INFO:tasks.workunit.client.1.smithi203.stdout:7/580: mkdir d1/d9/d2c/d79/dc7 0 2024-05-25T06:11:32.786 INFO:tasks.workunit.client.0.smithi160.stdout:0/199: dwrite d2b/d31/d35/d36/f3c [0,4194304] 0 2024-05-25T06:11:32.787 INFO:tasks.workunit.client.0.smithi160.stdout:7/184: fsync d6/d2c/f37 0 2024-05-25T06:11:32.787 INFO:tasks.workunit.client.0.smithi160.stdout:5/231: creat db/dc/d13/d1d/d23/f46 x:0 0 0 2024-05-25T06:11:32.788 INFO:tasks.workunit.client.0.smithi160.stdout:3/164: mknod d19/d2f/d38/c3b 0 2024-05-25T06:11:32.789 INFO:tasks.workunit.client.0.smithi160.stdout:8/274: mknod d5/d44/d1f/d1e/d28/d4a/d4e/c5a 0 2024-05-25T06:11:32.789 INFO:tasks.workunit.client.1.smithi203.stdout:3/724: symlink d10/d16/le8 0 2024-05-25T06:11:32.792 INFO:tasks.workunit.client.0.smithi160.stdout:7/185: chown d6/dc/l36 1967177 1 2024-05-25T06:11:32.793 INFO:tasks.workunit.client.0.smithi160.stdout:9/144: truncate f2 7014122 0 2024-05-25T06:11:32.794 INFO:tasks.workunit.client.1.smithi203.stdout:7/581: unlink d1/l1b 0 2024-05-25T06:11:32.795 INFO:tasks.workunit.client.1.smithi203.stdout:9/527: dread d0/f6f [0,4194304] 0 2024-05-25T06:11:32.797 INFO:tasks.workunit.client.1.smithi203.stdout:6/866: dread d8/d11/f90 [0,4194304] 0 2024-05-25T06:11:32.799 INFO:tasks.workunit.client.1.smithi203.stdout:5/850: unlink dc/d11/d17/d1a/caf 0 2024-05-25T06:11:32.799 INFO:tasks.workunit.client.0.smithi160.stdout:7/186: creat d6/d2c/f38 x:0 0 0 2024-05-25T06:11:32.800 INFO:tasks.workunit.client.0.smithi160.stdout:2/196: creat d12/f3d x:0 0 0 2024-05-25T06:11:32.800 INFO:tasks.workunit.client.0.smithi160.stdout:7/187: readlink d6/l23 0 2024-05-25T06:11:32.800 INFO:tasks.workunit.client.1.smithi203.stdout:6/867: stat d8/d5a/c74 0 2024-05-25T06:11:32.801 INFO:tasks.workunit.client.0.smithi160.stdout:2/197: read f8 [199847,79694] 0 2024-05-25T06:11:32.802 INFO:tasks.workunit.client.0.smithi160.stdout:3/165: symlink d19/d2f/d34/l3c 0 2024-05-25T06:11:32.802 INFO:tasks.workunit.client.0.smithi160.stdout:9/145: rename d4/d9 to d4/d11/d1e/d24 0 2024-05-25T06:11:32.803 INFO:tasks.workunit.client.0.smithi160.stdout:5/232: dread db/dc/d13/f2c [0,4194304] 0 2024-05-25T06:11:32.803 INFO:tasks.workunit.client.0.smithi160.stdout:9/146: fdatasync d4/d11/d1e/f1f 0 2024-05-25T06:11:32.805 INFO:tasks.workunit.client.1.smithi203.stdout:2/619: dwrite d2/d4/d10/f9b [0,4194304] 0 2024-05-25T06:11:32.806 INFO:tasks.workunit.client.1.smithi203.stdout:3/725: unlink d10/d16/d50/lb2 0 2024-05-25T06:11:32.807 INFO:tasks.workunit.client.1.smithi203.stdout:5/851: write dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdd [564886,48746] 0 2024-05-25T06:11:32.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/713: write d2/d5/d10/d11/d1f/f72 [3403749,56549] 0 2024-05-25T06:11:32.810 INFO:tasks.workunit.client.0.smithi160.stdout:7/188: symlink d6/dc/d21/l39 0 2024-05-25T06:11:32.810 INFO:tasks.workunit.client.0.smithi160.stdout:3/166: dwrite f11 [4194304,4194304] 0 2024-05-25T06:11:32.813 INFO:tasks.workunit.client.0.smithi160.stdout:5/233: mkdir db/d47 0 2024-05-25T06:11:32.816 INFO:tasks.workunit.client.1.smithi203.stdout:9/528: creat d0/d17/d20/fbc x:0 0 0 2024-05-25T06:11:32.816 INFO:tasks.workunit.client.0.smithi160.stdout:0/200: dwrite f5 [0,4194304] 0 2024-05-25T06:11:32.817 INFO:tasks.workunit.client.1.smithi203.stdout:9/529: fdatasync d0/df/d37/f70 0 2024-05-25T06:11:32.817 INFO:tasks.workunit.client.0.smithi160.stdout:8/275: rmdir d5/d9/d53 0 2024-05-25T06:11:32.817 INFO:tasks.workunit.client.1.smithi203.stdout:9/530: stat d0/d17/d20/f2c 0 2024-05-25T06:11:32.818 INFO:tasks.workunit.client.0.smithi160.stdout:0/201: fsync d2b/f2e 0 2024-05-25T06:11:32.818 INFO:tasks.workunit.client.1.smithi203.stdout:7/582: mknod d1/d11/d3d/d9a/da4/cc8 0 2024-05-25T06:11:32.819 INFO:tasks.workunit.client.0.smithi160.stdout:0/202: write f5 [5065744,101086] 0 2024-05-25T06:11:32.820 INFO:tasks.workunit.client.1.smithi203.stdout:2/620: mkdir d2/d1a/dd3 0 2024-05-25T06:11:32.821 INFO:tasks.workunit.client.0.smithi160.stdout:5/234: fdatasync db/dc/d13/f2c 0 2024-05-25T06:11:32.822 INFO:tasks.workunit.client.0.smithi160.stdout:7/189: mknod d6/d2b/d2d/c3a 0 2024-05-25T06:11:32.824 INFO:tasks.workunit.client.0.smithi160.stdout:8/276: rmdir d5/d44/d1f/d1e 39 2024-05-25T06:11:32.825 INFO:tasks.workunit.client.1.smithi203.stdout:9/531: creat d0/d17/d20/d3c/d5c/fbd x:0 0 0 2024-05-25T06:11:32.826 INFO:tasks.workunit.client.0.smithi160.stdout:0/203: rename c29 to d2b/d31/d35/d36/d37/c3f 0 2024-05-25T06:11:32.827 INFO:tasks.workunit.client.1.smithi203.stdout:7/583: creat d1/d9/d2c/d62/d9b/fc9 x:0 0 0 2024-05-25T06:11:32.827 INFO:tasks.workunit.client.0.smithi160.stdout:9/147: getdents d4/d11/d14 0 2024-05-25T06:11:32.827 INFO:tasks.workunit.client.1.smithi203.stdout:2/621: symlink d2/d1a/db0/ld4 0 2024-05-25T06:11:32.828 INFO:tasks.workunit.client.0.smithi160.stdout:0/204: chown l2a 2326 1 2024-05-25T06:11:32.828 INFO:tasks.workunit.client.0.smithi160.stdout:8/277: write d5/fb [4222146,68579] 0 2024-05-25T06:11:32.829 INFO:tasks.workunit.client.1.smithi203.stdout:8/714: mknod d2/d5/d10/d11/cf1 0 2024-05-25T06:11:32.829 INFO:tasks.workunit.client.1.smithi203.stdout:5/852: link dc/d11/d42/leb dc/dcb/l12f 0 2024-05-25T06:11:32.831 INFO:tasks.workunit.client.0.smithi160.stdout:9/148: fdatasync d4/fa 0 2024-05-25T06:11:32.834 INFO:tasks.workunit.client.1.smithi203.stdout:7/584: truncate d1/fa 3352325 0 2024-05-25T06:11:32.834 INFO:tasks.workunit.client.1.smithi203.stdout:8/715: mknod d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/cf2 0 2024-05-25T06:11:32.836 INFO:tasks.workunit.client.1.smithi203.stdout:7/585: dread - d1/d9/d2c/fc5 zero size 2024-05-25T06:11:32.837 INFO:tasks.workunit.client.0.smithi160.stdout:0/205: mknod d2b/d31/d35/d36/d37/c40 0 2024-05-25T06:11:32.838 INFO:tasks.workunit.client.1.smithi203.stdout:5/853: dwrite dc/d11/d17/d44/f8f [0,4194304] 0 2024-05-25T06:11:32.839 INFO:tasks.workunit.client.0.smithi160.stdout:2/198: truncate f10 1039522 0 2024-05-25T06:11:32.843 INFO:tasks.workunit.client.1.smithi203.stdout:8/716: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/ff3 x:0 0 0 2024-05-25T06:11:32.844 INFO:tasks.workunit.client.0.smithi160.stdout:0/206: mkdir d2b/d31/d35/d41 0 2024-05-25T06:11:32.844 INFO:tasks.workunit.client.0.smithi160.stdout:8/278: chown d5/d44/d1f/d1e/d28/l2b 5902631 1 2024-05-25T06:11:32.845 INFO:tasks.workunit.client.0.smithi160.stdout:7/190: dread d6/dc/f15 [0,4194304] 0 2024-05-25T06:11:32.846 INFO:tasks.workunit.client.0.smithi160.stdout:8/279: write d5/d44/d1f/d1e/d28/d4a/f59 [826490,8899] 0 2024-05-25T06:11:32.848 INFO:tasks.workunit.client.1.smithi203.stdout:5/854: symlink dc/d11/d17/d1a/d2d/d4d/ded/l130 0 2024-05-25T06:11:32.851 INFO:tasks.workunit.client.1.smithi203.stdout:9/532: dread d0/f51 [0,4194304] 0 2024-05-25T06:11:32.851 INFO:tasks.workunit.client.1.smithi203.stdout:7/586: rename d1/d9/d2c/d36/d3b/c5e to d1/d11/d3d/cca 0 2024-05-25T06:11:32.852 INFO:tasks.workunit.client.1.smithi203.stdout:6/868: dwrite d8/d11/d47/f83 [0,4194304] 0 2024-05-25T06:11:32.853 INFO:tasks.workunit.client.0.smithi160.stdout:8/280: rmdir d5/d7 39 2024-05-25T06:11:32.854 INFO:tasks.workunit.client.1.smithi203.stdout:8/717: link d2/d5/d2d/f3e d2/d5/d18/d31/dac/dc2/d8a/d8f/ff4 0 2024-05-25T06:11:32.855 INFO:tasks.workunit.client.0.smithi160.stdout:5/235: truncate db/dc/d13/d1d/d23/f2d 135175 0 2024-05-25T06:11:32.856 INFO:tasks.workunit.client.1.smithi203.stdout:9/533: fdatasync d0/df/d37/f58 0 2024-05-25T06:11:32.859 INFO:tasks.workunit.client.1.smithi203.stdout:8/718: dread - d2/d5/d28/d38/fa6 zero size 2024-05-25T06:11:32.859 INFO:tasks.workunit.client.1.smithi203.stdout:3/726: dwrite d10/d16/d23/f6e [0,4194304] 0 2024-05-25T06:11:32.860 INFO:tasks.workunit.client.1.smithi203.stdout:5/855: mknod dc/d11/d42/d94/dad/c131 0 2024-05-25T06:11:32.860 INFO:tasks.workunit.client.0.smithi160.stdout:8/281: dread d5/d44/f37 [0,4194304] 0 2024-05-25T06:11:32.860 INFO:tasks.workunit.client.1.smithi203.stdout:6/869: symlink d8/d3d/l119 0 2024-05-25T06:11:32.861 INFO:tasks.workunit.client.1.smithi203.stdout:8/719: chown d2/d5/d28/d38/d4e/lb7 29552 1 2024-05-25T06:11:32.863 INFO:tasks.workunit.client.0.smithi160.stdout:8/282: truncate d5/f48 134012 0 2024-05-25T06:11:32.864 INFO:tasks.workunit.client.0.smithi160.stdout:0/207: truncate f21 556841 0 2024-05-25T06:11:32.865 INFO:tasks.workunit.client.1.smithi203.stdout:7/587: creat d1/d9/dc2/fcb x:0 0 0 2024-05-25T06:11:32.870 INFO:tasks.workunit.client.0.smithi160.stdout:7/191: truncate d6/dc/f17 1888229 0 2024-05-25T06:11:32.870 INFO:tasks.workunit.client.0.smithi160.stdout:8/283: truncate d5/d44/f16 8604138 0 2024-05-25T06:11:32.870 INFO:tasks.workunit.client.1.smithi203.stdout:9/534: dread d0/d17/d20/d3c/db2/f84 [0,4194304] 0 2024-05-25T06:11:32.872 INFO:tasks.workunit.client.1.smithi203.stdout:8/720: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/ff5 x:0 0 0 2024-05-25T06:11:32.873 INFO:tasks.workunit.client.0.smithi160.stdout:9/149: dread d4/fd [0,4194304] 0 2024-05-25T06:11:32.873 INFO:tasks.workunit.client.0.smithi160.stdout:0/208: creat d2b/f42 x:0 0 0 2024-05-25T06:11:32.873 INFO:tasks.workunit.client.1.smithi203.stdout:3/727: mknod d10/d16/ce9 0 2024-05-25T06:11:32.874 INFO:tasks.workunit.client.0.smithi160.stdout:7/192: rename d6/l27 to d6/dc/d21/l3b 0 2024-05-25T06:11:32.875 INFO:tasks.workunit.client.1.smithi203.stdout:7/588: mknod d1/d9/d2c/d36/d3b/ccc 0 2024-05-25T06:11:32.875 INFO:tasks.workunit.client.1.smithi203.stdout:8/721: read d2/d5/d10/d4a/fed [1131345,17642] 0 2024-05-25T06:11:32.877 INFO:tasks.workunit.client.0.smithi160.stdout:7/193: fsync f1 0 2024-05-25T06:11:32.877 INFO:tasks.workunit.client.0.smithi160.stdout:2/199: dwrite d12/f1d [0,4194304] 0 2024-05-25T06:11:32.877 INFO:tasks.workunit.client.0.smithi160.stdout:0/209: mkdir d2b/d31/d35/d43 0 2024-05-25T06:11:32.878 INFO:tasks.workunit.client.0.smithi160.stdout:9/150: write d4/d11/d1e/d24/f1b [2633518,50400] 0 2024-05-25T06:11:32.879 INFO:tasks.workunit.client.0.smithi160.stdout:0/210: fdatasync d2b/f2e 0 2024-05-25T06:11:32.882 INFO:tasks.workunit.client.1.smithi203.stdout:6/870: link d8/d11/d15/d28/c95 d8/d11/d1b/c11a 0 2024-05-25T06:11:32.883 INFO:tasks.workunit.client.0.smithi160.stdout:7/194: creat d6/d2c/f3c x:0 0 0 2024-05-25T06:11:32.883 INFO:tasks.workunit.client.0.smithi160.stdout:2/200: unlink d12/f3d 0 2024-05-25T06:11:32.883 INFO:tasks.workunit.client.1.smithi203.stdout:6/871: chown d8/d11/d1b/d75/dcb/dcf/f7e 2353 1 2024-05-25T06:11:32.883 INFO:tasks.workunit.client.1.smithi203.stdout:8/722: rename d2/d5/d28/d38/d4e/d97/ddf/le1 to d2/d5/d18/d31/lf6 0 2024-05-25T06:11:32.886 INFO:tasks.workunit.client.0.smithi160.stdout:8/284: dwrite d5/d44/d1f/f42 [0,4194304] 0 2024-05-25T06:11:32.886 INFO:tasks.workunit.client.0.smithi160.stdout:0/211: write d2b/d31/d35/d36/d37/f3b [783868,63709] 0 2024-05-25T06:11:32.886 INFO:tasks.workunit.client.0.smithi160.stdout:2/201: readlink d12/d13/l22 0 2024-05-25T06:11:32.887 INFO:tasks.workunit.client.1.smithi203.stdout:3/728: unlink d10/d16/f44 0 2024-05-25T06:11:32.889 INFO:tasks.workunit.client.0.smithi160.stdout:5/236: write db/dc/d13/d1d/d23/f2d [281791,85509] 0 2024-05-25T06:11:32.891 INFO:tasks.workunit.client.0.smithi160.stdout:8/285: write d5/d44/d1f/d1e/d28/d4a/f3d [3678319,70270] 0 2024-05-25T06:11:32.891 INFO:tasks.workunit.client.1.smithi203.stdout:7/589: unlink d1/d11/l1a 0 2024-05-25T06:11:32.893 INFO:tasks.workunit.client.0.smithi160.stdout:9/151: rename d4/d11/d14/d23/d22 to d4/d11/d14/d23/d25 0 2024-05-25T06:11:32.894 INFO:tasks.workunit.client.0.smithi160.stdout:9/152: readlink d4/lb 0 2024-05-25T06:11:32.894 INFO:tasks.workunit.client.1.smithi203.stdout:9/535: link d0/d17/d20/d29/l4e d0/d17/lbe 0 2024-05-25T06:11:32.898 INFO:tasks.workunit.client.1.smithi203.stdout:8/723: mknod d2/d5/dae/cf7 0 2024-05-25T06:11:32.898 INFO:tasks.workunit.client.1.smithi203.stdout:5/856: dread dc/f10 [0,4194304] 0 2024-05-25T06:11:32.899 INFO:tasks.workunit.client.0.smithi160.stdout:5/237: fdatasync f9 0 2024-05-25T06:11:32.900 INFO:tasks.workunit.client.1.smithi203.stdout:6/872: fsync d8/d11/d1b/d75/dcb/dcf/d46/fd2 0 2024-05-25T06:11:32.902 INFO:tasks.workunit.client.1.smithi203.stdout:7/590: mknod d1/d9/d2c/d3a/d4e/d50/ccd 0 2024-05-25T06:11:32.903 INFO:tasks.workunit.client.0.smithi160.stdout:0/212: rename d2b/f3a to d2b/d31/d35/d36/d37/f44 0 2024-05-25T06:11:32.905 INFO:tasks.workunit.client.1.smithi203.stdout:3/729: rename d10/d16/d50/d8b/db3 to d10/dea 0 2024-05-25T06:11:32.905 INFO:tasks.workunit.client.1.smithi203.stdout:6/873: symlink d8/d5a/d9b/dcc/l11b 0 2024-05-25T06:11:32.906 INFO:tasks.workunit.client.0.smithi160.stdout:2/202: mkdir d12/d3e 0 2024-05-25T06:11:32.906 INFO:tasks.workunit.client.1.smithi203.stdout:3/730: truncate d10/d22/d2a/f81 264064 0 2024-05-25T06:11:32.907 INFO:tasks.workunit.client.0.smithi160.stdout:5/238: dread db/dc/d13/d1d/d23/f33 [0,4194304] 0 2024-05-25T06:11:32.907 INFO:tasks.workunit.client.1.smithi203.stdout:5/857: creat dc/d11/d17/d1a/d75/ddb/f132 x:0 0 0 2024-05-25T06:11:32.908 INFO:tasks.workunit.client.1.smithi203.stdout:9/536: dread d0/f1e [0,4194304] 0 2024-05-25T06:11:32.909 INFO:tasks.workunit.client.0.smithi160.stdout:5/239: truncate db/dc/d13/d1d/d23/f33 1521465 0 2024-05-25T06:11:32.909 INFO:tasks.workunit.client.1.smithi203.stdout:6/874: mknod d8/d5a/d9b/c11c 0 2024-05-25T06:11:32.910 INFO:tasks.workunit.client.0.smithi160.stdout:5/240: chown db/dc/c3d 13181048 1 2024-05-25T06:11:32.911 INFO:tasks.workunit.client.1.smithi203.stdout:7/591: mknod d1/d11/cce 0 2024-05-25T06:11:32.911 INFO:tasks.workunit.client.1.smithi203.stdout:7/592: dread - d1/d9/d2c/d36/d54/fc3 zero size 2024-05-25T06:11:32.912 INFO:tasks.workunit.client.1.smithi203.stdout:7/593: dread - d1/d9/d2c/d36/d3b/faa zero size 2024-05-25T06:11:32.913 INFO:tasks.workunit.client.0.smithi160.stdout:8/286: rename d5/d44/d1f/l41 to d5/d44/d1f/d1e/d28/d4a/d4e/l5b 0 2024-05-25T06:11:32.913 INFO:tasks.workunit.client.1.smithi203.stdout:3/731: mkdir d10/d16/d23/d4a/d88/deb 0 2024-05-25T06:11:32.914 INFO:tasks.workunit.client.0.smithi160.stdout:9/153: link d4/d11/d1e/d24/fc d4/d11/f26 0 2024-05-25T06:11:32.916 INFO:tasks.workunit.client.1.smithi203.stdout:9/537: symlink d0/d17/d23/lbf 0 2024-05-25T06:11:32.917 INFO:tasks.workunit.client.1.smithi203.stdout:9/538: readlink d0/d17/d23/l26 0 2024-05-25T06:11:32.917 INFO:tasks.workunit.client.0.smithi160.stdout:7/195: stat d6/dc/f17 0 2024-05-25T06:11:32.918 INFO:tasks.workunit.client.1.smithi203.stdout:6/875: mkdir d8/d11/d47/d60/d11d 0 2024-05-25T06:11:32.918 INFO:tasks.workunit.client.0.smithi160.stdout:7/196: chown d6/dc/f19 4196 1 2024-05-25T06:11:32.918 INFO:tasks.workunit.client.0.smithi160.stdout:7/197: fdatasync d6/d2c/f38 0 2024-05-25T06:11:32.919 INFO:tasks.workunit.client.0.smithi160.stdout:0/213: rename d2b/d31/d35/d36/d37/c40 to d2b/d31/d35/d43/c45 0 2024-05-25T06:11:32.921 INFO:tasks.workunit.client.1.smithi203.stdout:8/724: rename d2/d5/d28/d38/d4e/d97/fbd to d2/d5/d18/d31/dac/dc2/d88/ff8 0 2024-05-25T06:11:32.922 INFO:tasks.workunit.client.1.smithi203.stdout:8/725: chown d2/d5/d2d/c89 5 1 2024-05-25T06:11:32.923 INFO:tasks.workunit.client.0.smithi160.stdout:0/214: dread d2b/d31/f34 [0,4194304] 0 2024-05-25T06:11:32.924 INFO:tasks.workunit.client.1.smithi203.stdout:9/539: dwrite d0/d17/f81 [0,4194304] 0 2024-05-25T06:11:32.928 INFO:tasks.workunit.client.0.smithi160.stdout:9/154: dwrite d4/d11/f26 [0,4194304] 0 2024-05-25T06:11:32.928 INFO:tasks.workunit.client.1.smithi203.stdout:6/876: rmdir d8/d11/d15/d28/d52/d43/d73/db7 39 2024-05-25T06:11:32.934 INFO:tasks.workunit.client.0.smithi160.stdout:0/215: mkdir d2b/d46 0 2024-05-25T06:11:32.935 INFO:tasks.workunit.client.0.smithi160.stdout:5/241: write f8 [2776979,119403] 0 2024-05-25T06:11:32.936 INFO:tasks.workunit.client.1.smithi203.stdout:6/877: mknod d8/d11/d15/c11e 0 2024-05-25T06:11:32.937 INFO:tasks.workunit.client.0.smithi160.stdout:9/155: mkdir d4/d11/d14/d23/d25/d27 0 2024-05-25T06:11:32.938 INFO:tasks.workunit.client.1.smithi203.stdout:3/732: rmdir d10/d16/d23/d6c/de4 0 2024-05-25T06:11:32.940 INFO:tasks.workunit.client.1.smithi203.stdout:5/858: dwrite dc/d11/d17/d1a/f2e [4194304,4194304] 0 2024-05-25T06:11:32.941 INFO:tasks.workunit.client.1.smithi203.stdout:3/733: chown d10/d16/d50/d7d/f9d 389 1 2024-05-25T06:11:32.941 INFO:tasks.workunit.client.1.smithi203.stdout:5/859: chown dc/d11/d17/d1a/d72/de4 459 1 2024-05-25T06:11:32.942 INFO:tasks.workunit.client.0.smithi160.stdout:0/216: symlink d2b/d31/d35/d43/l47 0 2024-05-25T06:11:32.944 INFO:tasks.workunit.client.1.smithi203.stdout:6/878: creat d8/d11/d1b/d75/dcb/dcf/d46/f11f x:0 0 0 2024-05-25T06:11:32.945 INFO:tasks.workunit.client.0.smithi160.stdout:8/287: link d5/d44/d1f/d1e/d28/f3f d5/d44/d1f/d1e/d28/d4a/f5c 0 2024-05-25T06:11:32.946 INFO:tasks.workunit.client.1.smithi203.stdout:8/726: creat d2/d5/d28/ff9 x:0 0 0 2024-05-25T06:11:32.946 INFO:tasks.workunit.client.0.smithi160.stdout:0/217: read f10 [6831176,89776] 0 2024-05-25T06:11:32.947 INFO:tasks.workunit.client.1.smithi203.stdout:3/734: chown d10/d16/d71/fc8 2056 1 2024-05-25T06:11:32.947 INFO:tasks.workunit.client.1.smithi203.stdout:8/727: chown d2/d5/d2d/c89 2659 1 2024-05-25T06:11:32.948 INFO:tasks.workunit.client.0.smithi160.stdout:8/288: write d5/d44/d1f/d1e/d28/f38 [2457485,62001] 0 2024-05-25T06:11:32.948 INFO:tasks.workunit.client.0.smithi160.stdout:7/198: truncate d6/dc/f24 899748 0 2024-05-25T06:11:32.949 INFO:tasks.workunit.client.1.smithi203.stdout:6/879: readlink d8/d11/d1b/d75/dcb/dcf/d46/d88/l103 0 2024-05-25T06:11:32.951 INFO:tasks.workunit.client.1.smithi203.stdout:6/880: write d8/fa0 [146519,127832] 0 2024-05-25T06:11:32.952 INFO:tasks.workunit.client.0.smithi160.stdout:7/199: rmdir d6/dc 39 2024-05-25T06:11:32.952 INFO:tasks.workunit.client.1.smithi203.stdout:5/860: getdents dc/d11/d17/d1a/d2d/d4d/d64/dd4 0 2024-05-25T06:11:32.953 INFO:tasks.workunit.client.1.smithi203.stdout:9/540: truncate d0/d17/d20/f44 588461 0 2024-05-25T06:11:32.954 INFO:tasks.workunit.client.0.smithi160.stdout:8/289: unlink d5/d39/f3b 0 2024-05-25T06:11:32.955 INFO:tasks.workunit.client.1.smithi203.stdout:9/541: write d0/d17/d20/d6e/d75/f98 [709637,127750] 0 2024-05-25T06:11:32.955 INFO:tasks.workunit.client.1.smithi203.stdout:8/728: mkdir d2/d5/d10/dfa 0 2024-05-25T06:11:32.956 INFO:tasks.workunit.client.0.smithi160.stdout:0/218: rename d2b/d31/d35/d43/c45 to d2b/d46/c48 0 2024-05-25T06:11:32.956 INFO:tasks.workunit.client.1.smithi203.stdout:9/542: readlink d0/df/d2d/l40 0 2024-05-25T06:11:32.958 INFO:tasks.workunit.client.1.smithi203.stdout:9/543: write d0/d17/d20/f3a [2189495,115642] 0 2024-05-25T06:11:32.958 INFO:tasks.workunit.client.0.smithi160.stdout:0/219: chown d2b/d31/d35/f3e 443655102 1 2024-05-25T06:11:32.961 INFO:tasks.workunit.client.0.smithi160.stdout:8/290: rename d5/d44/d1f/d1e/d28/c50 to d5/d52/c5d 0 2024-05-25T06:11:32.961 INFO:tasks.workunit.client.0.smithi160.stdout:7/200: dwrite d6/d2c/f38 [0,4194304] 0 2024-05-25T06:11:32.961 INFO:tasks.workunit.client.1.smithi203.stdout:3/735: dread d10/d16/d23/d4a/d88/f95 [0,4194304] 0 2024-05-25T06:11:32.962 INFO:tasks.workunit.client.0.smithi160.stdout:5/242: dread f8 [0,4194304] 0 2024-05-25T06:11:32.963 INFO:tasks.workunit.client.0.smithi160.stdout:8/291: write d5/d44/d1f/f42 [484129,109677] 0 2024-05-25T06:11:32.963 INFO:tasks.workunit.client.0.smithi160.stdout:5/243: chown db/dc/d13/c25 125 1 2024-05-25T06:11:32.966 INFO:tasks.workunit.client.0.smithi160.stdout:0/220: creat d2b/d31/d35/f49 x:0 0 0 2024-05-25T06:11:32.966 INFO:tasks.workunit.client.0.smithi160.stdout:9/156: write d4/fd [2086437,61308] 0 2024-05-25T06:11:32.967 INFO:tasks.workunit.client.1.smithi203.stdout:6/881: getdents d8/d2a/d63 0 2024-05-25T06:11:32.968 INFO:tasks.workunit.client.1.smithi203.stdout:5/861: getdents dc/d11/d17 0 2024-05-25T06:11:32.970 INFO:tasks.workunit.client.1.smithi203.stdout:8/729: getdents d2/d5/d10/d11/d1f/d75 0 2024-05-25T06:11:32.970 INFO:tasks.workunit.client.1.smithi203.stdout:9/544: dwrite d0/d57/f86 [0,4194304] 0 2024-05-25T06:11:32.970 INFO:tasks.workunit.client.1.smithi203.stdout:5/862: chown dc/d11/d17/d44/f51 3123897 1 2024-05-25T06:11:32.973 INFO:tasks.workunit.client.0.smithi160.stdout:8/292: dwrite d5/f3e [0,4194304] 0 2024-05-25T06:11:32.973 INFO:tasks.workunit.client.1.smithi203.stdout:9/545: chown d0/d17/d20/d29/l50 2 1 2024-05-25T06:11:32.974 INFO:tasks.workunit.client.0.smithi160.stdout:7/201: fsync d6/dc/f25 0 2024-05-25T06:11:32.979 INFO:tasks.workunit.client.1.smithi203.stdout:6/882: dwrite d8/fe0 [0,4194304] 0 2024-05-25T06:11:32.980 INFO:tasks.workunit.client.0.smithi160.stdout:0/221: rename f21 to d2b/d31/d35/d36/f4a 0 2024-05-25T06:11:32.981 INFO:tasks.workunit.client.1.smithi203.stdout:5/863: creat dc/d11/d17/d1a/d2d/d4d/d101/f133 x:0 0 0 2024-05-25T06:11:32.981 INFO:tasks.workunit.client.0.smithi160.stdout:9/157: getdents d4/d11/d1e/d24 0 2024-05-25T06:11:32.982 INFO:tasks.workunit.client.1.smithi203.stdout:3/736: link d10/d16/d23/d46/fd5 d10/d16/d23/d4a/fec 0 2024-05-25T06:11:32.984 INFO:tasks.workunit.client.1.smithi203.stdout:3/737: stat d10/d16/le8 0 2024-05-25T06:11:32.984 INFO:tasks.workunit.client.0.smithi160.stdout:9/158: fsync d4/d11/d1e/f1f 0 2024-05-25T06:11:32.985 INFO:tasks.workunit.client.0.smithi160.stdout:9/159: rename d4/d11 to d4/d11/d12/d28 22 2024-05-25T06:11:32.985 INFO:tasks.workunit.client.1.smithi203.stdout:3/738: read - d10/fa2 zero size 2024-05-25T06:11:32.989 INFO:tasks.workunit.client.0.smithi160.stdout:5/244: dwrite db/dc/d13/d1d/d31/d3b/f41 [0,4194304] 0 2024-05-25T06:11:32.989 INFO:tasks.workunit.client.0.smithi160.stdout:8/293: creat d5/d44/d1f/d1e/d28/f5e x:0 0 0 2024-05-25T06:11:32.991 INFO:tasks.workunit.client.1.smithi203.stdout:3/739: symlink d10/d16/d23/d6c/led 0 2024-05-25T06:11:32.991 INFO:tasks.workunit.client.0.smithi160.stdout:5/245: chown f7 245 1 2024-05-25T06:11:32.991 INFO:tasks.workunit.client.0.smithi160.stdout:5/246: readlink db/lf 0 2024-05-25T06:11:32.992 INFO:tasks.workunit.client.0.smithi160.stdout:8/294: truncate d5/d44/d1f/d1e/d28/f5e 513792 0 2024-05-25T06:11:32.994 INFO:tasks.workunit.client.0.smithi160.stdout:7/202: truncate d6/dc/f17 1931322 0 2024-05-25T06:11:32.995 INFO:tasks.workunit.client.1.smithi203.stdout:8/730: write d2/d5/f29 [253874,102769] 0 2024-05-25T06:11:32.995 INFO:tasks.workunit.client.0.smithi160.stdout:9/160: mknod d4/c29 0 2024-05-25T06:11:32.996 INFO:tasks.workunit.client.0.smithi160.stdout:7/203: truncate d6/d2c/f3c 209548 0 2024-05-25T06:11:32.996 INFO:tasks.workunit.client.0.smithi160.stdout:5/247: chown db/dc/d13/d1d/d23/d24 19422971 1 2024-05-25T06:11:32.997 INFO:tasks.workunit.client.1.smithi203.stdout:8/731: stat d2/d5/d28/d38/d4e/lb7 0 2024-05-25T06:11:32.999 INFO:tasks.workunit.client.0.smithi160.stdout:9/161: symlink d4/d11/d1e/d24/l2a 0 2024-05-25T06:11:33.000 INFO:tasks.workunit.client.1.smithi203.stdout:3/740: dwrite d10/d16/d23/d67/f94 [0,4194304] 0 2024-05-25T06:11:33.000 INFO:tasks.workunit.client.1.smithi203.stdout:8/732: chown d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/cf2 924219 1 2024-05-25T06:11:33.001 INFO:tasks.workunit.client.0.smithi160.stdout:7/204: truncate d6/fb 870610 0 2024-05-25T06:11:33.001 INFO:tasks.workunit.client.0.smithi160.stdout:7/205: stat c2 0 2024-05-25T06:11:33.001 INFO:tasks.workunit.client.0.smithi160.stdout:5/248: mknod db/d47/c48 0 2024-05-25T06:11:33.001 INFO:tasks.workunit.client.0.smithi160.stdout:5/249: read f7 [498598,607] 0 2024-05-25T06:11:33.002 INFO:tasks.workunit.client.0.smithi160.stdout:8/295: creat d5/d7/f5f x:0 0 0 2024-05-25T06:11:33.002 INFO:tasks.workunit.client.0.smithi160.stdout:5/250: stat db/dc/d13/d1d/d23/d24/f3e 0 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.1.smithi203.stdout:8/733: dread - d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/fee zero size 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.1.smithi203.stdout:5/864: dread dc/d11/d42/f9b [0,4194304] 0 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.1.smithi203.stdout:3/741: creat d10/d22/fee x:0 0 0 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.0.smithi160.stdout:8/296: stat d5/d44 0 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.0.smithi160.stdout:9/162: rename d4/d11/d1e/d24/fe to d4/d11/d12/f2b 0 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.0.smithi160.stdout:7/206: rmdir d6/d2b 39 2024-05-25T06:11:33.006 INFO:tasks.workunit.client.0.smithi160.stdout:9/163: readlink d4/d11/d1e/d24/l2a 0 2024-05-25T06:11:33.007 INFO:tasks.workunit.client.1.smithi203.stdout:5/865: chown dc/d11/d17/d1a/d2d/d53/da2/f118 14 1 2024-05-25T06:11:33.008 INFO:tasks.workunit.client.1.smithi203.stdout:8/734: mkdir d2/d5/d10/dfa/dfb 0 2024-05-25T06:11:33.009 INFO:tasks.workunit.client.1.smithi203.stdout:8/735: stat d2/d5/d18/d31/c43 0 2024-05-25T06:11:33.010 INFO:tasks.workunit.client.1.smithi203.stdout:3/742: mkdir d10/d16/d23/def 0 2024-05-25T06:11:33.013 INFO:tasks.workunit.client.0.smithi160.stdout:5/251: dwrite db/f2a [4194304,4194304] 0 2024-05-25T06:11:33.016 INFO:tasks.workunit.client.0.smithi160.stdout:8/297: dwrite d5/d44/f37 [0,4194304] 0 2024-05-25T06:11:33.018 INFO:tasks.workunit.client.0.smithi160.stdout:5/252: rename l3 to db/d47/l49 0 2024-05-25T06:11:33.020 INFO:tasks.workunit.client.1.smithi203.stdout:3/743: creat d10/d16/d23/ff0 x:0 0 0 2024-05-25T06:11:33.021 INFO:tasks.workunit.client.0.smithi160.stdout:8/298: chown d5/d44/d1f/d1e/c31 1071058 1 2024-05-25T06:11:33.021 INFO:tasks.workunit.client.1.smithi203.stdout:8/736: dwrite d2/d5/d10/d45/d5d/f5c [0,4194304] 0 2024-05-25T06:11:33.022 INFO:tasks.workunit.client.0.smithi160.stdout:8/299: write d5/d9/f4b [4095103,40186] 0 2024-05-25T06:11:33.023 INFO:tasks.workunit.client.1.smithi203.stdout:9/546: dwrite d0/d17/d23/f8e [0,4194304] 0 2024-05-25T06:11:33.024 INFO:tasks.workunit.client.0.smithi160.stdout:5/253: dwrite db/dc/d13/f2c [0,4194304] 0 2024-05-25T06:11:33.025 INFO:tasks.workunit.client.0.smithi160.stdout:5/254: chown db/lf 199374 1 2024-05-25T06:11:33.028 INFO:tasks.workunit.client.0.smithi160.stdout:8/300: unlink d5/d9/f4b 0 2024-05-25T06:11:33.028 INFO:tasks.workunit.client.1.smithi203.stdout:0/509: sync 2024-05-25T06:11:33.030 INFO:tasks.workunit.client.0.smithi160.stdout:8/301: truncate d5/d44/d1f/f22 1007254 0 2024-05-25T06:11:33.030 INFO:tasks.workunit.client.0.smithi160.stdout:5/255: fsync db/dc/d13/f44 0 2024-05-25T06:11:33.031 INFO:tasks.workunit.client.0.smithi160.stdout:8/302: write d5/d44/d1f/d1e/d28/f4d [4366387,94335] 0 2024-05-25T06:11:33.039 INFO:tasks.workunit.client.1.smithi203.stdout:3/744: dwrite d10/d16/d23/d46/fce [0,4194304] 0 2024-05-25T06:11:33.042 INFO:tasks.workunit.client.0.smithi160.stdout:9/164: dread d4/d11/d12/f2b [0,4194304] 0 2024-05-25T06:11:33.044 INFO:tasks.workunit.client.1.smithi203.stdout:6/883: dwrite d8/d70/f7c [0,4194304] 0 2024-05-25T06:11:33.046 INFO:tasks.workunit.client.1.smithi203.stdout:0/510: mknod dd/d5c/cb3 0 2024-05-25T06:11:33.047 INFO:tasks.workunit.client.1.smithi203.stdout:6/884: truncate d8/d11/d47/fee 1885109 0 2024-05-25T06:11:33.054 INFO:tasks.workunit.client.1.smithi203.stdout:3/745: rename d10/d16/d50/lac to d10/d16/d23/d4a/d88/dca/dda/lf1 0 2024-05-25T06:11:33.054 INFO:tasks.workunit.client.0.smithi160.stdout:9/165: creat d4/d11/f2c x:0 0 0 2024-05-25T06:11:33.056 INFO:tasks.workunit.client.0.smithi160.stdout:9/166: chown d4/d11/d1e/d24/c18 229055217 1 2024-05-25T06:11:33.057 INFO:tasks.workunit.client.0.smithi160.stdout:5/256: mkdir db/dc/d13/d1d/d3f/d4a 0 2024-05-25T06:11:33.057 INFO:tasks.workunit.client.1.smithi203.stdout:8/737: dwrite d2/d5/d18/d31/dac/dc2/f98 [0,4194304] 0 2024-05-25T06:11:33.058 INFO:tasks.workunit.client.0.smithi160.stdout:8/303: creat d5/d44/d1f/d58/f60 x:0 0 0 2024-05-25T06:11:33.059 INFO:tasks.workunit.client.0.smithi160.stdout:8/304: fdatasync d5/d7/f5f 0 2024-05-25T06:11:33.060 INFO:tasks.workunit.client.1.smithi203.stdout:6/885: write d8/d11/d15/d28/d52/d43/d73/fb5 [543088,10436] 0 2024-05-25T06:11:33.062 INFO:tasks.workunit.client.1.smithi203.stdout:8/738: dread - d2/d5/d28/d38/d81/fb3 zero size 2024-05-25T06:11:33.064 INFO:tasks.workunit.client.0.smithi160.stdout:9/167: dwrite d4/fd [0,4194304] 0 2024-05-25T06:11:33.064 INFO:tasks.workunit.client.1.smithi203.stdout:0/511: mknod dd/d3d/d48/d91/dad/cb4 0 2024-05-25T06:11:33.066 INFO:tasks.workunit.client.1.smithi203.stdout:8/739: dread - d2/d5/d10/d45/d5d/faf zero size 2024-05-25T06:11:33.067 INFO:tasks.workunit.client.0.smithi160.stdout:8/305: mkdir d5/d44/d1f/d1e/d28/d49/d61 0 2024-05-25T06:11:33.068 INFO:tasks.workunit.client.0.smithi160.stdout:5/257: link l2 db/dc/d13/d1d/d31/l4b 0 2024-05-25T06:11:33.068 INFO:tasks.workunit.client.1.smithi203.stdout:3/746: truncate d10/d22/f35 3705260 0 2024-05-25T06:11:33.069 INFO:tasks.workunit.client.0.smithi160.stdout:9/168: mknod d4/d11/d14/d23/d25/d27/c2d 0 2024-05-25T06:11:33.069 INFO:tasks.workunit.client.0.smithi160.stdout:5/258: chown db/dc/d13/d1d/d23 271 1 2024-05-25T06:11:33.072 INFO:tasks.workunit.client.0.smithi160.stdout:9/169: mkdir d4/d11/d12/d2e 0 2024-05-25T06:11:33.072 INFO:tasks.workunit.client.1.smithi203.stdout:8/740: mkdir d2/d5/d18/d31/dac/dc2/d8a/d8f/dfc 0 2024-05-25T06:11:33.074 INFO:tasks.workunit.client.1.smithi203.stdout:3/747: creat d10/d16/d23/d4a/d88/deb/ff2 x:0 0 0 2024-05-25T06:11:33.075 INFO:tasks.workunit.client.0.smithi160.stdout:9/170: mkdir d4/d11/d14/d23/d2f 0 2024-05-25T06:11:33.075 INFO:tasks.workunit.client.0.smithi160.stdout:8/306: link d5/d44/d1f/d1e/d28/d49/c4c d5/d44/d1f/d1e/d28/d49/d61/c62 0 2024-05-25T06:11:33.075 INFO:tasks.workunit.client.1.smithi203.stdout:0/512: rmdir dd/d8c 0 2024-05-25T06:11:33.076 INFO:tasks.workunit.client.0.smithi160.stdout:9/171: write d4/d11/d1e/d24/f1b [4729640,26875] 0 2024-05-25T06:11:33.076 INFO:tasks.workunit.client.1.smithi203.stdout:0/513: dread - dd/d3a/fae zero size 2024-05-25T06:11:33.076 INFO:tasks.workunit.client.1.smithi203.stdout:8/741: symlink d2/d5/d10/dfa/dfb/lfd 0 2024-05-25T06:11:33.077 INFO:tasks.workunit.client.1.smithi203.stdout:3/748: symlink d10/d16/d23/d38/lf3 0 2024-05-25T06:11:33.078 INFO:tasks.workunit.client.1.smithi203.stdout:0/514: readlink dd/d3a/l6c 0 2024-05-25T06:11:33.078 INFO:tasks.workunit.client.1.smithi203.stdout:9/547: write d0/f19 [1950427,29859] 0 2024-05-25T06:11:33.082 INFO:tasks.workunit.client.0.smithi160.stdout:5/259: link db/dc/d13/l18 db/dc/d13/d1d/d23/l4c 0 2024-05-25T06:11:33.082 INFO:tasks.workunit.client.0.smithi160.stdout:5/260: truncate f1 4675534 0 2024-05-25T06:11:33.083 INFO:tasks.workunit.client.0.smithi160.stdout:5/261: readlink db/dc/d13/d1d/d23/d24/d2e/l36 0 2024-05-25T06:11:33.084 INFO:tasks.workunit.client.0.smithi160.stdout:9/172: mkdir d4/d30 0 2024-05-25T06:11:33.085 INFO:tasks.workunit.client.0.smithi160.stdout:8/307: mknod d5/d44/d1f/d1e/d28/d49/d61/c63 0 2024-05-25T06:11:33.085 INFO:tasks.workunit.client.0.smithi160.stdout:5/262: chown db/dc/d13/d1d/d23/d24/d2e/l36 23329967 1 2024-05-25T06:11:33.088 INFO:tasks.workunit.client.0.smithi160.stdout:5/263: mknod db/dc/d13/d1d/d23/d24/d2e/c4d 0 2024-05-25T06:11:33.090 INFO:tasks.workunit.client.0.smithi160.stdout:5/264: write f8 [1156429,70217] 0 2024-05-25T06:11:33.090 INFO:tasks.workunit.client.0.smithi160.stdout:9/173: dwrite d4/f1c [0,4194304] 0 2024-05-25T06:11:33.091 INFO:tasks.workunit.client.0.smithi160.stdout:9/174: read - d4/d11/f2c zero size 2024-05-25T06:11:33.094 INFO:tasks.workunit.client.0.smithi160.stdout:4/295: sync 2024-05-25T06:11:33.095 INFO:tasks.workunit.client.0.smithi160.stdout:9/175: creat d4/d11/d14/f31 x:0 0 0 2024-05-25T06:11:33.096 INFO:tasks.workunit.client.0.smithi160.stdout:4/296: read - d4/d6/d28/f4e zero size 2024-05-25T06:11:33.100 INFO:tasks.workunit.client.0.smithi160.stdout:9/176: creat d4/d11/d12/d2e/f32 x:0 0 0 2024-05-25T06:11:33.102 INFO:tasks.workunit.client.0.smithi160.stdout:4/297: write d4/d6/d21/d42/d49/f4b [31298,113864] 0 2024-05-25T06:11:33.104 INFO:tasks.workunit.client.0.smithi160.stdout:8/308: dread d5/fe [0,4194304] 0 2024-05-25T06:11:33.106 INFO:tasks.workunit.client.0.smithi160.stdout:7/207: dwrite d6/fb [0,4194304] 0 2024-05-25T06:11:33.108 INFO:tasks.workunit.client.0.smithi160.stdout:4/298: rename d4/d1a/f38 to d4/d6/d35/f58 0 2024-05-25T06:11:33.109 INFO:tasks.workunit.client.0.smithi160.stdout:8/309: symlink d5/l64 0 2024-05-25T06:11:33.113 INFO:tasks.workunit.client.0.smithi160.stdout:9/177: dwrite d4/d11/f2c [0,4194304] 0 2024-05-25T06:11:33.115 INFO:tasks.workunit.client.0.smithi160.stdout:7/208: dread d6/d2c/f38 [0,4194304] 0 2024-05-25T06:11:33.116 INFO:tasks.workunit.client.0.smithi160.stdout:9/178: truncate d4/d11/d12/d2e/f32 754142 0 2024-05-25T06:11:33.116 INFO:tasks.workunit.client.0.smithi160.stdout:4/299: creat d4/d1a/f59 x:0 0 0 2024-05-25T06:11:33.122 INFO:tasks.workunit.client.0.smithi160.stdout:7/209: mknod d6/d32/c3d 0 2024-05-25T06:11:33.122 INFO:tasks.workunit.client.0.smithi160.stdout:8/310: dread d5/d44/d1f/d1e/d28/d4a/f3a [0,4194304] 0 2024-05-25T06:11:33.123 INFO:tasks.workunit.client.0.smithi160.stdout:5/265: dwrite db/dc/f15 [4194304,4194304] 0 2024-05-25T06:11:33.123 INFO:tasks.workunit.client.0.smithi160.stdout:9/179: mknod d4/d30/c33 0 2024-05-25T06:11:33.125 INFO:tasks.workunit.client.0.smithi160.stdout:7/210: write d6/d2c/f38 [1707735,76582] 0 2024-05-25T06:11:33.127 INFO:tasks.workunit.client.0.smithi160.stdout:8/311: mkdir d5/d44/d1f/d58/d65 0 2024-05-25T06:11:33.128 INFO:tasks.workunit.client.0.smithi160.stdout:9/180: creat d4/d30/f34 x:0 0 0 2024-05-25T06:11:33.129 INFO:tasks.workunit.client.0.smithi160.stdout:8/312: fdatasync d5/d44/d1f/d58/f60 0 2024-05-25T06:11:33.130 INFO:tasks.workunit.client.0.smithi160.stdout:5/266: rename db/dc/d13/d1d/d31/f40 to db/dc/d13/d1d/d23/d24/d2e/f4e 0 2024-05-25T06:11:33.131 INFO:tasks.workunit.client.0.smithi160.stdout:9/181: read - d4/d11/d14/f31 zero size 2024-05-25T06:11:33.131 INFO:tasks.workunit.client.0.smithi160.stdout:8/313: stat d5/d44/d1f/d1e 0 2024-05-25T06:11:33.131 INFO:tasks.workunit.client.0.smithi160.stdout:9/182: readlink d4/l10 0 2024-05-25T06:11:33.131 INFO:tasks.workunit.client.0.smithi160.stdout:8/314: fdatasync d5/d7/f19 0 2024-05-25T06:11:33.132 INFO:tasks.workunit.client.0.smithi160.stdout:5/267: write db/dc/d13/d1d/d23/d24/f3e [914283,103422] 0 2024-05-25T06:11:33.133 INFO:tasks.workunit.client.0.smithi160.stdout:8/315: write d5/d44/d1f/d1e/d28/f5e [247414,96173] 0 2024-05-25T06:11:33.135 INFO:tasks.workunit.client.0.smithi160.stdout:8/316: write d5/d9/f24 [571050,111803] 0 2024-05-25T06:11:33.137 INFO:tasks.workunit.client.0.smithi160.stdout:5/268: mknod db/dc/d13/d1d/d23/d45/c4f 0 2024-05-25T06:11:33.138 INFO:tasks.workunit.client.0.smithi160.stdout:5/269: readlink db/dc/l1b 0 2024-05-25T06:11:33.139 INFO:tasks.workunit.client.0.smithi160.stdout:8/317: creat d5/d52/f66 x:0 0 0 2024-05-25T06:11:33.140 INFO:tasks.workunit.client.0.smithi160.stdout:8/318: write d5/d44/d1f/f55 [202177,78236] 0 2024-05-25T06:11:33.140 INFO:tasks.workunit.client.0.smithi160.stdout:5/270: write db/dc/d13/d1d/d23/d24/d2e/f32 [11605,60746] 0 2024-05-25T06:11:33.143 INFO:tasks.workunit.client.0.smithi160.stdout:5/271: creat db/dc/d13/f50 x:0 0 0 2024-05-25T06:11:33.144 INFO:tasks.workunit.client.0.smithi160.stdout:8/319: chown d5/d44/d1f/d1e/d28/d49/d61/c62 157814824 1 2024-05-25T06:11:33.145 INFO:tasks.workunit.client.0.smithi160.stdout:9/183: truncate d4/d11/d1e/d24/fc 3201772 0 2024-05-25T06:11:33.145 INFO:tasks.workunit.client.0.smithi160.stdout:4/300: dread d4/f26 [0,4194304] 0 2024-05-25T06:11:33.146 INFO:tasks.workunit.client.0.smithi160.stdout:5/272: creat db/d47/f51 x:0 0 0 2024-05-25T06:11:33.147 INFO:tasks.workunit.client.0.smithi160.stdout:9/184: stat d4/d11/d14/d23/d25/d27/c2d 0 2024-05-25T06:11:33.147 INFO:tasks.workunit.client.0.smithi160.stdout:7/211: dread d6/f12 [0,4194304] 0 2024-05-25T06:11:33.148 INFO:tasks.workunit.client.0.smithi160.stdout:5/273: fsync f7 0 2024-05-25T06:11:33.150 INFO:tasks.workunit.client.0.smithi160.stdout:9/185: creat d4/d11/d12/f35 x:0 0 0 2024-05-25T06:11:33.151 INFO:tasks.workunit.client.0.smithi160.stdout:4/301: mknod d4/d1c/d54/c5a 0 2024-05-25T06:11:33.152 INFO:tasks.workunit.client.0.smithi160.stdout:4/302: dread - d4/d6/d21/f3f zero size 2024-05-25T06:11:33.152 INFO:tasks.workunit.client.0.smithi160.stdout:5/274: read db/f22 [2385627,19499] 0 2024-05-25T06:11:33.153 INFO:tasks.workunit.client.0.smithi160.stdout:9/186: mknod d4/d11/d1e/c36 0 2024-05-25T06:11:33.153 INFO:tasks.workunit.client.0.smithi160.stdout:7/212: dread d6/f12 [0,4194304] 0 2024-05-25T06:11:33.155 INFO:tasks.workunit.client.0.smithi160.stdout:9/187: unlink d4/d30/f34 0 2024-05-25T06:11:33.156 INFO:tasks.workunit.client.0.smithi160.stdout:5/275: write db/dc/d13/d1d/d23/d24/d2e/f4e [2490115,7377] 0 2024-05-25T06:11:33.157 INFO:tasks.workunit.client.0.smithi160.stdout:7/213: read d6/dc/f17 [1660721,62762] 0 2024-05-25T06:11:33.157 INFO:tasks.workunit.client.0.smithi160.stdout:7/214: chown c3 520125440 1 2024-05-25T06:11:33.157 INFO:tasks.workunit.client.0.smithi160.stdout:9/188: creat d4/d11/d14/f37 x:0 0 0 2024-05-25T06:11:33.158 INFO:tasks.workunit.client.0.smithi160.stdout:9/189: write d4/fd [3122226,17066] 0 2024-05-25T06:11:33.181 INFO:tasks.workunit.client.0.smithi160.stdout:6/179: sync 2024-05-25T06:11:33.181 INFO:tasks.workunit.client.0.smithi160.stdout:1/121: sync 2024-05-25T06:11:33.197 INFO:tasks.workunit.client.1.smithi203.stdout:9/548: chown d0/df/f5f 38438552 1 2024-05-25T06:11:33.198 INFO:tasks.workunit.client.1.smithi203.stdout:0/515: rmdir dd/d3d/d4f/d62 39 2024-05-25T06:11:33.199 INFO:tasks.workunit.client.1.smithi203.stdout:0/516: dread - dd/d5c/f73 zero size 2024-05-25T06:11:33.200 INFO:tasks.workunit.client.1.smithi203.stdout:8/742: link d2/d5/d18/dbf/cd3 d2/d5/d28/d38/d4e/d97/da1/cfe 0 2024-05-25T06:11:33.202 INFO:tasks.workunit.client.1.smithi203.stdout:3/749: dread d10/d16/d23/f6e [0,4194304] 0 2024-05-25T06:11:33.202 INFO:tasks.workunit.client.1.smithi203.stdout:0/517: mknod dd/d32/d37/cb5 0 2024-05-25T06:11:33.226 INFO:tasks.workunit.client.0.smithi160.stdout:7/215: mkdir d6/d3e 0 2024-05-25T06:11:33.227 INFO:tasks.workunit.client.0.smithi160.stdout:8/320: dread d5/d44/d1f/d1e/d28/d4a/f5c [0,4194304] 0 2024-05-25T06:11:33.229 INFO:tasks.workunit.client.0.smithi160.stdout:9/190: creat d4/d11/f38 x:0 0 0 2024-05-25T06:11:33.232 INFO:tasks.workunit.client.0.smithi160.stdout:1/122: creat de/d1b/d22/f24 x:0 0 0 2024-05-25T06:11:33.235 INFO:tasks.workunit.client.0.smithi160.stdout:7/216: dwrite d6/fb [4194304,4194304] 0 2024-05-25T06:11:33.235 INFO:tasks.workunit.client.0.smithi160.stdout:7/217: chown d6 186208857 1 2024-05-25T06:11:33.236 INFO:tasks.workunit.client.0.smithi160.stdout:7/218: fdatasync d6/dc/f25 0 2024-05-25T06:11:33.238 INFO:tasks.workunit.client.0.smithi160.stdout:1/123: mknod de/c25 0 2024-05-25T06:11:33.243 INFO:tasks.workunit.client.0.smithi160.stdout:9/191: link d4/fd d4/d11/d1e/d24/f39 0 2024-05-25T06:11:33.244 INFO:tasks.workunit.client.0.smithi160.stdout:1/124: dwrite de/f19 [0,4194304] 0 2024-05-25T06:11:33.248 INFO:tasks.workunit.client.0.smithi160.stdout:6/180: link d5/c31 d5/de/d32/d1b/d2b/c43 0 2024-05-25T06:11:33.249 INFO:tasks.workunit.client.0.smithi160.stdout:6/181: read d5/f11 [562574,36436] 0 2024-05-25T06:11:33.251 INFO:tasks.workunit.client.0.smithi160.stdout:9/192: creat d4/d11/d14/f3a x:0 0 0 2024-05-25T06:11:33.253 INFO:tasks.workunit.client.0.smithi160.stdout:6/182: dread d5/de/f17 [0,4194304] 0 2024-05-25T06:11:33.253 INFO:tasks.workunit.client.0.smithi160.stdout:7/219: fdatasync d6/dc/f15 0 2024-05-25T06:11:33.254 INFO:tasks.workunit.client.0.smithi160.stdout:6/183: stat d5/de/d32/d1b/d2b 0 2024-05-25T06:11:33.256 INFO:tasks.workunit.client.0.smithi160.stdout:1/125: rename de/f14 to de/d1b/d22/d23/f26 0 2024-05-25T06:11:33.259 INFO:tasks.workunit.client.0.smithi160.stdout:8/321: getdents d5/d44/d1f/d1e/d28 0 2024-05-25T06:11:33.259 INFO:tasks.workunit.client.0.smithi160.stdout:5/276: truncate db/dc/d13/d1d/d31/d3b/f38 793200 0 2024-05-25T06:11:33.264 INFO:tasks.workunit.client.0.smithi160.stdout:7/220: creat d6/d32/f3f x:0 0 0 2024-05-25T06:11:33.266 INFO:tasks.workunit.client.0.smithi160.stdout:5/277: write db/dc/d13/d1d/d23/d24/d2e/f4e [754740,109393] 0 2024-05-25T06:11:33.266 INFO:tasks.workunit.client.0.smithi160.stdout:1/126: dwrite de/f15 [0,4194304] 0 2024-05-25T06:11:33.268 INFO:tasks.workunit.client.0.smithi160.stdout:5/278: write db/dc/d13/d1d/d31/d3b/f42 [899498,14565] 0 2024-05-25T06:11:33.269 INFO:tasks.workunit.client.0.smithi160.stdout:5/279: stat db 0 2024-05-25T06:11:33.269 INFO:tasks.workunit.client.0.smithi160.stdout:8/322: rename d5/f3e to d5/d44/d1f/d1e/d28/f67 0 2024-05-25T06:11:33.271 INFO:tasks.workunit.client.0.smithi160.stdout:7/221: chown d6/dc/d21/l3b 848 1 2024-05-25T06:11:33.276 INFO:tasks.workunit.client.0.smithi160.stdout:1/127: rename de/f11 to de/d1b/f27 0 2024-05-25T06:11:33.278 INFO:tasks.workunit.client.0.smithi160.stdout:5/280: symlink db/dc/d13/d1d/d23/d24/d2e/l52 0 2024-05-25T06:11:33.280 INFO:tasks.workunit.client.0.smithi160.stdout:5/281: write db/dc/f15 [6640077,84677] 0 2024-05-25T06:11:33.280 INFO:tasks.workunit.client.0.smithi160.stdout:1/128: fsync de/d1b/f1d 0 2024-05-25T06:11:33.281 INFO:tasks.workunit.client.0.smithi160.stdout:7/222: creat d6/d2b/d2d/f40 x:0 0 0 2024-05-25T06:11:33.283 INFO:tasks.workunit.client.0.smithi160.stdout:5/282: creat db/f53 x:0 0 0 2024-05-25T06:11:33.285 INFO:tasks.workunit.client.0.smithi160.stdout:8/323: dwrite d5/d44/d1f/f22 [0,4194304] 0 2024-05-25T06:11:33.285 INFO:tasks.workunit.client.0.smithi160.stdout:1/129: mkdir de/d1b/d28 0 2024-05-25T06:11:33.287 INFO:tasks.workunit.client.0.smithi160.stdout:7/223: creat d6/dc/f41 x:0 0 0 2024-05-25T06:11:33.290 INFO:tasks.workunit.client.1.smithi203.stdout:9/549: dwrite d0/df/d37/f58 [0,4194304] 0 2024-05-25T06:11:33.292 INFO:tasks.workunit.client.0.smithi160.stdout:6/184: dread d5/de/ff [0,4194304] 0 2024-05-25T06:11:33.292 INFO:tasks.workunit.client.0.smithi160.stdout:5/283: dwrite db/dc/d13/d1d/d23/d24/d2e/f2f [0,4194304] 0 2024-05-25T06:11:33.294 INFO:tasks.workunit.client.0.smithi160.stdout:5/284: write f7 [1188947,76616] 0 2024-05-25T06:11:33.296 INFO:tasks.workunit.client.1.smithi203.stdout:6/886: dread d8/d3d/f6a [0,4194304] 0 2024-05-25T06:11:33.296 INFO:tasks.workunit.client.0.smithi160.stdout:7/224: truncate d6/dc/f24 160699 0 2024-05-25T06:11:33.297 INFO:tasks.workunit.client.1.smithi203.stdout:3/750: getdents d10/d16/d23/d4a 0 2024-05-25T06:11:33.300 INFO:tasks.workunit.client.0.smithi160.stdout:3/167: sync 2024-05-25T06:11:33.300 INFO:tasks.workunit.client.0.smithi160.stdout:0/222: sync 2024-05-25T06:11:33.300 INFO:tasks.workunit.client.0.smithi160.stdout:2/203: sync 2024-05-25T06:11:33.301 INFO:tasks.workunit.client.1.smithi203.stdout:8/743: link d2/d5/d28/d38/d4e/d97/l65 d2/d5/d10/d11/lff 0 2024-05-25T06:11:33.301 INFO:tasks.workunit.client.0.smithi160.stdout:3/168: chown f11 74464597 1 2024-05-25T06:11:33.302 INFO:tasks.workunit.client.0.smithi160.stdout:0/223: write f12 [73291,63485] 0 2024-05-25T06:11:33.302 INFO:tasks.workunit.client.0.smithi160.stdout:6/185: dwrite d5/f14 [4194304,4194304] 0 2024-05-25T06:11:33.305 INFO:tasks.workunit.client.1.smithi203.stdout:9/550: dwrite d0/d17/d20/d3c/db2/d7a/fb4 [0,4194304] 0 2024-05-25T06:11:33.308 INFO:tasks.workunit.client.1.smithi203.stdout:6/887: rename d8/d11/d1b/d75/fd7 to d8/d2a/d6c/d87/d113/f120 0 2024-05-25T06:11:33.309 INFO:tasks.workunit.client.0.smithi160.stdout:3/169: read f13 [4013824,83710] 0 2024-05-25T06:11:33.309 INFO:tasks.workunit.client.1.smithi203.stdout:8/744: rmdir d2/d5/d10/dfa/dfb 39 2024-05-25T06:11:33.310 INFO:tasks.workunit.client.1.smithi203.stdout:3/751: getdents d10/d22/d2a/d99 0 2024-05-25T06:11:33.311 INFO:tasks.workunit.client.0.smithi160.stdout:0/224: dread d2b/f30 [0,4194304] 0 2024-05-25T06:11:33.311 INFO:tasks.workunit.client.0.smithi160.stdout:0/225: readlink l2a 0 2024-05-25T06:11:33.314 INFO:tasks.workunit.client.1.smithi203.stdout:9/551: creat d0/d17/d20/d6e/d75/fc0 x:0 0 0 2024-05-25T06:11:33.314 INFO:tasks.workunit.client.1.smithi203.stdout:3/752: fdatasync d10/d16/d23/d4a/d88/fd8 0 2024-05-25T06:11:33.314 INFO:tasks.workunit.client.0.smithi160.stdout:5/285: mkdir db/dc/d13/d54 0 2024-05-25T06:11:33.314 INFO:tasks.workunit.client.0.smithi160.stdout:5/286: stat db/f2a 0 2024-05-25T06:11:33.315 INFO:tasks.workunit.client.1.smithi203.stdout:8/745: mkdir d2/d5/d10/d45/d90/d100 0 2024-05-25T06:11:33.316 INFO:tasks.workunit.client.1.smithi203.stdout:9/552: creat d0/df/fc1 x:0 0 0 2024-05-25T06:11:33.317 INFO:tasks.workunit.client.1.smithi203.stdout:6/888: dread d8/d70/d38/f4c [0,4194304] 0 2024-05-25T06:11:33.317 INFO:tasks.workunit.client.0.smithi160.stdout:2/204: mknod d12/d13/c3f 0 2024-05-25T06:11:33.319 INFO:tasks.workunit.client.1.smithi203.stdout:3/753: creat d10/d16/d23/d4a/d88/dca/ff4 x:0 0 0 2024-05-25T06:11:33.325 INFO:tasks.workunit.client.0.smithi160.stdout:7/225: rmdir d6/d2b/d2d 39 2024-05-25T06:11:33.325 INFO:tasks.workunit.client.1.smithi203.stdout:8/746: link d2/d5/l32 d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/l101 0 2024-05-25T06:11:33.328 INFO:tasks.workunit.client.0.smithi160.stdout:3/170: unlink d19/d2f/f30 0 2024-05-25T06:11:33.328 INFO:tasks.workunit.client.1.smithi203.stdout:8/747: creat d2/d5/d28/d38/d4e/d97/f102 x:0 0 0 2024-05-25T06:11:33.329 INFO:tasks.workunit.client.1.smithi203.stdout:3/754: dwrite d10/d16/d23/d45/f9b [0,4194304] 0 2024-05-25T06:11:33.330 INFO:tasks.workunit.client.0.smithi160.stdout:5/287: readlink l2 0 2024-05-25T06:11:33.331 INFO:tasks.workunit.client.0.smithi160.stdout:7/226: dwrite d6/dc/f25 [0,4194304] 0 2024-05-25T06:11:33.331 INFO:tasks.workunit.client.0.smithi160.stdout:5/288: readlink db/l14 0 2024-05-25T06:11:33.332 INFO:tasks.workunit.client.0.smithi160.stdout:7/227: fsync d6/f12 0 2024-05-25T06:11:33.335 INFO:tasks.workunit.client.0.smithi160.stdout:1/130: getdents de/d1b 0 2024-05-25T06:11:33.335 INFO:tasks.workunit.client.0.smithi160.stdout:1/131: stat cc 0 2024-05-25T06:11:33.339 INFO:tasks.workunit.client.0.smithi160.stdout:0/226: rename d2b/d31/d35/d36/d37/c3f to d2b/d46/c4b 0 2024-05-25T06:11:33.341 INFO:tasks.workunit.client.0.smithi160.stdout:2/205: unlink d12/l16 0 2024-05-25T06:11:33.344 INFO:tasks.workunit.client.0.smithi160.stdout:3/171: write d19/f29 [3040757,25222] 0 2024-05-25T06:11:33.344 INFO:tasks.workunit.client.0.smithi160.stdout:1/132: unlink de/d1b/d22/d23/f26 0 2024-05-25T06:11:33.345 INFO:tasks.workunit.client.0.smithi160.stdout:0/227: creat d2b/d31/d35/d43/f4c x:0 0 0 2024-05-25T06:11:33.347 INFO:tasks.workunit.client.0.smithi160.stdout:0/228: write d2b/d31/d35/f3e [82453,65292] 0 2024-05-25T06:11:33.349 INFO:tasks.workunit.client.1.smithi203.stdout:6/889: dread d8/d11/d15/d28/f5e [0,4194304] 0 2024-05-25T06:11:33.349 INFO:tasks.workunit.client.0.smithi160.stdout:2/206: dwrite d12/d1c/f2b [0,4194304] 0 2024-05-25T06:11:33.353 INFO:tasks.workunit.client.0.smithi160.stdout:1/133: creat de/f29 x:0 0 0 2024-05-25T06:11:33.354 INFO:tasks.workunit.client.1.smithi203.stdout:9/553: truncate d0/d17/f81 2456371 0 2024-05-25T06:11:33.354 INFO:tasks.workunit.client.0.smithi160.stdout:1/134: chown de/d1b/d22/d23 3 1 2024-05-25T06:11:33.355 INFO:tasks.workunit.client.1.smithi203.stdout:9/554: readlink d0/d17/d20/d3c/l62 0 2024-05-25T06:11:33.355 INFO:tasks.workunit.client.0.smithi160.stdout:2/207: unlink d12/d13/f35 0 2024-05-25T06:11:33.356 INFO:tasks.workunit.client.1.smithi203.stdout:9/555: truncate d0/d57/f76 910188 0 2024-05-25T06:11:33.357 INFO:tasks.workunit.client.0.smithi160.stdout:6/186: link d5/de/d32/c25 d5/de/c44 0 2024-05-25T06:11:33.360 INFO:tasks.workunit.client.0.smithi160.stdout:0/229: creat d2b/d31/d35/d36/d37/f4d x:0 0 0 2024-05-25T06:11:33.360 INFO:tasks.workunit.client.1.smithi203.stdout:9/556: creat d0/df/d2d/d92/fc2 x:0 0 0 2024-05-25T06:11:33.361 INFO:tasks.workunit.client.1.smithi203.stdout:8/748: truncate d2/d5/d2d/f4b 4339984 0 2024-05-25T06:11:33.361 INFO:tasks.workunit.client.0.smithi160.stdout:3/172: unlink d19/c1c 0 2024-05-25T06:11:33.361 INFO:tasks.workunit.client.0.smithi160.stdout:6/187: write d5/f1e [325935,34514] 0 2024-05-25T06:11:33.363 INFO:tasks.workunit.client.1.smithi203.stdout:6/890: truncate d8/d11/f19 3305082 0 2024-05-25T06:11:33.364 INFO:tasks.workunit.client.0.smithi160.stdout:0/230: write d2b/d31/f38 [600067,314] 0 2024-05-25T06:11:33.365 INFO:tasks.workunit.client.0.smithi160.stdout:6/188: fsync d5/f34 0 2024-05-25T06:11:33.366 INFO:tasks.workunit.client.1.smithi203.stdout:3/755: dwrite d10/d16/d23/d4a/fe3 [0,4194304] 0 2024-05-25T06:11:33.367 INFO:tasks.workunit.client.1.smithi203.stdout:6/891: stat d8/d11/d15/d28/d52/d43/d73/c102 0 2024-05-25T06:11:33.368 INFO:tasks.workunit.client.0.smithi160.stdout:1/135: dwrite de/f19 [0,4194304] 0 2024-05-25T06:11:33.370 INFO:tasks.workunit.client.0.smithi160.stdout:6/189: chown d5/d39/f41 14903974 1 2024-05-25T06:11:33.371 INFO:tasks.workunit.client.0.smithi160.stdout:9/193: dwrite d4/d11/d1e/d24/fc [0,4194304] 0 2024-05-25T06:11:33.371 INFO:tasks.workunit.client.1.smithi203.stdout:3/756: dread - d10/d16/d23/d4a/d88/dca/ff4 zero size 2024-05-25T06:11:33.372 INFO:tasks.workunit.client.0.smithi160.stdout:2/208: dwrite d12/d1c/f2f [0,4194304] 0 2024-05-25T06:11:33.375 INFO:tasks.workunit.client.1.smithi203.stdout:9/557: dwrite d0/df/d37/f79 [0,4194304] 0 2024-05-25T06:11:33.377 INFO:tasks.workunit.client.1.smithi203.stdout:6/892: unlink d8/d11/d1b/d75/dcb/dcf/d46/d88/da5/fe2 0 2024-05-25T06:11:33.380 INFO:tasks.workunit.client.0.smithi160.stdout:3/173: rename d19/d2f/d34/l3c to d19/d2f/d34/l3d 0 2024-05-25T06:11:33.383 INFO:tasks.workunit.client.0.smithi160.stdout:4/303: sync 2024-05-25T06:11:33.383 INFO:tasks.workunit.client.0.smithi160.stdout:4/304: stat d4/d6/d21/d42/d49/f4b 0 2024-05-25T06:11:33.388 INFO:tasks.workunit.client.0.smithi160.stdout:6/190: chown d5/de/c29 0 1 2024-05-25T06:11:33.389 INFO:tasks.workunit.client.0.smithi160.stdout:5/289: dwrite db/dc/d13/d1d/d23/f33 [0,4194304] 0 2024-05-25T06:11:33.389 INFO:tasks.workunit.client.0.smithi160.stdout:7/228: truncate d6/dc/f25 2456887 0 2024-05-25T06:11:33.389 INFO:tasks.workunit.client.1.smithi203.stdout:3/757: readlink d10/d16/d23/d4a/d88/dca/ld0 0 2024-05-25T06:11:33.389 INFO:tasks.workunit.client.1.smithi203.stdout:9/558: dread d0/d57/f76 [0,4194304] 0 2024-05-25T06:11:33.390 INFO:tasks.workunit.client.1.smithi203.stdout:6/893: creat d8/d11/d1b/d75/f121 x:0 0 0 2024-05-25T06:11:33.392 INFO:tasks.workunit.client.0.smithi160.stdout:7/229: chown f1 16 1 2024-05-25T06:11:33.392 INFO:tasks.workunit.client.1.smithi203.stdout:6/894: dread - d8/d11/d1b/d75/dcb/fcd zero size 2024-05-25T06:11:33.394 INFO:tasks.workunit.client.0.smithi160.stdout:1/136: mknod de/d1b/d28/c2a 0 2024-05-25T06:11:33.394 INFO:tasks.workunit.client.0.smithi160.stdout:9/194: mknod d4/d11/d12/d2e/c3b 0 2024-05-25T06:11:33.394 INFO:tasks.workunit.client.0.smithi160.stdout:1/137: chown de/f10 288293 1 2024-05-25T06:11:33.394 INFO:tasks.workunit.client.1.smithi203.stdout:6/895: dread - d8/d11/d1b/d75/f121 zero size 2024-05-25T06:11:33.395 INFO:tasks.workunit.client.0.smithi160.stdout:3/174: creat d19/d2f/d38/f3e x:0 0 0 2024-05-25T06:11:33.396 INFO:tasks.workunit.client.1.smithi203.stdout:9/559: dread d0/df/d37/f58 [0,4194304] 0 2024-05-25T06:11:33.397 INFO:tasks.workunit.client.1.smithi203.stdout:9/560: fdatasync d0/df/d37/da8/db8/fb9 0 2024-05-25T06:11:33.399 INFO:tasks.workunit.client.1.smithi203.stdout:9/561: truncate d0/df/d73/f53 1512382 0 2024-05-25T06:11:33.403 INFO:tasks.workunit.client.0.smithi160.stdout:6/191: mkdir d5/d45 0 2024-05-25T06:11:33.407 INFO:tasks.workunit.client.1.smithi203.stdout:4/674: sync 2024-05-25T06:11:33.407 INFO:tasks.workunit.client.1.smithi203.stdout:1/525: sync 2024-05-25T06:11:33.408 INFO:tasks.workunit.client.0.smithi160.stdout:3/175: mknod d19/d2f/c3f 0 2024-05-25T06:11:33.410 INFO:tasks.workunit.client.1.smithi203.stdout:8/749: write d2/d5/d28/d38/d4e/d97/f77 [279501,9153] 0 2024-05-25T06:11:33.411 INFO:tasks.workunit.client.1.smithi203.stdout:8/750: chown d2/d5/d10/d4a/l6b 5087125 1 2024-05-25T06:11:33.413 INFO:tasks.workunit.client.0.smithi160.stdout:4/305: rename d4/d6/d21/d42/d49 to d4/d6/d5b 0 2024-05-25T06:11:33.414 INFO:tasks.workunit.client.0.smithi160.stdout:5/290: creat db/dc/d13/d1d/d23/f55 x:0 0 0 2024-05-25T06:11:33.414 INFO:tasks.workunit.client.0.smithi160.stdout:8/324: sync 2024-05-25T06:11:33.415 INFO:tasks.workunit.client.0.smithi160.stdout:1/138: rmdir de/d1b/d22/d23 0 2024-05-25T06:11:33.416 INFO:tasks.workunit.client.1.smithi203.stdout:1/526: dwrite d14/d57/d7e/fa7 [0,4194304] 0 2024-05-25T06:11:33.417 INFO:tasks.workunit.client.0.smithi160.stdout:3/176: rename d19/d22/f28 to d19/d2f/d34/f40 0 2024-05-25T06:11:33.417 INFO:tasks.workunit.client.0.smithi160.stdout:1/139: chown f9 113 1 2024-05-25T06:11:33.420 INFO:tasks.workunit.client.0.smithi160.stdout:5/291: mkdir db/dc/d13/d1d/d31/d56 0 2024-05-25T06:11:33.421 INFO:tasks.workunit.client.0.smithi160.stdout:7/230: dread d6/dc/f17 [0,4194304] 0 2024-05-25T06:11:33.421 INFO:tasks.workunit.client.0.smithi160.stdout:0/231: write d2b/d31/f34 [3819198,33864] 0 2024-05-25T06:11:33.425 INFO:tasks.workunit.client.0.smithi160.stdout:4/306: dwrite d4/d6/d35/f53 [4194304,4194304] 0 2024-05-25T06:11:33.425 INFO:tasks.workunit.client.0.smithi160.stdout:8/325: rename d5/d44/d1f/d1e/d28/d4a/f59 to d5/d44/f68 0 2024-05-25T06:11:33.425 INFO:tasks.workunit.client.0.smithi160.stdout:7/231: dread - d6/dc/f41 zero size 2024-05-25T06:11:33.428 INFO:tasks.workunit.client.0.smithi160.stdout:0/232: readlink d2b/d31/d35/d43/l47 0 2024-05-25T06:11:33.428 INFO:tasks.workunit.client.0.smithi160.stdout:2/209: truncate fb 2302932 0 2024-05-25T06:11:33.429 INFO:tasks.workunit.client.0.smithi160.stdout:7/232: truncate d6/d32/f3f 16800 0 2024-05-25T06:11:33.431 INFO:tasks.workunit.client.0.smithi160.stdout:1/140: link de/f17 de/d1b/f2b 0 2024-05-25T06:11:33.432 INFO:tasks.workunit.client.0.smithi160.stdout:9/195: write f2 [5444692,130653] 0 2024-05-25T06:11:33.435 INFO:tasks.workunit.client.0.smithi160.stdout:2/210: rmdir d12 39 2024-05-25T06:11:33.437 INFO:tasks.workunit.client.0.smithi160.stdout:3/177: dwrite d19/d2f/f35 [0,4194304] 0 2024-05-25T06:11:33.440 INFO:tasks.workunit.client.0.smithi160.stdout:9/196: write d4/d11/d1e/d24/fc [3481920,129322] 0 2024-05-25T06:11:33.442 INFO:tasks.workunit.client.1.smithi203.stdout:3/758: mkdir d10/d16/d50/d7d/df5 0 2024-05-25T06:11:33.443 INFO:tasks.workunit.client.0.smithi160.stdout:0/233: mkdir d2b/d4e 0 2024-05-25T06:11:33.443 INFO:tasks.workunit.client.0.smithi160.stdout:6/192: truncate d5/fa 2478806 0 2024-05-25T06:11:33.444 INFO:tasks.workunit.client.1.smithi203.stdout:3/759: truncate d10/d16/d23/d4a/d88/fd8 1036896 0 2024-05-25T06:11:33.444 INFO:tasks.workunit.client.1.smithi203.stdout:6/896: mkdir d8/d11/d1b/d75/dcb/dcf/d46/d88/d122 0 2024-05-25T06:11:33.445 INFO:tasks.workunit.client.1.smithi203.stdout:9/562: creat d0/d17/d20/d6e/fc3 x:0 0 0 2024-05-25T06:11:33.447 INFO:tasks.workunit.client.1.smithi203.stdout:9/563: chown d0/d17/d23/f90 0 1 2024-05-25T06:11:33.447 INFO:tasks.workunit.client.0.smithi160.stdout:8/326: symlink d5/d44/d1f/d1e/d28/d4a/d4e/l69 0 2024-05-25T06:11:33.448 INFO:tasks.workunit.client.1.smithi203.stdout:9/564: readlink d0/d17/d20/d29/l50 0 2024-05-25T06:11:33.448 INFO:tasks.workunit.client.0.smithi160.stdout:6/193: fsync d5/f7 0 2024-05-25T06:11:33.449 INFO:tasks.workunit.client.1.smithi203.stdout:9/565: chown d0/df/d73/f53 2469391 1 2024-05-25T06:11:33.450 INFO:tasks.workunit.client.0.smithi160.stdout:1/141: symlink de/l2c 0 2024-05-25T06:11:33.451 INFO:tasks.workunit.client.0.smithi160.stdout:5/292: dread f4 [4194304,4194304] 0 2024-05-25T06:11:33.451 INFO:tasks.workunit.client.0.smithi160.stdout:6/194: stat d5/d30/d36/c2e 0 2024-05-25T06:11:33.451 INFO:tasks.workunit.client.0.smithi160.stdout:7/233: dwrite f1 [0,4194304] 0 2024-05-25T06:11:33.454 INFO:tasks.workunit.client.0.smithi160.stdout:4/307: dwrite d4/d1c/f3e [4194304,4194304] 0 2024-05-25T06:11:33.455 INFO:tasks.workunit.client.1.smithi203.stdout:6/897: rename d8/d2a/d6c/dc1 to d8/d70/d38/d108/d123 0 2024-05-25T06:11:33.457 INFO:tasks.workunit.client.1.smithi203.stdout:6/898: dread - d8/d11/d15/d28/d52/d43/fea zero size 2024-05-25T06:11:33.457 INFO:tasks.workunit.client.0.smithi160.stdout:3/178: rename l4 to d19/d2f/l41 0 2024-05-25T06:11:33.458 INFO:tasks.workunit.client.1.smithi203.stdout:8/751: symlink d2/d5/d10/d45/d5d/l103 0 2024-05-25T06:11:33.458 INFO:tasks.workunit.client.1.smithi203.stdout:3/760: dwrite d10/d16/d23/f6e [0,4194304] 0 2024-05-25T06:11:33.460 INFO:tasks.workunit.client.0.smithi160.stdout:5/293: write db/dc/d13/d1d/d31/d3b/f41 [5399369,29487] 0 2024-05-25T06:11:33.462 INFO:tasks.workunit.client.1.smithi203.stdout:8/752: write d2/d5/d18/fb1 [656990,74753] 0 2024-05-25T06:11:33.463 INFO:tasks.workunit.client.1.smithi203.stdout:9/566: read d0/df/f15 [1865571,90285] 0 2024-05-25T06:11:33.465 INFO:tasks.workunit.client.0.smithi160.stdout:6/195: write d5/de/d32/f37 [544852,45407] 0 2024-05-25T06:11:33.465 INFO:tasks.workunit.client.0.smithi160.stdout:8/327: mkdir d5/d39/d6a 0 2024-05-25T06:11:33.466 INFO:tasks.workunit.client.1.smithi203.stdout:1/527: getdents d14/d57/d6b 0 2024-05-25T06:11:33.466 INFO:tasks.workunit.client.1.smithi203.stdout:4/675: dread d0/d8/d3a/f53 [0,4194304] 0 2024-05-25T06:11:33.467 INFO:tasks.workunit.client.1.smithi203.stdout:1/528: fdatasync d14/f32 0 2024-05-25T06:11:33.469 INFO:tasks.workunit.client.1.smithi203.stdout:8/753: rmdir d2/d5/d10/d11/d59 39 2024-05-25T06:11:33.469 INFO:tasks.workunit.client.0.smithi160.stdout:9/197: mknod d4/d11/d14/d23/c3c 0 2024-05-25T06:11:33.470 INFO:tasks.workunit.client.0.smithi160.stdout:9/198: stat d4/d11/d14/d23/d25 0 2024-05-25T06:11:33.474 INFO:tasks.workunit.client.1.smithi203.stdout:3/761: link d10/d16/d23/d3a/f55 d10/d16/d23/d4a/d88/dca/ff6 0 2024-05-25T06:11:33.475 INFO:tasks.workunit.client.0.smithi160.stdout:9/199: write d4/f1c [2241622,110041] 0 2024-05-25T06:11:33.475 INFO:tasks.workunit.client.0.smithi160.stdout:9/200: write d4/d11/d1e/d24/f39 [6871078,20649] 0 2024-05-25T06:11:33.476 INFO:tasks.workunit.client.0.smithi160.stdout:1/142: dwrite f7 [0,4194304] 0 2024-05-25T06:11:33.478 INFO:tasks.workunit.client.0.smithi160.stdout:1/143: read fa [2046264,43510] 0 2024-05-25T06:11:33.478 INFO:tasks.workunit.client.0.smithi160.stdout:1/144: stat de/f29 0 2024-05-25T06:11:33.478 INFO:tasks.workunit.client.1.smithi203.stdout:4/676: symlink d0/d9/d50/d6b/lda 0 2024-05-25T06:11:33.479 INFO:tasks.workunit.client.1.smithi203.stdout:6/899: read d8/fb [911014,89719] 0 2024-05-25T06:11:33.480 INFO:tasks.workunit.client.1.smithi203.stdout:6/900: chown d8/d11/d47/d60/l6d 3 1 2024-05-25T06:11:33.482 INFO:tasks.workunit.client.0.smithi160.stdout:6/196: rename d5/l13 to d5/de/d32/d1b/l46 0 2024-05-25T06:11:33.482 INFO:tasks.workunit.client.0.smithi160.stdout:1/145: mknod de/c2d 0 2024-05-25T06:11:33.482 INFO:tasks.workunit.client.0.smithi160.stdout:4/308: write d4/f3b [4181369,120684] 0 2024-05-25T06:11:33.483 INFO:tasks.workunit.client.1.smithi203.stdout:3/762: creat d10/d16/d23/d3a/dc6/ff7 x:0 0 0 2024-05-25T06:11:33.483 INFO:tasks.workunit.client.0.smithi160.stdout:9/201: rename d4/d30 to d4/d30/d3d 22 2024-05-25T06:11:33.483 INFO:tasks.workunit.client.0.smithi160.stdout:0/234: link l2a d2b/d31/d35/d41/l4f 0 2024-05-25T06:11:33.484 INFO:tasks.workunit.client.0.smithi160.stdout:1/146: write de/f13 [394113,48619] 0 2024-05-25T06:11:33.485 INFO:tasks.workunit.client.0.smithi160.stdout:1/147: chown de 2 1 2024-05-25T06:11:33.486 INFO:tasks.workunit.client.1.smithi203.stdout:3/763: chown d10/d16/d23/d38/c7c 13711164 1 2024-05-25T06:11:33.486 INFO:tasks.workunit.client.1.smithi203.stdout:6/901: write d8/d5a/d9b/dcc/f100 [733715,3465] 0 2024-05-25T06:11:33.487 INFO:tasks.workunit.client.0.smithi160.stdout:8/328: mkdir d5/d44/d1f/d6b 0 2024-05-25T06:11:33.488 INFO:tasks.workunit.client.1.smithi203.stdout:1/529: creat d14/d37/d5e/fb1 x:0 0 0 2024-05-25T06:11:33.489 INFO:tasks.workunit.client.1.smithi203.stdout:3/764: readlink d10/d22/d2a/l41 0 2024-05-25T06:11:33.490 INFO:tasks.workunit.client.0.smithi160.stdout:0/235: truncate d2b/d31/d35/d36/d37/f44 78194 0 2024-05-25T06:11:33.491 INFO:tasks.workunit.client.0.smithi160.stdout:4/309: mknod d4/d6/d21/c5c 0 2024-05-25T06:11:33.491 INFO:tasks.workunit.client.1.smithi203.stdout:6/902: dread - d8/d2a/d6c/d87/daa/fb4 zero size 2024-05-25T06:11:33.493 INFO:tasks.workunit.client.1.smithi203.stdout:4/677: dwrite d0/d8/d1f/f2b [4194304,4194304] 0 2024-05-25T06:11:33.493 INFO:tasks.workunit.client.0.smithi160.stdout:0/236: dread - d2b/d31/d35/f49 zero size 2024-05-25T06:11:33.495 INFO:tasks.workunit.client.1.smithi203.stdout:6/903: stat d8/d11/d15/fbc 0 2024-05-25T06:11:33.495 INFO:tasks.workunit.client.1.smithi203.stdout:9/567: truncate d0/df/d37/d4f/f80 728988 0 2024-05-25T06:11:33.495 INFO:tasks.workunit.client.1.smithi203.stdout:8/754: dwrite d2/d5/d10/d11/d59/fd4 [0,4194304] 0 2024-05-25T06:11:33.497 INFO:tasks.workunit.client.0.smithi160.stdout:2/211: getdents d12/d1c 0 2024-05-25T06:11:33.498 INFO:tasks.workunit.client.0.smithi160.stdout:4/310: write d4/d6/d28/f17 [5305736,112329] 0 2024-05-25T06:11:33.498 INFO:tasks.workunit.client.1.smithi203.stdout:2/622: sync 2024-05-25T06:11:33.503 INFO:tasks.workunit.client.0.smithi160.stdout:9/202: dwrite d4/d11/d14/f3a [0,4194304] 0 2024-05-25T06:11:33.504 INFO:tasks.workunit.client.1.smithi203.stdout:9/568: fdatasync d0/d17/d20/d6e/d75/f98 0 2024-05-25T06:11:33.504 INFO:tasks.workunit.client.1.smithi203.stdout:6/904: stat d8/d11/d15/d28/d52/d43/c6b 0 2024-05-25T06:11:33.505 INFO:tasks.workunit.client.0.smithi160.stdout:0/237: write d2b/d31/f38 [1262571,46123] 0 2024-05-25T06:11:33.505 INFO:tasks.workunit.client.1.smithi203.stdout:8/755: dread - d2/d5/d28/d38/fde zero size 2024-05-25T06:11:33.507 INFO:tasks.workunit.client.1.smithi203.stdout:1/530: truncate d14/f32 857863 0 2024-05-25T06:11:33.507 INFO:tasks.workunit.client.1.smithi203.stdout:6/905: chown d8/d11/d15/d28/d52/d43/d73/c102 3 1 2024-05-25T06:11:33.508 INFO:tasks.workunit.client.1.smithi203.stdout:3/765: dwrite d10/f51 [0,4194304] 0 2024-05-25T06:11:33.508 INFO:tasks.workunit.client.0.smithi160.stdout:5/294: truncate db/dc/d13/d1d/d31/d3b/f2b 1065228 0 2024-05-25T06:11:33.508 INFO:tasks.workunit.client.0.smithi160.stdout:8/329: dread d5/d44/d1f/d1e/d28/f3f [0,4194304] 0 2024-05-25T06:11:33.509 INFO:tasks.workunit.client.0.smithi160.stdout:7/234: write d6/dc/d21/f2a [122099,71931] 0 2024-05-25T06:11:33.511 INFO:tasks.workunit.client.1.smithi203.stdout:4/678: getdents d0/d9/d50/d79/d9a/dbe 0 2024-05-25T06:11:33.512 INFO:tasks.workunit.client.0.smithi160.stdout:2/212: truncate d12/d1c/f1e 565054 0 2024-05-25T06:11:33.513 INFO:tasks.workunit.client.0.smithi160.stdout:1/148: dwrite f2 [4194304,4194304] 0 2024-05-25T06:11:33.514 INFO:tasks.workunit.client.0.smithi160.stdout:9/203: chown d4/d11/d1e/d24/f39 11752228 1 2024-05-25T06:11:33.516 INFO:tasks.workunit.client.1.smithi203.stdout:8/756: fsync d2/d5/f8 0 2024-05-25T06:11:33.523 INFO:tasks.workunit.client.1.smithi203.stdout:9/569: dwrite d0/d57/d8f/f9e [4194304,4194304] 0 2024-05-25T06:11:33.524 INFO:tasks.workunit.client.1.smithi203.stdout:1/531: rename d14/d57/c79 to d14/d7c/d9d/cb2 0 2024-05-25T06:11:33.526 INFO:tasks.workunit.client.0.smithi160.stdout:0/238: mknod d2b/d31/d35/d36/d37/c50 0 2024-05-25T06:11:33.527 INFO:tasks.workunit.client.1.smithi203.stdout:8/757: mknod d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/c104 0 2024-05-25T06:11:33.529 INFO:tasks.workunit.client.0.smithi160.stdout:2/213: symlink d12/d13/l40 0 2024-05-25T06:11:33.531 INFO:tasks.workunit.client.1.smithi203.stdout:3/766: getdents d10/d16/da1/de6 0 2024-05-25T06:11:33.531 INFO:tasks.workunit.client.1.smithi203.stdout:6/906: dwrite d8/d11/d15/fbc [0,4194304] 0 2024-05-25T06:11:33.532 INFO:tasks.workunit.client.0.smithi160.stdout:9/204: rename d4/d11/d14/d23 to d4/d11/d1e/d3e 0 2024-05-25T06:11:33.533 INFO:tasks.workunit.client.0.smithi160.stdout:4/311: dwrite d4/d6/d35/d3d/d55/f56 [0,4194304] 0 2024-05-25T06:11:33.534 INFO:tasks.workunit.client.1.smithi203.stdout:2/623: write d2/f6c [2724276,82844] 0 2024-05-25T06:11:33.539 INFO:tasks.workunit.client.1.smithi203.stdout:9/570: rename d0/l83 to d0/df/d37/lc4 0 2024-05-25T06:11:33.542 INFO:tasks.workunit.client.1.smithi203.stdout:2/624: mknod d2/d1a/db0/cd5 0 2024-05-25T06:11:33.545 INFO:tasks.workunit.client.1.smithi203.stdout:8/758: creat d2/d5/d18/d31/dac/dc2/d8a/d8f/dfc/f105 x:0 0 0 2024-05-25T06:11:33.546 INFO:tasks.workunit.client.0.smithi160.stdout:7/235: dwrite d6/d2c/f3c [0,4194304] 0 2024-05-25T06:11:33.548 INFO:tasks.workunit.client.0.smithi160.stdout:4/312: symlink d4/d1c/d54/l5d 0 2024-05-25T06:11:33.548 INFO:tasks.workunit.client.0.smithi160.stdout:0/239: mknod d2b/d31/d35/d36/d37/c51 0 2024-05-25T06:11:33.550 INFO:tasks.workunit.client.0.smithi160.stdout:0/240: write d2b/d31/f34 [2766679,95577] 0 2024-05-25T06:11:33.551 INFO:tasks.workunit.client.1.smithi203.stdout:1/532: rename d14/d37 to d14/d57/d1d/d3e/d36/db3 0 2024-05-25T06:11:33.552 INFO:tasks.workunit.client.1.smithi203.stdout:4/679: dread d0/d8/d1f/d26/f1e [0,4194304] 0 2024-05-25T06:11:33.552 INFO:tasks.workunit.client.1.smithi203.stdout:3/767: rmdir d10/d16/d50/d7d/df5 0 2024-05-25T06:11:33.554 INFO:tasks.workunit.client.0.smithi160.stdout:7/236: creat d6/d32/f42 x:0 0 0 2024-05-25T06:11:33.557 INFO:tasks.workunit.client.1.smithi203.stdout:8/759: rename d2/d5/d10/d11/cf1 to d2/d5/d2d/c106 0 2024-05-25T06:11:33.557 INFO:tasks.workunit.client.0.smithi160.stdout:3/179: dread d19/f2b [0,4194304] 0 2024-05-25T06:11:33.558 INFO:tasks.workunit.client.1.smithi203.stdout:8/760: write d2/d5/d10/fc6 [562235,62571] 0 2024-05-25T06:11:33.558 INFO:tasks.workunit.client.1.smithi203.stdout:2/625: symlink d2/d4/d10/dc8/ld6 0 2024-05-25T06:11:33.559 INFO:tasks.workunit.client.0.smithi160.stdout:2/214: dwrite d12/f25 [0,4194304] 0 2024-05-25T06:11:33.560 INFO:tasks.workunit.client.0.smithi160.stdout:7/237: creat d6/d2c/f43 x:0 0 0 2024-05-25T06:11:33.560 INFO:tasks.workunit.client.1.smithi203.stdout:4/680: mknod d0/d9/d11/d54/cdb 0 2024-05-25T06:11:33.561 INFO:tasks.workunit.client.1.smithi203.stdout:2/626: truncate d2/d38/d43/d60/f96 4795212 0 2024-05-25T06:11:33.562 INFO:tasks.workunit.client.0.smithi160.stdout:0/241: dread d2b/d31/d35/d36/d37/f3b [0,4194304] 0 2024-05-25T06:11:33.563 INFO:tasks.workunit.client.1.smithi203.stdout:3/768: creat d10/d16/d23/d3a/ff8 x:0 0 0 2024-05-25T06:11:33.563 INFO:tasks.workunit.client.0.smithi160.stdout:9/205: dread d4/d11/d1e/d24/f39 [0,4194304] 0 2024-05-25T06:11:33.564 INFO:tasks.workunit.client.1.smithi203.stdout:1/533: creat d14/d57/d1d/da8/fb4 x:0 0 0 2024-05-25T06:11:33.565 INFO:tasks.workunit.client.0.smithi160.stdout:7/238: write d6/dc/d21/f2a [2997241,91355] 0 2024-05-25T06:11:33.566 INFO:tasks.workunit.client.0.smithi160.stdout:4/313: rename d4/d6/d28/l39 to d4/d1a/l5e 0 2024-05-25T06:11:33.567 INFO:tasks.workunit.client.1.smithi203.stdout:3/769: fdatasync d10/d16/d50/fc2 0 2024-05-25T06:11:33.568 INFO:tasks.workunit.client.1.smithi203.stdout:4/681: rename d0/d9/d11/d54/lb4 to d0/d8/d1f/d26/d3b/ldc 0 2024-05-25T06:11:33.570 INFO:tasks.workunit.client.0.smithi160.stdout:0/242: mkdir d2b/d31/d35/d36/d52 0 2024-05-25T06:11:33.574 INFO:tasks.workunit.client.1.smithi203.stdout:2/627: dwrite d2/d6/f26 [4194304,4194304] 0 2024-05-25T06:11:33.574 INFO:tasks.workunit.client.0.smithi160.stdout:2/215: creat d12/d3e/f41 x:0 0 0 2024-05-25T06:11:33.575 INFO:tasks.workunit.client.0.smithi160.stdout:4/314: symlink d4/d6/d5b/l5f 0 2024-05-25T06:11:33.575 INFO:tasks.workunit.client.0.smithi160.stdout:5/295: dread f9 [0,4194304] 0 2024-05-25T06:11:33.576 INFO:tasks.workunit.client.0.smithi160.stdout:8/330: dread d5/f35 [0,4194304] 0 2024-05-25T06:11:33.577 INFO:tasks.workunit.client.1.smithi203.stdout:2/628: write d2/d38/d43/d60/f96 [1052200,99848] 0 2024-05-25T06:11:33.578 INFO:tasks.workunit.client.1.smithi203.stdout:2/629: chown d2/d4/f2e 57376032 1 2024-05-25T06:11:33.580 INFO:tasks.workunit.client.1.smithi203.stdout:4/682: symlink d0/d9/d50/d79/d9a/ldd 0 2024-05-25T06:11:33.585 INFO:tasks.workunit.client.0.smithi160.stdout:2/216: mknod d12/d1c/c42 0 2024-05-25T06:11:33.585 INFO:tasks.workunit.client.1.smithi203.stdout:1/534: link d14/f1a d14/d57/d7e/fb5 0 2024-05-25T06:11:33.587 INFO:tasks.workunit.client.0.smithi160.stdout:5/296: fsync db/dc/d13/d1d/d23/f2d 0 2024-05-25T06:11:33.588 INFO:tasks.workunit.client.1.smithi203.stdout:3/770: creat d10/d16/d23/def/ff9 x:0 0 0 2024-05-25T06:11:33.593 INFO:tasks.workunit.client.1.smithi203.stdout:8/761: write d2/d5/f44 [1151307,34978] 0 2024-05-25T06:11:33.596 INFO:tasks.workunit.client.0.smithi160.stdout:7/239: dwrite f1 [0,4194304] 0 2024-05-25T06:11:33.598 INFO:tasks.workunit.client.0.smithi160.stdout:0/243: dwrite f24 [0,4194304] 0 2024-05-25T06:11:33.598 INFO:tasks.workunit.client.1.smithi203.stdout:8/762: dread d2/d5/d18/d31/dac/dc2/f98 [0,4194304] 0 2024-05-25T06:11:33.601 INFO:tasks.workunit.client.1.smithi203.stdout:2/630: fsync d2/d38/f1d 0 2024-05-25T06:11:33.602 INFO:tasks.workunit.client.0.smithi160.stdout:3/180: dwrite f18 [0,4194304] 0 2024-05-25T06:11:33.603 INFO:tasks.workunit.client.0.smithi160.stdout:1/149: dwrite de/f15 [0,4194304] 0 2024-05-25T06:11:33.603 INFO:tasks.workunit.client.1.smithi203.stdout:3/771: mkdir d10/d16/d71/dfa 0 2024-05-25T06:11:33.604 INFO:tasks.workunit.client.1.smithi203.stdout:1/535: readlink d14/d57/d1d/d3e/d36/db3/l49 0 2024-05-25T06:11:33.604 INFO:tasks.workunit.client.0.smithi160.stdout:5/297: creat db/f57 x:0 0 0 2024-05-25T06:11:33.604 INFO:tasks.workunit.client.0.smithi160.stdout:8/331: dwrite d5/f35 [0,4194304] 0 2024-05-25T06:11:33.605 INFO:tasks.workunit.client.1.smithi203.stdout:3/772: chown d10/d16/d50/d8b/l5a 745560 1 2024-05-25T06:11:33.605 INFO:tasks.workunit.client.1.smithi203.stdout:1/536: write d14/d57/d29/f7b [88814,113245] 0 2024-05-25T06:11:33.607 INFO:tasks.workunit.client.1.smithi203.stdout:1/537: write d14/d57/d1d/f67 [1065730,49058] 0 2024-05-25T06:11:33.608 INFO:tasks.workunit.client.0.smithi160.stdout:5/298: read - db/dc/d13/f50 zero size 2024-05-25T06:11:33.614 INFO:tasks.workunit.client.0.smithi160.stdout:3/181: write d19/d22/f2a [256382,117469] 0 2024-05-25T06:11:33.616 INFO:tasks.workunit.client.1.smithi203.stdout:4/683: symlink d0/d8/lde 0 2024-05-25T06:11:33.619 INFO:tasks.workunit.client.1.smithi203.stdout:8/763: dwrite d2/d5/d28/d38/f3c [0,4194304] 0 2024-05-25T06:11:33.622 INFO:tasks.workunit.client.1.smithi203.stdout:2/631: creat d2/d38/d2f/d69/fd7 x:0 0 0 2024-05-25T06:11:33.629 INFO:tasks.workunit.client.1.smithi203.stdout:8/764: truncate d2/d5/d28/d38/d4e/d97/f66 13329014 0 2024-05-25T06:11:33.630 INFO:tasks.workunit.client.1.smithi203.stdout:4/684: rename d0/d8/d3a/d81/d93/d98/d99/cca to d0/d8/d52/cdf 0 2024-05-25T06:11:33.633 INFO:tasks.workunit.client.0.smithi160.stdout:9/206: dread d4/d11/d1e/d24/f1b [0,4194304] 0 2024-05-25T06:11:33.635 INFO:tasks.workunit.client.0.smithi160.stdout:2/217: dread d12/f1d [0,4194304] 0 2024-05-25T06:11:33.639 INFO:tasks.workunit.client.0.smithi160.stdout:5/299: rename db/dc/d13/d1d/d23/f55 to db/dc/d13/d1d/d3f/d4a/f58 0 2024-05-25T06:11:33.639 INFO:tasks.workunit.client.1.smithi203.stdout:2/632: dwrite d2/f6c [0,4194304] 0 2024-05-25T06:11:33.640 INFO:tasks.workunit.client.1.smithi203.stdout:4/685: mkdir d0/d9/d50/d79/d9a/dbe/de0 0 2024-05-25T06:11:33.641 INFO:tasks.workunit.client.0.smithi160.stdout:5/300: fdatasync db/dc/d13/d1d/d23/d24/f3e 0 2024-05-25T06:11:33.642 INFO:tasks.workunit.client.0.smithi160.stdout:1/150: dread de/d1b/f27 [0,4194304] 0 2024-05-25T06:11:33.643 INFO:tasks.workunit.client.1.smithi203.stdout:8/765: dwrite d2/d5/d10/f52 [0,4194304] 0 2024-05-25T06:11:33.644 INFO:tasks.workunit.client.0.smithi160.stdout:2/218: creat d12/d1c/d37/f43 x:0 0 0 2024-05-25T06:11:33.646 INFO:tasks.workunit.client.1.smithi203.stdout:4/686: write d0/d9/d50/d79/f40 [674245,65192] 0 2024-05-25T06:11:33.648 INFO:tasks.workunit.client.0.smithi160.stdout:9/207: rename d4/d11/d12/d2e/c3b to d4/d11/d12/c3f 0 2024-05-25T06:11:33.650 INFO:tasks.workunit.client.0.smithi160.stdout:9/208: chown d4/d11/d1e/d24/fc 21435 1 2024-05-25T06:11:33.651 INFO:tasks.workunit.client.0.smithi160.stdout:1/151: mknod de/d1b/c2e 0 2024-05-25T06:11:33.652 INFO:tasks.workunit.client.1.smithi203.stdout:2/633: dwrite d2/d38/d2f/f4f [0,4194304] 0 2024-05-25T06:11:33.652 INFO:tasks.workunit.client.1.smithi203.stdout:8/766: read d2/d5/f8 [3445811,69402] 0 2024-05-25T06:11:33.654 INFO:tasks.workunit.client.0.smithi160.stdout:2/219: symlink d12/d13/l44 0 2024-05-25T06:11:33.656 INFO:tasks.workunit.client.0.smithi160.stdout:9/209: rmdir d4/d11/d1e/d3e/d25 39 2024-05-25T06:11:33.657 INFO:tasks.workunit.client.1.smithi203.stdout:4/687: creat d0/d8/d10/d91/da2/fe1 x:0 0 0 2024-05-25T06:11:33.658 INFO:tasks.workunit.client.1.smithi203.stdout:2/634: symlink d2/d6/ld8 0 2024-05-25T06:11:33.660 INFO:tasks.workunit.client.0.smithi160.stdout:1/152: dwrite f7 [0,4194304] 0 2024-05-25T06:11:33.661 INFO:tasks.workunit.client.1.smithi203.stdout:2/635: chown d2/d4/l7 14600623 1 2024-05-25T06:11:33.662 INFO:tasks.workunit.client.0.smithi160.stdout:9/210: symlink d4/d11/d1e/d3e/d2f/l40 0 2024-05-25T06:11:33.663 INFO:tasks.workunit.client.0.smithi160.stdout:9/211: dread - d4/d11/d14/f37 zero size 2024-05-25T06:11:33.667 INFO:tasks.workunit.client.0.smithi160.stdout:2/220: link fe d12/f45 0 2024-05-25T06:11:33.667 INFO:tasks.workunit.client.0.smithi160.stdout:2/221: readlink d12/l36 0 2024-05-25T06:11:33.668 INFO:tasks.workunit.client.1.smithi203.stdout:2/636: mkdir d2/d1a/d87/dd9 0 2024-05-25T06:11:33.668 INFO:tasks.workunit.client.0.smithi160.stdout:1/153: mknod de/d1b/d22/c2f 0 2024-05-25T06:11:33.669 INFO:tasks.workunit.client.1.smithi203.stdout:1/538: dwrite d14/f25 [0,4194304] 0 2024-05-25T06:11:33.670 INFO:tasks.workunit.client.0.smithi160.stdout:9/212: write d4/d11/d1e/d24/f1b [2399820,4301] 0 2024-05-25T06:11:33.670 INFO:tasks.workunit.client.1.smithi203.stdout:4/688: dwrite d0/d9/d9d/fc6 [0,4194304] 0 2024-05-25T06:11:33.672 INFO:tasks.workunit.client.0.smithi160.stdout:2/222: symlink d12/d1c/d2a/l46 0 2024-05-25T06:11:33.676 INFO:tasks.workunit.client.0.smithi160.stdout:1/154: rename de/c1a to de/c30 0 2024-05-25T06:11:33.676 INFO:tasks.workunit.client.0.smithi160.stdout:1/155: fsync fa 0 2024-05-25T06:11:33.677 INFO:tasks.workunit.client.0.smithi160.stdout:9/213: write d4/d11/d1e/d24/f39 [2163788,99571] 0 2024-05-25T06:11:33.678 INFO:tasks.workunit.client.1.smithi203.stdout:2/637: link d2/d38/d2f/d69/fd7 d2/d4/d10/fda 0 2024-05-25T06:11:33.680 INFO:tasks.workunit.client.0.smithi160.stdout:7/240: truncate d6/d2c/f38 36914 0 2024-05-25T06:11:33.681 INFO:tasks.workunit.client.1.smithi203.stdout:8/767: dwrite d2/d5/d10/d45/d5d/d68/f87 [0,4194304] 0 2024-05-25T06:11:33.682 INFO:tasks.workunit.client.0.smithi160.stdout:2/223: mkdir d12/d47 0 2024-05-25T06:11:33.684 INFO:tasks.workunit.client.0.smithi160.stdout:5/301: rename db/dc/d13/d1d/d3f/d4a/f58 to db/dc/d13/d1d/d23/f59 0 2024-05-25T06:11:33.684 INFO:tasks.workunit.client.1.smithi203.stdout:1/539: write d14/d57/d1d/faf [2473253,124106] 0 2024-05-25T06:11:33.684 INFO:tasks.workunit.client.1.smithi203.stdout:2/638: fsync d2/d38/d2f/f31 0 2024-05-25T06:11:33.686 INFO:tasks.workunit.client.1.smithi203.stdout:2/639: chown d2/d1a/d89/fd2 103785 1 2024-05-25T06:11:33.687 INFO:tasks.workunit.client.0.smithi160.stdout:3/182: write f13 [1829011,121074] 0 2024-05-25T06:11:33.687 INFO:tasks.workunit.client.1.smithi203.stdout:2/640: chown d2/d4/d10/d5f/f7b 190402 1 2024-05-25T06:11:33.688 INFO:tasks.workunit.client.1.smithi203.stdout:8/768: rename d2/d5/d18/d31/dac/dc2/f9a to d2/d5/d10/d45/d90/f107 0 2024-05-25T06:11:33.688 INFO:tasks.workunit.client.1.smithi203.stdout:1/540: dread d14/d57/d7e/fb5 [0,4194304] 0 2024-05-25T06:11:33.688 INFO:tasks.workunit.client.0.smithi160.stdout:0/244: truncate d2b/d31/f38 489673 0 2024-05-25T06:11:33.689 INFO:tasks.workunit.client.0.smithi160.stdout:0/245: fdatasync f25 0 2024-05-25T06:11:33.689 INFO:tasks.workunit.client.1.smithi203.stdout:2/641: write d2/d38/d2f/f4f [2968345,133] 0 2024-05-25T06:11:33.689 INFO:tasks.workunit.client.1.smithi203.stdout:4/689: dread d0/d8/d52/d59/d7a/d83/fd6 [0,4194304] 0 2024-05-25T06:11:33.690 INFO:tasks.workunit.client.0.smithi160.stdout:7/241: write d6/dc/f25 [3366377,123261] 0 2024-05-25T06:11:33.692 INFO:tasks.workunit.client.1.smithi203.stdout:2/642: write d2/d4/d10/f21 [462956,5468] 0 2024-05-25T06:11:33.692 INFO:tasks.workunit.client.0.smithi160.stdout:7/242: chown d6/c26 55 1 2024-05-25T06:11:33.693 INFO:tasks.workunit.client.1.smithi203.stdout:8/769: symlink d2/d5/d10/d11/l108 0 2024-05-25T06:11:33.695 INFO:tasks.workunit.client.0.smithi160.stdout:0/246: unlink l22 0 2024-05-25T06:11:33.695 INFO:tasks.workunit.client.1.smithi203.stdout:2/643: getdents d2/d1a/d87/dd9 0 2024-05-25T06:11:33.696 INFO:tasks.workunit.client.0.smithi160.stdout:6/197: sync 2024-05-25T06:11:33.697 INFO:tasks.workunit.client.1.smithi203.stdout:1/541: getdents d14/d57/d1d/d3e/d36 0 2024-05-25T06:11:33.698 INFO:tasks.workunit.client.1.smithi203.stdout:1/542: chown d14/f54 0 1 2024-05-25T06:11:33.699 INFO:tasks.workunit.client.0.smithi160.stdout:7/243: mknod d6/d2c/c44 0 2024-05-25T06:11:33.699 INFO:tasks.workunit.client.1.smithi203.stdout:1/543: truncate d14/d57/d97/fa3 4569454 0 2024-05-25T06:11:33.700 INFO:tasks.workunit.client.0.smithi160.stdout:7/244: fsync d6/dc/f19 0 2024-05-25T06:11:33.701 INFO:tasks.workunit.client.1.smithi203.stdout:1/544: write d14/d57/d6b/f8e [334756,78005] 0 2024-05-25T06:11:33.701 INFO:tasks.workunit.client.0.smithi160.stdout:7/245: readlink d6/dc/d21/l30 0 2024-05-25T06:11:33.704 INFO:tasks.workunit.client.1.smithi203.stdout:8/770: getdents d2/d5/d18/d31/dac/dc2/d8a/d8f/dfc 0 2024-05-25T06:11:33.705 INFO:tasks.workunit.client.0.smithi160.stdout:2/224: link d12/c1a d12/d47/c48 0 2024-05-25T06:11:33.706 INFO:tasks.workunit.client.1.smithi203.stdout:1/545: fsync d14/d23/f42 0 2024-05-25T06:11:33.707 INFO:tasks.workunit.client.0.smithi160.stdout:7/246: creat d6/d2c/f45 x:0 0 0 2024-05-25T06:11:33.708 INFO:tasks.workunit.client.0.smithi160.stdout:7/247: fdatasync d6/d32/f3f 0 2024-05-25T06:11:33.709 INFO:tasks.workunit.client.0.smithi160.stdout:2/225: mkdir d12/d1c/d37/d49 0 2024-05-25T06:11:33.709 INFO:tasks.workunit.client.1.smithi203.stdout:2/644: write d2/d38/d43/d60/d6b/dae/f82 [1216857,109574] 0 2024-05-25T06:11:33.709 INFO:tasks.workunit.client.1.smithi203.stdout:7/594: sync 2024-05-25T06:11:33.709 INFO:tasks.workunit.client.1.smithi203.stdout:0/518: sync 2024-05-25T06:11:33.709 INFO:tasks.workunit.client.1.smithi203.stdout:5/866: sync 2024-05-25T06:11:33.710 INFO:tasks.workunit.client.0.smithi160.stdout:9/214: dread d4/fa [0,4194304] 0 2024-05-25T06:11:33.712 INFO:tasks.workunit.client.1.smithi203.stdout:1/546: symlink d14/d57/d1d/da8/lb6 0 2024-05-25T06:11:33.714 INFO:tasks.workunit.client.1.smithi203.stdout:5/867: chown dc/d11/d42/d94/f123 1987441 1 2024-05-25T06:11:33.714 INFO:tasks.workunit.client.0.smithi160.stdout:7/248: mkdir d6/d32/d46 0 2024-05-25T06:11:33.715 INFO:tasks.workunit.client.0.smithi160.stdout:7/249: chown d6/d32/f3f 1142 1 2024-05-25T06:11:33.715 INFO:tasks.workunit.client.0.smithi160.stdout:5/302: dread db/dc/d13/d1d/f20 [0,4194304] 0 2024-05-25T06:11:33.716 INFO:tasks.workunit.client.1.smithi203.stdout:0/519: stat dd/d10/f26 0 2024-05-25T06:11:33.717 INFO:tasks.workunit.client.0.smithi160.stdout:2/226: mknod d12/d1c/d37/c4a 0 2024-05-25T06:11:33.717 INFO:tasks.workunit.client.1.smithi203.stdout:7/595: symlink d1/d9/d2c/d36/d3b/lcf 0 2024-05-25T06:11:33.717 INFO:tasks.workunit.client.1.smithi203.stdout:2/645: chown d2/d4/d10/d58/f6d 31814832 1 2024-05-25T06:11:33.719 INFO:tasks.workunit.client.1.smithi203.stdout:5/868: readlink dc/d11/d42/d94/dad/lc3 0 2024-05-25T06:11:33.721 INFO:tasks.workunit.client.1.smithi203.stdout:7/596: mkdir d1/d9/d2c/d36/d54/dd0 0 2024-05-25T06:11:33.724 INFO:tasks.workunit.client.0.smithi160.stdout:5/303: unlink db/dc/d13/d1d/d31/d3b/f42 0 2024-05-25T06:11:33.726 INFO:tasks.workunit.client.0.smithi160.stdout:6/198: dread d5/fa [0,4194304] 0 2024-05-25T06:11:33.731 INFO:tasks.workunit.client.0.smithi160.stdout:7/250: rename d6/dc/l13 to d6/d2b/d2d/l47 0 2024-05-25T06:11:33.731 INFO:tasks.workunit.client.1.smithi203.stdout:7/597: mkdir d1/d9/d2c/d3a/dd1 0 2024-05-25T06:11:33.732 INFO:tasks.workunit.client.0.smithi160.stdout:7/251: chown c4 736 1 2024-05-25T06:11:33.732 INFO:tasks.workunit.client.0.smithi160.stdout:3/183: truncate f18 3142784 0 2024-05-25T06:11:33.732 INFO:tasks.workunit.client.1.smithi203.stdout:7/598: chown d1/d9/f18 17601 1 2024-05-25T06:11:33.733 INFO:tasks.workunit.client.1.smithi203.stdout:2/646: dwrite d2/d42/d4e/d68/fcd [0,4194304] 0 2024-05-25T06:11:33.734 INFO:tasks.workunit.client.0.smithi160.stdout:6/199: dwrite d5/de/d32/d1b/f2d [0,4194304] 0 2024-05-25T06:11:33.735 INFO:tasks.workunit.client.0.smithi160.stdout:5/304: write f9 [191199,64605] 0 2024-05-25T06:11:33.736 INFO:tasks.workunit.client.1.smithi203.stdout:8/771: dread d2/d5/d18/d31/dac/dc2/d88/ff8 [0,4194304] 0 2024-05-25T06:11:33.736 INFO:tasks.workunit.client.0.smithi160.stdout:0/247: write d2b/f30 [4695407,71131] 0 2024-05-25T06:11:33.738 INFO:tasks.workunit.client.0.smithi160.stdout:9/215: creat d4/f41 x:0 0 0 2024-05-25T06:11:33.738 INFO:tasks.workunit.client.1.smithi203.stdout:1/547: dread d14/d57/d1d/f66 [0,4194304] 0 2024-05-25T06:11:33.739 INFO:tasks.workunit.client.1.smithi203.stdout:2/647: rename d2/cd to d2/d4/da6/cdb 0 2024-05-25T06:11:33.743 INFO:tasks.workunit.client.1.smithi203.stdout:8/772: unlink d2/d5/d10/d11/de/fa2 0 2024-05-25T06:11:33.744 INFO:tasks.workunit.client.0.smithi160.stdout:7/252: symlink d6/dc/d21/l48 0 2024-05-25T06:11:33.744 INFO:tasks.workunit.client.0.smithi160.stdout:3/184: fdatasync d19/d22/f39 0 2024-05-25T06:11:33.745 INFO:tasks.workunit.client.0.smithi160.stdout:5/305: mknod db/dc/d13/d1d/d3f/c5a 0 2024-05-25T06:11:33.746 INFO:tasks.workunit.client.0.smithi160.stdout:6/200: rmdir d5/de/d32/d1b/d2b 39 2024-05-25T06:11:33.748 INFO:tasks.workunit.client.1.smithi203.stdout:1/548: mkdir d14/d57/db7 0 2024-05-25T06:11:33.748 INFO:tasks.workunit.client.1.smithi203.stdout:7/599: dwrite d1/d9/d2c/f5d [0,4194304] 0 2024-05-25T06:11:33.749 INFO:tasks.workunit.client.0.smithi160.stdout:6/201: dread d5/de/ff [0,4194304] 0 2024-05-25T06:11:33.749 INFO:tasks.workunit.client.0.smithi160.stdout:7/253: unlink d6/c2e 0 2024-05-25T06:11:33.751 INFO:tasks.workunit.client.1.smithi203.stdout:8/773: creat d2/d5/d28/d38/d4e/d97/ddf/df0/f109 x:0 0 0 2024-05-25T06:11:33.752 INFO:tasks.workunit.client.1.smithi203.stdout:7/600: dread - d1/f78 zero size 2024-05-25T06:11:33.753 INFO:tasks.workunit.client.1.smithi203.stdout:7/601: stat d1/d9/l92 0 2024-05-25T06:11:33.753 INFO:tasks.workunit.client.1.smithi203.stdout:8/774: readlink d2/d5/d18/d31/dac/dc2/d88/ld5 0 2024-05-25T06:11:33.754 INFO:tasks.workunit.client.0.smithi160.stdout:9/216: mknod d4/d11/d1e/d3e/d25/c42 0 2024-05-25T06:11:33.754 INFO:tasks.workunit.client.1.smithi203.stdout:2/648: rename d2/d38/d43/d60/da3/cad to d2/d1a/cdc 0 2024-05-25T06:11:33.754 INFO:tasks.workunit.client.1.smithi203.stdout:2/649: stat d2/d4/l2b 0 2024-05-25T06:11:33.757 INFO:tasks.workunit.client.0.smithi160.stdout:3/185: unlink d19/d2f/l41 0 2024-05-25T06:11:33.758 INFO:tasks.workunit.client.0.smithi160.stdout:6/202: symlink d5/de/d32/d35/l47 0 2024-05-25T06:11:33.758 INFO:tasks.workunit.client.0.smithi160.stdout:6/203: dread - d5/f15 zero size 2024-05-25T06:11:33.758 INFO:tasks.workunit.client.1.smithi203.stdout:8/775: mknod d2/d5/d18/d31/c10a 0 2024-05-25T06:11:33.759 INFO:tasks.workunit.client.0.smithi160.stdout:0/248: rmdir d2b/d4e 0 2024-05-25T06:11:33.759 INFO:tasks.workunit.client.0.smithi160.stdout:0/249: stat d2b/d46 0 2024-05-25T06:11:33.761 INFO:tasks.workunit.client.1.smithi203.stdout:1/549: rename d14/d57/d1d/d9f to d14/d57/d29/d94/db8 0 2024-05-25T06:11:33.761 INFO:tasks.workunit.client.0.smithi160.stdout:9/217: rename d4/c29 to d4/d11/d1e/d3e/d2f/c43 0 2024-05-25T06:11:33.763 INFO:tasks.workunit.client.0.smithi160.stdout:7/254: creat d6/d3e/f49 x:0 0 0 2024-05-25T06:11:33.763 INFO:tasks.workunit.client.0.smithi160.stdout:6/204: mknod d5/d30/d36/c48 0 2024-05-25T06:11:33.763 INFO:tasks.workunit.client.0.smithi160.stdout:7/255: stat d6/c33 0 2024-05-25T06:11:33.764 INFO:tasks.workunit.client.0.smithi160.stdout:3/186: mknod d19/c42 0 2024-05-25T06:11:33.765 INFO:tasks.workunit.client.1.smithi203.stdout:2/650: rename d2/d38/d43/d60/d6b/dae/f88 to d2/d1a/d89/daa/fdd 0 2024-05-25T06:11:33.765 INFO:tasks.workunit.client.0.smithi160.stdout:6/205: chown d5/d39 1 1 2024-05-25T06:11:33.767 INFO:tasks.workunit.client.0.smithi160.stdout:7/256: read d6/d32/f3f [1701,99047] 0 2024-05-25T06:11:33.768 INFO:tasks.workunit.client.1.smithi203.stdout:1/550: mkdir d14/d57/d6b/db9 0 2024-05-25T06:11:33.769 INFO:tasks.workunit.client.1.smithi203.stdout:0/520: write dd/d3d/d4f/fa2 [180066,118059] 0 2024-05-25T06:11:33.770 INFO:tasks.workunit.client.1.smithi203.stdout:2/651: symlink d2/d1a/d89/daa/lde 0 2024-05-25T06:11:33.771 INFO:tasks.workunit.client.0.smithi160.stdout:0/250: dwrite d2b/f42 [0,4194304] 0 2024-05-25T06:11:33.773 INFO:tasks.workunit.client.1.smithi203.stdout:2/652: write d2/d4/d10/d32/f52 [5117924,37803] 0 2024-05-25T06:11:33.774 INFO:tasks.workunit.client.1.smithi203.stdout:2/653: write d2/d6/f26 [5629756,46226] 0 2024-05-25T06:11:33.775 INFO:tasks.workunit.client.0.smithi160.stdout:7/257: creat d6/dc/f4a x:0 0 0 2024-05-25T06:11:33.775 INFO:tasks.workunit.client.1.smithi203.stdout:2/654: dread - d2/d1a/d87/fbe zero size 2024-05-25T06:11:33.779 INFO:tasks.workunit.client.1.smithi203.stdout:2/655: truncate d2/d38/d2f/d69/fd7 944626 0 2024-05-25T06:11:33.779 INFO:tasks.workunit.client.0.smithi160.stdout:7/258: stat d6/d2c/c44 0 2024-05-25T06:11:33.780 INFO:tasks.workunit.client.0.smithi160.stdout:2/227: truncate d12/d1c/f1e 303864 0 2024-05-25T06:11:33.781 INFO:tasks.workunit.client.0.smithi160.stdout:9/218: dwrite f2 [0,4194304] 0 2024-05-25T06:11:33.781 INFO:tasks.workunit.client.1.smithi203.stdout:7/602: write d1/d9/d2c/d3a/d4e/f5b [1028651,78693] 0 2024-05-25T06:11:33.784 INFO:tasks.workunit.client.1.smithi203.stdout:2/656: fdatasync d2/fe 0 2024-05-25T06:11:33.786 INFO:tasks.workunit.client.0.smithi160.stdout:3/187: dwrite d19/d2f/d38/f3e [0,4194304] 0 2024-05-25T06:11:33.789 INFO:tasks.workunit.client.0.smithi160.stdout:7/259: creat d6/d3e/f4b x:0 0 0 2024-05-25T06:11:33.789 INFO:tasks.workunit.client.0.smithi160.stdout:5/306: truncate db/dc/f15 3300680 0 2024-05-25T06:11:33.791 INFO:tasks.workunit.client.1.smithi203.stdout:0/521: write dd/d10/f4a [216902,120407] 0 2024-05-25T06:11:33.792 INFO:tasks.workunit.client.0.smithi160.stdout:7/260: chown d6/dc 13496528 1 2024-05-25T06:11:33.792 INFO:tasks.workunit.client.0.smithi160.stdout:6/206: dwrite d5/f26 [4194304,4194304] 0 2024-05-25T06:11:33.792 INFO:tasks.workunit.client.0.smithi160.stdout:2/228: mkdir d12/d1c/d37/d4b 0 2024-05-25T06:11:33.794 INFO:tasks.workunit.client.0.smithi160.stdout:7/261: chown d6/dc/d21 14970119 1 2024-05-25T06:11:33.794 INFO:tasks.workunit.client.1.smithi203.stdout:2/657: mkdir d2/d1a/d33/d80/dd1/ddf 0 2024-05-25T06:11:33.795 INFO:tasks.workunit.client.1.smithi203.stdout:7/603: creat d1/d9/d2c/d62/d9b/fd2 x:0 0 0 2024-05-25T06:11:33.796 INFO:tasks.workunit.client.0.smithi160.stdout:9/219: dread d4/fa [0,4194304] 0 2024-05-25T06:11:33.796 INFO:tasks.workunit.client.0.smithi160.stdout:3/188: mkdir d19/d2f/d38/d43 0 2024-05-25T06:11:33.798 INFO:tasks.workunit.client.0.smithi160.stdout:2/229: write d12/d1c/d37/f3b [378629,42132] 0 2024-05-25T06:11:33.801 INFO:tasks.workunit.client.0.smithi160.stdout:6/207: readlink d5/d30/d36/l33 0 2024-05-25T06:11:33.801 INFO:tasks.workunit.client.1.smithi203.stdout:8/776: dread d2/d5/fc [0,4194304] 0 2024-05-25T06:11:33.802 INFO:tasks.workunit.client.0.smithi160.stdout:5/307: creat db/dc/d13/d1d/d23/d24/d2e/f5b x:0 0 0 2024-05-25T06:11:33.802 INFO:tasks.workunit.client.0.smithi160.stdout:3/189: truncate d19/d22/f2d 952139 0 2024-05-25T06:11:33.804 INFO:tasks.workunit.client.1.smithi203.stdout:0/522: rename dd/d32/l60 to dd/d3d/da0/lb6 0 2024-05-25T06:11:33.805 INFO:tasks.workunit.client.0.smithi160.stdout:6/208: dread - d5/f3f zero size 2024-05-25T06:11:33.807 INFO:tasks.workunit.client.0.smithi160.stdout:7/262: symlink d6/d3e/l4c 0 2024-05-25T06:11:33.808 INFO:tasks.workunit.client.0.smithi160.stdout:3/190: truncate d19/d22/f2d 1630570 0 2024-05-25T06:11:33.809 INFO:tasks.workunit.client.1.smithi203.stdout:8/777: symlink d2/d5/d10/d45/d5d/d68/l10b 0 2024-05-25T06:11:33.809 INFO:tasks.workunit.client.0.smithi160.stdout:5/308: symlink db/dc/d13/d1d/d3f/d4a/l5c 0 2024-05-25T06:11:33.811 INFO:tasks.workunit.client.0.smithi160.stdout:6/209: creat d5/d30/d36/f49 x:0 0 0 2024-05-25T06:11:33.813 INFO:tasks.workunit.client.0.smithi160.stdout:9/220: getdents d4 0 2024-05-25T06:11:33.813 INFO:tasks.workunit.client.0.smithi160.stdout:0/251: dread f2 [0,4194304] 0 2024-05-25T06:11:33.813 INFO:tasks.workunit.client.0.smithi160.stdout:5/309: creat db/f5d x:0 0 0 2024-05-25T06:11:33.813 INFO:tasks.workunit.client.0.smithi160.stdout:7/263: dread d6/dc/f24 [0,4194304] 0 2024-05-25T06:11:33.815 INFO:tasks.workunit.client.0.smithi160.stdout:7/264: chown d6/d32/c3d 80522221 1 2024-05-25T06:11:33.815 INFO:tasks.workunit.client.0.smithi160.stdout:9/221: write d4/fd [3308401,103064] 0 2024-05-25T06:11:33.816 INFO:tasks.workunit.client.0.smithi160.stdout:5/310: mkdir db/d47/d5e 0 2024-05-25T06:11:33.816 INFO:tasks.workunit.client.0.smithi160.stdout:9/222: readlink d4/d11/d1e/d3e/d2f/l40 0 2024-05-25T06:11:33.816 INFO:tasks.workunit.client.0.smithi160.stdout:3/191: creat d19/f44 x:0 0 0 2024-05-25T06:11:33.817 INFO:tasks.workunit.client.0.smithi160.stdout:0/252: symlink d2b/d31/d35/d36/l53 0 2024-05-25T06:11:33.822 INFO:tasks.workunit.client.0.smithi160.stdout:7/265: dread f1 [0,4194304] 0 2024-05-25T06:11:33.823 INFO:tasks.workunit.client.0.smithi160.stdout:9/223: getdents d4/d11/d1e/d3e/d25/d27 0 2024-05-25T06:11:33.825 INFO:tasks.workunit.client.0.smithi160.stdout:9/224: write d4/d11/d14/f31 [548922,128479] 0 2024-05-25T06:11:33.825 INFO:tasks.workunit.client.0.smithi160.stdout:7/266: read d6/dc/f15 [160507,66422] 0 2024-05-25T06:11:33.826 INFO:tasks.workunit.client.1.smithi203.stdout:7/604: write d1/d9/f76 [760450,30414] 0 2024-05-25T06:11:33.828 INFO:tasks.workunit.client.0.smithi160.stdout:5/311: dwrite db/dc/d13/d1d/d23/d24/f3e [0,4194304] 0 2024-05-25T06:11:33.830 INFO:tasks.workunit.client.0.smithi160.stdout:9/225: chown d4/d11/d12/d2e/f32 323309 1 2024-05-25T06:11:33.831 INFO:tasks.workunit.client.1.smithi203.stdout:2/658: dwrite d2/d38/d2f/d69/fd7 [0,4194304] 0 2024-05-25T06:11:33.831 INFO:tasks.workunit.client.1.smithi203.stdout:7/605: dread - d1/d9/d2c/fa1 zero size 2024-05-25T06:11:33.832 INFO:tasks.workunit.client.0.smithi160.stdout:9/226: readlink d4/d11/d1e/d24/l15 0 2024-05-25T06:11:33.832 INFO:tasks.workunit.client.0.smithi160.stdout:9/227: chown d4/d30 275925 1 2024-05-25T06:11:33.833 INFO:tasks.workunit.client.0.smithi160.stdout:7/267: symlink d6/dc/d21/l4d 0 2024-05-25T06:11:33.834 INFO:tasks.workunit.client.1.smithi203.stdout:7/606: write d1/d9/d2c/d3a/d4e/d50/fbe [787666,53844] 0 2024-05-25T06:11:33.834 INFO:tasks.workunit.client.0.smithi160.stdout:4/315: sync 2024-05-25T06:11:33.837 INFO:tasks.workunit.client.0.smithi160.stdout:4/316: chown d4/c29 9 1 2024-05-25T06:11:33.842 INFO:tasks.workunit.client.0.smithi160.stdout:4/317: stat d4/d6/l3a 0 2024-05-25T06:11:33.842 INFO:tasks.workunit.client.0.smithi160.stdout:9/228: dread f2 [0,4194304] 0 2024-05-25T06:11:33.845 INFO:tasks.workunit.client.0.smithi160.stdout:2/230: dwrite fb [0,4194304] 0 2024-05-25T06:11:33.847 INFO:tasks.workunit.client.0.smithi160.stdout:2/231: write d12/d3e/f41 [846903,89033] 0 2024-05-25T06:11:33.904 INFO:tasks.workunit.client.0.smithi160.stdout:8/332: sync 2024-05-25T06:11:33.905 INFO:tasks.workunit.client.0.smithi160.stdout:1/156: sync 2024-05-25T06:11:33.905 INFO:tasks.workunit.client.0.smithi160.stdout:8/333: write d5/d44/d1f/d1e/d28/d4a/f3d [2707765,1791] 0 2024-05-25T06:11:33.906 INFO:tasks.workunit.client.0.smithi160.stdout:8/334: write d5/f48 [952950,125252] 0 2024-05-25T06:11:33.958 INFO:tasks.workunit.client.1.smithi203.stdout:2/659: rename d2/d38/f1d to d2/d1a/d33/d80/dd1/ddf/fe0 0 2024-05-25T06:11:33.958 INFO:tasks.workunit.client.0.smithi160.stdout:7/268: mknod d6/d2b/d2d/c4e 0 2024-05-25T06:11:33.959 INFO:tasks.workunit.client.1.smithi203.stdout:0/523: dread dd/f23 [0,4194304] 0 2024-05-25T06:11:33.959 INFO:tasks.workunit.client.1.smithi203.stdout:7/607: mkdir d1/d9/d2c/d36/dd3 0 2024-05-25T06:11:33.960 INFO:tasks.workunit.client.0.smithi160.stdout:4/318: fsync d4/f26 0 2024-05-25T06:11:33.961 INFO:tasks.workunit.client.1.smithi203.stdout:7/608: fdatasync d1/d9/f2e 0 2024-05-25T06:11:33.963 INFO:tasks.workunit.client.1.smithi203.stdout:0/524: mkdir dd/d3d/d4f/db7 0 2024-05-25T06:11:33.963 INFO:tasks.workunit.client.0.smithi160.stdout:1/157: rmdir de/d1b/d28 39 2024-05-25T06:11:33.965 INFO:tasks.workunit.client.0.smithi160.stdout:6/210: dwrite d5/fa [0,4194304] 0 2024-05-25T06:11:33.965 INFO:tasks.workunit.client.0.smithi160.stdout:3/192: getdents d19 0 2024-05-25T06:11:33.966 INFO:tasks.workunit.client.0.smithi160.stdout:6/211: write d5/de/d32/d1b/f2d [4116107,11179] 0 2024-05-25T06:11:33.967 INFO:tasks.workunit.client.1.smithi203.stdout:7/609: link d1/d11/d3d/d4c/f63 d1/d9/d2c/d79/dc7/fd4 0 2024-05-25T06:11:33.968 INFO:tasks.workunit.client.0.smithi160.stdout:8/335: rename d5/d7/c1d to d5/d44/d1f/d1e/d28/d4a/c6c 0 2024-05-25T06:11:33.969 INFO:tasks.workunit.client.1.smithi203.stdout:2/660: getdents d2/d38 0 2024-05-25T06:11:33.970 INFO:tasks.workunit.client.1.smithi203.stdout:0/525: mkdir dd/d3d/d59/db8 0 2024-05-25T06:11:33.972 INFO:tasks.workunit.client.0.smithi160.stdout:2/232: link d12/d1c/f31 d12/d13/f4c 0 2024-05-25T06:11:33.972 INFO:tasks.workunit.client.0.smithi160.stdout:0/253: dwrite f10 [0,4194304] 0 2024-05-25T06:11:33.974 INFO:tasks.workunit.client.1.smithi203.stdout:2/661: symlink d2/d1a/db0/le1 0 2024-05-25T06:11:33.974 INFO:tasks.workunit.client.0.smithi160.stdout:3/193: creat d19/f45 x:0 0 0 2024-05-25T06:11:33.975 INFO:tasks.workunit.client.1.smithi203.stdout:2/662: chown d2/d1a/l5a 162646 1 2024-05-25T06:11:33.976 INFO:tasks.workunit.client.1.smithi203.stdout:0/526: mkdir dd/d3d/da0/db9 0 2024-05-25T06:11:33.976 INFO:tasks.workunit.client.1.smithi203.stdout:7/610: getdents d1/d9/d60/dac 0 2024-05-25T06:11:33.976 INFO:tasks.workunit.client.0.smithi160.stdout:5/312: getdents db/dc/d13/d1d/d23/d45 0 2024-05-25T06:11:33.978 INFO:tasks.workunit.client.1.smithi203.stdout:2/663: creat d2/d38/fe2 x:0 0 0 2024-05-25T06:11:33.978 INFO:tasks.workunit.client.0.smithi160.stdout:6/212: dwrite d5/f26 [4194304,4194304] 0 2024-05-25T06:11:33.979 INFO:tasks.workunit.client.0.smithi160.stdout:7/269: link d6/d2c/f37 d6/dc/d21/f4f 0 2024-05-25T06:11:33.979 INFO:tasks.workunit.client.0.smithi160.stdout:5/313: write db/dc/d13/d1d/d23/f2d [246476,31706] 0 2024-05-25T06:11:33.980 INFO:tasks.workunit.client.0.smithi160.stdout:4/319: rename d4/c29 to d4/d1c/d54/c60 0 2024-05-25T06:11:33.980 INFO:tasks.workunit.client.0.smithi160.stdout:5/314: dread - db/f53 zero size 2024-05-25T06:11:33.984 INFO:tasks.workunit.client.1.smithi203.stdout:0/527: dread dd/d10/f4a [0,4194304] 0 2024-05-25T06:11:33.986 INFO:tasks.workunit.client.0.smithi160.stdout:9/229: creat d4/d11/d1e/d3e/f44 x:0 0 0 2024-05-25T06:11:33.986 INFO:tasks.workunit.client.0.smithi160.stdout:7/270: write d6/dc/ff [4093211,8351] 0 2024-05-25T06:11:33.986 INFO:tasks.workunit.client.1.smithi203.stdout:0/528: rmdir dd/d32 39 2024-05-25T06:11:33.987 INFO:tasks.workunit.client.1.smithi203.stdout:7/611: dread d1/d9/d2c/d36/f40 [0,4194304] 0 2024-05-25T06:11:33.988 INFO:tasks.workunit.client.1.smithi203.stdout:0/529: dread - dd/d45/d96/f9f zero size 2024-05-25T06:11:33.989 INFO:tasks.workunit.client.1.smithi203.stdout:2/664: dwrite d2/d42/fc3 [0,4194304] 0 2024-05-25T06:11:33.990 INFO:tasks.workunit.client.0.smithi160.stdout:8/336: mknod d5/c6d 0 2024-05-25T06:11:33.991 INFO:tasks.workunit.client.1.smithi203.stdout:0/530: truncate dd/d45/d96/f9f 712504 0 2024-05-25T06:11:33.991 INFO:tasks.workunit.client.0.smithi160.stdout:9/230: write d4/d11/f26 [3999441,77150] 0 2024-05-25T06:11:33.992 INFO:tasks.workunit.client.1.smithi203.stdout:2/665: write d2/d1a/d33/d80/fc9 [56943,112593] 0 2024-05-25T06:11:33.994 INFO:tasks.workunit.client.0.smithi160.stdout:6/213: dread d5/fa [0,4194304] 0 2024-05-25T06:11:33.995 INFO:tasks.workunit.client.1.smithi203.stdout:7/612: truncate d1/d11/d3d/f6a 787537 0 2024-05-25T06:11:33.997 INFO:tasks.workunit.client.1.smithi203.stdout:7/613: truncate d1/d9/fb4 814143 0 2024-05-25T06:11:33.998 INFO:tasks.workunit.client.1.smithi203.stdout:0/531: rename dd/l43 to dd/d3d/d48/lba 0 2024-05-25T06:11:33.998 INFO:tasks.workunit.client.1.smithi203.stdout:7/614: write d1/d9/d2c/d3a/d4e/f37 [733954,2865] 0 2024-05-25T06:11:33.998 INFO:tasks.workunit.client.0.smithi160.stdout:5/315: readlink db/dc/d13/d1d/d23/l4c 0 2024-05-25T06:11:33.999 INFO:tasks.workunit.client.1.smithi203.stdout:2/666: unlink d2/c9d 0 2024-05-25T06:11:33.999 INFO:tasks.workunit.client.0.smithi160.stdout:4/320: rename d4/d6/d5b/f4b to d4/d1c/d54/f61 0 2024-05-25T06:11:34.002 INFO:tasks.workunit.client.0.smithi160.stdout:3/194: dread f18 [0,4194304] 0 2024-05-25T06:11:34.002 INFO:tasks.workunit.client.1.smithi203.stdout:0/532: rmdir dd/d45/d96 39 2024-05-25T06:11:34.003 INFO:tasks.workunit.client.1.smithi203.stdout:2/667: dread d2/f6c [0,4194304] 0 2024-05-25T06:11:34.003 INFO:tasks.workunit.client.0.smithi160.stdout:0/254: mkdir d2b/d31/d35/d36/d52/d54 0 2024-05-25T06:11:34.003 INFO:tasks.workunit.client.1.smithi203.stdout:0/533: truncate f9 3721743 0 2024-05-25T06:11:34.005 INFO:tasks.workunit.client.0.smithi160.stdout:6/214: readlink d5/de/l1c 0 2024-05-25T06:11:34.006 INFO:tasks.workunit.client.1.smithi203.stdout:0/534: stat dd/d10/l22 0 2024-05-25T06:11:34.007 INFO:tasks.workunit.client.0.smithi160.stdout:7/271: dread d6/dc/f25 [0,4194304] 0 2024-05-25T06:11:34.007 INFO:tasks.workunit.client.0.smithi160.stdout:9/231: dwrite d4/d11/d12/d2e/f32 [0,4194304] 0 2024-05-25T06:11:34.010 INFO:tasks.workunit.client.0.smithi160.stdout:6/215: write d5/f3f [499175,56355] 0 2024-05-25T06:11:34.011 INFO:tasks.workunit.client.0.smithi160.stdout:0/255: chown d2b/d46/c4b 104867 1 2024-05-25T06:11:34.011 INFO:tasks.workunit.client.1.smithi203.stdout:0/535: mknod dd/d3a/d66/d9d/cbb 0 2024-05-25T06:11:34.012 INFO:tasks.workunit.client.1.smithi203.stdout:2/668: link d2/d4/d10/d58/da0/cb1 d2/d1a/d33/ce3 0 2024-05-25T06:11:34.013 INFO:tasks.workunit.client.0.smithi160.stdout:2/233: rename d12/l19 to d12/l4d 0 2024-05-25T06:11:34.013 INFO:tasks.workunit.client.1.smithi203.stdout:2/669: readlink d2/d4/l7 0 2024-05-25T06:11:34.014 INFO:tasks.workunit.client.0.smithi160.stdout:1/158: getdents de/d1b/d28 0 2024-05-25T06:11:34.014 INFO:tasks.workunit.client.0.smithi160.stdout:5/316: link db/dc/d13/d1d/d23/f59 db/dc/d13/d1d/d23/d45/f5f 0 2024-05-25T06:11:34.015 INFO:tasks.workunit.client.0.smithi160.stdout:6/216: chown d5/d30/d36/l33 118230323 1 2024-05-25T06:11:34.015 INFO:tasks.workunit.client.1.smithi203.stdout:2/670: truncate d2/d42/d8e/fcb 219646 0 2024-05-25T06:11:34.015 INFO:tasks.workunit.client.1.smithi203.stdout:0/536: getdents dd/d3d/d59/db8 0 2024-05-25T06:11:34.016 INFO:tasks.workunit.client.1.smithi203.stdout:0/537: truncate dd/d10/d34/d4c/f7e 1004413 0 2024-05-25T06:11:34.016 INFO:tasks.workunit.client.1.smithi203.stdout:2/671: mknod d2/d38/ce4 0 2024-05-25T06:11:34.017 INFO:tasks.workunit.client.0.smithi160.stdout:9/232: mkdir d4/d11/d12/d2e/d45 0 2024-05-25T06:11:34.018 INFO:tasks.workunit.client.0.smithi160.stdout:9/233: dread - d4/d11/d12/f35 zero size 2024-05-25T06:11:34.019 INFO:tasks.workunit.client.0.smithi160.stdout:7/272: dread d6/dc/f25 [0,4194304] 0 2024-05-25T06:11:34.020 INFO:tasks.workunit.client.0.smithi160.stdout:8/337: getdents d5/d44/d1f/d1e/d28/d49/d61 0 2024-05-25T06:11:34.021 INFO:tasks.workunit.client.1.smithi203.stdout:0/538: creat dd/d3d/d48/d91/fbc x:0 0 0 2024-05-25T06:11:34.022 INFO:tasks.workunit.client.0.smithi160.stdout:1/159: truncate de/f29 19004 0 2024-05-25T06:11:34.022 INFO:tasks.workunit.client.1.smithi203.stdout:7/615: getdents d1/d9/d2c/d79/dc7 0 2024-05-25T06:11:34.022 INFO:tasks.workunit.client.0.smithi160.stdout:7/273: write d6/f12 [4810321,17569] 0 2024-05-25T06:11:34.024 INFO:tasks.workunit.client.0.smithi160.stdout:1/160: dread - de/f16 zero size 2024-05-25T06:11:34.024 INFO:tasks.workunit.client.0.smithi160.stdout:6/217: mkdir d5/d45/d4a 0 2024-05-25T06:11:34.025 INFO:tasks.workunit.client.0.smithi160.stdout:2/234: creat d12/d1c/d37/d49/f4e x:0 0 0 2024-05-25T06:11:34.026 INFO:tasks.workunit.client.0.smithi160.stdout:1/161: fsync de/f15 0 2024-05-25T06:11:34.028 INFO:tasks.workunit.client.0.smithi160.stdout:3/195: getdents d19/d2f 0 2024-05-25T06:11:34.031 INFO:tasks.workunit.client.0.smithi160.stdout:0/256: link d2b/d31/d35/d36/d37/f44 d2b/d31/d35/f55 0 2024-05-25T06:11:34.031 INFO:tasks.workunit.client.1.smithi203.stdout:0/539: getdents dd/d10/d34/d4c/d99/d70 0 2024-05-25T06:11:34.032 INFO:tasks.workunit.client.0.smithi160.stdout:5/317: dwrite db/dc/d13/d1d/d23/f46 [0,4194304] 0 2024-05-25T06:11:34.032 INFO:tasks.workunit.client.0.smithi160.stdout:6/218: fdatasync f2 0 2024-05-25T06:11:34.033 INFO:tasks.workunit.client.1.smithi203.stdout:2/672: dread d2/d38/d2f/fab [0,4194304] 0 2024-05-25T06:11:34.035 INFO:tasks.workunit.client.0.smithi160.stdout:6/219: stat d5/f26 0 2024-05-25T06:11:34.036 INFO:tasks.workunit.client.1.smithi203.stdout:2/673: mkdir d2/d42/de5 0 2024-05-25T06:11:34.038 INFO:tasks.workunit.client.0.smithi160.stdout:2/235: creat d12/d1c/d37/f4f x:0 0 0 2024-05-25T06:11:34.038 INFO:tasks.workunit.client.1.smithi203.stdout:0/540: unlink dd/d32/f7c 0 2024-05-25T06:11:34.039 INFO:tasks.workunit.client.1.smithi203.stdout:2/674: symlink d2/d4/da6/le6 0 2024-05-25T06:11:34.040 INFO:tasks.workunit.client.0.smithi160.stdout:4/321: dwrite d4/d6/d21/f32 [0,4194304] 0 2024-05-25T06:11:34.042 INFO:tasks.workunit.client.1.smithi203.stdout:0/541: rmdir dd/d3d/da0 39 2024-05-25T06:11:34.043 INFO:tasks.workunit.client.1.smithi203.stdout:0/542: fdatasync dd/d10/d34/d4c/d99/d70/fb0 0 2024-05-25T06:11:34.044 INFO:tasks.workunit.client.1.smithi203.stdout:2/675: creat d2/d4/d10/d32/fe7 x:0 0 0 2024-05-25T06:11:34.045 INFO:tasks.workunit.client.0.smithi160.stdout:0/257: unlink f25 0 2024-05-25T06:11:34.046 INFO:tasks.workunit.client.1.smithi203.stdout:0/543: mknod dd/d3a/cbd 0 2024-05-25T06:11:34.048 INFO:tasks.workunit.client.1.smithi203.stdout:7/616: write d1/d9/fc [2632887,130614] 0 2024-05-25T06:11:34.048 INFO:tasks.workunit.client.1.smithi203.stdout:0/544: dread - dd/d10/d34/d4c/d99/d70/fb0 zero size 2024-05-25T06:11:34.049 INFO:tasks.workunit.client.0.smithi160.stdout:9/234: truncate f2 1499781 0 2024-05-25T06:11:34.051 INFO:tasks.workunit.client.1.smithi203.stdout:2/676: dread d2/d4/d10/fda [0,4194304] 0 2024-05-25T06:11:34.051 INFO:tasks.workunit.client.0.smithi160.stdout:7/274: link d6/dc/d21/l28 d6/dc/d21/l50 0 2024-05-25T06:11:34.054 INFO:tasks.workunit.client.1.smithi203.stdout:2/677: write d2/d42/fa9 [3273259,91191] 0 2024-05-25T06:11:34.054 INFO:tasks.workunit.client.0.smithi160.stdout:5/318: dwrite db/dc/d13/d1d/d23/d24/d2e/f5b [0,4194304] 0 2024-05-25T06:11:34.055 INFO:tasks.workunit.client.0.smithi160.stdout:0/258: read f5 [1692212,52444] 0 2024-05-25T06:11:34.055 INFO:tasks.workunit.client.0.smithi160.stdout:2/236: mknod d12/c50 0 2024-05-25T06:11:34.056 INFO:tasks.workunit.client.0.smithi160.stdout:1/162: getdents de/d1b 0 2024-05-25T06:11:34.058 INFO:tasks.workunit.client.0.smithi160.stdout:4/322: dwrite d4/d6/d28/f17 [8388608,4194304] 0 2024-05-25T06:11:34.058 INFO:tasks.workunit.client.0.smithi160.stdout:7/275: write d6/d2c/f43 [313398,52521] 0 2024-05-25T06:11:34.058 INFO:tasks.workunit.client.0.smithi160.stdout:6/220: mknod d5/de/d32/d1b/c4b 0 2024-05-25T06:11:34.059 INFO:tasks.workunit.client.0.smithi160.stdout:5/319: chown f7 179 1 2024-05-25T06:11:34.059 INFO:tasks.workunit.client.1.smithi203.stdout:7/617: dwrite d1/d9/d2c/d62/d9b/fd2 [0,4194304] 0 2024-05-25T06:11:34.060 INFO:tasks.workunit.client.0.smithi160.stdout:9/235: mknod d4/d11/d12/d2e/c46 0 2024-05-25T06:11:34.062 INFO:tasks.workunit.client.0.smithi160.stdout:2/237: dread - d12/d1c/d37/f4f zero size 2024-05-25T06:11:34.063 INFO:tasks.workunit.client.0.smithi160.stdout:6/221: truncate d5/f34 757140 0 2024-05-25T06:11:34.065 INFO:tasks.workunit.client.0.smithi160.stdout:8/338: dread d5/d44/d1f/f42 [0,4194304] 0 2024-05-25T06:11:34.066 INFO:tasks.workunit.client.0.smithi160.stdout:0/259: symlink d2b/d46/l56 0 2024-05-25T06:11:34.074 INFO:tasks.workunit.client.0.smithi160.stdout:4/323: dwrite d4/f4f [0,4194304] 0 2024-05-25T06:11:34.079 INFO:tasks.workunit.client.1.smithi203.stdout:0/545: dread dd/d5c/f90 [0,4194304] 0 2024-05-25T06:11:34.079 INFO:tasks.workunit.client.0.smithi160.stdout:6/222: symlink d5/de/d32/d35/l4c 0 2024-05-25T06:11:34.082 INFO:tasks.workunit.client.0.smithi160.stdout:6/223: chown d5/d30/d36/l33 403321710 1 2024-05-25T06:11:34.085 INFO:tasks.workunit.client.0.smithi160.stdout:8/339: symlink d5/d44/d1f/l6e 0 2024-05-25T06:11:34.087 INFO:tasks.workunit.client.0.smithi160.stdout:4/324: dwrite d4/d6/d35/d3d/d55/f56 [4194304,4194304] 0 2024-05-25T06:11:34.088 INFO:tasks.workunit.client.0.smithi160.stdout:7/276: creat d6/d2b/f51 x:0 0 0 2024-05-25T06:11:34.089 INFO:tasks.workunit.client.1.smithi203.stdout:0/546: dwrite dd/f44 [4194304,4194304] 0 2024-05-25T06:11:34.090 INFO:tasks.workunit.client.0.smithi160.stdout:6/224: unlink d5/f15 0 2024-05-25T06:11:34.094 INFO:tasks.workunit.client.0.smithi160.stdout:8/340: mknod d5/d39/c6f 0 2024-05-25T06:11:34.094 INFO:tasks.workunit.client.1.smithi203.stdout:7/618: dread d1/d9/f18 [0,4194304] 0 2024-05-25T06:11:34.095 INFO:tasks.workunit.client.0.smithi160.stdout:4/325: fdatasync d4/f11 0 2024-05-25T06:11:34.095 INFO:tasks.workunit.client.0.smithi160.stdout:3/196: dread d19/d22/f2c [0,4194304] 0 2024-05-25T06:11:34.097 INFO:tasks.workunit.client.0.smithi160.stdout:7/277: mknod d6/dc/d21/c52 0 2024-05-25T06:11:34.098 INFO:tasks.workunit.client.1.smithi203.stdout:0/547: chown dd/d3d/d4f/d62/d78/fab 11 1 2024-05-25T06:11:34.098 INFO:tasks.workunit.client.1.smithi203.stdout:7/619: creat d1/d9/d2c/d62/d9b/fd5 x:0 0 0 2024-05-25T06:11:34.101 INFO:tasks.workunit.client.1.smithi203.stdout:0/548: rmdir dd/d3d/d59 39 2024-05-25T06:11:34.101 INFO:tasks.workunit.client.1.smithi203.stdout:0/549: readlink dd/d10/l3f 0 2024-05-25T06:11:34.102 INFO:tasks.workunit.client.1.smithi203.stdout:7/620: mkdir d1/d9/d2c/d79/dd6 0 2024-05-25T06:11:34.106 INFO:tasks.workunit.client.1.smithi203.stdout:7/621: creat d1/d9/d2c/d79/dd6/fd7 x:0 0 0 2024-05-25T06:11:34.108 INFO:tasks.workunit.client.0.smithi160.stdout:4/326: dread - d4/d6/d28/f4e zero size 2024-05-25T06:11:34.109 INFO:tasks.workunit.client.1.smithi203.stdout:7/622: truncate d1/d11/d20/f27 2118842 0 2024-05-25T06:11:34.109 INFO:tasks.workunit.client.1.smithi203.stdout:0/550: link dd/d10/d34/f67 dd/d10/d34/d4c/d99/fbe 0 2024-05-25T06:11:34.110 INFO:tasks.workunit.client.1.smithi203.stdout:0/551: chown dd/d3d/d4f/d62/f74 1805241671 1 2024-05-25T06:11:34.112 INFO:tasks.workunit.client.1.smithi203.stdout:7/623: creat d1/dbd/dc1/fd8 x:0 0 0 2024-05-25T06:11:34.112 INFO:tasks.workunit.client.0.smithi160.stdout:7/278: truncate d6/dc/f15 1180533 0 2024-05-25T06:11:34.114 INFO:tasks.workunit.client.1.smithi203.stdout:0/552: link dd/d3a/f7f dd/d3a/d66/d86/fbf 0 2024-05-25T06:11:34.115 INFO:tasks.workunit.client.0.smithi160.stdout:0/260: truncate f1f 567067 0 2024-05-25T06:11:34.117 INFO:tasks.workunit.client.1.smithi203.stdout:0/553: creat dd/d3d/d48/d91/fc0 x:0 0 0 2024-05-25T06:11:34.117 INFO:tasks.workunit.client.0.smithi160.stdout:9/236: dread d4/d11/f2c [0,4194304] 0 2024-05-25T06:11:34.117 INFO:tasks.workunit.client.0.smithi160.stdout:7/279: readlink d6/l22 0 2024-05-25T06:11:34.120 INFO:tasks.workunit.client.0.smithi160.stdout:8/341: dread d5/d44/d1f/f22 [0,4194304] 0 2024-05-25T06:11:34.123 INFO:tasks.workunit.client.0.smithi160.stdout:6/225: dwrite d5/de/d32/d1b/f2d [4194304,4194304] 0 2024-05-25T06:11:34.123 INFO:tasks.workunit.client.0.smithi160.stdout:5/320: dwrite db/dc/d13/d1d/d23/f59 [0,4194304] 0 2024-05-25T06:11:34.124 INFO:tasks.workunit.client.1.smithi203.stdout:0/554: dwrite dd/d10/d34/d4c/d99/d70/fb0 [0,4194304] 0 2024-05-25T06:11:34.125 INFO:tasks.workunit.client.0.smithi160.stdout:4/327: link d4/d6/d28/f4e d4/d1c/f62 0 2024-05-25T06:11:34.127 INFO:tasks.workunit.client.0.smithi160.stdout:5/321: fdatasync f7 0 2024-05-25T06:11:34.128 INFO:tasks.workunit.client.0.smithi160.stdout:3/197: dwrite d19/d2f/d38/f3e [0,4194304] 0 2024-05-25T06:11:34.128 INFO:tasks.workunit.client.0.smithi160.stdout:9/237: write d4/d11/d14/f37 [968587,52731] 0 2024-05-25T06:11:34.129 INFO:tasks.workunit.client.0.smithi160.stdout:2/238: dread f11 [0,4194304] 0 2024-05-25T06:11:34.130 INFO:tasks.workunit.client.1.smithi203.stdout:0/555: symlink dd/d10/d34/d4c/d5d/lc1 0 2024-05-25T06:11:34.131 INFO:tasks.workunit.client.0.smithi160.stdout:7/280: rename d6/dc/f17 to d6/d2c/f53 0 2024-05-25T06:11:34.131 INFO:tasks.workunit.client.0.smithi160.stdout:1/163: dread f2 [0,4194304] 0 2024-05-25T06:11:34.132 INFO:tasks.workunit.client.1.smithi203.stdout:0/556: truncate dd/d10/f4a 826544 0 2024-05-25T06:11:34.141 INFO:tasks.workunit.client.0.smithi160.stdout:7/281: truncate d6/d32/f42 18885 0 2024-05-25T06:11:34.141 INFO:tasks.workunit.client.0.smithi160.stdout:0/261: dwrite f16 [0,4194304] 0 2024-05-25T06:11:34.146 INFO:tasks.workunit.client.0.smithi160.stdout:5/322: rmdir db/dc/d13/d1d/d3f/d4a 39 2024-05-25T06:11:34.147 INFO:tasks.workunit.client.0.smithi160.stdout:2/239: symlink d12/d3e/l51 0 2024-05-25T06:11:34.148 INFO:tasks.workunit.client.0.smithi160.stdout:4/328: mkdir d4/d63 0 2024-05-25T06:11:34.149 INFO:tasks.workunit.client.0.smithi160.stdout:3/198: creat d19/d2f/d38/d43/f46 x:0 0 0 2024-05-25T06:11:34.152 INFO:tasks.workunit.client.0.smithi160.stdout:2/240: write d12/d1c/f27 [212733,2379] 0 2024-05-25T06:11:34.153 INFO:tasks.workunit.client.0.smithi160.stdout:4/329: read d4/d6/d28/f25 [378416,84143] 0 2024-05-25T06:11:34.154 INFO:tasks.workunit.client.0.smithi160.stdout:1/164: dwrite de/f13 [0,4194304] 0 2024-05-25T06:11:34.155 INFO:tasks.workunit.client.0.smithi160.stdout:0/262: chown d2b/d31/d35/d36/f4a 188 1 2024-05-25T06:11:34.156 INFO:tasks.workunit.client.0.smithi160.stdout:5/323: rmdir db/dc/d13/d1d/d23 39 2024-05-25T06:11:34.156 INFO:tasks.workunit.client.0.smithi160.stdout:6/226: dread d5/fa [0,4194304] 0 2024-05-25T06:11:34.158 INFO:tasks.workunit.client.0.smithi160.stdout:2/241: chown f8 193921 1 2024-05-25T06:11:34.159 INFO:tasks.workunit.client.1.smithi203.stdout:0/557: write dd/d10/f33 [1223433,21495] 0 2024-05-25T06:11:34.161 INFO:tasks.workunit.client.0.smithi160.stdout:7/282: dwrite d6/dc/f41 [0,4194304] 0 2024-05-25T06:11:34.165 INFO:tasks.workunit.client.1.smithi203.stdout:0/558: truncate dd/d32/d37/f4e 684770 0 2024-05-25T06:11:34.165 INFO:tasks.workunit.client.1.smithi203.stdout:0/559: dread - dd/d10/d34/fa8 zero size 2024-05-25T06:11:34.166 INFO:tasks.workunit.client.0.smithi160.stdout:0/263: link f10 d2b/f57 0 2024-05-25T06:11:34.169 INFO:tasks.workunit.client.1.smithi203.stdout:0/560: rmdir dd/d3d 39 2024-05-25T06:11:34.170 INFO:tasks.workunit.client.0.smithi160.stdout:4/330: dread d4/d6/d28/f17 [4194304,4194304] 0 2024-05-25T06:11:34.172 INFO:tasks.workunit.client.1.smithi203.stdout:0/561: mknod dd/d3a/cc2 0 2024-05-25T06:11:34.173 INFO:tasks.workunit.client.1.smithi203.stdout:0/562: fdatasync dd/d10/f33 0 2024-05-25T06:11:34.174 INFO:tasks.workunit.client.1.smithi203.stdout:0/563: chown dd/d3a/d66/d9d 0 1 2024-05-25T06:11:34.175 INFO:tasks.workunit.client.0.smithi160.stdout:2/242: creat d12/d1c/d2a/f52 x:0 0 0 2024-05-25T06:11:34.175 INFO:tasks.workunit.client.0.smithi160.stdout:6/227: dwrite d5/f2c [0,4194304] 0 2024-05-25T06:11:34.176 INFO:tasks.workunit.client.0.smithi160.stdout:9/238: write d4/fd [2143614,118110] 0 2024-05-25T06:11:34.176 INFO:tasks.workunit.client.0.smithi160.stdout:0/264: mkdir d2b/d58 0 2024-05-25T06:11:34.178 INFO:tasks.workunit.client.0.smithi160.stdout:1/165: creat de/d21/f31 x:0 0 0 2024-05-25T06:11:34.179 INFO:tasks.workunit.client.0.smithi160.stdout:7/283: rename d6/f12 to d6/dc/d21/f54 0 2024-05-25T06:11:34.179 INFO:tasks.workunit.client.1.smithi203.stdout:0/564: rename dd/d10/d34/d4c/d99/l77 to dd/d10/d34/lc3 0 2024-05-25T06:11:34.180 INFO:tasks.workunit.client.0.smithi160.stdout:5/324: mknod db/dc/d13/d1d/d3f/d4a/c60 0 2024-05-25T06:11:34.183 INFO:tasks.workunit.client.0.smithi160.stdout:3/199: write d19/d2f/d34/f40 [721550,8331] 0 2024-05-25T06:11:34.185 INFO:tasks.workunit.client.1.smithi203.stdout:0/565: creat dd/d3d/da0/fc4 x:0 0 0 2024-05-25T06:11:34.185 INFO:tasks.workunit.client.0.smithi160.stdout:2/243: write f11 [4716395,31210] 0 2024-05-25T06:11:34.186 INFO:tasks.workunit.client.0.smithi160.stdout:3/200: chown d19/d2f/d38 107391846 1 2024-05-25T06:11:34.186 INFO:tasks.workunit.client.0.smithi160.stdout:0/265: chown c26 2960 1 2024-05-25T06:11:34.188 INFO:tasks.workunit.client.1.smithi203.stdout:0/566: symlink dd/d3a/lc5 0 2024-05-25T06:11:34.188 INFO:tasks.workunit.client.0.smithi160.stdout:9/239: dread d4/d11/f2c [0,4194304] 0 2024-05-25T06:11:34.188 INFO:tasks.workunit.client.0.smithi160.stdout:6/228: rename d5/fa to d5/d39/f4d 0 2024-05-25T06:11:34.189 INFO:tasks.workunit.client.0.smithi160.stdout:4/331: mkdir d4/d63/d64 0 2024-05-25T06:11:34.189 INFO:tasks.workunit.client.0.smithi160.stdout:3/201: readlink d19/d2f/l33 0 2024-05-25T06:11:34.189 INFO:tasks.workunit.client.0.smithi160.stdout:8/342: dread d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:34.190 INFO:tasks.workunit.client.0.smithi160.stdout:1/166: symlink de/d1b/d28/l32 0 2024-05-25T06:11:34.191 INFO:tasks.workunit.client.1.smithi203.stdout:0/567: chown dd/d10/l14 1 1 2024-05-25T06:11:34.193 INFO:tasks.workunit.client.0.smithi160.stdout:2/244: mknod d12/d13/c53 0 2024-05-25T06:11:34.194 INFO:tasks.workunit.client.1.smithi203.stdout:0/568: mknod dd/d3d/d75/d93/cc6 0 2024-05-25T06:11:34.195 INFO:tasks.workunit.client.0.smithi160.stdout:3/202: chown d19/l1f 111969703 1 2024-05-25T06:11:34.196 INFO:tasks.workunit.client.0.smithi160.stdout:7/284: rename d6/d3e/f49 to d6/d2b/d2d/f55 0 2024-05-25T06:11:34.196 INFO:tasks.workunit.client.1.smithi203.stdout:0/569: mknod dd/d3a/d66/cc7 0 2024-05-25T06:11:34.198 INFO:tasks.workunit.client.0.smithi160.stdout:5/325: rmdir db/dc/d13/d1d/d23/d24 39 2024-05-25T06:11:34.199 INFO:tasks.workunit.client.0.smithi160.stdout:1/167: mknod de/d1b/d1c/c33 0 2024-05-25T06:11:34.201 INFO:tasks.workunit.client.1.smithi203.stdout:0/570: dread dd/f24 [0,4194304] 0 2024-05-25T06:11:34.205 INFO:tasks.workunit.client.0.smithi160.stdout:1/168: fsync f2 0 2024-05-25T06:11:34.206 INFO:tasks.workunit.client.0.smithi160.stdout:3/203: truncate d19/f44 178613 0 2024-05-25T06:11:34.208 INFO:tasks.workunit.client.0.smithi160.stdout:9/240: dwrite d4/d11/d12/f35 [0,4194304] 0 2024-05-25T06:11:34.208 INFO:tasks.workunit.client.0.smithi160.stdout:1/169: read - de/d21/f31 zero size 2024-05-25T06:11:34.209 INFO:tasks.workunit.client.0.smithi160.stdout:4/332: link d4/d1a/f59 d4/d1c/d54/f65 0 2024-05-25T06:11:34.210 INFO:tasks.workunit.client.0.smithi160.stdout:4/333: fdatasync d4/d6/d21/f3f 0 2024-05-25T06:11:34.210 INFO:tasks.workunit.client.1.smithi203.stdout:0/571: write dd/d3a/f7f [982734,17424] 0 2024-05-25T06:11:34.214 INFO:tasks.workunit.client.0.smithi160.stdout:7/285: dwrite d6/fb [0,4194304] 0 2024-05-25T06:11:34.214 INFO:tasks.workunit.client.0.smithi160.stdout:3/204: mkdir d19/d2f/d38/d47 0 2024-05-25T06:11:34.214 INFO:tasks.workunit.client.0.smithi160.stdout:2/245: write fe [2821452,34491] 0 2024-05-25T06:11:34.215 INFO:tasks.workunit.client.0.smithi160.stdout:5/326: creat db/dc/d13/d1d/d31/d56/f61 x:0 0 0 2024-05-25T06:11:34.218 INFO:tasks.workunit.client.0.smithi160.stdout:1/170: symlink de/d1b/l34 0 2024-05-25T06:11:34.218 INFO:tasks.workunit.client.1.smithi203.stdout:0/572: dwrite dd/d3a/d66/f69 [0,4194304] 0 2024-05-25T06:11:34.219 INFO:tasks.workunit.client.0.smithi160.stdout:8/343: link d5/d44/d1f/d1e/d28/d4a/f3a d5/d44/d1f/d1e/d28/d49/f70 0 2024-05-25T06:11:34.219 INFO:tasks.workunit.client.0.smithi160.stdout:9/241: rmdir d4/d11/d1e/d24 39 2024-05-25T06:11:34.223 INFO:tasks.workunit.client.1.smithi203.stdout:0/573: creat dd/d3a/fc8 x:0 0 0 2024-05-25T06:11:34.225 INFO:tasks.workunit.client.0.smithi160.stdout:2/246: dread - d12/d1c/d37/d49/f4e zero size 2024-05-25T06:11:34.225 INFO:tasks.workunit.client.0.smithi160.stdout:4/334: mkdir d4/d6/d5b/d66 0 2024-05-25T06:11:34.226 INFO:tasks.workunit.client.0.smithi160.stdout:3/205: creat d19/d22/f48 x:0 0 0 2024-05-25T06:11:34.226 INFO:tasks.workunit.client.0.smithi160.stdout:0/266: dwrite d2b/d31/d35/d36/f4a [0,4194304] 0 2024-05-25T06:11:34.227 INFO:tasks.workunit.client.1.smithi203.stdout:0/574: dread dd/d3d/d48/f7a [0,4194304] 0 2024-05-25T06:11:34.227 INFO:tasks.workunit.client.0.smithi160.stdout:7/286: creat d6/d2b/d2d/f56 x:0 0 0 2024-05-25T06:11:34.229 INFO:tasks.workunit.client.0.smithi160.stdout:6/229: dread d5/f1e [0,4194304] 0 2024-05-25T06:11:34.230 INFO:tasks.workunit.client.1.smithi203.stdout:0/575: mknod dd/d3d/d48/db1/cc9 0 2024-05-25T06:11:34.233 INFO:tasks.workunit.client.0.smithi160.stdout:8/344: mknod d5/d44/d1f/d58/c71 0 2024-05-25T06:11:34.234 INFO:tasks.workunit.client.0.smithi160.stdout:9/242: dread - d4/d11/d1e/d24/f13 zero size 2024-05-25T06:11:34.234 INFO:tasks.workunit.client.0.smithi160.stdout:5/327: creat db/d47/d5e/f62 x:0 0 0 2024-05-25T06:11:34.234 INFO:tasks.workunit.client.1.smithi203.stdout:0/576: symlink dd/d3d/d48/d91/lca 0 2024-05-25T06:11:34.235 INFO:tasks.workunit.client.1.smithi203.stdout:0/577: chown dd/d45/d96/f9f 243009 1 2024-05-25T06:11:34.236 INFO:tasks.workunit.client.1.smithi203.stdout:0/578: dread - dd/d5c/f73 zero size 2024-05-25T06:11:34.238 INFO:tasks.workunit.client.0.smithi160.stdout:9/243: fsync d4/d11/d1e/d3e/f44 0 2024-05-25T06:11:34.239 INFO:tasks.workunit.client.0.smithi160.stdout:5/328: write db/f57 [861518,80192] 0 2024-05-25T06:11:34.240 INFO:tasks.workunit.client.0.smithi160.stdout:6/230: dread d5/f3f [0,4194304] 0 2024-05-25T06:11:34.243 INFO:tasks.workunit.client.0.smithi160.stdout:3/206: dwrite f11 [4194304,4194304] 0 2024-05-25T06:11:34.243 INFO:tasks.workunit.client.0.smithi160.stdout:8/345: mkdir d5/d9/d72 0 2024-05-25T06:11:34.244 INFO:tasks.workunit.client.1.smithi203.stdout:0/579: dwrite dd/d10/f33 [0,4194304] 0 2024-05-25T06:11:34.245 INFO:tasks.workunit.client.0.smithi160.stdout:0/267: mknod d2b/d58/c59 0 2024-05-25T06:11:34.245 INFO:tasks.workunit.client.0.smithi160.stdout:7/287: read d6/dc/d21/f54 [4043803,4153] 0 2024-05-25T06:11:34.248 INFO:tasks.workunit.client.0.smithi160.stdout:4/335: dwrite d4/d6/d21/f24 [0,4194304] 0 2024-05-25T06:11:34.248 INFO:tasks.workunit.client.0.smithi160.stdout:9/244: creat d4/f47 x:0 0 0 2024-05-25T06:11:34.252 INFO:tasks.workunit.client.0.smithi160.stdout:9/245: fdatasync d4/fd 0 2024-05-25T06:11:34.253 INFO:tasks.workunit.client.1.smithi203.stdout:0/580: dwrite dd/d32/f54 [4194304,4194304] 0 2024-05-25T06:11:34.253 INFO:tasks.workunit.client.0.smithi160.stdout:2/247: dwrite d12/f1d [0,4194304] 0 2024-05-25T06:11:34.254 INFO:tasks.workunit.client.0.smithi160.stdout:7/288: write d6/d2c/f43 [1120190,6476] 0 2024-05-25T06:11:34.256 INFO:tasks.workunit.client.0.smithi160.stdout:6/231: creat d5/de/d32/d35/f4e x:0 0 0 2024-05-25T06:11:34.258 INFO:tasks.workunit.client.1.smithi203.stdout:0/581: truncate dd/d10/d34/d4c/d99/f82 242273 0 2024-05-25T06:11:34.259 INFO:tasks.workunit.client.0.smithi160.stdout:0/268: write d2b/d31/d35/d36/d37/f4d [85239,76246] 0 2024-05-25T06:11:34.260 INFO:tasks.workunit.client.1.smithi203.stdout:0/582: read dd/fe [569991,29863] 0 2024-05-25T06:11:34.264 INFO:tasks.workunit.client.0.smithi160.stdout:7/289: truncate d6/dc/f4a 150682 0 2024-05-25T06:11:34.266 INFO:tasks.workunit.client.1.smithi203.stdout:0/583: mknod dd/d5c/ccb 0 2024-05-25T06:11:34.268 INFO:tasks.workunit.client.0.smithi160.stdout:2/248: rename d12/l36 to d12/d3c/l54 0 2024-05-25T06:11:34.268 INFO:tasks.workunit.client.1.smithi203.stdout:0/584: chown dd/d3d/d59/l94 13774689 1 2024-05-25T06:11:34.269 INFO:tasks.workunit.client.0.smithi160.stdout:9/246: creat d4/d11/d12/f48 x:0 0 0 2024-05-25T06:11:34.269 INFO:tasks.workunit.client.0.smithi160.stdout:5/329: dwrite db/f5d [0,4194304] 0 2024-05-25T06:11:34.271 INFO:tasks.workunit.client.1.smithi203.stdout:0/585: chown dd/d10/d34/d4c/d99/c85 27090873 1 2024-05-25T06:11:34.272 INFO:tasks.workunit.client.1.smithi203.stdout:0/586: write dd/d45/f5b [4424728,64537] 0 2024-05-25T06:11:34.275 INFO:tasks.workunit.client.1.smithi203.stdout:0/587: mknod dd/d10/d34/d4c/ccc 0 2024-05-25T06:11:34.276 INFO:tasks.workunit.client.0.smithi160.stdout:0/269: creat d2b/d46/f5a x:0 0 0 2024-05-25T06:11:34.278 INFO:tasks.workunit.client.1.smithi203.stdout:0/588: creat dd/d45/d96/fcd x:0 0 0 2024-05-25T06:11:34.279 INFO:tasks.workunit.client.0.smithi160.stdout:3/207: link d19/d2f/f35 d19/d2f/d38/f49 0 2024-05-25T06:11:34.280 INFO:tasks.workunit.client.1.smithi203.stdout:0/589: read dd/d10/d34/d4c/d99/d70/fb0 [1072501,19496] 0 2024-05-25T06:11:34.281 INFO:tasks.workunit.client.0.smithi160.stdout:7/290: creat d6/dc/d21/f57 x:0 0 0 2024-05-25T06:11:34.281 INFO:tasks.workunit.client.0.smithi160.stdout:4/336: creat d4/d6/d5b/d66/f67 x:0 0 0 2024-05-25T06:11:34.282 INFO:tasks.workunit.client.1.smithi203.stdout:0/590: read dd/d3a/f7f [329137,100671] 0 2024-05-25T06:11:34.283 INFO:tasks.workunit.client.0.smithi160.stdout:1/171: rename de/d1b/d22 to de/d1b/d35 0 2024-05-25T06:11:34.285 INFO:tasks.workunit.client.0.smithi160.stdout:8/346: dwrite d5/d44/d1f/d1e/d28/f4d [0,4194304] 0 2024-05-25T06:11:34.285 INFO:tasks.workunit.client.0.smithi160.stdout:5/330: creat db/f63 x:0 0 0 2024-05-25T06:11:34.289 INFO:tasks.workunit.client.0.smithi160.stdout:4/337: chown d4/d6/d28/f25 145 1 2024-05-25T06:11:34.289 INFO:tasks.workunit.client.1.smithi203.stdout:0/591: link dd/d10/d34/d4c/ccc dd/d10/d34/d4c/cce 0 2024-05-25T06:11:34.292 INFO:tasks.workunit.client.1.smithi203.stdout:0/592: creat dd/d3d/d75/fcf x:0 0 0 2024-05-25T06:11:34.293 INFO:tasks.workunit.client.1.smithi203.stdout:0/593: read - dd/d3d/d48/d91/fc0 zero size 2024-05-25T06:11:34.295 INFO:tasks.workunit.client.0.smithi160.stdout:2/249: dwrite d12/d1c/d2a/f3a [0,4194304] 0 2024-05-25T06:11:34.300 INFO:tasks.workunit.client.0.smithi160.stdout:0/270: getdents d2b/d31/d35/d36/d37 0 2024-05-25T06:11:34.302 INFO:tasks.workunit.client.0.smithi160.stdout:6/232: write d5/de/fd [504572,31774] 0 2024-05-25T06:11:34.309 INFO:tasks.workunit.client.0.smithi160.stdout:5/331: dwrite db/f63 [0,4194304] 0 2024-05-25T06:11:34.309 INFO:tasks.workunit.client.0.smithi160.stdout:7/291: dwrite d6/d3e/f4b [0,4194304] 0 2024-05-25T06:11:34.309 INFO:tasks.workunit.client.0.smithi160.stdout:3/208: getdents d19/d2f/d38 0 2024-05-25T06:11:34.311 INFO:tasks.workunit.client.0.smithi160.stdout:4/338: rename d4/d6/d21/l4d to d4/d6/l68 0 2024-05-25T06:11:34.312 INFO:tasks.workunit.client.0.smithi160.stdout:6/233: mknod d5/d30/c4f 0 2024-05-25T06:11:34.313 INFO:tasks.workunit.client.0.smithi160.stdout:7/292: chown d6/d2b/f51 6 1 2024-05-25T06:11:34.314 INFO:tasks.workunit.client.0.smithi160.stdout:3/209: dread d19/f44 [0,4194304] 0 2024-05-25T06:11:34.315 INFO:tasks.workunit.client.0.smithi160.stdout:9/247: link d4/d11/d1e/d3e/d25/d27/c2d d4/d11/d1e/c49 0 2024-05-25T06:11:34.320 INFO:tasks.workunit.client.0.smithi160.stdout:2/250: rename f5 to d12/d13/f55 0 2024-05-25T06:11:34.322 INFO:tasks.workunit.client.0.smithi160.stdout:0/271: creat d2b/d31/d35/d41/f5b x:0 0 0 2024-05-25T06:11:34.323 INFO:tasks.workunit.client.0.smithi160.stdout:1/172: chown de/d1b/d35 443 1 2024-05-25T06:11:34.326 INFO:tasks.workunit.client.0.smithi160.stdout:6/234: dwrite d5/d30/f3d [0,4194304] 0 2024-05-25T06:11:34.326 INFO:tasks.workunit.client.0.smithi160.stdout:5/332: mknod db/dc/d13/d1d/d3f/c64 0 2024-05-25T06:11:34.326 INFO:tasks.workunit.client.0.smithi160.stdout:1/173: write de/f19 [1160074,129882] 0 2024-05-25T06:11:34.328 INFO:tasks.workunit.client.0.smithi160.stdout:8/347: dread d5/d44/d1f/d1e/d28/d49/f70 [4194304,4194304] 0 2024-05-25T06:11:34.328 INFO:tasks.workunit.client.0.smithi160.stdout:6/235: read d5/f2c [400942,22499] 0 2024-05-25T06:11:34.328 INFO:tasks.workunit.client.0.smithi160.stdout:3/210: symlink d19/d2f/d38/l4a 0 2024-05-25T06:11:34.329 INFO:tasks.workunit.client.0.smithi160.stdout:3/211: stat fe 0 2024-05-25T06:11:34.329 INFO:tasks.workunit.client.0.smithi160.stdout:6/236: write d5/f26 [2290771,83296] 0 2024-05-25T06:11:34.330 INFO:tasks.workunit.client.0.smithi160.stdout:9/248: rmdir d4/d11/d1e/d24 39 2024-05-25T06:11:34.332 INFO:tasks.workunit.client.0.smithi160.stdout:2/251: unlink d12/d1c/d2a/f52 0 2024-05-25T06:11:34.335 INFO:tasks.workunit.client.1.smithi203.stdout:0/594: dread dd/d3d/d59/f9b [0,4194304] 0 2024-05-25T06:11:34.336 INFO:tasks.workunit.client.0.smithi160.stdout:4/339: write d4/f11 [8222627,72281] 0 2024-05-25T06:11:34.337 INFO:tasks.workunit.client.0.smithi160.stdout:0/272: symlink d2b/d31/d35/d36/l5c 0 2024-05-25T06:11:34.338 INFO:tasks.workunit.client.1.smithi203.stdout:0/595: creat dd/d10/d34/d4c/fd0 x:0 0 0 2024-05-25T06:11:34.340 INFO:tasks.workunit.client.0.smithi160.stdout:1/174: mknod de/d21/c36 0 2024-05-25T06:11:34.340 INFO:tasks.workunit.client.1.smithi203.stdout:0/596: truncate dd/d3d/d59/fac 61750 0 2024-05-25T06:11:34.340 INFO:tasks.workunit.client.0.smithi160.stdout:1/175: rename de to de/d37 22 2024-05-25T06:11:34.341 INFO:tasks.workunit.client.0.smithi160.stdout:6/237: mknod d5/c50 0 2024-05-25T06:11:34.343 INFO:tasks.workunit.client.0.smithi160.stdout:9/249: mknod d4/d11/c4a 0 2024-05-25T06:11:34.344 INFO:tasks.workunit.client.0.smithi160.stdout:2/252: creat d12/d1c/f56 x:0 0 0 2024-05-25T06:11:34.345 INFO:tasks.workunit.client.0.smithi160.stdout:7/293: link d6/dc/c14 d6/d32/d46/c58 0 2024-05-25T06:11:34.346 INFO:tasks.workunit.client.1.smithi203.stdout:0/597: dwrite dd/d10/f29 [0,4194304] 0 2024-05-25T06:11:34.347 INFO:tasks.workunit.client.0.smithi160.stdout:4/340: mknod d4/d6/d21/d42/c69 0 2024-05-25T06:11:34.349 INFO:tasks.workunit.client.1.smithi203.stdout:6/907: sync 2024-05-25T06:11:34.353 INFO:tasks.workunit.client.1.smithi203.stdout:0/598: dwrite dd/f44 [4194304,4194304] 0 2024-05-25T06:11:34.353 INFO:tasks.workunit.client.0.smithi160.stdout:6/238: creat d5/d39/f51 x:0 0 0 2024-05-25T06:11:34.356 INFO:tasks.workunit.client.0.smithi160.stdout:7/294: mkdir d6/d3e/d59 0 2024-05-25T06:11:34.356 INFO:tasks.workunit.client.0.smithi160.stdout:5/333: dread db/dc/d13/d1d/d31/d3b/f2b [0,4194304] 0 2024-05-25T06:11:34.357 INFO:tasks.workunit.client.0.smithi160.stdout:0/273: dwrite f2 [0,4194304] 0 2024-05-25T06:11:34.357 INFO:tasks.workunit.client.0.smithi160.stdout:1/176: mknod de/d1b/c38 0 2024-05-25T06:11:34.358 INFO:tasks.workunit.client.1.smithi203.stdout:0/599: symlink dd/d3d/ld1 0 2024-05-25T06:11:34.360 INFO:tasks.workunit.client.0.smithi160.stdout:2/253: dwrite d12/f45 [0,4194304] 0 2024-05-25T06:11:34.361 INFO:tasks.workunit.client.1.smithi203.stdout:0/600: write dd/d3d/d59/f9b [398992,95662] 0 2024-05-25T06:11:34.362 INFO:tasks.workunit.client.0.smithi160.stdout:5/334: write db/dc/d13/d1d/d31/d3b/f41 [4887563,68369] 0 2024-05-25T06:11:34.363 INFO:tasks.workunit.client.0.smithi160.stdout:2/254: chown d12/d3e 2 1 2024-05-25T06:11:34.363 INFO:tasks.workunit.client.0.smithi160.stdout:1/177: write fa [1263442,106806] 0 2024-05-25T06:11:34.364 INFO:tasks.workunit.client.0.smithi160.stdout:6/239: symlink d5/d45/l52 0 2024-05-25T06:11:34.365 INFO:tasks.workunit.client.1.smithi203.stdout:0/601: symlink dd/d10/d34/d4c/ld2 0 2024-05-25T06:11:34.369 INFO:tasks.workunit.client.1.smithi203.stdout:6/908: write d8/d11/d15/f2e [658854,110352] 0 2024-05-25T06:11:34.369 INFO:tasks.workunit.client.1.smithi203.stdout:0/602: creat dd/d3d/d48/d91/fd3 x:0 0 0 2024-05-25T06:11:34.371 INFO:tasks.workunit.client.0.smithi160.stdout:3/212: getdents d19/d2f/d38/d43 0 2024-05-25T06:11:34.371 INFO:tasks.workunit.client.0.smithi160.stdout:8/348: getdents d5/d44/d1f/d1e/d28 0 2024-05-25T06:11:34.372 INFO:tasks.workunit.client.0.smithi160.stdout:6/240: write d5/de/f17 [915875,24377] 0 2024-05-25T06:11:34.373 INFO:tasks.workunit.client.0.smithi160.stdout:0/274: symlink d2b/d31/d35/d43/l5d 0 2024-05-25T06:11:34.373 INFO:tasks.workunit.client.1.smithi203.stdout:0/603: creat dd/d3d/d48/d91/dad/fd4 x:0 0 0 2024-05-25T06:11:34.374 INFO:tasks.workunit.client.0.smithi160.stdout:7/295: dwrite d6/d2c/f43 [0,4194304] 0 2024-05-25T06:11:34.374 INFO:tasks.workunit.client.0.smithi160.stdout:5/335: creat db/dc/d13/f65 x:0 0 0 2024-05-25T06:11:34.374 INFO:tasks.workunit.client.0.smithi160.stdout:0/275: fdatasync f12 0 2024-05-25T06:11:34.377 INFO:tasks.workunit.client.0.smithi160.stdout:1/178: readlink de/d1b/d1c/l1f 0 2024-05-25T06:11:34.378 INFO:tasks.workunit.client.0.smithi160.stdout:9/250: write d4/d11/d12/f2b [691043,18086] 0 2024-05-25T06:11:34.378 INFO:tasks.workunit.client.1.smithi203.stdout:6/909: dwrite d8/d2a/d6c/d87/f10a [0,4194304] 0 2024-05-25T06:11:34.380 INFO:tasks.workunit.client.1.smithi203.stdout:3/773: sync 2024-05-25T06:11:34.380 INFO:tasks.workunit.client.1.smithi203.stdout:1/551: sync 2024-05-25T06:11:34.380 INFO:tasks.workunit.client.1.smithi203.stdout:4/690: sync 2024-05-25T06:11:34.380 INFO:tasks.workunit.client.1.smithi203.stdout:8/778: sync 2024-05-25T06:11:34.380 INFO:tasks.workunit.client.1.smithi203.stdout:2/678: sync 2024-05-25T06:11:34.381 INFO:tasks.workunit.client.1.smithi203.stdout:0/604: truncate dd/d10/d34/d4c/d99/f82 680392 0 2024-05-25T06:11:34.382 INFO:tasks.workunit.client.0.smithi160.stdout:9/251: write d4/d11/f38 [356484,105032] 0 2024-05-25T06:11:34.382 INFO:tasks.workunit.client.1.smithi203.stdout:7/624: sync 2024-05-25T06:11:34.382 INFO:tasks.workunit.client.1.smithi203.stdout:9/571: sync 2024-05-25T06:11:34.382 INFO:tasks.workunit.client.1.smithi203.stdout:5/869: sync 2024-05-25T06:11:34.384 INFO:tasks.workunit.client.0.smithi160.stdout:2/255: link d12/d13/f15 d12/d1c/d37/d4b/f57 0 2024-05-25T06:11:34.385 INFO:tasks.workunit.client.1.smithi203.stdout:4/691: write d0/d8/d1f/d26/d3b/f8c [431904,6898] 0 2024-05-25T06:11:34.386 INFO:tasks.workunit.client.0.smithi160.stdout:0/276: fsync f15 0 2024-05-25T06:11:34.388 INFO:tasks.workunit.client.1.smithi203.stdout:9/572: fsync d0/df/d73/f53 0 2024-05-25T06:11:34.390 INFO:tasks.workunit.client.0.smithi160.stdout:8/349: mkdir d5/d44/d1f/d1e/d28/d4a/d73 0 2024-05-25T06:11:34.392 INFO:tasks.workunit.client.0.smithi160.stdout:6/241: creat d5/d45/d4a/f53 x:0 0 0 2024-05-25T06:11:34.394 INFO:tasks.workunit.client.0.smithi160.stdout:5/336: dread db/dc/d13/d1d/d23/d45/f5f [0,4194304] 0 2024-05-25T06:11:34.394 INFO:tasks.workunit.client.0.smithi160.stdout:3/213: creat d19/d2f/d38/d47/f4b x:0 0 0 2024-05-25T06:11:34.396 INFO:tasks.workunit.client.0.smithi160.stdout:2/256: dread f11 [4194304,4194304] 0 2024-05-25T06:11:34.398 INFO:tasks.workunit.client.0.smithi160.stdout:0/277: creat d2b/d58/f5e x:0 0 0 2024-05-25T06:11:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:34 smithi203 ceph-mon[36030]: pgmap v17: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 57 MiB/s rd, 231 MiB/s wr, 531 op/s 2024-05-25T06:11:34.399 INFO:tasks.workunit.client.1.smithi203.stdout:4/692: dread - d0/d9/f94 zero size 2024-05-25T06:11:34.399 INFO:tasks.workunit.client.0.smithi160.stdout:8/350: creat d5/d39/f74 x:0 0 0 2024-05-25T06:11:34.401 INFO:tasks.workunit.client.0.smithi160.stdout:9/252: mkdir d4/d11/d1e/d4b 0 2024-05-25T06:11:34.401 INFO:tasks.workunit.client.1.smithi203.stdout:6/910: dread d8/d70/d38/f4c [0,4194304] 0 2024-05-25T06:11:34.402 INFO:tasks.workunit.client.1.smithi203.stdout:4/693: fsync d0/d8/d1f/fb7 0 2024-05-25T06:11:34.404 INFO:tasks.workunit.client.0.smithi160.stdout:7/296: dwrite d6/d2b/d2d/f56 [0,4194304] 0 2024-05-25T06:11:34.405 INFO:tasks.workunit.client.1.smithi203.stdout:2/679: dwrite d2/d4/d10/fac [0,4194304] 0 2024-05-25T06:11:34.405 INFO:tasks.workunit.client.0.smithi160.stdout:1/179: dwrite f5 [4194304,4194304] 0 2024-05-25T06:11:34.406 INFO:tasks.workunit.client.0.smithi160.stdout:5/337: creat db/d47/f66 x:0 0 0 2024-05-25T06:11:34.407 INFO:tasks.workunit.client.1.smithi203.stdout:2/680: chown d2/d1a/f8b 7 1 2024-05-25T06:11:34.407 INFO:tasks.workunit.client.1.smithi203.stdout:4/694: write d0/d9/d50/d6b/fc1 [2265807,45028] 0 2024-05-25T06:11:34.408 INFO:tasks.workunit.client.0.smithi160.stdout:4/341: dwrite d4/d1c/f62 [0,4194304] 0 2024-05-25T06:11:34.410 INFO:tasks.workunit.client.1.smithi203.stdout:2/681: dread - d2/d1a/d87/fbe zero size 2024-05-25T06:11:34.410 INFO:tasks.workunit.client.1.smithi203.stdout:9/573: dwrite d0/df/d37/fb3 [0,4194304] 0 2024-05-25T06:11:34.411 INFO:tasks.workunit.client.0.smithi160.stdout:3/214: creat d19/d2f/d38/d47/f4c x:0 0 0 2024-05-25T06:11:34.411 INFO:tasks.workunit.client.0.smithi160.stdout:2/257: mknod d12/d1c/d2a/c58 0 2024-05-25T06:11:34.412 INFO:tasks.workunit.client.0.smithi160.stdout:8/351: mkdir d5/d39/d75 0 2024-05-25T06:11:34.414 INFO:tasks.workunit.client.0.smithi160.stdout:0/278: rmdir d2b/d31/d35/d36 39 2024-05-25T06:11:34.414 INFO:tasks.workunit.client.0.smithi160.stdout:0/279: fdatasync f16 0 2024-05-25T06:11:34.415 INFO:tasks.workunit.client.0.smithi160.stdout:5/338: truncate db/dc/d13/f2c 4462222 0 2024-05-25T06:11:34.419 INFO:tasks.workunit.client.0.smithi160.stdout:1/180: mkdir de/d39 0 2024-05-25T06:11:34.421 INFO:tasks.workunit.client.0.smithi160.stdout:6/242: symlink d5/de/d32/l54 0 2024-05-25T06:11:34.421 INFO:tasks.workunit.client.0.smithi160.stdout:1/181: dread - de/d21/f31 zero size 2024-05-25T06:11:34.423 INFO:tasks.workunit.client.0.smithi160.stdout:7/297: mkdir d6/d2c/d5a 0 2024-05-25T06:11:34.427 INFO:tasks.workunit.client.0.smithi160.stdout:4/342: mknod d4/d6/d5b/d66/c6a 0 2024-05-25T06:11:34.427 INFO:tasks.workunit.client.1.smithi203.stdout:3/774: dwrite d10/d16/d23/d45/f76 [0,4194304] 0 2024-05-25T06:11:34.428 INFO:tasks.workunit.client.0.smithi160.stdout:9/253: dread d4/d11/d14/f31 [0,4194304] 0 2024-05-25T06:11:34.429 INFO:tasks.workunit.client.0.smithi160.stdout:2/258: mkdir d12/d1c/d59 0 2024-05-25T06:11:34.432 INFO:tasks.workunit.client.0.smithi160.stdout:9/254: readlink d4/d11/l21 0 2024-05-25T06:11:34.434 INFO:tasks.workunit.client.0.smithi160.stdout:3/215: creat d19/d2f/d34/f4d x:0 0 0 2024-05-25T06:11:34.437 INFO:tasks.workunit.client.0.smithi160.stdout:0/280: dwrite d2b/d31/d35/f3e [0,4194304] 0 2024-05-25T06:11:34.437 INFO:tasks.workunit.client.0.smithi160.stdout:8/352: write d5/d44/f68 [705561,64867] 0 2024-05-25T06:11:34.437 INFO:tasks.workunit.client.0.smithi160.stdout:3/216: dread - d19/f45 zero size 2024-05-25T06:11:34.439 INFO:tasks.workunit.client.0.smithi160.stdout:2/259: mkdir d12/d3e/d5a 0 2024-05-25T06:11:34.443 INFO:tasks.workunit.client.0.smithi160.stdout:7/298: dwrite d6/fb [0,4194304] 0 2024-05-25T06:11:34.444 INFO:tasks.workunit.client.0.smithi160.stdout:0/281: read f20 [512818,92606] 0 2024-05-25T06:11:34.445 INFO:tasks.workunit.client.0.smithi160.stdout:7/299: readlink d6/dc/d21/l39 0 2024-05-25T06:11:34.446 INFO:tasks.workunit.client.0.smithi160.stdout:9/255: link d4/f47 d4/d11/f4c 0 2024-05-25T06:11:34.446 INFO:tasks.workunit.client.0.smithi160.stdout:6/243: creat d5/de/d32/d1b/f55 x:0 0 0 2024-05-25T06:11:34.446 INFO:tasks.workunit.client.0.smithi160.stdout:9/256: read - d4/d11/f4c zero size 2024-05-25T06:11:34.447 INFO:tasks.workunit.client.0.smithi160.stdout:2/260: dread - d12/d13/f24 zero size 2024-05-25T06:11:34.450 INFO:tasks.workunit.client.0.smithi160.stdout:7/300: stat d6/d2c 0 2024-05-25T06:11:34.450 INFO:tasks.workunit.client.0.smithi160.stdout:3/217: dread d19/f44 [0,4194304] 0 2024-05-25T06:11:34.452 INFO:tasks.workunit.client.0.smithi160.stdout:2/261: write fe [2830621,16783] 0 2024-05-25T06:11:34.454 INFO:tasks.workunit.client.1.smithi203.stdout:8/779: rename d2/l21 to d2/d5/dae/l10c 0 2024-05-25T06:11:34.454 INFO:tasks.workunit.client.0.smithi160.stdout:4/343: getdents d4/d6/d21 0 2024-05-25T06:11:34.455 INFO:tasks.workunit.client.0.smithi160.stdout:3/218: fdatasync f13 0 2024-05-25T06:11:34.455 INFO:tasks.workunit.client.0.smithi160.stdout:7/301: mkdir d6/d32/d5b 0 2024-05-25T06:11:34.456 INFO:tasks.workunit.client.1.smithi203.stdout:0/605: mkdir dd/d5c/dd5 0 2024-05-25T06:11:34.456 INFO:tasks.workunit.client.0.smithi160.stdout:6/244: rename d5/de/d32/d1b/l3c to d5/de/d18/l56 0 2024-05-25T06:11:34.458 INFO:tasks.workunit.client.1.smithi203.stdout:1/552: fdatasync d14/d57/d1d/d3e/d36/db3/d5e/f55 0 2024-05-25T06:11:34.460 INFO:tasks.workunit.client.0.smithi160.stdout:6/245: write d5/d30/d36/f49 [67128,91466] 0 2024-05-25T06:11:34.460 INFO:tasks.workunit.client.1.smithi203.stdout:5/870: unlink dc/d11/d42/d94/dad/ff7 0 2024-05-25T06:11:34.461 INFO:tasks.workunit.client.0.smithi160.stdout:9/257: truncate d4/fd 821737 0 2024-05-25T06:11:34.462 INFO:tasks.workunit.client.0.smithi160.stdout:4/344: write d4/d6/d21/f48 [359235,122948] 0 2024-05-25T06:11:34.462 INFO:tasks.workunit.client.0.smithi160.stdout:8/353: dwrite d5/d44/d1f/f22 [0,4194304] 0 2024-05-25T06:11:34.463 INFO:tasks.workunit.client.1.smithi203.stdout:6/911: unlink d8/d70/c34 0 2024-05-25T06:11:34.466 INFO:tasks.workunit.client.0.smithi160.stdout:3/219: chown c14 4 1 2024-05-25T06:11:34.467 INFO:tasks.workunit.client.0.smithi160.stdout:2/262: creat d12/d1c/d59/f5b x:0 0 0 2024-05-25T06:11:34.467 INFO:tasks.workunit.client.1.smithi203.stdout:4/695: mknod d0/d9/d50/d79/ce2 0 2024-05-25T06:11:34.467 INFO:tasks.workunit.client.0.smithi160.stdout:0/282: rename c13 to d2b/d31/d35/d36/c5f 0 2024-05-25T06:11:34.467 INFO:tasks.workunit.client.0.smithi160.stdout:7/302: write d6/d2b/d2d/f40 [432784,14436] 0 2024-05-25T06:11:34.468 INFO:tasks.workunit.client.1.smithi203.stdout:9/574: mkdir d0/df/dc5 0 2024-05-25T06:11:34.469 INFO:tasks.workunit.client.0.smithi160.stdout:9/258: read - d4/d11/d1e/d3e/f44 zero size 2024-05-25T06:11:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:34 smithi160 ceph-mon[29041]: pgmap v17: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 57 MiB/s rd, 231 MiB/s wr, 531 op/s 2024-05-25T06:11:34.470 INFO:tasks.workunit.client.1.smithi203.stdout:2/682: mkdir d2/d4/d10/d32/dca/de8 0 2024-05-25T06:11:34.470 INFO:tasks.workunit.client.0.smithi160.stdout:0/283: write f2 [66472,51212] 0 2024-05-25T06:11:34.472 INFO:tasks.workunit.client.1.smithi203.stdout:7/625: creat d1/fd9 x:0 0 0 2024-05-25T06:11:34.474 INFO:tasks.workunit.client.1.smithi203.stdout:8/780: creat d2/f10d x:0 0 0 2024-05-25T06:11:34.477 INFO:tasks.workunit.client.1.smithi203.stdout:0/606: rename dd/d3d/d59/l94 to dd/d3d/d48/d91/ld6 0 2024-05-25T06:11:34.477 INFO:tasks.workunit.client.0.smithi160.stdout:1/182: dread de/ff [4194304,4194304] 0 2024-05-25T06:11:34.478 INFO:tasks.workunit.client.0.smithi160.stdout:3/220: rename f18 to d19/d2f/d38/d43/f4e 0 2024-05-25T06:11:34.480 INFO:tasks.workunit.client.0.smithi160.stdout:4/345: dwrite d4/f2a [0,4194304] 0 2024-05-25T06:11:34.481 INFO:tasks.workunit.client.0.smithi160.stdout:5/339: dread db/dc/f15 [0,4194304] 0 2024-05-25T06:11:34.482 INFO:tasks.workunit.client.1.smithi203.stdout:1/553: rmdir d14/d81 39 2024-05-25T06:11:34.483 INFO:tasks.workunit.client.0.smithi160.stdout:2/263: mkdir d12/d5c 0 2024-05-25T06:11:34.483 INFO:tasks.workunit.client.1.smithi203.stdout:1/554: write d14/d57/d6b/da1/faa [435469,128829] 0 2024-05-25T06:11:34.486 INFO:tasks.workunit.client.1.smithi203.stdout:1/555: write d14/d57/d1d/f67 [3914372,93427] 0 2024-05-25T06:11:34.486 INFO:tasks.workunit.client.0.smithi160.stdout:3/221: mknod d19/d2f/c4f 0 2024-05-25T06:11:34.486 INFO:tasks.workunit.client.0.smithi160.stdout:5/340: read - db/d47/f66 zero size 2024-05-25T06:11:34.488 INFO:tasks.workunit.client.0.smithi160.stdout:0/284: dread d2b/d31/f38 [0,4194304] 0 2024-05-25T06:11:34.489 INFO:tasks.workunit.client.0.smithi160.stdout:3/222: stat fe 0 2024-05-25T06:11:34.490 INFO:tasks.workunit.client.0.smithi160.stdout:1/183: dread - de/d1b/d35/f24 zero size 2024-05-25T06:11:34.490 INFO:tasks.workunit.client.1.smithi203.stdout:9/575: creat d0/d17/d20/d3c/db2/fc6 x:0 0 0 2024-05-25T06:11:34.491 INFO:tasks.workunit.client.1.smithi203.stdout:9/576: stat d0/d17/d23/lbf 0 2024-05-25T06:11:34.492 INFO:tasks.workunit.client.1.smithi203.stdout:2/683: creat d2/d42/fe9 x:0 0 0 2024-05-25T06:11:34.493 INFO:tasks.workunit.client.0.smithi160.stdout:2/264: write d12/d1c/d37/f43 [100884,9673] 0 2024-05-25T06:11:34.494 INFO:tasks.workunit.client.0.smithi160.stdout:8/354: write d5/d44/d1f/d1e/d28/d4a/f5c [2848089,78526] 0 2024-05-25T06:11:34.495 INFO:tasks.workunit.client.0.smithi160.stdout:6/246: write d5/de/f22 [1954902,16674] 0 2024-05-25T06:11:34.497 INFO:tasks.workunit.client.0.smithi160.stdout:9/259: dwrite d4/d11/d1e/d24/f1b [0,4194304] 0 2024-05-25T06:11:34.499 INFO:tasks.workunit.client.0.smithi160.stdout:1/184: write f9 [1590397,96784] 0 2024-05-25T06:11:34.500 INFO:tasks.workunit.client.0.smithi160.stdout:5/341: symlink db/dc/d13/d1d/d3f/l67 0 2024-05-25T06:11:34.500 INFO:tasks.workunit.client.1.smithi203.stdout:8/781: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/f10e x:0 0 0 2024-05-25T06:11:34.501 INFO:tasks.workunit.client.0.smithi160.stdout:2/265: symlink d12/d1c/l5d 0 2024-05-25T06:11:34.502 INFO:tasks.workunit.client.1.smithi203.stdout:5/871: dread dc/d11/d17/d1a/d22/fb4 [0,4194304] 0 2024-05-25T06:11:34.503 INFO:tasks.workunit.client.1.smithi203.stdout:7/626: dwrite d1/d9/d2c/f56 [0,4194304] 0 2024-05-25T06:11:34.503 INFO:tasks.workunit.client.0.smithi160.stdout:5/342: stat f9 0 2024-05-25T06:11:34.505 INFO:tasks.workunit.client.1.smithi203.stdout:8/782: read d2/d5/d2d/fbe [1712772,118010] 0 2024-05-25T06:11:34.505 INFO:tasks.workunit.client.0.smithi160.stdout:6/247: unlink d5/de/d32/d1b/f2d 0 2024-05-25T06:11:34.506 INFO:tasks.workunit.client.0.smithi160.stdout:5/343: write db/dc/d13/f65 [251382,43438] 0 2024-05-25T06:11:34.506 INFO:tasks.workunit.client.0.smithi160.stdout:3/223: dwrite d19/d2f/d34/f4d [0,4194304] 0 2024-05-25T06:11:34.509 INFO:tasks.workunit.client.0.smithi160.stdout:5/344: stat f1 0 2024-05-25T06:11:34.509 INFO:tasks.workunit.client.0.smithi160.stdout:1/185: write f2 [2451005,114119] 0 2024-05-25T06:11:34.512 INFO:tasks.workunit.client.1.smithi203.stdout:4/696: rename d0/d8/d10/l1a to d0/d8/d3a/le3 0 2024-05-25T06:11:34.513 INFO:tasks.workunit.client.1.smithi203.stdout:4/697: chown d0/d9/d50/d6b 337328 1 2024-05-25T06:11:34.514 INFO:tasks.workunit.client.0.smithi160.stdout:1/186: creat de/f3a x:0 0 0 2024-05-25T06:11:34.521 INFO:tasks.workunit.client.0.smithi160.stdout:2/266: write f11 [3597818,43313] 0 2024-05-25T06:11:34.523 INFO:tasks.workunit.client.0.smithi160.stdout:1/187: creat de/d1b/d35/f3b x:0 0 0 2024-05-25T06:11:34.524 INFO:tasks.workunit.client.0.smithi160.stdout:8/355: dwrite d5/d44/d1f/d58/f60 [0,4194304] 0 2024-05-25T06:11:34.524 INFO:tasks.workunit.client.0.smithi160.stdout:0/285: truncate f20 3893827 0 2024-05-25T06:11:34.525 INFO:tasks.workunit.client.0.smithi160.stdout:6/248: rename d5/d39/c3e to d5/de/d32/d1b/d2b/c57 0 2024-05-25T06:11:34.529 INFO:tasks.workunit.client.0.smithi160.stdout:5/345: truncate db/dc/d13/d1d/d31/d3b/f38 923839 0 2024-05-25T06:11:34.529 INFO:tasks.workunit.client.1.smithi203.stdout:2/684: read d2/d4/d10/d32/f28 [261851,29060] 0 2024-05-25T06:11:34.529 INFO:tasks.workunit.client.1.smithi203.stdout:5/872: symlink dc/d11/d17/d1a/d2d/d4d/ded/l134 0 2024-05-25T06:11:34.530 INFO:tasks.workunit.client.0.smithi160.stdout:3/224: dwrite f13 [0,4194304] 0 2024-05-25T06:11:34.530 INFO:tasks.workunit.client.1.smithi203.stdout:1/556: dwrite d14/d57/f58 [0,4194304] 0 2024-05-25T06:11:34.530 INFO:tasks.workunit.client.0.smithi160.stdout:2/267: chown d12/l4d 0 1 2024-05-25T06:11:34.531 INFO:tasks.workunit.client.0.smithi160.stdout:6/249: readlink d5/de/d18/l56 0 2024-05-25T06:11:34.533 INFO:tasks.workunit.client.1.smithi203.stdout:8/783: creat d2/d5/d28/d38/d4e/f10f x:0 0 0 2024-05-25T06:11:34.534 INFO:tasks.workunit.client.0.smithi160.stdout:5/346: write db/dc/d13/f44 [3030891,101243] 0 2024-05-25T06:11:34.534 INFO:tasks.workunit.client.0.smithi160.stdout:6/250: readlink d5/de/l27 0 2024-05-25T06:11:34.537 INFO:tasks.workunit.client.0.smithi160.stdout:3/225: mknod d19/d2f/d38/d43/c50 0 2024-05-25T06:11:34.539 INFO:tasks.workunit.client.1.smithi203.stdout:9/577: rename d0/df/d2b/c8c to d0/d17/d20/d3c/db2/cc7 0 2024-05-25T06:11:34.541 INFO:tasks.workunit.client.0.smithi160.stdout:8/356: creat d5/d39/d75/f76 x:0 0 0 2024-05-25T06:11:34.543 INFO:tasks.workunit.client.0.smithi160.stdout:2/268: creat d12/d1c/d37/d4b/f5e x:0 0 0 2024-05-25T06:11:34.545 INFO:tasks.workunit.client.1.smithi203.stdout:3/775: getdents d10/d16/d23/d45 0 2024-05-25T06:11:34.546 INFO:tasks.workunit.client.0.smithi160.stdout:2/269: write d12/d13/f55 [552016,16168] 0 2024-05-25T06:11:34.547 INFO:tasks.workunit.client.1.smithi203.stdout:3/776: dread - d10/d16/d23/ff0 zero size 2024-05-25T06:11:34.547 INFO:tasks.workunit.client.1.smithi203.stdout:7/627: creat d1/d9/d2c/d3a/dd1/fda x:0 0 0 2024-05-25T06:11:34.547 INFO:tasks.workunit.client.0.smithi160.stdout:1/188: getdents de/d1b/d1c 0 2024-05-25T06:11:34.549 INFO:tasks.workunit.client.0.smithi160.stdout:8/357: mknod d5/d44/d1f/d58/d65/c77 0 2024-05-25T06:11:34.549 INFO:tasks.workunit.client.0.smithi160.stdout:2/270: dread - d12/d13/f24 zero size 2024-05-25T06:11:34.549 INFO:tasks.workunit.client.0.smithi160.stdout:9/260: dwrite d4/d11/d1e/d24/f39 [0,4194304] 0 2024-05-25T06:11:34.550 INFO:tasks.workunit.client.1.smithi203.stdout:9/578: dwrite d0/d17/d20/d6e/fc3 [0,4194304] 0 2024-05-25T06:11:34.550 INFO:tasks.workunit.client.0.smithi160.stdout:3/226: mknod d19/c51 0 2024-05-25T06:11:34.550 INFO:tasks.workunit.client.1.smithi203.stdout:9/579: dread - d0/df/f5f zero size 2024-05-25T06:11:34.551 INFO:tasks.workunit.client.0.smithi160.stdout:1/189: truncate de/f10 302237 0 2024-05-25T06:11:34.551 INFO:tasks.workunit.client.0.smithi160.stdout:3/227: dread - d19/d22/f48 zero size 2024-05-25T06:11:34.552 INFO:tasks.workunit.client.0.smithi160.stdout:2/271: mkdir d12/d3e/d5f 0 2024-05-25T06:11:34.553 INFO:tasks.workunit.client.0.smithi160.stdout:1/190: truncate de/f29 161691 0 2024-05-25T06:11:34.554 INFO:tasks.workunit.client.1.smithi203.stdout:5/873: truncate dc/d11/d17/d1a/d2d/d4d/d64/dd4/fdc 650658 0 2024-05-25T06:11:34.557 INFO:tasks.workunit.client.0.smithi160.stdout:1/191: write de/f13 [338176,73527] 0 2024-05-25T06:11:34.557 INFO:tasks.workunit.client.0.smithi160.stdout:8/358: unlink d5/d44/d1f/d1e/d28/d4a/d4e/l69 0 2024-05-25T06:11:34.561 INFO:tasks.workunit.client.0.smithi160.stdout:2/272: creat d12/d13/f60 x:0 0 0 2024-05-25T06:11:34.562 INFO:tasks.workunit.client.0.smithi160.stdout:2/273: chown d12/d1c/d37/f43 3938 1 2024-05-25T06:11:34.562 INFO:tasks.workunit.client.0.smithi160.stdout:2/274: chown f11 15 1 2024-05-25T06:11:34.562 INFO:tasks.workunit.client.0.smithi160.stdout:3/228: unlink d19/d2f/d38/l4a 0 2024-05-25T06:11:34.563 INFO:tasks.workunit.client.1.smithi203.stdout:1/557: mknod d14/d57/cba 0 2024-05-25T06:11:34.567 INFO:tasks.workunit.client.1.smithi203.stdout:9/580: truncate d0/f13 850278 0 2024-05-25T06:11:34.568 INFO:tasks.workunit.client.0.smithi160.stdout:9/261: creat d4/d11/d1e/f4d x:0 0 0 2024-05-25T06:11:34.568 INFO:tasks.workunit.client.0.smithi160.stdout:2/275: dread d12/f1d [0,4194304] 0 2024-05-25T06:11:34.568 INFO:tasks.workunit.client.1.smithi203.stdout:4/698: creat d0/d9/fe4 x:0 0 0 2024-05-25T06:11:34.569 INFO:tasks.workunit.client.0.smithi160.stdout:9/262: write d4/d11/d14/f37 [338959,39559] 0 2024-05-25T06:11:34.569 INFO:tasks.workunit.client.0.smithi160.stdout:2/276: chown d12/d13/c18 1887 1 2024-05-25T06:11:34.570 INFO:tasks.workunit.client.1.smithi203.stdout:6/912: write d8/d3d/f9a [3949159,129687] 0 2024-05-25T06:11:34.571 INFO:tasks.workunit.client.1.smithi203.stdout:3/777: mknod d10/d16/cfb 0 2024-05-25T06:11:34.573 INFO:tasks.workunit.client.1.smithi203.stdout:2/685: creat d2/fea x:0 0 0 2024-05-25T06:11:34.574 INFO:tasks.workunit.client.0.smithi160.stdout:1/192: creat de/d1b/d28/f3c x:0 0 0 2024-05-25T06:11:34.575 INFO:tasks.workunit.client.0.smithi160.stdout:6/251: write d5/f11 [1141484,26153] 0 2024-05-25T06:11:34.576 INFO:tasks.workunit.client.1.smithi203.stdout:0/607: dwrite dd/d10/d34/d4c/d99/d70/fb0 [0,4194304] 0 2024-05-25T06:11:34.580 INFO:tasks.workunit.client.0.smithi160.stdout:0/286: dwrite d2b/d31/d35/d36/f3c [0,4194304] 0 2024-05-25T06:11:34.580 INFO:tasks.workunit.client.0.smithi160.stdout:3/229: creat d19/f52 x:0 0 0 2024-05-25T06:11:34.581 INFO:tasks.workunit.client.0.smithi160.stdout:2/277: dread d12/f1d [0,4194304] 0 2024-05-25T06:11:34.581 INFO:tasks.workunit.client.1.smithi203.stdout:7/628: creat d1/d9/fdb x:0 0 0 2024-05-25T06:11:34.582 INFO:tasks.workunit.client.0.smithi160.stdout:3/230: stat d19/d2f/l33 0 2024-05-25T06:11:34.582 INFO:tasks.workunit.client.0.smithi160.stdout:0/287: write f1b [5206365,9714] 0 2024-05-25T06:11:34.584 INFO:tasks.workunit.client.0.smithi160.stdout:5/347: dread db/dc/d13/d1d/d23/d24/d2e/f4e [0,4194304] 0 2024-05-25T06:11:34.586 INFO:tasks.workunit.client.0.smithi160.stdout:1/193: link de/f19 de/f3d 0 2024-05-25T06:11:34.587 INFO:tasks.workunit.client.1.smithi203.stdout:3/778: dwrite d10/d22/d2a/d99/fb6 [0,4194304] 0 2024-05-25T06:11:34.588 INFO:tasks.workunit.client.0.smithi160.stdout:6/252: mkdir d5/de/d58 0 2024-05-25T06:11:34.589 INFO:tasks.workunit.client.0.smithi160.stdout:1/194: truncate de/f10 672275 0 2024-05-25T06:11:34.589 INFO:tasks.workunit.client.0.smithi160.stdout:3/231: dread d19/d22/f2a [0,4194304] 0 2024-05-25T06:11:34.591 INFO:tasks.workunit.client.1.smithi203.stdout:2/686: readlink d2/d1a/d33/l76 0 2024-05-25T06:11:34.592 INFO:tasks.workunit.client.0.smithi160.stdout:8/359: dread d5/d44/f25 [0,4194304] 0 2024-05-25T06:11:34.593 INFO:tasks.workunit.client.1.smithi203.stdout:8/784: truncate d2/f14 4519661 0 2024-05-25T06:11:34.593 INFO:tasks.workunit.client.1.smithi203.stdout:2/687: write d2/d4/d10/f21 [1490152,3963] 0 2024-05-25T06:11:34.594 INFO:tasks.workunit.client.1.smithi203.stdout:5/874: link dc/d11/d18/l88 dc/d11/d17/d1a/dda/l135 0 2024-05-25T06:11:34.597 INFO:tasks.workunit.client.0.smithi160.stdout:5/348: creat db/d47/f68 x:0 0 0 2024-05-25T06:11:34.598 INFO:tasks.workunit.client.1.smithi203.stdout:0/608: mknod dd/d3d/d75/cd7 0 2024-05-25T06:11:34.599 INFO:tasks.workunit.client.1.smithi203.stdout:1/558: dread d14/f25 [4194304,4194304] 0 2024-05-25T06:11:34.600 INFO:tasks.workunit.client.1.smithi203.stdout:6/913: mkdir d8/d11/dfa/d124 0 2024-05-25T06:11:34.600 INFO:tasks.workunit.client.1.smithi203.stdout:0/609: write dd/d45/f5b [2092724,48866] 0 2024-05-25T06:11:34.600 INFO:tasks.workunit.client.0.smithi160.stdout:1/195: creat de/d1b/d1c/f3e x:0 0 0 2024-05-25T06:11:34.601 INFO:tasks.workunit.client.1.smithi203.stdout:9/581: write d0/d17/d20/d29/f8a [395856,71132] 0 2024-05-25T06:11:34.602 INFO:tasks.workunit.client.0.smithi160.stdout:1/196: readlink de/l18 0 2024-05-25T06:11:34.605 INFO:tasks.workunit.client.1.smithi203.stdout:9/582: write d0/d17/f33 [6055238,103148] 0 2024-05-25T06:11:34.605 INFO:tasks.workunit.client.1.smithi203.stdout:4/699: link d0/f3 d0/d8/d1f/d26/d3b/d9c/fe5 0 2024-05-25T06:11:34.607 INFO:tasks.workunit.client.0.smithi160.stdout:2/278: truncate f10 299675 0 2024-05-25T06:11:34.609 INFO:tasks.workunit.client.1.smithi203.stdout:3/779: dread d10/d22/f57 [0,4194304] 0 2024-05-25T06:11:34.610 INFO:tasks.workunit.client.1.smithi203.stdout:8/785: write d2/d5/d18/f1b [5937274,97143] 0 2024-05-25T06:11:34.612 INFO:tasks.workunit.client.0.smithi160.stdout:6/253: rmdir d5/de 39 2024-05-25T06:11:34.612 INFO:tasks.workunit.client.1.smithi203.stdout:2/688: fsync d2/d38/d2f/d69/fd7 0 2024-05-25T06:11:34.614 INFO:tasks.workunit.client.0.smithi160.stdout:3/232: dwrite d19/f27 [0,4194304] 0 2024-05-25T06:11:34.617 INFO:tasks.workunit.client.0.smithi160.stdout:3/233: dread - d19/f52 zero size 2024-05-25T06:11:34.617 INFO:tasks.workunit.client.1.smithi203.stdout:6/914: creat d8/d2a/d63/f125 x:0 0 0 2024-05-25T06:11:34.618 INFO:tasks.workunit.client.0.smithi160.stdout:0/288: rmdir d2b/d31/d35/d36/d52/d54 0 2024-05-25T06:11:34.618 INFO:tasks.workunit.client.0.smithi160.stdout:5/349: dwrite db/f63 [4194304,4194304] 0 2024-05-25T06:11:34.621 INFO:tasks.workunit.client.1.smithi203.stdout:1/559: stat d14/d57/d29/c93 0 2024-05-25T06:11:34.623 INFO:tasks.workunit.client.1.smithi203.stdout:1/560: write d14/d57/d1d/d3e/d36/f59 [55995,6542] 0 2024-05-25T06:11:34.623 INFO:tasks.workunit.client.0.smithi160.stdout:0/289: dread - d2b/d31/d35/f49 zero size 2024-05-25T06:11:34.624 INFO:tasks.workunit.client.0.smithi160.stdout:3/234: readlink d19/l1f 0 2024-05-25T06:11:34.627 INFO:tasks.workunit.client.0.smithi160.stdout:0/290: fsync d2b/d58/f5e 0 2024-05-25T06:11:34.627 INFO:tasks.workunit.client.1.smithi203.stdout:6/915: dwrite d8/d70/d38/f39 [0,4194304] 0 2024-05-25T06:11:34.628 INFO:tasks.workunit.client.0.smithi160.stdout:6/254: stat d5/de/d32/d1b/f55 0 2024-05-25T06:11:34.628 INFO:tasks.workunit.client.0.smithi160.stdout:2/279: dwrite d12/d1c/d37/f43 [0,4194304] 0 2024-05-25T06:11:34.634 INFO:tasks.workunit.client.1.smithi203.stdout:6/916: dread - d8/d11/d1b/d75/dcb/dcf/d46/f11f zero size 2024-05-25T06:11:34.635 INFO:tasks.workunit.client.0.smithi160.stdout:6/255: fdatasync d5/de/d32/d35/f4e 0 2024-05-25T06:11:34.635 INFO:tasks.workunit.client.0.smithi160.stdout:8/360: dread d5/fe [0,4194304] 0 2024-05-25T06:11:34.635 INFO:tasks.workunit.client.0.smithi160.stdout:1/197: link de/d1b/f1d de/d1b/d1c/f3f 0 2024-05-25T06:11:34.636 INFO:tasks.workunit.client.1.smithi203.stdout:2/689: fsync d2/f6c 0 2024-05-25T06:11:34.638 INFO:tasks.workunit.client.0.smithi160.stdout:2/280: write d12/d3e/f41 [382526,15158] 0 2024-05-25T06:11:34.639 INFO:tasks.workunit.client.1.smithi203.stdout:5/875: link dc/d3c/d3f/fae dc/d11/d17/d1a/d2d/d4d/d101/f136 0 2024-05-25T06:11:34.639 INFO:tasks.workunit.client.0.smithi160.stdout:0/291: creat d2b/d58/f60 x:0 0 0 2024-05-25T06:11:34.640 INFO:tasks.workunit.client.0.smithi160.stdout:6/256: read d5/d30/d36/f49 [68781,83641] 0 2024-05-25T06:11:34.640 INFO:tasks.workunit.client.0.smithi160.stdout:3/235: creat d19/d2f/d38/d43/f53 x:0 0 0 2024-05-25T06:11:34.643 INFO:tasks.workunit.client.1.smithi203.stdout:7/629: dread d1/d9/d2c/d3a/f3e [0,4194304] 0 2024-05-25T06:11:34.645 INFO:tasks.workunit.client.0.smithi160.stdout:2/281: unlink d12/d1c/d59/f5b 0 2024-05-25T06:11:34.645 INFO:tasks.workunit.client.0.smithi160.stdout:5/350: write f4 [2185473,31503] 0 2024-05-25T06:11:34.646 INFO:tasks.workunit.client.1.smithi203.stdout:8/786: link d2/d5/d18/d31/f5e d2/d5/d28/d38/d4e/d97/f110 0 2024-05-25T06:11:34.646 INFO:tasks.workunit.client.0.smithi160.stdout:5/351: chown db/dc/d13/d1d/d31/d3b/c43 206011 1 2024-05-25T06:11:34.647 INFO:tasks.workunit.client.1.smithi203.stdout:8/787: dread - d2/d5/d28/d38/fa6 zero size 2024-05-25T06:11:34.648 INFO:tasks.workunit.client.0.smithi160.stdout:0/292: creat d2b/d31/d35/d36/f61 x:0 0 0 2024-05-25T06:11:34.648 INFO:tasks.workunit.client.1.smithi203.stdout:8/788: dread - d2/d5/d28/ff9 zero size 2024-05-25T06:11:34.649 INFO:tasks.workunit.client.0.smithi160.stdout:0/293: read d2b/d31/d35/d36/f4a [3672323,94442] 0 2024-05-25T06:11:34.649 INFO:tasks.workunit.client.0.smithi160.stdout:3/236: creat d19/d2f/d38/d47/f54 x:0 0 0 2024-05-25T06:11:34.649 INFO:tasks.workunit.client.0.smithi160.stdout:8/361: read d5/d44/d1f/d1e/d28/d4a/f3d [151826,73048] 0 2024-05-25T06:11:34.650 INFO:tasks.workunit.client.1.smithi203.stdout:8/789: dread - d2/d5/d18/d31/dac/dc2/d88/fea zero size 2024-05-25T06:11:34.651 INFO:tasks.workunit.client.1.smithi203.stdout:1/561: mkdir d14/d57/dbb 0 2024-05-25T06:11:34.652 INFO:tasks.workunit.client.0.smithi160.stdout:2/282: rename d12/d1c/f32 to d12/d3c/f61 0 2024-05-25T06:11:34.654 INFO:tasks.workunit.client.0.smithi160.stdout:5/352: rmdir db/dc/d13/d1d/d23/d45 39 2024-05-25T06:11:34.654 INFO:tasks.workunit.client.1.smithi203.stdout:5/876: fdatasync dc/d11/d17/d1a/d2d/d53/da2/fd6 0 2024-05-25T06:11:34.655 INFO:tasks.workunit.client.0.smithi160.stdout:6/257: mknod d5/de/d32/d35/d40/c59 0 2024-05-25T06:11:34.655 INFO:tasks.workunit.client.1.smithi203.stdout:9/583: dwrite d0/d17/d20/d3c/f43 [0,4194304] 0 2024-05-25T06:11:34.656 INFO:tasks.workunit.client.1.smithi203.stdout:4/700: getdents d0/d8/d1f/d26/d3b 0 2024-05-25T06:11:34.657 INFO:tasks.workunit.client.0.smithi160.stdout:6/258: write d5/f34 [1196264,97210] 0 2024-05-25T06:11:34.657 INFO:tasks.workunit.client.1.smithi203.stdout:8/790: fdatasync d2/d5/d10/d45/faa 0 2024-05-25T06:11:34.658 INFO:tasks.workunit.client.0.smithi160.stdout:0/294: creat d2b/d31/d35/d36/d52/f62 x:0 0 0 2024-05-25T06:11:34.659 INFO:tasks.workunit.client.0.smithi160.stdout:2/283: creat d12/d3e/f62 x:0 0 0 2024-05-25T06:11:34.660 INFO:tasks.workunit.client.1.smithi203.stdout:1/562: symlink d14/d57/d1d/d3e/d36/lbc 0 2024-05-25T06:11:34.660 INFO:tasks.workunit.client.0.smithi160.stdout:3/237: dwrite d19/d2f/d38/d43/f53 [0,4194304] 0 2024-05-25T06:11:34.660 INFO:tasks.workunit.client.0.smithi160.stdout:2/284: chown d12/d1c/d37 0 1 2024-05-25T06:11:34.661 INFO:tasks.workunit.client.1.smithi203.stdout:8/791: chown d2/d5/d28/f3a 0 1 2024-05-25T06:11:34.662 INFO:tasks.workunit.client.0.smithi160.stdout:8/362: dread d5/d9/f24 [0,4194304] 0 2024-05-25T06:11:34.664 INFO:tasks.workunit.client.1.smithi203.stdout:5/877: mknod dc/d11/d17/d1a/d22/c137 0 2024-05-25T06:11:34.665 INFO:tasks.workunit.client.0.smithi160.stdout:5/353: mkdir db/dc/d13/d1d/d31/d3b/d69 0 2024-05-25T06:11:34.665 INFO:tasks.workunit.client.1.smithi203.stdout:4/701: rmdir d0/d8/d1f/d26 39 2024-05-25T06:11:34.665 INFO:tasks.workunit.client.0.smithi160.stdout:6/259: creat d5/d30/f5a x:0 0 0 2024-05-25T06:11:34.666 INFO:tasks.workunit.client.0.smithi160.stdout:5/354: readlink db/dc/d13/d1d/d31/l4b 0 2024-05-25T06:11:34.668 INFO:tasks.workunit.client.1.smithi203.stdout:4/702: stat d0/d8/f44 0 2024-05-25T06:11:34.668 INFO:tasks.workunit.client.0.smithi160.stdout:5/355: write db/dc/d13/d1d/d31/d3b/f41 [3030152,14160] 0 2024-05-25T06:11:34.669 INFO:tasks.workunit.client.0.smithi160.stdout:5/356: chown db/dc/d13/f1f 0 1 2024-05-25T06:11:34.669 INFO:tasks.workunit.client.1.smithi203.stdout:6/917: getdents d8/d11/d1b/d75/dcb/dcf/d46 0 2024-05-25T06:11:34.670 INFO:tasks.workunit.client.0.smithi160.stdout:8/363: symlink d5/d44/d1f/d1e/d28/d49/l78 0 2024-05-25T06:11:34.672 INFO:tasks.workunit.client.1.smithi203.stdout:6/918: readlink d8/d11/d15/l7d 0 2024-05-25T06:11:34.672 INFO:tasks.workunit.client.1.smithi203.stdout:5/878: creat dc/d11/d42/d94/d10a/f138 x:0 0 0 2024-05-25T06:11:34.673 INFO:tasks.workunit.client.0.smithi160.stdout:8/364: write d5/d44/d1f/d1e/d28/f3f [618493,12391] 0 2024-05-25T06:11:34.674 INFO:tasks.workunit.client.0.smithi160.stdout:2/285: dread d12/d1c/d37/f3b [0,4194304] 0 2024-05-25T06:11:34.675 INFO:tasks.workunit.client.1.smithi203.stdout:8/792: truncate d2/d5/d18/d31/f50 2975460 0 2024-05-25T06:11:34.675 INFO:tasks.workunit.client.1.smithi203.stdout:1/563: dwrite d14/d57/d6b/f8e [0,4194304] 0 2024-05-25T06:11:34.676 INFO:tasks.workunit.client.0.smithi160.stdout:3/238: getdents d19/d2f 0 2024-05-25T06:11:34.676 INFO:tasks.workunit.client.1.smithi203.stdout:3/780: write d10/d16/d23/d4a/fe5 [93109,39948] 0 2024-05-25T06:11:34.679 INFO:tasks.workunit.client.0.smithi160.stdout:8/365: unlink d5/d39/f74 0 2024-05-25T06:11:34.680 INFO:tasks.workunit.client.0.smithi160.stdout:5/357: dwrite db/dc/d13/f65 [0,4194304] 0 2024-05-25T06:11:34.682 INFO:tasks.workunit.client.0.smithi160.stdout:8/366: chown d5/d44/d1f/d1e/d28/l2b 127265 1 2024-05-25T06:11:34.683 INFO:tasks.workunit.client.1.smithi203.stdout:3/781: write d10/d16/d23/d67/f6f [953229,6597] 0 2024-05-25T06:11:34.683 INFO:tasks.workunit.client.0.smithi160.stdout:3/239: creat d19/d2f/d34/f55 x:0 0 0 2024-05-25T06:11:34.685 INFO:tasks.workunit.client.1.smithi203.stdout:5/879: write dc/d11/d17/d1a/d2d/d53/da2/f118 [455298,95001] 0 2024-05-25T06:11:34.686 INFO:tasks.workunit.client.0.smithi160.stdout:2/286: link d12/f25 d12/d3e/d5f/f63 0 2024-05-25T06:11:34.687 INFO:tasks.workunit.client.1.smithi203.stdout:4/703: dwrite d0/d8/d3a/f46 [0,4194304] 0 2024-05-25T06:11:34.689 INFO:tasks.workunit.client.0.smithi160.stdout:5/358: creat db/dc/d13/f6a x:0 0 0 2024-05-25T06:11:34.690 INFO:tasks.workunit.client.0.smithi160.stdout:3/240: symlink d19/d2f/d38/l56 0 2024-05-25T06:11:34.691 INFO:tasks.workunit.client.1.smithi203.stdout:6/919: dread d8/d2a/f107 [0,4194304] 0 2024-05-25T06:11:34.696 INFO:tasks.workunit.client.1.smithi203.stdout:9/584: dread d0/f21 [0,4194304] 0 2024-05-25T06:11:34.697 INFO:tasks.workunit.client.1.smithi203.stdout:7/630: dread d1/d9/d2c/d3a/d4e/f37 [0,4194304] 0 2024-05-25T06:11:34.698 INFO:tasks.workunit.client.1.smithi203.stdout:6/920: chown d8/d5a/d9b/ce8 1026 1 2024-05-25T06:11:34.700 INFO:tasks.workunit.client.1.smithi203.stdout:3/782: write d10/d16/d23/d4a/d88/f95 [1174956,78317] 0 2024-05-25T06:11:34.701 INFO:tasks.workunit.client.1.smithi203.stdout:7/631: readlink d1/d9/d2c/d3a/d4e/l67 0 2024-05-25T06:11:34.702 INFO:tasks.workunit.client.0.smithi160.stdout:2/287: dwrite d12/d1c/f27 [0,4194304] 0 2024-05-25T06:11:34.702 INFO:tasks.workunit.client.0.smithi160.stdout:3/241: creat d19/d2f/d38/d47/f57 x:0 0 0 2024-05-25T06:11:34.704 INFO:tasks.workunit.client.0.smithi160.stdout:5/359: dwrite db/dc/f15 [0,4194304] 0 2024-05-25T06:11:34.706 INFO:tasks.workunit.client.0.smithi160.stdout:8/367: link d5/d44/d1f/d1e/c45 d5/d44/d1f/d6b/c79 0 2024-05-25T06:11:34.706 INFO:tasks.workunit.client.1.smithi203.stdout:8/793: dread d2/d5/d28/d38/d4e/f51 [0,4194304] 0 2024-05-25T06:11:34.706 INFO:tasks.workunit.client.1.smithi203.stdout:6/921: truncate d8/d70/d38/f39 4541110 0 2024-05-25T06:11:34.707 INFO:tasks.workunit.client.1.smithi203.stdout:0/610: dwrite dd/d10/d34/f67 [0,4194304] 0 2024-05-25T06:11:34.714 INFO:tasks.workunit.client.0.smithi160.stdout:1/198: write de/f17 [58963,96993] 0 2024-05-25T06:11:34.715 INFO:tasks.workunit.client.0.smithi160.stdout:1/199: chown c8 1493 1 2024-05-25T06:11:34.716 INFO:tasks.workunit.client.0.smithi160.stdout:3/242: rename d19/d2f/d38/d43/c50 to d19/d22/c58 0 2024-05-25T06:11:34.717 INFO:tasks.workunit.client.1.smithi203.stdout:2/690: dwrite d2/d1a/f8b [0,4194304] 0 2024-05-25T06:11:34.719 INFO:tasks.workunit.client.0.smithi160.stdout:8/368: dwrite d5/d44/f68 [0,4194304] 0 2024-05-25T06:11:34.721 INFO:tasks.workunit.client.0.smithi160.stdout:8/369: write d5/d7/f19 [3943728,104963] 0 2024-05-25T06:11:34.721 INFO:tasks.workunit.client.1.smithi203.stdout:4/704: symlink d0/d8/d52/d59/le6 0 2024-05-25T06:11:34.725 INFO:tasks.workunit.client.1.smithi203.stdout:2/691: creat d2/d1a/db0/feb x:0 0 0 2024-05-25T06:11:34.726 INFO:tasks.workunit.client.0.smithi160.stdout:2/288: mknod d12/d3e/c64 0 2024-05-25T06:11:34.727 INFO:tasks.workunit.client.0.smithi160.stdout:3/243: symlink d19/d2f/d34/l59 0 2024-05-25T06:11:34.729 INFO:tasks.workunit.client.0.smithi160.stdout:2/289: dread d12/d1c/d37/f3b [0,4194304] 0 2024-05-25T06:11:34.730 INFO:tasks.workunit.client.1.smithi203.stdout:6/922: dwrite d8/d2a/d63/fe9 [0,4194304] 0 2024-05-25T06:11:34.733 INFO:tasks.workunit.client.0.smithi160.stdout:1/200: rename de/ff to de/d1b/d35/f40 0 2024-05-25T06:11:34.734 INFO:tasks.workunit.client.0.smithi160.stdout:3/244: mknod d19/d2f/d38/c5a 0 2024-05-25T06:11:34.735 INFO:tasks.workunit.client.0.smithi160.stdout:2/290: symlink d12/d13/l65 0 2024-05-25T06:11:34.738 INFO:tasks.workunit.client.1.smithi203.stdout:1/564: dwrite d14/d23/f78 [0,4194304] 0 2024-05-25T06:11:34.738 INFO:tasks.workunit.client.1.smithi203.stdout:3/783: rename d10/d16/d23/d38/lf3 to d10/d16/lfc 0 2024-05-25T06:11:34.741 INFO:tasks.workunit.client.0.smithi160.stdout:8/370: chown d5/d44/d1f/d1e/c45 21943736 1 2024-05-25T06:11:34.741 INFO:tasks.workunit.client.1.smithi203.stdout:8/794: getdents d2 0 2024-05-25T06:11:34.744 INFO:tasks.workunit.client.0.smithi160.stdout:3/245: mkdir d19/d2f/d38/d43/d5b 0 2024-05-25T06:11:34.744 INFO:tasks.workunit.client.1.smithi203.stdout:4/705: unlink d0/d9/d50/d79/l73 0 2024-05-25T06:11:34.745 INFO:tasks.workunit.client.0.smithi160.stdout:1/201: read de/f19 [3651143,37985] 0 2024-05-25T06:11:34.747 INFO:tasks.workunit.client.1.smithi203.stdout:6/923: creat d8/d11/dfa/d124/f126 x:0 0 0 2024-05-25T06:11:34.748 INFO:tasks.workunit.client.1.smithi203.stdout:4/706: stat d0/d9/lbb 0 2024-05-25T06:11:34.750 INFO:tasks.workunit.client.0.smithi160.stdout:6/260: dwrite d5/de/ff [0,4194304] 0 2024-05-25T06:11:34.751 INFO:tasks.workunit.client.1.smithi203.stdout:4/707: write d0/d8/d3a/f46 [2455267,17614] 0 2024-05-25T06:11:34.751 INFO:tasks.workunit.client.1.smithi203.stdout:3/784: getdents d10/d16/d71/dfa 0 2024-05-25T06:11:34.752 INFO:tasks.workunit.client.1.smithi203.stdout:8/795: creat d2/d5/d10/d45/d90/d100/f111 x:0 0 0 2024-05-25T06:11:34.754 INFO:tasks.workunit.client.1.smithi203.stdout:8/796: chown d2/d5/d18/d31/f34 1415037257 1 2024-05-25T06:11:34.755 INFO:tasks.workunit.client.1.smithi203.stdout:8/797: truncate d2/f10d 24213 0 2024-05-25T06:11:34.756 INFO:tasks.workunit.client.0.smithi160.stdout:1/202: mkdir de/d1b/d1c/d41 0 2024-05-25T06:11:34.757 INFO:tasks.workunit.client.1.smithi203.stdout:8/798: write d2/d5/d28/d38/d4e/fbb [2714123,36590] 0 2024-05-25T06:11:34.758 INFO:tasks.workunit.client.1.smithi203.stdout:2/692: rename d2/d38/d43/d60/ca5 to d2/cec 0 2024-05-25T06:11:34.759 INFO:tasks.workunit.client.0.smithi160.stdout:0/295: dwrite f20 [0,4194304] 0 2024-05-25T06:11:34.759 INFO:tasks.workunit.client.0.smithi160.stdout:3/246: rename d19/d2f/f37 to d19/d2f/d34/f5c 0 2024-05-25T06:11:34.761 INFO:tasks.workunit.client.1.smithi203.stdout:4/708: chown d0/d8/d1f/d26/d3b/c4e 9184 1 2024-05-25T06:11:34.761 INFO:tasks.workunit.client.1.smithi203.stdout:3/785: creat d10/d16/d23/d67/ffd x:0 0 0 2024-05-25T06:11:34.761 INFO:tasks.workunit.client.0.smithi160.stdout:1/203: creat de/d1b/d35/f42 x:0 0 0 2024-05-25T06:11:34.762 INFO:tasks.workunit.client.1.smithi203.stdout:4/709: chown d0/d8/d10/d91/fba 2 1 2024-05-25T06:11:34.766 INFO:tasks.workunit.client.0.smithi160.stdout:0/296: mkdir d2b/d46/d63 0 2024-05-25T06:11:34.766 INFO:tasks.workunit.client.1.smithi203.stdout:2/693: creat d2/d1a/d33/d80/fed x:0 0 0 2024-05-25T06:11:34.767 INFO:tasks.workunit.client.0.smithi160.stdout:2/291: read f10 [62056,124685] 0 2024-05-25T06:11:34.767 INFO:tasks.workunit.client.1.smithi203.stdout:8/799: mknod d2/d5/d28/d38/d4e/c112 0 2024-05-25T06:11:34.768 INFO:tasks.workunit.client.1.smithi203.stdout:2/694: write d2/d1a/fc7 [865636,3840] 0 2024-05-25T06:11:34.769 INFO:tasks.workunit.client.0.smithi160.stdout:6/261: creat d5/de/d58/f5b x:0 0 0 2024-05-25T06:11:34.769 INFO:tasks.workunit.client.1.smithi203.stdout:1/565: rename d14/d57/d1d/c6d to d14/d57/d97/da4/cbd 0 2024-05-25T06:11:34.771 INFO:tasks.workunit.client.0.smithi160.stdout:5/360: dread f1 [0,4194304] 0 2024-05-25T06:11:34.773 INFO:tasks.workunit.client.0.smithi160.stdout:0/297: mknod d2b/d31/d35/c64 0 2024-05-25T06:11:34.774 INFO:tasks.workunit.client.1.smithi203.stdout:2/695: dread d2/d4/d10/fac [0,4194304] 0 2024-05-25T06:11:34.775 INFO:tasks.workunit.client.1.smithi203.stdout:1/566: unlink d14/d57/d1d/f66 0 2024-05-25T06:11:34.775 INFO:tasks.workunit.client.0.smithi160.stdout:2/292: symlink d12/d1c/d2a/l66 0 2024-05-25T06:11:34.775 INFO:tasks.workunit.client.0.smithi160.stdout:6/262: creat d5/d30/d36/f5c x:0 0 0 2024-05-25T06:11:34.776 INFO:tasks.workunit.client.0.smithi160.stdout:1/204: dread de/d1b/d35/f40 [4194304,4194304] 0 2024-05-25T06:11:34.777 INFO:tasks.workunit.client.0.smithi160.stdout:2/293: write d12/d3e/f62 [39025,15440] 0 2024-05-25T06:11:34.778 INFO:tasks.workunit.client.0.smithi160.stdout:0/298: read d2b/f42 [1441521,98647] 0 2024-05-25T06:11:34.778 INFO:tasks.workunit.client.0.smithi160.stdout:6/263: creat d5/d45/d4a/f5d x:0 0 0 2024-05-25T06:11:34.779 INFO:tasks.workunit.client.0.smithi160.stdout:2/294: write d12/d1c/f2b [327822,56616] 0 2024-05-25T06:11:34.780 INFO:tasks.workunit.client.0.smithi160.stdout:5/361: mknod db/dc/d13/d1d/c6b 0 2024-05-25T06:11:34.781 INFO:tasks.workunit.client.1.smithi203.stdout:8/800: mkdir d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/d113 0 2024-05-25T06:11:34.783 INFO:tasks.workunit.client.1.smithi203.stdout:6/924: link d8/d2a/d6c/lef d8/d11/d15/d28/d52/d43/d73/db7/ddc/l127 0 2024-05-25T06:11:34.784 INFO:tasks.workunit.client.1.smithi203.stdout:1/567: dread d14/d57/d6b/f8e [0,4194304] 0 2024-05-25T06:11:34.785 INFO:tasks.workunit.client.0.smithi160.stdout:5/362: readlink db/d47/l49 0 2024-05-25T06:11:34.786 INFO:tasks.workunit.client.1.smithi203.stdout:2/696: dwrite d2/d1a/db0/feb [0,4194304] 0 2024-05-25T06:11:34.786 INFO:tasks.workunit.client.1.smithi203.stdout:4/710: link d0/d9/d50/d6b/f4c d0/d8/d10/d91/fe7 0 2024-05-25T06:11:34.786 INFO:tasks.workunit.client.0.smithi160.stdout:5/363: chown db/dc/d13/d1d 185227815 1 2024-05-25T06:11:34.786 INFO:tasks.workunit.client.0.smithi160.stdout:2/295: symlink d12/d3e/d5a/l67 0 2024-05-25T06:11:34.789 INFO:tasks.workunit.client.0.smithi160.stdout:6/264: dwrite d5/de/f42 [0,4194304] 0 2024-05-25T06:11:34.790 INFO:tasks.workunit.client.1.smithi203.stdout:5/880: truncate dc/d11/d18/f37 7819552 0 2024-05-25T06:11:34.790 INFO:tasks.workunit.client.0.smithi160.stdout:2/296: chown d12/d1c/f1e 1500323468 1 2024-05-25T06:11:34.791 INFO:tasks.workunit.client.0.smithi160.stdout:2/297: read - d12/d1c/d37/d49/f4e zero size 2024-05-25T06:11:34.792 INFO:tasks.workunit.client.0.smithi160.stdout:5/364: unlink db/dc/d13/d1d/d3f/l67 0 2024-05-25T06:11:34.792 INFO:tasks.workunit.client.0.smithi160.stdout:2/298: chown d12/d1c/f2c 1213673 1 2024-05-25T06:11:34.793 INFO:tasks.workunit.client.1.smithi203.stdout:2/697: chown d2/d4/d10/d32/fb6 688 1 2024-05-25T06:11:34.794 INFO:tasks.workunit.client.0.smithi160.stdout:6/265: mknod d5/de/d32/d1b/d2b/c5e 0 2024-05-25T06:11:34.796 INFO:tasks.workunit.client.1.smithi203.stdout:8/801: unlink d2/d5/l32 0 2024-05-25T06:11:34.796 INFO:tasks.workunit.client.0.smithi160.stdout:2/299: fdatasync d12/d1c/f31 0 2024-05-25T06:11:34.801 INFO:tasks.workunit.client.1.smithi203.stdout:1/568: creat d14/d81/fbe x:0 0 0 2024-05-25T06:11:34.802 INFO:tasks.workunit.client.0.smithi160.stdout:6/266: creat d5/de/d32/d35/f5f x:0 0 0 2024-05-25T06:11:34.802 INFO:tasks.workunit.client.1.smithi203.stdout:2/698: write d2/d1a/fc7 [412677,90724] 0 2024-05-25T06:11:34.804 INFO:tasks.workunit.client.0.smithi160.stdout:5/365: dread db/dc/d13/d1d/d23/f2d [0,4194304] 0 2024-05-25T06:11:34.806 INFO:tasks.workunit.client.0.smithi160.stdout:2/300: chown d12/c29 320 1 2024-05-25T06:11:34.815 INFO:tasks.workunit.client.1.smithi203.stdout:7/632: dwrite d1/d9/d2c/d3a/f49 [4194304,4194304] 0 2024-05-25T06:11:34.816 INFO:tasks.workunit.client.0.smithi160.stdout:2/301: creat d12/d3e/f68 x:0 0 0 2024-05-25T06:11:34.817 INFO:tasks.workunit.client.1.smithi203.stdout:6/925: dwrite d8/d5a/d9b/dcc/fe7 [0,4194304] 0 2024-05-25T06:11:34.817 INFO:tasks.workunit.client.1.smithi203.stdout:4/711: dwrite d0/d8/d1f/d26/d3b/fac [0,4194304] 0 2024-05-25T06:11:34.821 INFO:tasks.workunit.client.1.smithi203.stdout:7/633: chown d1/d9/d2c/d62/d9b/fc9 637 1 2024-05-25T06:11:34.822 INFO:tasks.workunit.client.1.smithi203.stdout:7/634: readlink d1/d11/l2f 0 2024-05-25T06:11:34.822 INFO:tasks.workunit.client.1.smithi203.stdout:0/611: dwrite dd/d10/d34/d4c/d99/f82 [0,4194304] 0 2024-05-25T06:11:34.822 INFO:tasks.workunit.client.0.smithi160.stdout:2/302: symlink d12/d1c/d2a/l69 0 2024-05-25T06:11:34.824 INFO:tasks.workunit.client.0.smithi160.stdout:2/303: fsync d12/d3e/f62 0 2024-05-25T06:11:34.828 INFO:tasks.workunit.client.1.smithi203.stdout:3/786: write d10/d22/f35 [493867,38825] 0 2024-05-25T06:11:34.831 INFO:tasks.workunit.client.0.smithi160.stdout:2/304: mknod d12/d3e/c6a 0 2024-05-25T06:11:34.834 INFO:tasks.workunit.client.1.smithi203.stdout:4/712: dwrite d0/d9/d50/fc5 [0,4194304] 0 2024-05-25T06:11:34.835 INFO:tasks.workunit.client.0.smithi160.stdout:8/371: dwrite d5/d44/d1f/d1e/d28/d49/f70 [4194304,4194304] 0 2024-05-25T06:11:34.837 INFO:tasks.workunit.client.0.smithi160.stdout:8/372: write d5/d7/f19 [2814122,102787] 0 2024-05-25T06:11:34.837 INFO:tasks.workunit.client.1.smithi203.stdout:4/713: fsync d0/d9/d9d/fc6 0 2024-05-25T06:11:34.838 INFO:tasks.workunit.client.1.smithi203.stdout:3/787: write d10/d16/d50/fc2 [140714,70956] 0 2024-05-25T06:11:34.838 INFO:tasks.workunit.client.1.smithi203.stdout:7/635: unlink f0 0 2024-05-25T06:11:34.841 INFO:tasks.workunit.client.1.smithi203.stdout:4/714: write d0/d9/fa4 [7835358,39282] 0 2024-05-25T06:11:34.842 INFO:tasks.workunit.client.0.smithi160.stdout:2/305: rename d12/d1c/d37/d49/f4e to d12/d5c/f6b 0 2024-05-25T06:11:34.844 INFO:tasks.workunit.client.0.smithi160.stdout:8/373: mkdir d5/d39/d7a 0 2024-05-25T06:11:34.845 INFO:tasks.workunit.client.1.smithi203.stdout:6/926: dread d8/d3d/f6a [0,4194304] 0 2024-05-25T06:11:34.846 INFO:tasks.workunit.client.0.smithi160.stdout:0/299: dread f10 [4194304,4194304] 0 2024-05-25T06:11:34.847 INFO:tasks.workunit.client.0.smithi160.stdout:1/205: truncate de/d1b/f27 918928 0 2024-05-25T06:11:34.848 INFO:tasks.workunit.client.1.smithi203.stdout:1/569: symlink d14/d57/dbb/lbf 0 2024-05-25T06:11:34.849 INFO:tasks.workunit.client.0.smithi160.stdout:2/306: creat d12/d1c/f6c x:0 0 0 2024-05-25T06:11:34.850 INFO:tasks.workunit.client.1.smithi203.stdout:0/612: unlink dd/d10/d34/f67 0 2024-05-25T06:11:34.850 INFO:tasks.workunit.client.1.smithi203.stdout:6/927: fdatasync d8/d11/d47/fbe 0 2024-05-25T06:11:34.851 INFO:tasks.workunit.client.1.smithi203.stdout:5/881: getdents dc/d11/d17/d1a/d72/de4 0 2024-05-25T06:11:34.854 INFO:tasks.workunit.client.1.smithi203.stdout:8/802: rename d2/d5/d10/d11/lff to d2/d5/d18/d31/dac/dc2/d8a/l114 0 2024-05-25T06:11:34.854 INFO:tasks.workunit.client.0.smithi160.stdout:0/300: mknod d2b/d31/c65 0 2024-05-25T06:11:34.854 INFO:tasks.workunit.client.1.smithi203.stdout:0/613: read dd/d10/f31 [2712625,82011] 0 2024-05-25T06:11:34.855 INFO:tasks.workunit.client.1.smithi203.stdout:1/570: readlink d14/d57/d1d/d3e/l52 0 2024-05-25T06:11:34.856 INFO:tasks.workunit.client.0.smithi160.stdout:2/307: creat d12/d1c/d59/f6d x:0 0 0 2024-05-25T06:11:34.856 INFO:tasks.workunit.client.0.smithi160.stdout:1/206: dread de/d1b/f2b [0,4194304] 0 2024-05-25T06:11:34.857 INFO:tasks.workunit.client.1.smithi203.stdout:5/882: creat dc/d11/d17/d1a/d2d/d53/da2/f139 x:0 0 0 2024-05-25T06:11:34.857 INFO:tasks.workunit.client.1.smithi203.stdout:8/803: read d2/d5/d10/d45/d5d/d68/f7d [2155260,32401] 0 2024-05-25T06:11:34.858 INFO:tasks.workunit.client.0.smithi160.stdout:2/308: read f10 [35397,62681] 0 2024-05-25T06:11:34.858 INFO:tasks.workunit.client.1.smithi203.stdout:7/636: creat d1/d9/d2c/d36/fdc x:0 0 0 2024-05-25T06:11:34.859 INFO:tasks.workunit.client.1.smithi203.stdout:5/883: write dc/d11/d17/d1a/fc1 [159327,127391] 0 2024-05-25T06:11:34.860 INFO:tasks.workunit.client.1.smithi203.stdout:2/699: rename d2/d4/d10/dc8/ld6 to d2/d4/d10/d58/da0/lee 0 2024-05-25T06:11:34.861 INFO:tasks.workunit.client.0.smithi160.stdout:0/301: rename f10 to d2b/d31/d35/d41/f66 0 2024-05-25T06:11:34.863 INFO:tasks.workunit.client.1.smithi203.stdout:6/928: symlink d8/d11/d15/d28/d52/l128 0 2024-05-25T06:11:34.863 INFO:tasks.workunit.client.1.smithi203.stdout:5/884: chown dc/d11/d17/d1a/d22/c137 2 1 2024-05-25T06:11:34.863 INFO:tasks.workunit.client.1.smithi203.stdout:2/700: chown d2/d38/d2f/fa7 44863 1 2024-05-25T06:11:34.863 INFO:tasks.workunit.client.0.smithi160.stdout:2/309: dread d12/d13/f55 [0,4194304] 0 2024-05-25T06:11:34.864 INFO:tasks.workunit.client.0.smithi160.stdout:0/302: dread - d2b/d58/f60 zero size 2024-05-25T06:11:34.864 INFO:tasks.workunit.client.0.smithi160.stdout:1/207: fdatasync fb 0 2024-05-25T06:11:34.864 INFO:tasks.workunit.client.1.smithi203.stdout:2/701: fsync d2/d38/f6e 0 2024-05-25T06:11:34.864 INFO:tasks.workunit.client.0.smithi160.stdout:8/374: getdents d5/d52 0 2024-05-25T06:11:34.865 INFO:tasks.workunit.client.1.smithi203.stdout:8/804: write d2/d5/d10/d45/d5d/d68/f7d [2816050,74880] 0 2024-05-25T06:11:34.865 INFO:tasks.workunit.client.1.smithi203.stdout:2/702: chown d2/d1a/d87 314978 1 2024-05-25T06:11:34.866 INFO:tasks.workunit.client.0.smithi160.stdout:1/208: write de/f16 [115734,16111] 0 2024-05-25T06:11:34.868 INFO:tasks.workunit.client.0.smithi160.stdout:0/303: mkdir d2b/d58/d67 0 2024-05-25T06:11:34.868 INFO:tasks.workunit.client.0.smithi160.stdout:6/267: dread d5/f26 [0,4194304] 0 2024-05-25T06:11:34.868 INFO:tasks.workunit.client.1.smithi203.stdout:3/788: rename d10/cbd to d10/d16/d23/d6c/cfe 0 2024-05-25T06:11:34.868 INFO:tasks.workunit.client.1.smithi203.stdout:7/637: dread d1/d9/d2c/d62/d9b/fd2 [0,4194304] 0 2024-05-25T06:11:34.868 INFO:tasks.workunit.client.1.smithi203.stdout:2/703: fsync d2/d1a/d89/fd2 0 2024-05-25T06:11:34.868 INFO:tasks.workunit.client.1.smithi203.stdout:6/929: truncate d8/d11/d47/f99 145135 0 2024-05-25T06:11:34.869 INFO:tasks.workunit.client.1.smithi203.stdout:7/638: chown d1/c87 1303351 1 2024-05-25T06:11:34.869 INFO:tasks.workunit.client.0.smithi160.stdout:6/268: write d5/f11 [2606695,21171] 0 2024-05-25T06:11:34.870 INFO:tasks.workunit.client.0.smithi160.stdout:2/310: creat d12/f6e x:0 0 0 2024-05-25T06:11:34.870 INFO:tasks.workunit.client.1.smithi203.stdout:7/639: dread - d1/d11/f80 zero size 2024-05-25T06:11:34.871 INFO:tasks.workunit.client.0.smithi160.stdout:0/304: chown d2b/d31/d35/d43/f4c 26664 1 2024-05-25T06:11:34.871 INFO:tasks.workunit.client.1.smithi203.stdout:8/805: mknod d2/d5/d10/d11/de/c115 0 2024-05-25T06:11:34.874 INFO:tasks.workunit.client.0.smithi160.stdout:1/209: dread de/f10 [0,4194304] 0 2024-05-25T06:11:34.874 INFO:tasks.workunit.client.1.smithi203.stdout:0/614: rename dd to dd/d3d/d4f/dd8 22 2024-05-25T06:11:34.875 INFO:tasks.workunit.client.0.smithi160.stdout:2/311: creat d12/d1c/f6f x:0 0 0 2024-05-25T06:11:34.875 INFO:tasks.workunit.client.0.smithi160.stdout:5/366: dread db/dc/d13/d1d/d23/d24/d2e/f2f [0,4194304] 0 2024-05-25T06:11:34.876 INFO:tasks.workunit.client.0.smithi160.stdout:6/269: mkdir d5/de/d32/d1b/d60 0 2024-05-25T06:11:34.876 INFO:tasks.workunit.client.1.smithi203.stdout:6/930: mkdir d8/d3d/d129 0 2024-05-25T06:11:34.876 INFO:tasks.workunit.client.0.smithi160.stdout:0/305: truncate f12 3495659 0 2024-05-25T06:11:34.877 INFO:tasks.workunit.client.0.smithi160.stdout:1/210: truncate de/d1b/d35/f3b 163288 0 2024-05-25T06:11:34.878 INFO:tasks.workunit.client.0.smithi160.stdout:6/270: fsync d5/de/fd 0 2024-05-25T06:11:34.879 INFO:tasks.workunit.client.0.smithi160.stdout:5/367: truncate db/dc/f15 4197207 0 2024-05-25T06:11:34.884 INFO:tasks.workunit.client.0.smithi160.stdout:6/271: write d5/f3f [694466,22994] 0 2024-05-25T06:11:34.888 INFO:tasks.workunit.client.1.smithi203.stdout:6/931: symlink d8/d70/d38/d108/l12a 0 2024-05-25T06:11:34.889 INFO:tasks.workunit.client.0.smithi160.stdout:0/306: dwrite d2b/f30 [0,4194304] 0 2024-05-25T06:11:34.889 INFO:tasks.workunit.client.0.smithi160.stdout:1/211: dwrite de/d1b/d35/f42 [0,4194304] 0 2024-05-25T06:11:34.890 INFO:tasks.workunit.client.0.smithi160.stdout:1/212: dread - de/d21/f31 zero size 2024-05-25T06:11:34.891 INFO:tasks.workunit.client.1.smithi203.stdout:4/715: dread d0/d8/d3a/f56 [0,4194304] 0 2024-05-25T06:11:34.893 INFO:tasks.workunit.client.0.smithi160.stdout:5/368: getdents db/dc/d13 0 2024-05-25T06:11:34.893 INFO:tasks.workunit.client.0.smithi160.stdout:6/272: mkdir d5/de/d32/d1b/d60/d61 0 2024-05-25T06:11:34.893 INFO:tasks.workunit.client.1.smithi203.stdout:0/615: creat dd/d5c/dd5/fd9 x:0 0 0 2024-05-25T06:11:34.895 INFO:tasks.workunit.client.1.smithi203.stdout:6/932: dwrite d8/d5a/d9b/fd4 [0,4194304] 0 2024-05-25T06:11:34.895 INFO:tasks.workunit.client.0.smithi160.stdout:1/213: dread de/d1b/f1d [0,4194304] 0 2024-05-25T06:11:34.896 INFO:tasks.workunit.client.1.smithi203.stdout:6/933: read d8/d2a/f107 [33961,115318] 0 2024-05-25T06:11:34.902 INFO:tasks.workunit.client.0.smithi160.stdout:1/214: rmdir de/d1b 39 2024-05-25T06:11:34.903 INFO:tasks.workunit.client.1.smithi203.stdout:3/789: creat d10/d22/fff x:0 0 0 2024-05-25T06:11:34.904 INFO:tasks.workunit.client.0.smithi160.stdout:5/369: fsync f1 0 2024-05-25T06:11:34.906 INFO:tasks.workunit.client.1.smithi203.stdout:7/640: link d1/f32 d1/d9/d2c/d3a/dd1/fdd 0 2024-05-25T06:11:34.907 INFO:tasks.workunit.client.0.smithi160.stdout:6/273: symlink d5/de/d32/d35/d40/l62 0 2024-05-25T06:11:34.907 INFO:tasks.workunit.client.0.smithi160.stdout:1/215: chown de/d1b/d35/f24 7 1 2024-05-25T06:11:34.908 INFO:tasks.workunit.client.0.smithi160.stdout:5/370: write db/f22 [3842251,85047] 0 2024-05-25T06:11:34.908 INFO:tasks.workunit.client.1.smithi203.stdout:4/716: unlink d0/d8/d10/d91/da2/fe1 0 2024-05-25T06:11:34.908 INFO:tasks.workunit.client.1.smithi203.stdout:1/571: dwrite d14/d57/d1d/f2f [4194304,4194304] 0 2024-05-25T06:11:34.909 INFO:tasks.workunit.client.0.smithi160.stdout:2/312: dwrite d12/d1c/f1e [0,4194304] 0 2024-05-25T06:11:34.909 INFO:tasks.workunit.client.0.smithi160.stdout:0/307: getdents d2b/d58 0 2024-05-25T06:11:34.910 INFO:tasks.workunit.client.0.smithi160.stdout:5/371: chown db/f22 59011820 1 2024-05-25T06:11:34.914 INFO:tasks.workunit.client.1.smithi203.stdout:0/616: symlink dd/d3d/d48/lda 0 2024-05-25T06:11:34.915 INFO:tasks.workunit.client.1.smithi203.stdout:6/934: mkdir d8/d11/d1b/d12b 0 2024-05-25T06:11:34.915 INFO:tasks.workunit.client.1.smithi203.stdout:5/885: dwrite f1 [0,4194304] 0 2024-05-25T06:11:34.916 INFO:tasks.workunit.client.1.smithi203.stdout:3/790: dwrite d10/d16/d23/d6c/f63 [0,4194304] 0 2024-05-25T06:11:34.917 INFO:tasks.workunit.client.0.smithi160.stdout:2/313: write d12/d1c/f6c [642838,39456] 0 2024-05-25T06:11:34.917 INFO:tasks.workunit.client.0.smithi160.stdout:6/274: rename d5/de/d32/d1b/d60 to d5/d45/d63 0 2024-05-25T06:11:34.917 INFO:tasks.workunit.client.1.smithi203.stdout:1/572: symlink d14/d81/lc0 0 2024-05-25T06:11:34.919 INFO:tasks.workunit.client.1.smithi203.stdout:1/573: write d14/d57/d1d/d3e/d36/f59 [959037,43003] 0 2024-05-25T06:11:34.920 INFO:tasks.workunit.client.0.smithi160.stdout:6/275: write d5/f11 [3825219,103952] 0 2024-05-25T06:11:34.924 INFO:tasks.workunit.client.1.smithi203.stdout:4/717: mknod d0/d9/d50/d79/d9a/ce8 0 2024-05-25T06:11:34.925 INFO:tasks.workunit.client.1.smithi203.stdout:1/574: chown d14/l21 31300 1 2024-05-25T06:11:34.926 INFO:tasks.workunit.client.0.smithi160.stdout:1/216: mkdir de/d43 0 2024-05-25T06:11:34.929 INFO:tasks.workunit.client.0.smithi160.stdout:6/276: write f2 [3750101,54720] 0 2024-05-25T06:11:34.934 INFO:tasks.workunit.client.1.smithi203.stdout:8/806: dwrite d2/d5/d28/f3a [0,4194304] 0 2024-05-25T06:11:34.934 INFO:tasks.workunit.client.0.smithi160.stdout:5/372: dwrite db/f53 [0,4194304] 0 2024-05-25T06:11:34.935 INFO:tasks.workunit.client.0.smithi160.stdout:1/217: chown de/l18 762 1 2024-05-25T06:11:34.938 INFO:tasks.workunit.client.1.smithi203.stdout:1/575: rename d14/d57/d1d/d3e/d36/lbc to d14/d57/d29/d94/lc1 0 2024-05-25T06:11:34.939 INFO:tasks.workunit.client.1.smithi203.stdout:4/718: dwrite d0/d9/d50/d79/f40 [0,4194304] 0 2024-05-25T06:11:34.940 INFO:tasks.workunit.client.0.smithi160.stdout:0/308: dwrite f20 [0,4194304] 0 2024-05-25T06:11:34.940 INFO:tasks.workunit.client.0.smithi160.stdout:5/373: write db/dc/f15 [4708560,365] 0 2024-05-25T06:11:34.940 INFO:tasks.workunit.client.1.smithi203.stdout:1/576: stat d14/d57/d7e 0 2024-05-25T06:11:34.942 INFO:tasks.workunit.client.1.smithi203.stdout:0/617: rmdir dd/d3d/d59/db8 0 2024-05-25T06:11:34.942 INFO:tasks.workunit.client.1.smithi203.stdout:3/791: dwrite d10/d16/d23/d67/fae [0,4194304] 0 2024-05-25T06:11:34.942 INFO:tasks.workunit.client.1.smithi203.stdout:1/577: chown d14/d81/c9c 45 1 2024-05-25T06:11:34.943 INFO:tasks.workunit.client.1.smithi203.stdout:0/618: chown dd/d3a/f46 178600831 1 2024-05-25T06:11:34.944 INFO:tasks.workunit.client.0.smithi160.stdout:6/277: creat d5/d45/d63/d61/f64 x:0 0 0 2024-05-25T06:11:34.950 INFO:tasks.workunit.client.0.smithi160.stdout:6/278: write d5/d30/d36/f5c [116005,129827] 0 2024-05-25T06:11:34.951 INFO:tasks.workunit.client.0.smithi160.stdout:1/218: rename de/d1b/d28/l32 to de/d39/l44 0 2024-05-25T06:11:34.951 INFO:tasks.workunit.client.1.smithi203.stdout:8/807: creat d2/d5/d10/f116 x:0 0 0 2024-05-25T06:11:34.951 INFO:tasks.workunit.client.0.smithi160.stdout:6/279: write d5/f14 [6554382,103106] 0 2024-05-25T06:11:34.952 INFO:tasks.workunit.client.0.smithi160.stdout:0/309: dwrite d2b/d31/d35/f3e [0,4194304] 0 2024-05-25T06:11:34.952 INFO:tasks.workunit.client.1.smithi203.stdout:5/886: link dc/d11/d17/d1a/d2d/d53/c9a dc/d11/d17/d1a/d103/d98/c13a 0 2024-05-25T06:11:34.953 INFO:tasks.workunit.client.1.smithi203.stdout:3/792: dwrite d10/fb5 [0,4194304] 0 2024-05-25T06:11:34.955 INFO:tasks.workunit.client.1.smithi203.stdout:0/619: unlink dd/d3d/d48/d91/fd3 0 2024-05-25T06:11:34.955 INFO:tasks.workunit.client.1.smithi203.stdout:3/793: stat d10/d16/d50/d8b/ca6 0 2024-05-25T06:11:34.956 INFO:tasks.workunit.client.1.smithi203.stdout:0/620: chown dd/d10/d34/d4c/fd0 963 1 2024-05-25T06:11:34.963 INFO:tasks.workunit.client.0.smithi160.stdout:1/219: mkdir de/d1b/d1c/d41/d45 0 2024-05-25T06:11:34.963 INFO:tasks.workunit.client.0.smithi160.stdout:0/310: dread d2b/d31/d35/d36/d37/f44 [0,4194304] 0 2024-05-25T06:11:34.965 INFO:tasks.workunit.client.0.smithi160.stdout:5/374: mknod db/dc/d13/d1d/d3f/c6c 0 2024-05-25T06:11:34.968 INFO:tasks.workunit.client.1.smithi203.stdout:4/719: rmdir d0/d8 39 2024-05-25T06:11:34.970 INFO:tasks.workunit.client.0.smithi160.stdout:5/375: symlink db/dc/d13/d1d/d3f/d4a/l6d 0 2024-05-25T06:11:34.976 INFO:tasks.workunit.client.1.smithi203.stdout:4/720: symlink d0/d8/d3a/le9 0 2024-05-25T06:11:34.979 INFO:tasks.workunit.client.0.smithi160.stdout:5/376: dwrite db/f5d [0,4194304] 0 2024-05-25T06:11:34.980 INFO:tasks.workunit.client.1.smithi203.stdout:4/721: creat d0/d8/d10/d91/fea x:0 0 0 2024-05-25T06:11:34.982 INFO:tasks.workunit.client.0.smithi160.stdout:5/377: write db/d47/f66 [901545,54374] 0 2024-05-25T06:11:34.983 INFO:tasks.workunit.client.0.smithi160.stdout:0/311: dwrite d2b/d31/d35/d43/f4c [0,4194304] 0 2024-05-25T06:11:34.988 INFO:tasks.workunit.client.0.smithi160.stdout:5/378: mkdir db/d47/d5e/d6e 0 2024-05-25T06:11:34.989 INFO:tasks.workunit.client.0.smithi160.stdout:5/379: write db/f2a [7262670,112239] 0 2024-05-25T06:11:34.995 INFO:tasks.workunit.client.1.smithi203.stdout:4/722: dread d0/d8/d52/d59/d7a/d83/fd6 [0,4194304] 0 2024-05-25T06:11:34.995 INFO:tasks.workunit.client.1.smithi203.stdout:6/935: write d8/d11/d15/f79 [114586,44520] 0 2024-05-25T06:11:34.997 INFO:tasks.workunit.client.0.smithi160.stdout:5/380: symlink db/dc/d13/d1d/d3f/l6f 0 2024-05-25T06:11:35.001 INFO:tasks.workunit.client.0.smithi160.stdout:5/381: unlink db/dc/d13/d1d/d3f/d4a/l6d 0 2024-05-25T06:11:35.002 INFO:tasks.workunit.client.0.smithi160.stdout:5/382: write f7 [2034689,92003] 0 2024-05-25T06:11:35.003 INFO:tasks.workunit.client.1.smithi203.stdout:4/723: creat d0/d8/d3a/d81/d93/d98/d99/feb x:0 0 0 2024-05-25T06:11:35.006 INFO:tasks.workunit.client.1.smithi203.stdout:4/724: symlink d0/d8/d52/d59/d7a/lec 0 2024-05-25T06:11:35.007 INFO:tasks.workunit.client.1.smithi203.stdout:0/621: dread dd/d10/f31 [0,4194304] 0 2024-05-25T06:11:35.007 INFO:tasks.workunit.client.1.smithi203.stdout:4/725: rmdir d0/d9/d9d 39 2024-05-25T06:11:35.009 INFO:tasks.workunit.client.1.smithi203.stdout:0/622: creat dd/d3d/d4f/d62/d78/fdb x:0 0 0 2024-05-25T06:11:35.014 INFO:tasks.workunit.client.1.smithi203.stdout:0/623: link dd/d32/c35 dd/d3a/d66/d86/cdc 0 2024-05-25T06:11:35.016 INFO:tasks.workunit.client.1.smithi203.stdout:1/578: rename d14/d57/d1d/d3e to d14/d57/d29/d94/db8/dc2 0 2024-05-25T06:11:35.020 INFO:tasks.workunit.client.0.smithi160.stdout:1/220: write de/d1b/f2b [527533,93288] 0 2024-05-25T06:11:35.020 INFO:tasks.workunit.client.0.smithi160.stdout:1/221: fdatasync de/f16 0 2024-05-25T06:11:35.021 INFO:tasks.workunit.client.1.smithi203.stdout:5/887: rename dc/d11/d17/d1a/d72/de4/f121 to dc/d11/d18/dc0/d107/f13b 0 2024-05-25T06:11:35.022 INFO:tasks.workunit.client.0.smithi160.stdout:0/312: truncate d2b/d31/d35/d43/f4c 2185520 0 2024-05-25T06:11:35.023 INFO:tasks.workunit.client.1.smithi203.stdout:6/936: dread d8/fa0 [0,4194304] 0 2024-05-25T06:11:35.024 INFO:tasks.workunit.client.1.smithi203.stdout:1/579: dwrite d14/d57/d6b/da1/faa [0,4194304] 0 2024-05-25T06:11:35.024 INFO:tasks.workunit.client.1.smithi203.stdout:5/888: readlink dc/d11/d17/d1a/d2d/d53/da2/l7f 0 2024-05-25T06:11:35.025 INFO:tasks.workunit.client.1.smithi203.stdout:6/937: fdatasync d8/d11/d1b/d75/dcb/fcd 0 2024-05-25T06:11:35.026 INFO:tasks.workunit.client.1.smithi203.stdout:0/624: creat dd/d10/fdd x:0 0 0 2024-05-25T06:11:35.027 INFO:tasks.workunit.client.1.smithi203.stdout:1/580: write d14/d57/f58 [125182,8937] 0 2024-05-25T06:11:35.028 INFO:tasks.workunit.client.1.smithi203.stdout:3/794: truncate d10/d22/d2a/d99/fb6 3690497 0 2024-05-25T06:11:35.030 INFO:tasks.workunit.client.1.smithi203.stdout:6/938: creat d8/d2a/d63/f12c x:0 0 0 2024-05-25T06:11:35.031 INFO:tasks.workunit.client.1.smithi203.stdout:4/726: write d0/d9/d50/dce/f7d [416491,122898] 0 2024-05-25T06:11:35.031 INFO:tasks.workunit.client.0.smithi160.stdout:5/383: truncate db/dc/d13/f1f 2061267 0 2024-05-25T06:11:35.031 INFO:tasks.workunit.client.1.smithi203.stdout:1/581: chown d14/d57/d97/da4/dad 6422062 1 2024-05-25T06:11:35.032 INFO:tasks.workunit.client.0.smithi160.stdout:5/384: dread - db/d47/f68 zero size 2024-05-25T06:11:35.032 INFO:tasks.workunit.client.1.smithi203.stdout:0/625: mkdir dd/d5c/dd5/dde 0 2024-05-25T06:11:35.033 INFO:tasks.workunit.client.0.smithi160.stdout:0/313: rmdir d2b 39 2024-05-25T06:11:35.033 INFO:tasks.workunit.client.1.smithi203.stdout:3/795: read d10/d16/d23/d67/f94 [3873865,40174] 0 2024-05-25T06:11:35.034 INFO:tasks.workunit.client.0.smithi160.stdout:1/222: write de/d1b/d1c/f3f [912779,67790] 0 2024-05-25T06:11:35.035 INFO:tasks.workunit.client.0.smithi160.stdout:0/314: readlink d2b/d46/l56 0 2024-05-25T06:11:35.035 INFO:tasks.workunit.client.1.smithi203.stdout:5/889: link dc/d11/d17/d1a/d22/c36 dc/d11/d17/d1a/d2d/d53/da2/c13c 0 2024-05-25T06:11:35.036 INFO:tasks.workunit.client.0.smithi160.stdout:0/315: truncate d2b/d46/f5a 524225 0 2024-05-25T06:11:35.037 INFO:tasks.workunit.client.1.smithi203.stdout:1/582: chown d14/d57/d29/d94/db8/dc2/d36/db3/d5e/l86 1015 1 2024-05-25T06:11:35.037 INFO:tasks.workunit.client.0.smithi160.stdout:5/385: rename db/d47/d5e/f62 to db/dc/d13/d1d/d31/d3b/f70 0 2024-05-25T06:11:35.040 INFO:tasks.workunit.client.1.smithi203.stdout:5/890: creat dc/d11/d106/f13d x:0 0 0 2024-05-25T06:11:35.042 INFO:tasks.workunit.client.1.smithi203.stdout:1/583: unlink d14/d57/d1d/faf 0 2024-05-25T06:11:35.043 INFO:tasks.workunit.client.0.smithi160.stdout:5/386: rename db/dc/d13/d1d/d23/d24/d2e/l36 to db/dc/d13/d1d/d31/d3b/d69/l71 0 2024-05-25T06:11:35.045 INFO:tasks.workunit.client.1.smithi203.stdout:5/891: mknod dc/d11/d17/d1a/d72/c13e 0 2024-05-25T06:11:35.049 INFO:tasks.workunit.client.0.smithi160.stdout:0/316: truncate f2 2788754 0 2024-05-25T06:11:35.050 INFO:tasks.workunit.client.1.smithi203.stdout:4/727: rename d0/d8/d1f/d26/d3b/d9c/dd8 to d0/d8/d3a/d81/d93/d98/d99/ded 0 2024-05-25T06:11:35.051 INFO:tasks.workunit.client.0.smithi160.stdout:0/317: read f1f [237904,75274] 0 2024-05-25T06:11:35.053 INFO:tasks.workunit.client.0.smithi160.stdout:1/223: dwrite de/f3d [0,4194304] 0 2024-05-25T06:11:35.053 INFO:tasks.workunit.client.0.smithi160.stdout:5/387: getdents db/dc/d13/d1d/d23/d24/d2e 0 2024-05-25T06:11:35.055 INFO:tasks.workunit.client.1.smithi203.stdout:5/892: dwrite dc/f80 [0,4194304] 0 2024-05-25T06:11:35.056 INFO:tasks.workunit.client.1.smithi203.stdout:6/939: dread d8/f12 [0,4194304] 0 2024-05-25T06:11:35.056 INFO:tasks.workunit.client.0.smithi160.stdout:0/318: mkdir d2b/d58/d68 0 2024-05-25T06:11:35.058 INFO:tasks.workunit.client.0.smithi160.stdout:0/319: dread - d2b/f2e zero size 2024-05-25T06:11:35.058 INFO:tasks.workunit.client.0.smithi160.stdout:0/320: chown f1e 2598 1 2024-05-25T06:11:35.058 INFO:tasks.workunit.client.1.smithi203.stdout:0/626: truncate dd/d10/d34/f3c 2384782 0 2024-05-25T06:11:35.058 INFO:tasks.workunit.client.1.smithi203.stdout:5/893: stat dc/d11 0 2024-05-25T06:11:35.059 INFO:tasks.workunit.client.0.smithi160.stdout:5/388: fsync db/dc/d13/d1d/f20 0 2024-05-25T06:11:35.068 INFO:tasks.workunit.client.0.smithi160.stdout:0/321: link d2b/d46/c4b d2b/d31/d35/d36/d37/c69 0 2024-05-25T06:11:35.069 INFO:tasks.workunit.client.1.smithi203.stdout:4/728: getdents d0/d9 0 2024-05-25T06:11:35.070 INFO:tasks.workunit.client.1.smithi203.stdout:5/894: write dc/d11/d17/d1a/d2d/d4d/dcc/f126 [26777,86917] 0 2024-05-25T06:11:35.070 INFO:tasks.workunit.client.1.smithi203.stdout:0/627: write dd/d5c/dd5/fd9 [729523,86721] 0 2024-05-25T06:11:35.070 INFO:tasks.workunit.client.0.smithi160.stdout:5/389: getdents db/dc/d13/d1d/d23 0 2024-05-25T06:11:35.071 INFO:tasks.workunit.client.1.smithi203.stdout:3/796: dwrite d10/d16/d23/fdd [0,4194304] 0 2024-05-25T06:11:35.071 INFO:tasks.workunit.client.1.smithi203.stdout:6/940: read d8/d11/d1b/d75/dcb/dcf/d46/fa4 [560168,109094] 0 2024-05-25T06:11:35.075 INFO:tasks.workunit.client.0.smithi160.stdout:1/224: dread de/d1b/d35/f40 [0,4194304] 0 2024-05-25T06:11:35.076 INFO:tasks.workunit.client.0.smithi160.stdout:5/390: mkdir db/dc/d13/d1d/d3f/d72 0 2024-05-25T06:11:35.076 INFO:tasks.workunit.client.0.smithi160.stdout:1/225: fsync de/f29 0 2024-05-25T06:11:35.076 INFO:tasks.workunit.client.1.smithi203.stdout:0/628: mknod dd/d10/d34/d4c/cdf 0 2024-05-25T06:11:35.081 INFO:tasks.workunit.client.0.smithi160.stdout:5/391: mknod db/dc/d13/d1d/d31/d56/c73 0 2024-05-25T06:11:35.081 INFO:tasks.workunit.client.1.smithi203.stdout:5/895: rename dc/d11/d17/d1a/d2d/d4d/d64 to dc/d11/d17/d1a/d2d/d4d/ded/d13f 0 2024-05-25T06:11:35.082 INFO:tasks.workunit.client.0.smithi160.stdout:1/226: rename de/f17 to de/d43/f46 0 2024-05-25T06:11:35.083 INFO:tasks.workunit.client.1.smithi203.stdout:0/629: stat dd/d32/c3e 0 2024-05-25T06:11:35.084 INFO:tasks.workunit.client.0.smithi160.stdout:1/227: write de/d1b/f1d [1676046,120584] 0 2024-05-25T06:11:35.085 INFO:tasks.workunit.client.1.smithi203.stdout:3/797: dwrite d10/d16/d23/d45/fcd [0,4194304] 0 2024-05-25T06:11:35.086 INFO:tasks.workunit.client.0.smithi160.stdout:5/392: mknod db/dc/d13/d1d/d23/d45/c74 0 2024-05-25T06:11:35.086 INFO:tasks.workunit.client.1.smithi203.stdout:4/729: rename d0/d9/fa4 to d0/d9/d9d/fee 0 2024-05-25T06:11:35.088 INFO:tasks.workunit.client.1.smithi203.stdout:5/896: fdatasync dc/d11/d42/fce 0 2024-05-25T06:11:35.089 INFO:tasks.workunit.client.1.smithi203.stdout:6/941: dwrite d8/d11/d15/d28/d52/d43/d73/fb5 [0,4194304] 0 2024-05-25T06:11:35.090 INFO:tasks.workunit.client.0.smithi160.stdout:5/393: mknod db/dc/d13/d1d/d3f/d4a/c75 0 2024-05-25T06:11:35.090 INFO:tasks.workunit.client.1.smithi203.stdout:0/630: symlink dd/d10/d34/le0 0 2024-05-25T06:11:35.090 INFO:tasks.workunit.client.1.smithi203.stdout:3/798: write d10/d16/d23/d45/fcd [3438511,108361] 0 2024-05-25T06:11:35.091 INFO:tasks.workunit.client.0.smithi160.stdout:0/322: write f12 [2069843,63518] 0 2024-05-25T06:11:35.091 INFO:tasks.workunit.client.0.smithi160.stdout:5/394: chown db/lf 449470571 1 2024-05-25T06:11:35.096 INFO:tasks.workunit.client.0.smithi160.stdout:5/395: rmdir db/d47/d5e 39 2024-05-25T06:11:35.097 INFO:tasks.workunit.client.0.smithi160.stdout:5/396: write db/d47/f51 [1017082,37908] 0 2024-05-25T06:11:35.100 INFO:tasks.workunit.client.1.smithi203.stdout:6/942: creat d8/d5a/d9b/daf/d116/f12d x:0 0 0 2024-05-25T06:11:35.102 INFO:tasks.workunit.client.0.smithi160.stdout:0/323: rmdir d2b/d58/d68 0 2024-05-25T06:11:35.103 INFO:tasks.workunit.client.1.smithi203.stdout:6/943: mkdir d8/d5a/d12e 0 2024-05-25T06:11:35.103 INFO:tasks.workunit.client.0.smithi160.stdout:0/324: read - d2b/d31/d35/d36/d52/f62 zero size 2024-05-25T06:11:35.107 INFO:tasks.workunit.client.0.smithi160.stdout:0/325: getdents d2b/d31/d35/d36/d37 0 2024-05-25T06:11:35.107 INFO:tasks.workunit.client.0.smithi160.stdout:0/326: chown c26 539090757 1 2024-05-25T06:11:35.109 INFO:tasks.workunit.client.1.smithi203.stdout:5/897: write dc/d11/d17/d1a/d103/fd9 [117851,81399] 0 2024-05-25T06:11:35.113 INFO:tasks.workunit.client.1.smithi203.stdout:5/898: unlink dc/d11/d17/d1a/d2d/d4d/ded/d13f/fa1 0 2024-05-25T06:11:35.114 INFO:tasks.workunit.client.0.smithi160.stdout:0/327: dwrite d2b/f57 [4194304,4194304] 0 2024-05-25T06:11:35.116 INFO:tasks.workunit.client.0.smithi160.stdout:0/328: chown d2b/c2d 46 1 2024-05-25T06:11:35.118 INFO:tasks.workunit.client.1.smithi203.stdout:5/899: rmdir dc/d11/d42/d94/d10a 39 2024-05-25T06:11:35.119 INFO:tasks.workunit.client.0.smithi160.stdout:0/329: mkdir d2b/d58/d67/d6a 0 2024-05-25T06:11:35.119 INFO:tasks.workunit.client.0.smithi160.stdout:0/330: write f12 [4246500,85144] 0 2024-05-25T06:11:35.120 INFO:tasks.workunit.client.1.smithi203.stdout:4/730: dread d0/d8/d1f/d26/d3b/f7b [0,4194304] 0 2024-05-25T06:11:35.123 INFO:tasks.workunit.client.0.smithi160.stdout:0/331: getdents d2b/d46/d63 0 2024-05-25T06:11:35.123 INFO:tasks.workunit.client.1.smithi203.stdout:6/944: dwrite d8/d70/fa1 [0,4194304] 0 2024-05-25T06:11:35.125 INFO:tasks.workunit.client.1.smithi203.stdout:3/799: dwrite d10/d22/f33 [0,4194304] 0 2024-05-25T06:11:35.125 INFO:tasks.workunit.client.1.smithi203.stdout:5/900: truncate dc/d11/d17/d44/fbe 473568 0 2024-05-25T06:11:35.127 INFO:tasks.workunit.client.0.smithi160.stdout:0/332: fdatasync d2b/d31/d35/d36/d37/f3b 0 2024-05-25T06:11:35.127 INFO:tasks.workunit.client.0.smithi160.stdout:0/333: chown d2b/d58 96051 1 2024-05-25T06:11:35.128 INFO:tasks.workunit.client.0.smithi160.stdout:0/334: chown d2b/d31/d35/d36/d52 203 1 2024-05-25T06:11:35.133 INFO:tasks.workunit.client.1.smithi203.stdout:5/901: write dc/d11/d17/d1a/d2d/d4d/ded/d13f/f9e [1439436,20670] 0 2024-05-25T06:11:35.133 INFO:tasks.workunit.client.1.smithi203.stdout:4/731: mkdir d0/d9/d11/d76/def 0 2024-05-25T06:11:35.138 INFO:tasks.workunit.client.1.smithi203.stdout:4/732: readlink d0/d9/d50/d6b/lda 0 2024-05-25T06:11:35.140 INFO:tasks.workunit.client.1.smithi203.stdout:6/945: dwrite d8/d11/d47/fbf [0,4194304] 0 2024-05-25T06:11:35.142 INFO:tasks.workunit.client.1.smithi203.stdout:3/800: dwrite d10/d16/d23/d46/fbf [0,4194304] 0 2024-05-25T06:11:35.144 INFO:tasks.workunit.client.0.smithi160.stdout:0/335: write f2 [2604170,88272] 0 2024-05-25T06:11:35.145 INFO:tasks.workunit.client.1.smithi203.stdout:4/733: unlink d0/d9/d50/d79/ce2 0 2024-05-25T06:11:35.145 INFO:tasks.workunit.client.1.smithi203.stdout:6/946: symlink d8/d5a/d9b/dcc/dce/l12f 0 2024-05-25T06:11:35.147 INFO:tasks.workunit.client.1.smithi203.stdout:6/947: dread - d8/d2a/d6c/d87/daa/fb4 zero size 2024-05-25T06:11:35.147 INFO:tasks.workunit.client.0.smithi160.stdout:0/336: symlink d2b/d58/d67/d6a/l6b 0 2024-05-25T06:11:35.150 INFO:tasks.workunit.client.1.smithi203.stdout:4/734: mknod d0/d8/d10/da9/cf0 0 2024-05-25T06:11:35.151 INFO:tasks.workunit.client.0.smithi160.stdout:0/337: mknod d2b/d31/d35/c6c 0 2024-05-25T06:11:35.151 INFO:tasks.workunit.client.0.smithi160.stdout:0/338: read - d2b/f2e zero size 2024-05-25T06:11:35.152 INFO:tasks.workunit.client.1.smithi203.stdout:4/735: fsync d0/f16 0 2024-05-25T06:11:35.152 INFO:tasks.workunit.client.1.smithi203.stdout:4/736: readlink d0/d8/ld9 0 2024-05-25T06:11:35.155 INFO:tasks.workunit.client.0.smithi160.stdout:0/339: unlink d2b/d31/d35/d36/f3c 0 2024-05-25T06:11:35.158 INFO:tasks.workunit.client.0.smithi160.stdout:0/340: mkdir d2b/d31/d6d 0 2024-05-25T06:11:35.159 INFO:tasks.workunit.client.1.smithi203.stdout:4/737: rename d0/d8/d1f/fb7 to d0/d8/d3a/d81/dd4/ff1 0 2024-05-25T06:11:35.159 INFO:tasks.workunit.client.1.smithi203.stdout:4/738: chown d0/d9/d50/dce/f4d 6 1 2024-05-25T06:11:35.160 INFO:tasks.workunit.client.0.smithi160.stdout:0/341: dread d2b/d31/d35/d43/f4c [0,4194304] 0 2024-05-25T06:11:35.164 INFO:tasks.workunit.client.1.smithi203.stdout:4/739: dwrite d0/d9/d50/d79/f5f [0,4194304] 0 2024-05-25T06:11:35.168 INFO:tasks.workunit.client.1.smithi203.stdout:4/740: rename d0/d8/c1b to d0/d9/d11/d76/def/cf2 0 2024-05-25T06:11:35.175 INFO:tasks.workunit.client.1.smithi203.stdout:4/741: dwrite d0/d8/f8d [0,4194304] 0 2024-05-25T06:11:35.178 INFO:tasks.workunit.client.0.smithi160.stdout:0/342: truncate d2b/d31/f34 3504961 0 2024-05-25T06:11:35.179 INFO:tasks.workunit.client.0.smithi160.stdout:0/343: stat d2b/d31/d35/d36 0 2024-05-25T06:11:35.183 INFO:tasks.workunit.client.1.smithi203.stdout:6/948: dread d8/d11/f19 [0,4194304] 0 2024-05-25T06:11:35.186 INFO:tasks.workunit.client.0.smithi160.stdout:0/344: dwrite f16 [0,4194304] 0 2024-05-25T06:11:35.188 INFO:tasks.workunit.client.1.smithi203.stdout:4/742: write d0/f3 [4451799,28257] 0 2024-05-25T06:11:35.191 INFO:tasks.workunit.client.0.smithi160.stdout:0/345: creat d2b/d46/d63/f6e x:0 0 0 2024-05-25T06:11:35.191 INFO:tasks.workunit.client.1.smithi203.stdout:4/743: write d0/d9/fe4 [799294,75026] 0 2024-05-25T06:11:35.193 INFO:tasks.workunit.client.0.smithi160.stdout:0/346: rmdir d2b/d31/d35/d43 39 2024-05-25T06:11:35.194 INFO:tasks.workunit.client.1.smithi203.stdout:6/949: dwrite d8/d11/d15/d28/d52/d43/ff5 [0,4194304] 0 2024-05-25T06:11:35.196 INFO:tasks.workunit.client.1.smithi203.stdout:4/744: mknod d0/d8/d52/cf3 0 2024-05-25T06:11:35.198 INFO:tasks.workunit.client.0.smithi160.stdout:0/347: unlink d2b/d31/d35/d36/c5f 0 2024-05-25T06:11:35.199 INFO:tasks.workunit.client.1.smithi203.stdout:6/950: symlink d8/d11/d15/d28/d52/d43/d73/db7/ddc/l130 0 2024-05-25T06:11:35.200 INFO:tasks.workunit.client.1.smithi203.stdout:6/951: chown d8/d11/d15/f79 2438350 1 2024-05-25T06:11:35.202 INFO:tasks.workunit.client.0.smithi160.stdout:0/348: mknod d2b/d31/d35/c6f 0 2024-05-25T06:11:35.203 INFO:tasks.workunit.client.1.smithi203.stdout:4/745: rename d0/d8/d52/f71 to d0/d8/d3a/d81/d93/d98/d99/ff4 0 2024-05-25T06:11:35.204 INFO:tasks.workunit.client.0.smithi160.stdout:0/349: stat d2b/d31/d35/d43/l47 0 2024-05-25T06:11:35.207 INFO:tasks.workunit.client.1.smithi203.stdout:4/746: mkdir d0/d8/d1f/d26/d3b/df5 0 2024-05-25T06:11:35.207 INFO:tasks.workunit.client.1.smithi203.stdout:6/952: dwrite d8/d11/d47/f99 [0,4194304] 0 2024-05-25T06:11:35.208 INFO:tasks.workunit.client.0.smithi160.stdout:0/350: rename d2b/d31/d35/d43/l47 to d2b/l70 0 2024-05-25T06:11:35.208 INFO:tasks.workunit.client.1.smithi203.stdout:4/747: write d0/d8/d52/f6f [2587412,114585] 0 2024-05-25T06:11:35.209 INFO:tasks.workunit.client.1.smithi203.stdout:6/953: readlink d8/d2a/da9/le6 0 2024-05-25T06:11:35.209 INFO:tasks.workunit.client.1.smithi203.stdout:6/954: fsync d8/d70/d38/f39 0 2024-05-25T06:11:35.212 INFO:tasks.workunit.client.0.smithi160.stdout:0/351: unlink d2b/d31/c65 0 2024-05-25T06:11:35.212 INFO:tasks.workunit.client.1.smithi203.stdout:4/748: write d0/d8/d3a/f77 [2399870,103745] 0 2024-05-25T06:11:35.213 INFO:tasks.workunit.client.1.smithi203.stdout:6/955: write d8/d11/f8e [1275869,63270] 0 2024-05-25T06:11:35.213 INFO:tasks.workunit.client.1.smithi203.stdout:4/749: fdatasync d0/f16 0 2024-05-25T06:11:35.215 INFO:tasks.workunit.client.0.smithi160.stdout:0/352: symlink d2b/d31/d35/d36/l71 0 2024-05-25T06:11:35.216 INFO:tasks.workunit.client.1.smithi203.stdout:6/956: symlink d8/d5a/d9b/l131 0 2024-05-25T06:11:35.217 INFO:tasks.workunit.client.1.smithi203.stdout:4/750: dread - d0/d8/d10/d91/fe7 zero size 2024-05-25T06:11:35.217 INFO:tasks.workunit.client.0.smithi160.stdout:0/353: creat d2b/d58/f72 x:0 0 0 2024-05-25T06:11:35.218 INFO:tasks.workunit.client.0.smithi160.stdout:7/303: sync 2024-05-25T06:11:35.219 INFO:tasks.workunit.client.0.smithi160.stdout:7/304: chown d6/c10 7553 1 2024-05-25T06:11:35.220 INFO:tasks.workunit.client.1.smithi203.stdout:4/751: creat d0/d9/d50/d6b/ff6 x:0 0 0 2024-05-25T06:11:35.221 INFO:tasks.workunit.client.0.smithi160.stdout:0/354: rename d2b/d31/d35/d36/l71 to d2b/d31/d35/d43/l73 0 2024-05-25T06:11:35.222 INFO:tasks.workunit.client.0.smithi160.stdout:0/355: write d2b/d31/d35/f49 [292402,57877] 0 2024-05-25T06:11:35.223 INFO:tasks.workunit.client.0.smithi160.stdout:7/305: creat d6/d2b/f5c x:0 0 0 2024-05-25T06:11:35.223 INFO:tasks.workunit.client.1.smithi203.stdout:4/752: symlink d0/d9/d50/d79/d9a/dbe/lf7 0 2024-05-25T06:11:35.224 INFO:tasks.workunit.client.0.smithi160.stdout:7/306: truncate d6/dc/d21/f57 258700 0 2024-05-25T06:11:35.225 INFO:tasks.workunit.client.1.smithi203.stdout:6/957: write d8/d11/d15/d28/d52/d43/f32 [1129136,118868] 0 2024-05-25T06:11:35.226 INFO:tasks.workunit.client.1.smithi203.stdout:4/753: symlink d0/d9/d50/d79/lf8 0 2024-05-25T06:11:35.228 INFO:tasks.workunit.client.1.smithi203.stdout:4/754: mknod d0/d8/d3a/d81/d93/d98/d99/ded/cf9 0 2024-05-25T06:11:35.229 INFO:tasks.workunit.client.0.smithi160.stdout:7/307: dread d6/fb [0,4194304] 0 2024-05-25T06:11:35.231 INFO:tasks.workunit.client.1.smithi203.stdout:6/958: getdents d8/d11/d1b/d75 0 2024-05-25T06:11:35.232 INFO:tasks.workunit.client.0.smithi160.stdout:7/308: getdents d6/d2c/d5a 0 2024-05-25T06:11:35.233 INFO:tasks.workunit.client.1.smithi203.stdout:6/959: truncate d8/d2a/d63/f5b 788031 0 2024-05-25T06:11:35.235 INFO:tasks.workunit.client.0.smithi160.stdout:7/309: mkdir d6/d2b/d5d 0 2024-05-25T06:11:35.236 INFO:tasks.workunit.client.0.smithi160.stdout:0/356: write d2b/f57 [2376556,115055] 0 2024-05-25T06:11:35.238 INFO:tasks.workunit.client.1.smithi203.stdout:6/960: link d8/d5a/d9b/c11c d8/d11/d15/dd3/c132 0 2024-05-25T06:11:35.240 INFO:tasks.workunit.client.0.smithi160.stdout:7/310: creat d6/d2c/d5a/f5e x:0 0 0 2024-05-25T06:11:35.240 INFO:tasks.workunit.client.1.smithi203.stdout:6/961: mkdir d8/d2a/d6c/d87/d113/d133 0 2024-05-25T06:11:35.241 INFO:tasks.workunit.client.1.smithi203.stdout:6/962: write d8/d2a/d63/fe9 [2953932,112632] 0 2024-05-25T06:11:35.242 INFO:tasks.workunit.client.1.smithi203.stdout:6/963: stat d8/d2a/da9/le6 0 2024-05-25T06:11:35.246 INFO:tasks.workunit.client.1.smithi203.stdout:4/755: dread d0/d9/d50/fc4 [0,4194304] 0 2024-05-25T06:11:35.246 INFO:tasks.workunit.client.0.smithi160.stdout:0/357: dwrite d2b/f42 [4194304,4194304] 0 2024-05-25T06:11:35.247 INFO:tasks.workunit.client.0.smithi160.stdout:0/358: readlink l4 0 2024-05-25T06:11:35.249 INFO:tasks.workunit.client.0.smithi160.stdout:7/311: rename d6/d2b/d2d/l47 to d6/d2b/l5f 0 2024-05-25T06:11:35.254 INFO:tasks.workunit.client.1.smithi203.stdout:4/756: rename d0/d9/d11/d76 to d0/d8/d3a/d81/d93/d98/d99/ded/dfa 0 2024-05-25T06:11:35.254 INFO:tasks.workunit.client.0.smithi160.stdout:7/312: mknod d6/dc/d21/c60 0 2024-05-25T06:11:35.259 INFO:tasks.workunit.client.1.smithi203.stdout:4/757: dread d0/d9/d50/dce/f75 [0,4194304] 0 2024-05-25T06:11:35.261 INFO:tasks.workunit.client.1.smithi203.stdout:4/758: write d0/d9/d50/d6b/f8a [125120,89314] 0 2024-05-25T06:11:35.263 INFO:tasks.workunit.client.1.smithi203.stdout:4/759: write d0/d8/d3a/d81/f96 [2277993,95197] 0 2024-05-25T06:11:35.264 INFO:tasks.workunit.client.1.smithi203.stdout:6/964: dwrite d8/d11/d1b/d75/fda [0,4194304] 0 2024-05-25T06:11:35.265 INFO:tasks.workunit.client.1.smithi203.stdout:6/965: write d8/d5a/d9b/daf/ff4 [313175,98332] 0 2024-05-25T06:11:35.267 INFO:tasks.workunit.client.0.smithi160.stdout:0/359: write d2b/d31/d35/d36/f4a [2545668,110301] 0 2024-05-25T06:11:35.269 INFO:tasks.workunit.client.0.smithi160.stdout:0/360: mknod d2b/c74 0 2024-05-25T06:11:35.270 INFO:tasks.workunit.client.1.smithi203.stdout:4/760: chown d0/d9/lbb 497 1 2024-05-25T06:11:35.270 INFO:tasks.workunit.client.1.smithi203.stdout:6/966: dread d8/d5a/d8b/fe4 [0,4194304] 0 2024-05-25T06:11:35.271 INFO:tasks.workunit.client.1.smithi203.stdout:4/761: read d0/d8/d3a/d81/dd4/ff1 [594472,85811] 0 2024-05-25T06:11:35.272 INFO:tasks.workunit.client.1.smithi203.stdout:4/762: chown d0/d8/d3a/f46 1358 1 2024-05-25T06:11:35.272 INFO:tasks.workunit.client.0.smithi160.stdout:0/361: creat d2b/d31/d35/d36/d37/f75 x:0 0 0 2024-05-25T06:11:35.276 INFO:tasks.workunit.client.1.smithi203.stdout:4/763: mknod d0/d8/cfb 0 2024-05-25T06:11:35.277 INFO:tasks.workunit.client.1.smithi203.stdout:6/967: dwrite d8/d2a/d63/fe9 [4194304,4194304] 0 2024-05-25T06:11:35.279 INFO:tasks.workunit.client.1.smithi203.stdout:4/764: symlink d0/d8/d52/lfc 0 2024-05-25T06:11:35.279 INFO:tasks.workunit.client.0.smithi160.stdout:7/313: dread d6/d2c/f3c [0,4194304] 0 2024-05-25T06:11:35.281 INFO:tasks.workunit.client.1.smithi203.stdout:4/765: dread - d0/d8/d1f/d26/f8e zero size 2024-05-25T06:11:35.284 INFO:tasks.workunit.client.1.smithi203.stdout:4/766: dread d0/d9/d50/d6b/fc1 [0,4194304] 0 2024-05-25T06:11:35.285 INFO:tasks.workunit.client.0.smithi160.stdout:0/362: truncate d2b/f57 5795233 0 2024-05-25T06:11:35.287 INFO:tasks.workunit.client.0.smithi160.stdout:0/363: dread - d2b/d31/d35/d41/f5b zero size 2024-05-25T06:11:35.287 INFO:tasks.workunit.client.0.smithi160.stdout:7/314: dwrite d6/d32/f42 [0,4194304] 0 2024-05-25T06:11:35.289 INFO:tasks.workunit.client.1.smithi203.stdout:4/767: creat d0/d9/d11/d54/ffd x:0 0 0 2024-05-25T06:11:35.293 INFO:tasks.workunit.client.0.smithi160.stdout:7/315: getdents d6/d2b 0 2024-05-25T06:11:35.295 INFO:tasks.workunit.client.1.smithi203.stdout:4/768: dread d0/d8/f23 [0,4194304] 0 2024-05-25T06:11:35.297 INFO:tasks.workunit.client.0.smithi160.stdout:7/316: mknod d6/d2b/d5d/c61 0 2024-05-25T06:11:35.298 INFO:tasks.workunit.client.1.smithi203.stdout:4/769: creat d0/d9/d50/d6b/ffe x:0 0 0 2024-05-25T06:11:35.298 INFO:tasks.workunit.client.0.smithi160.stdout:7/317: truncate d6/d2b/d2d/f40 507327 0 2024-05-25T06:11:35.300 INFO:tasks.workunit.client.0.smithi160.stdout:0/364: truncate f1b 2953667 0 2024-05-25T06:11:35.300 INFO:tasks.workunit.client.1.smithi203.stdout:6/968: dread d8/fc [0,4194304] 0 2024-05-25T06:11:35.301 INFO:tasks.workunit.client.1.smithi203.stdout:6/969: readlink d8/d70/ld5 0 2024-05-25T06:11:35.301 INFO:tasks.workunit.client.0.smithi160.stdout:0/365: rmdir d2b/d31/d35/d43 39 2024-05-25T06:11:35.302 INFO:tasks.workunit.client.0.smithi160.stdout:0/366: fsync d2b/d31/d35/d41/f5b 0 2024-05-25T06:11:35.303 INFO:tasks.workunit.client.0.smithi160.stdout:0/367: write d2b/d58/f5e [101397,126807] 0 2024-05-25T06:11:35.305 INFO:tasks.workunit.client.1.smithi203.stdout:6/970: unlink d8/d11/d1b/d75/dcb/dcf/d80/ca3 0 2024-05-25T06:11:35.305 INFO:tasks.workunit.client.0.smithi160.stdout:0/368: chown l2a 8325435 1 2024-05-25T06:11:35.306 INFO:tasks.workunit.client.1.smithi203.stdout:4/770: dwrite d0/d9/f9e [4194304,4194304] 0 2024-05-25T06:11:35.311 INFO:tasks.workunit.client.1.smithi203.stdout:4/771: rename d0/d8/d1f/d26/c37 to d0/d8/d1f/d26/d3b/df5/cff 0 2024-05-25T06:11:35.312 INFO:tasks.workunit.client.0.smithi160.stdout:7/318: write d6/dc/d21/f4f [420707,21676] 0 2024-05-25T06:11:35.314 INFO:tasks.workunit.client.1.smithi203.stdout:6/971: dwrite d8/fe0 [0,4194304] 0 2024-05-25T06:11:35.317 INFO:tasks.workunit.client.1.smithi203.stdout:4/772: symlink d0/d8/d3a/d81/l100 0 2024-05-25T06:11:35.318 INFO:tasks.workunit.client.1.smithi203.stdout:4/773: chown d0/d8/d52/d59/d7a/d83 93 1 2024-05-25T06:11:35.321 INFO:tasks.workunit.client.0.smithi160.stdout:7/319: dwrite d6/dc/d21/f2a [0,4194304] 0 2024-05-25T06:11:35.321 INFO:tasks.workunit.client.1.smithi203.stdout:4/774: creat d0/d9/d11/f101 x:0 0 0 2024-05-25T06:11:35.322 INFO:tasks.workunit.client.0.smithi160.stdout:7/320: fsync d6/d2b/d2d/f56 0 2024-05-25T06:11:35.323 INFO:tasks.workunit.client.1.smithi203.stdout:4/775: symlink d0/d8/d10/l102 0 2024-05-25T06:11:35.326 INFO:tasks.workunit.client.0.smithi160.stdout:7/321: fdatasync d6/d2c/f53 0 2024-05-25T06:11:35.327 INFO:tasks.workunit.client.1.smithi203.stdout:4/776: symlink d0/d8/d52/d59/d7a/l103 0 2024-05-25T06:11:35.328 INFO:tasks.workunit.client.0.smithi160.stdout:0/369: dread f5 [0,4194304] 0 2024-05-25T06:11:35.330 INFO:tasks.workunit.client.0.smithi160.stdout:7/322: creat d6/d32/d5b/f62 x:0 0 0 2024-05-25T06:11:35.331 INFO:tasks.workunit.client.0.smithi160.stdout:0/370: mknod d2b/d46/c76 0 2024-05-25T06:11:35.332 INFO:tasks.workunit.client.0.smithi160.stdout:7/323: creat d6/dc/d21/f63 x:0 0 0 2024-05-25T06:11:35.333 INFO:tasks.workunit.client.0.smithi160.stdout:0/371: symlink d2b/d31/d6d/l77 0 2024-05-25T06:11:35.334 INFO:tasks.workunit.client.0.smithi160.stdout:0/372: write f23 [859540,74146] 0 2024-05-25T06:11:35.336 INFO:tasks.workunit.client.0.smithi160.stdout:7/324: rename d6/d2c to d6/dc/d64 0 2024-05-25T06:11:35.336 INFO:tasks.workunit.client.1.smithi203.stdout:6/972: write d8/d11/f19 [2557631,69817] 0 2024-05-25T06:11:35.336 INFO:tasks.workunit.client.1.smithi203.stdout:4/777: rmdir d0/d9/d11 39 2024-05-25T06:11:35.341 INFO:tasks.workunit.client.1.smithi203.stdout:6/973: link d8/d5a/d8b/f96 d8/d11/d1b/d75/dcb/dcf/d46/f134 0 2024-05-25T06:11:35.343 INFO:tasks.workunit.client.0.smithi160.stdout:0/373: write f1f [1384727,96893] 0 2024-05-25T06:11:35.346 INFO:tasks.workunit.client.0.smithi160.stdout:0/374: creat d2b/f78 x:0 0 0 2024-05-25T06:11:35.350 INFO:tasks.workunit.client.1.smithi203.stdout:6/974: dwrite d8/d5a/d8b/fa8 [0,4194304] 0 2024-05-25T06:11:35.352 INFO:tasks.workunit.client.0.smithi160.stdout:7/325: dread d6/dc/d64/f43 [0,4194304] 0 2024-05-25T06:11:35.352 INFO:tasks.workunit.client.0.smithi160.stdout:0/375: dwrite f5 [4194304,4194304] 0 2024-05-25T06:11:35.354 INFO:tasks.workunit.client.1.smithi203.stdout:6/975: symlink d8/d11/d15/d28/d52/d43/d73/l135 0 2024-05-25T06:11:35.355 INFO:tasks.workunit.client.0.smithi160.stdout:0/376: mknod d2b/d31/d35/d41/c79 0 2024-05-25T06:11:35.356 INFO:tasks.workunit.client.1.smithi203.stdout:6/976: read d8/d11/d1b/f4d [473641,116388] 0 2024-05-25T06:11:35.357 INFO:tasks.workunit.client.1.smithi203.stdout:4/778: dread d0/f16 [0,4194304] 0 2024-05-25T06:11:35.358 INFO:tasks.workunit.client.0.smithi160.stdout:7/326: mkdir d6/d3e/d59/d65 0 2024-05-25T06:11:35.360 INFO:tasks.workunit.client.1.smithi203.stdout:4/779: write d0/d8/d10/d91/da2/fb9 [4886436,15746] 0 2024-05-25T06:11:35.361 INFO:tasks.workunit.client.0.smithi160.stdout:7/327: creat d6/d3e/d59/f66 x:0 0 0 2024-05-25T06:11:35.363 INFO:tasks.workunit.client.1.smithi203.stdout:4/780: mknod d0/d8/d3a/d81/dd4/c104 0 2024-05-25T06:11:35.363 INFO:tasks.workunit.client.0.smithi160.stdout:7/328: mknod d6/d3e/c67 0 2024-05-25T06:11:35.364 INFO:tasks.workunit.client.0.smithi160.stdout:7/329: chown d6/d32/d5b 1167 1 2024-05-25T06:11:35.366 INFO:tasks.workunit.client.1.smithi203.stdout:6/977: dwrite d8/d11/d1b/d75/dcb/fdf [0,4194304] 0 2024-05-25T06:11:35.368 INFO:tasks.workunit.client.1.smithi203.stdout:4/781: dread d0/d9/d11/f66 [0,4194304] 0 2024-05-25T06:11:35.370 INFO:tasks.workunit.client.1.smithi203.stdout:4/782: truncate d0/d9/d50/dce/f7d 1007312 0 2024-05-25T06:11:35.370 INFO:tasks.workunit.client.0.smithi160.stdout:9/263: sync 2024-05-25T06:11:35.371 INFO:tasks.workunit.client.0.smithi160.stdout:4/346: sync 2024-05-25T06:11:35.371 INFO:tasks.workunit.client.0.smithi160.stdout:7/330: dwrite f1 [0,4194304] 0 2024-05-25T06:11:35.372 INFO:tasks.workunit.client.0.smithi160.stdout:9/264: chown d4/d11/d1e/f4d 34 1 2024-05-25T06:11:35.375 INFO:tasks.workunit.client.0.smithi160.stdout:4/347: mknod d4/d6/d28/c6b 0 2024-05-25T06:11:35.376 INFO:tasks.workunit.client.0.smithi160.stdout:9/265: read d4/d11/d1e/d24/fc [1155082,106511] 0 2024-05-25T06:11:35.377 INFO:tasks.workunit.client.0.smithi160.stdout:4/348: chown d4/d6/d35/l41 99 1 2024-05-25T06:11:35.377 INFO:tasks.workunit.client.0.smithi160.stdout:7/331: unlink d6/dc/f24 0 2024-05-25T06:11:35.378 INFO:tasks.workunit.client.0.smithi160.stdout:7/332: dread - d6/d2b/f51 zero size 2024-05-25T06:11:35.378 INFO:tasks.workunit.client.1.smithi203.stdout:4/783: dwrite d0/d8/d3a/d81/d93/d98/d99/feb [0,4194304] 0 2024-05-25T06:11:35.379 INFO:tasks.workunit.client.0.smithi160.stdout:4/349: rmdir d4/d1c 39 2024-05-25T06:11:35.382 INFO:tasks.workunit.client.0.smithi160.stdout:9/266: unlink d4/d11/d1e/d3e/d25/c42 0 2024-05-25T06:11:35.383 INFO:tasks.workunit.client.0.smithi160.stdout:7/333: rename d6/dc/d21/l4d to d6/d32/l68 0 2024-05-25T06:11:35.384 INFO:tasks.workunit.client.0.smithi160.stdout:9/267: truncate d4/f47 24867 0 2024-05-25T06:11:35.386 INFO:tasks.workunit.client.0.smithi160.stdout:7/334: symlink d6/d32/d46/l69 0 2024-05-25T06:11:35.388 INFO:tasks.workunit.client.0.smithi160.stdout:4/350: link d4/d1a/f40 d4/d1c/d54/f6c 0 2024-05-25T06:11:35.389 INFO:tasks.workunit.client.0.smithi160.stdout:4/351: write d4/d1a/f50 [238288,130380] 0 2024-05-25T06:11:35.389 INFO:tasks.workunit.client.0.smithi160.stdout:7/335: creat d6/d2b/d5d/f6a x:0 0 0 2024-05-25T06:11:35.390 INFO:tasks.workunit.client.0.smithi160.stdout:7/336: dread - d6/dc/d64/d5a/f5e zero size 2024-05-25T06:11:35.393 INFO:tasks.workunit.client.0.smithi160.stdout:7/337: link d6/d2b/d5d/f6a d6/d2b/f6b 0 2024-05-25T06:11:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:35 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:35.399 INFO:tasks.workunit.client.0.smithi160.stdout:7/338: dwrite d6/dc/f19 [0,4194304] 0 2024-05-25T06:11:35.404 INFO:tasks.workunit.client.1.smithi203.stdout:4/784: mknod d0/d8/d52/d59/d7a/d83/c105 0 2024-05-25T06:11:35.410 INFO:tasks.workunit.client.1.smithi203.stdout:6/978: dwrite d8/d11/d15/f42 [0,4194304] 0 2024-05-25T06:11:35.412 INFO:tasks.workunit.client.1.smithi203.stdout:6/979: dread - d8/d2a/d6c/d87/daa/fb4 zero size 2024-05-25T06:11:35.413 INFO:tasks.workunit.client.1.smithi203.stdout:6/980: chown d8/d11/d15/d28/c1c 1852730 1 2024-05-25T06:11:35.413 INFO:tasks.workunit.client.0.smithi160.stdout:7/339: creat d6/d2b/d2d/f6c x:0 0 0 2024-05-25T06:11:35.415 INFO:tasks.workunit.client.0.smithi160.stdout:9/268: dread d4/d11/d1e/d24/fc [0,4194304] 0 2024-05-25T06:11:35.416 INFO:tasks.workunit.client.0.smithi160.stdout:9/269: readlink d4/d11/d14/l1a 0 2024-05-25T06:11:35.419 INFO:tasks.workunit.client.1.smithi203.stdout:6/981: dread d8/d5a/d9b/dcc/f100 [0,4194304] 0 2024-05-25T06:11:35.420 INFO:tasks.workunit.client.0.smithi160.stdout:9/270: dread d4/d11/f38 [0,4194304] 0 2024-05-25T06:11:35.420 INFO:tasks.workunit.client.0.smithi160.stdout:7/340: link d6/l7 d6/dc/d64/d5a/l6d 0 2024-05-25T06:11:35.421 INFO:tasks.workunit.client.0.smithi160.stdout:9/271: dread - d4/d11/d1e/d24/f13 zero size 2024-05-25T06:11:35.424 INFO:tasks.workunit.client.1.smithi203.stdout:4/785: dread d0/d8/d52/f5e [0,4194304] 0 2024-05-25T06:11:35.429 INFO:tasks.workunit.client.0.smithi160.stdout:7/341: dwrite d6/dc/d21/f54 [4194304,4194304] 0 2024-05-25T06:11:35.429 INFO:tasks.workunit.client.0.smithi160.stdout:9/272: dwrite d4/d11/d1e/d24/f39 [0,4194304] 0 2024-05-25T06:11:35.433 INFO:tasks.workunit.client.0.smithi160.stdout:7/342: readlink d6/l11 0 2024-05-25T06:11:35.434 INFO:tasks.workunit.client.0.smithi160.stdout:9/273: creat d4/f4e x:0 0 0 2024-05-25T06:11:35.435 INFO:tasks.workunit.client.0.smithi160.stdout:7/343: dread d6/dc/f4a [0,4194304] 0 2024-05-25T06:11:35.438 INFO:tasks.workunit.client.0.smithi160.stdout:7/344: creat d6/d2b/d2d/f6e x:0 0 0 2024-05-25T06:11:35.439 INFO:tasks.workunit.client.0.smithi160.stdout:7/345: read d6/dc/d21/f2a [262522,50046] 0 2024-05-25T06:11:35.440 INFO:tasks.workunit.client.1.smithi203.stdout:4/786: creat d0/d8/d10/da9/f106 x:0 0 0 2024-05-25T06:11:35.441 INFO:tasks.workunit.client.0.smithi160.stdout:9/274: rename d4/fd to d4/d11/d1e/d24/f4f 0 2024-05-25T06:11:35.446 INFO:tasks.workunit.client.0.smithi160.stdout:7/346: dread d6/d2b/d2d/f40 [0,4194304] 0 2024-05-25T06:11:35.454 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:35 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:35.459 INFO:tasks.workunit.client.0.smithi160.stdout:7/347: rename d6/d2b/d2d to d6/d3e/d6f 0 2024-05-25T06:11:35.461 INFO:tasks.workunit.client.0.smithi160.stdout:7/348: creat d6/d3e/f70 x:0 0 0 2024-05-25T06:11:35.464 INFO:tasks.workunit.client.0.smithi160.stdout:7/349: dread d6/dc/d64/f3c [0,4194304] 0 2024-05-25T06:11:35.471 INFO:tasks.workunit.client.0.smithi160.stdout:7/350: dwrite d6/dc/d21/f4f [0,4194304] 0 2024-05-25T06:11:35.472 INFO:tasks.workunit.client.0.smithi160.stdout:9/275: dread d4/d11/d1e/d24/f1b [0,4194304] 0 2024-05-25T06:11:35.472 INFO:tasks.workunit.client.1.smithi203.stdout:4/787: write d0/d8/d52/d59/d7a/d83/fd6 [338867,55470] 0 2024-05-25T06:11:35.475 INFO:tasks.workunit.client.1.smithi203.stdout:6/982: dwrite d8/d5a/d8b/f96 [0,4194304] 0 2024-05-25T06:11:35.479 INFO:tasks.workunit.client.1.smithi203.stdout:6/983: rmdir d8/d11/d47/d60 39 2024-05-25T06:11:35.480 INFO:tasks.workunit.client.0.smithi160.stdout:7/351: dwrite d6/dc/d64/d5a/f5e [0,4194304] 0 2024-05-25T06:11:35.483 INFO:tasks.workunit.client.1.smithi203.stdout:6/984: unlink d8/d11/d15/d28/d52/d43/c9f 0 2024-05-25T06:11:35.483 INFO:tasks.workunit.client.1.smithi203.stdout:4/788: write d0/d9/d50/d6b/f51 [5160188,19659] 0 2024-05-25T06:11:35.485 INFO:tasks.workunit.client.1.smithi203.stdout:4/789: write d0/d9/d50/d79/f40 [2762254,81262] 0 2024-05-25T06:11:35.488 INFO:tasks.workunit.client.1.smithi203.stdout:6/985: rename d8/d2a/d6c/d87/d113 to d8/d11/d1b/d75/dcb/dcf/d46/d136 0 2024-05-25T06:11:35.489 INFO:tasks.workunit.client.1.smithi203.stdout:4/790: chown d0/d9/f17 0 1 2024-05-25T06:11:35.490 INFO:tasks.workunit.client.0.smithi160.stdout:7/352: chown d6/d2b/f5c 90503327 1 2024-05-25T06:11:35.491 INFO:tasks.workunit.client.0.smithi160.stdout:7/353: chown d6/dc/f25 91 1 2024-05-25T06:11:35.492 INFO:tasks.workunit.client.1.smithi203.stdout:4/791: fdatasync d0/d8/d1f/d26/f8e 0 2024-05-25T06:11:35.493 INFO:tasks.workunit.client.0.smithi160.stdout:7/354: chown d6/dc/d64/d5a/f5e 20027952 1 2024-05-25T06:11:35.493 INFO:tasks.workunit.client.0.smithi160.stdout:9/276: truncate d4/d11/d1e/d24/f4f 3724278 0 2024-05-25T06:11:35.495 INFO:tasks.workunit.client.0.smithi160.stdout:9/277: rmdir d4/d11/d14 39 2024-05-25T06:11:35.497 INFO:tasks.workunit.client.0.smithi160.stdout:9/278: write d4/d11/d12/f35 [4295194,112058] 0 2024-05-25T06:11:35.497 INFO:tasks.workunit.client.1.smithi203.stdout:4/792: truncate d0/d9/d9d/fc6 214590 0 2024-05-25T06:11:35.499 INFO:tasks.workunit.client.1.smithi203.stdout:4/793: write d0/d8/d3a/f56 [1621627,88242] 0 2024-05-25T06:11:35.501 INFO:tasks.workunit.client.0.smithi160.stdout:7/355: fdatasync d6/dc/d64/f37 0 2024-05-25T06:11:35.501 INFO:tasks.workunit.client.0.smithi160.stdout:9/279: link d4/d11/d12/f35 d4/d11/d1e/d3e/d25/f50 0 2024-05-25T06:11:35.502 INFO:tasks.workunit.client.1.smithi203.stdout:4/794: rename d0/d9/d50/d79/lf8 to d0/d8/d52/l107 0 2024-05-25T06:11:35.504 INFO:tasks.workunit.client.0.smithi160.stdout:7/356: getdents d6/d3e/d59/d65 0 2024-05-25T06:11:35.504 INFO:tasks.workunit.client.1.smithi203.stdout:4/795: dread - d0/d8/d10/d91/fe7 zero size 2024-05-25T06:11:35.505 INFO:tasks.workunit.client.0.smithi160.stdout:9/280: getdents d4 0 2024-05-25T06:11:35.507 INFO:tasks.workunit.client.1.smithi203.stdout:4/796: truncate d0/d8/d1f/d26/d3b/d9c/fe5 4572347 0 2024-05-25T06:11:35.507 INFO:tasks.workunit.client.1.smithi203.stdout:6/986: write d8/d2a/d6c/d87/f10a [4638305,41655] 0 2024-05-25T06:11:35.507 INFO:tasks.workunit.client.1.smithi203.stdout:4/797: stat d0/d8/d3a/f46 0 2024-05-25T06:11:35.507 INFO:tasks.workunit.client.0.smithi160.stdout:9/281: truncate d4/d11/d1e/d24/fc 4716826 0 2024-05-25T06:11:35.508 INFO:tasks.workunit.client.1.smithi203.stdout:6/987: write d8/d11/d47/fbe [1401174,118219] 0 2024-05-25T06:11:35.509 INFO:tasks.workunit.client.0.smithi160.stdout:7/357: getdents d6/d2b 0 2024-05-25T06:11:35.511 INFO:tasks.workunit.client.1.smithi203.stdout:6/988: write d8/d11/d15/d28/d52/d43/f84 [326226,39724] 0 2024-05-25T06:11:35.511 INFO:tasks.workunit.client.1.smithi203.stdout:4/798: rename d0/d8/d1f/d26/d3b/c95 to d0/d9/d50/d79/d9a/dbe/de0/c108 0 2024-05-25T06:11:35.512 INFO:tasks.workunit.client.0.smithi160.stdout:7/358: creat d6/dc/d21/f71 x:0 0 0 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:7/359: chown d6/d32/f42 642460 1 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:3/247: sync 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:6/280: sync 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:2/314: sync 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:1/228: sync 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:5/397: sync 2024-05-25T06:11:35.513 INFO:tasks.workunit.client.0.smithi160.stdout:8/375: sync 2024-05-25T06:11:35.514 INFO:tasks.workunit.client.1.smithi203.stdout:6/989: write d8/fc [4848958,49368] 0 2024-05-25T06:11:35.514 INFO:tasks.workunit.client.1.smithi203.stdout:6/990: stat d8/d2a/d6c/dde 0 2024-05-25T06:11:35.514 INFO:tasks.workunit.client.1.smithi203.stdout:4/799: creat d0/d8/d3a/f109 x:0 0 0 2024-05-25T06:11:35.515 INFO:tasks.workunit.client.0.smithi160.stdout:3/248: fdatasync d19/d2f/d34/f40 0 2024-05-25T06:11:35.515 INFO:tasks.workunit.client.0.smithi160.stdout:7/360: readlink d6/d32/l68 0 2024-05-25T06:11:35.515 INFO:tasks.workunit.client.1.smithi203.stdout:6/991: creat d8/d2a/d6c/f137 x:0 0 0 2024-05-25T06:11:35.516 INFO:tasks.workunit.client.0.smithi160.stdout:6/281: fsync d5/d45/d63/d61/f64 0 2024-05-25T06:11:35.516 INFO:tasks.workunit.client.0.smithi160.stdout:9/282: truncate d4/d11/d1e/d3e/d25/f50 1571496 0 2024-05-25T06:11:35.518 INFO:tasks.workunit.client.0.smithi160.stdout:3/249: read - d19/d2f/d38/d47/f4c zero size 2024-05-25T06:11:35.518 INFO:tasks.workunit.client.1.smithi203.stdout:6/992: symlink d8/d11/d15/d28/d52/d43/d73/db7/l138 0 2024-05-25T06:11:35.520 INFO:tasks.workunit.client.0.smithi160.stdout:1/229: rename de/d1b/d1c/f3e to de/d1b/d1c/f47 0 2024-05-25T06:11:35.520 INFO:tasks.workunit.client.1.smithi203.stdout:4/800: link d0/d8/d10/l57 d0/d8/d3a/d81/d93/d98/d99/ded/l10a 0 2024-05-25T06:11:35.521 INFO:tasks.workunit.client.0.smithi160.stdout:5/398: mknod db/dc/d13/d1d/c76 0 2024-05-25T06:11:35.522 INFO:tasks.workunit.client.1.smithi203.stdout:4/801: write d0/d8/d10/d91/da2/fb9 [3220276,11044] 0 2024-05-25T06:11:35.525 INFO:tasks.workunit.client.0.smithi160.stdout:8/376: truncate d5/d44/f68 4173512 0 2024-05-25T06:11:35.525 INFO:tasks.workunit.client.0.smithi160.stdout:6/282: mknod d5/d45/d63/d61/c65 0 2024-05-25T06:11:35.525 INFO:tasks.workunit.client.1.smithi203.stdout:4/802: mkdir d0/d8/d3a/d81/d93/d98/d99/ded/d10b 0 2024-05-25T06:11:35.528 INFO:tasks.workunit.client.0.smithi160.stdout:9/283: rmdir d4/d11/d1e/d3e 39 2024-05-25T06:11:35.529 INFO:tasks.workunit.client.0.smithi160.stdout:1/230: rmdir de/d1b/d35 39 2024-05-25T06:11:35.530 INFO:tasks.workunit.client.0.smithi160.stdout:2/315: dwrite d12/d3e/f68 [0,4194304] 0 2024-05-25T06:11:35.530 INFO:tasks.workunit.client.0.smithi160.stdout:7/361: dwrite d6/d3e/d6f/f6e [0,4194304] 0 2024-05-25T06:11:35.530 INFO:tasks.workunit.client.0.smithi160.stdout:6/283: write d5/de/d32/d35/f5f [585765,78770] 0 2024-05-25T06:11:35.533 INFO:tasks.workunit.client.0.smithi160.stdout:3/250: mkdir d19/d5d 0 2024-05-25T06:11:35.533 INFO:tasks.workunit.client.1.smithi203.stdout:6/993: getdents d8/d11/d15/d28/d52/d43/d73/db7 0 2024-05-25T06:11:35.535 INFO:tasks.workunit.client.1.smithi203.stdout:6/994: fdatasync d8/d3d/f9a 0 2024-05-25T06:11:35.536 INFO:tasks.workunit.client.0.smithi160.stdout:8/377: mknod d5/d52/c7b 0 2024-05-25T06:11:35.542 INFO:tasks.workunit.client.0.smithi160.stdout:2/316: rmdir d12/d3c 39 2024-05-25T06:11:35.544 INFO:tasks.workunit.client.0.smithi160.stdout:5/399: dwrite db/dc/d13/d1d/d23/d24/d2e/f2f [0,4194304] 0 2024-05-25T06:11:35.547 INFO:tasks.workunit.client.1.smithi203.stdout:6/995: getdents d8/d11/d1b/d75/dcb/dcf/d46/d88 0 2024-05-25T06:11:35.547 INFO:tasks.workunit.client.0.smithi160.stdout:1/231: readlink de/d39/l44 0 2024-05-25T06:11:35.547 INFO:tasks.workunit.client.0.smithi160.stdout:6/284: symlink d5/d39/l66 0 2024-05-25T06:11:35.547 INFO:tasks.workunit.client.1.smithi203.stdout:6/996: chown d8/l110 7 1 2024-05-25T06:11:35.547 INFO:tasks.workunit.client.0.smithi160.stdout:9/284: symlink d4/d11/d1e/d3e/d25/l51 0 2024-05-25T06:11:35.549 INFO:tasks.workunit.client.1.smithi203.stdout:4/803: dread d0/d9/f17 [0,4194304] 0 2024-05-25T06:11:35.549 INFO:tasks.workunit.client.0.smithi160.stdout:2/317: mknod d12/d3e/d5a/c70 0 2024-05-25T06:11:35.553 INFO:tasks.workunit.client.0.smithi160.stdout:5/400: dwrite db/dc/d13/d1d/d23/d24/f3e [0,4194304] 0 2024-05-25T06:11:35.553 INFO:tasks.workunit.client.1.smithi203.stdout:4/804: creat d0/d8/d1f/d26/d3b/df5/f10c x:0 0 0 2024-05-25T06:11:35.554 INFO:tasks.workunit.client.0.smithi160.stdout:1/232: symlink de/d1b/d28/l48 0 2024-05-25T06:11:35.555 INFO:tasks.workunit.client.1.smithi203.stdout:4/805: write d0/d9/d50/d6b/ffe [490905,37396] 0 2024-05-25T06:11:35.556 INFO:tasks.workunit.client.1.smithi203.stdout:4/806: chown d0/d9/d11/c8f 1 1 2024-05-25T06:11:35.556 INFO:tasks.workunit.client.0.smithi160.stdout:1/233: chown de/l1e 173466832 1 2024-05-25T06:11:35.560 INFO:tasks.workunit.client.1.smithi203.stdout:4/807: truncate d0/d8/d1f/d26/f4a 3452424 0 2024-05-25T06:11:35.561 INFO:tasks.workunit.client.0.smithi160.stdout:9/285: mknod d4/d11/d1e/d3e/c52 0 2024-05-25T06:11:35.563 INFO:tasks.workunit.client.0.smithi160.stdout:7/362: write d6/d2b/f6b [575153,72321] 0 2024-05-25T06:11:35.563 INFO:tasks.workunit.client.1.smithi203.stdout:6/997: write d8/d11/d1b/fb0 [125574,58085] 0 2024-05-25T06:11:35.565 INFO:tasks.workunit.client.0.smithi160.stdout:9/286: fdatasync d4/d11/d1e/f4d 0 2024-05-25T06:11:35.565 INFO:tasks.workunit.client.1.smithi203.stdout:6/998: creat d8/d5a/d9b/dcc/f139 x:0 0 0 2024-05-25T06:11:35.566 INFO:tasks.workunit.client.1.smithi203.stdout:6/999: fsync d8/d11/f8e 0 2024-05-25T06:11:35.566 INFO:tasks.workunit.client.0.smithi160.stdout:1/234: read - de/d1b/d35/f24 zero size 2024-05-25T06:11:35.567 INFO:tasks.workunit.client.0.smithi160.stdout:5/401: creat db/dc/d13/d1d/d23/d24/f77 x:0 0 0 2024-05-25T06:11:35.569 INFO:tasks.workunit.client.0.smithi160.stdout:2/318: dwrite d12/d3e/f62 [0,4194304] 0 2024-05-25T06:11:35.570 INFO:tasks.workunit.client.0.smithi160.stdout:5/402: write f9 [5033884,29376] 0 2024-05-25T06:11:35.573 INFO:tasks.workunit.client.0.smithi160.stdout:6/285: dwrite d5/d39/f4d [4194304,4194304] 0 2024-05-25T06:11:35.577 INFO:tasks.workunit.client.0.smithi160.stdout:0/377: sync 2024-05-25T06:11:35.577 INFO:tasks.workunit.client.0.smithi160.stdout:2/319: chown d12/d1c/f1e 271310 1 2024-05-25T06:11:35.582 INFO:tasks.workunit.client.1.smithi203.stdout:4/808: dread d0/d9/d11/d54/fc7 [0,4194304] 0 2024-05-25T06:11:35.582 INFO:tasks.workunit.client.0.smithi160.stdout:9/287: rmdir d4/d11 39 2024-05-25T06:11:35.584 INFO:tasks.workunit.client.1.smithi203.stdout:4/809: write d0/d8/d1f/d26/d3b/df5/f10c [296004,40654] 0 2024-05-25T06:11:35.584 INFO:tasks.workunit.client.0.smithi160.stdout:8/378: dread d5/d44/d1f/d1e/d28/f67 [0,4194304] 0 2024-05-25T06:11:35.586 INFO:tasks.workunit.client.0.smithi160.stdout:0/378: write d2b/d46/f5a [416515,110915] 0 2024-05-25T06:11:35.586 INFO:tasks.workunit.client.0.smithi160.stdout:5/403: creat db/dc/d13/d1d/d31/d56/f78 x:0 0 0 2024-05-25T06:11:35.588 INFO:tasks.workunit.client.1.smithi203.stdout:4/810: creat d0/d8/d3a/d81/d93/d98/d99/f10d x:0 0 0 2024-05-25T06:11:35.590 INFO:tasks.workunit.client.0.smithi160.stdout:1/235: dwrite de/f16 [0,4194304] 0 2024-05-25T06:11:35.590 INFO:tasks.workunit.client.0.smithi160.stdout:3/251: dwrite d19/f44 [0,4194304] 0 2024-05-25T06:11:35.593 INFO:tasks.workunit.client.1.smithi203.stdout:4/811: creat d0/d9/d50/f10e x:0 0 0 2024-05-25T06:11:35.594 INFO:tasks.workunit.client.1.smithi203.stdout:4/812: write d0/f5 [8183342,68670] 0 2024-05-25T06:11:35.595 INFO:tasks.workunit.client.0.smithi160.stdout:0/379: creat d2b/d31/f7a x:0 0 0 2024-05-25T06:11:35.599 INFO:tasks.workunit.client.1.smithi203.stdout:4/813: mknod d0/d8/d52/d59/d7a/c10f 0 2024-05-25T06:11:35.601 INFO:tasks.workunit.client.0.smithi160.stdout:7/363: dwrite d6/dc/d64/f3c [4194304,4194304] 0 2024-05-25T06:11:35.601 INFO:tasks.workunit.client.0.smithi160.stdout:3/252: dread - d19/d2f/d34/f5c zero size 2024-05-25T06:11:35.601 INFO:tasks.workunit.client.0.smithi160.stdout:6/286: write d5/d30/d36/f49 [776616,34014] 0 2024-05-25T06:11:35.602 INFO:tasks.workunit.client.0.smithi160.stdout:8/379: chown d5/d44/d1f/d1e/d28/d4a/f3d 99 1 2024-05-25T06:11:35.604 INFO:tasks.workunit.client.1.smithi203.stdout:4/814: truncate d0/d8/d3a/d81/d93/d98/d99/fcc 195330 0 2024-05-25T06:11:35.606 INFO:tasks.workunit.client.0.smithi160.stdout:0/380: fdatasync d2b/d46/f5a 0 2024-05-25T06:11:35.607 INFO:tasks.workunit.client.0.smithi160.stdout:1/236: write de/d1b/d35/f40 [2003146,105449] 0 2024-05-25T06:11:35.608 INFO:tasks.workunit.client.0.smithi160.stdout:2/320: dwrite d12/d1c/f2c [0,4194304] 0 2024-05-25T06:11:35.608 INFO:tasks.workunit.client.0.smithi160.stdout:3/253: chown d19/d2f/d34/f40 2148817 1 2024-05-25T06:11:35.609 INFO:tasks.workunit.client.1.smithi203.stdout:4/815: dwrite d0/d8/d1f/f2b [8388608,4194304] 0 2024-05-25T06:11:35.615 INFO:tasks.workunit.client.0.smithi160.stdout:6/287: creat d5/de/d32/f67 x:0 0 0 2024-05-25T06:11:35.615 INFO:tasks.workunit.client.0.smithi160.stdout:3/254: write d19/f44 [2554024,99155] 0 2024-05-25T06:11:35.618 INFO:tasks.workunit.client.1.smithi203.stdout:4/816: dwrite d0/d9/fe4 [0,4194304] 0 2024-05-25T06:11:35.619 INFO:tasks.workunit.client.0.smithi160.stdout:5/404: dwrite db/dc/d13/d1d/d31/d3b/f70 [0,4194304] 0 2024-05-25T06:11:35.620 INFO:tasks.workunit.client.0.smithi160.stdout:7/364: dread d6/dc/d64/f3c [4194304,4194304] 0 2024-05-25T06:11:35.622 INFO:tasks.workunit.client.0.smithi160.stdout:1/237: rename de/d1b to de/d21/d49 0 2024-05-25T06:11:35.623 INFO:tasks.workunit.client.1.smithi203.stdout:4/817: symlink d0/d8/d3a/db6/l110 0 2024-05-25T06:11:35.627 INFO:tasks.workunit.client.1.smithi203.stdout:4/818: creat d0/d9/d50/dce/f111 x:0 0 0 2024-05-25T06:11:35.627 INFO:tasks.workunit.client.0.smithi160.stdout:0/381: mkdir d2b/d31/d35/d7b 0 2024-05-25T06:11:35.628 INFO:tasks.workunit.client.0.smithi160.stdout:3/255: creat d19/d2f/d34/f5e x:0 0 0 2024-05-25T06:11:35.630 INFO:tasks.workunit.client.1.smithi203.stdout:4/819: mknod d0/d9/d50/dce/c112 0 2024-05-25T06:11:35.631 INFO:tasks.workunit.client.0.smithi160.stdout:0/382: write d2b/d31/f7a [220012,108254] 0 2024-05-25T06:11:35.634 INFO:tasks.workunit.client.0.smithi160.stdout:0/383: chown d2b/d46/l56 1612 1 2024-05-25T06:11:35.637 INFO:tasks.workunit.client.1.smithi203.stdout:4/820: dwrite d0/d9/d50/d79/f65 [0,4194304] 0 2024-05-25T06:11:35.639 INFO:tasks.workunit.client.0.smithi160.stdout:3/256: write d19/d2f/d34/f55 [789979,4050] 0 2024-05-25T06:11:35.640 INFO:tasks.workunit.client.0.smithi160.stdout:7/365: dwrite d6/dc/d21/f57 [0,4194304] 0 2024-05-25T06:11:35.642 INFO:tasks.workunit.client.0.smithi160.stdout:6/288: dwrite d5/de/f17 [0,4194304] 0 2024-05-25T06:11:35.645 INFO:tasks.workunit.client.0.smithi160.stdout:3/257: symlink d19/d2f/l5f 0 2024-05-25T06:11:35.646 INFO:tasks.workunit.client.0.smithi160.stdout:9/288: write d4/d11/d1e/d3e/d25/f50 [107716,59964] 0 2024-05-25T06:11:35.648 INFO:tasks.workunit.client.0.smithi160.stdout:7/366: unlink d6/dc/d64/d5a/f5e 0 2024-05-25T06:11:35.648 INFO:tasks.workunit.client.0.smithi160.stdout:3/258: truncate d19/d22/f39 4674472 0 2024-05-25T06:11:35.650 INFO:tasks.workunit.client.0.smithi160.stdout:3/259: chown d19/d2f/d38/d47/f54 0 1 2024-05-25T06:11:35.651 INFO:tasks.workunit.client.0.smithi160.stdout:3/260: chown d19/d2f/d38/f3e 0 1 2024-05-25T06:11:35.651 INFO:tasks.workunit.client.0.smithi160.stdout:7/367: chown d6/l7 32843591 1 2024-05-25T06:11:35.653 INFO:tasks.workunit.client.0.smithi160.stdout:9/289: creat d4/d11/d14/f53 x:0 0 0 2024-05-25T06:11:35.653 INFO:tasks.workunit.client.0.smithi160.stdout:8/380: dread d5/d44/f16 [0,4194304] 0 2024-05-25T06:11:35.655 INFO:tasks.workunit.client.0.smithi160.stdout:9/290: stat d4/d11/d1e/d3e/d25/d27 0 2024-05-25T06:11:35.655 INFO:tasks.workunit.client.0.smithi160.stdout:3/261: rmdir d19/d2f/d38 39 2024-05-25T06:11:35.658 INFO:tasks.workunit.client.0.smithi160.stdout:8/381: write d5/d44/f16 [3322883,92744] 0 2024-05-25T06:11:35.659 INFO:tasks.workunit.client.0.smithi160.stdout:7/368: dread d6/d3e/d6f/f40 [0,4194304] 0 2024-05-25T06:11:35.661 INFO:tasks.workunit.client.0.smithi160.stdout:9/291: symlink d4/d11/d1e/d3e/d25/l54 0 2024-05-25T06:11:35.662 INFO:tasks.workunit.client.0.smithi160.stdout:8/382: chown d5/d44/d1f/d1e/d28/d4a/f3a 11498 1 2024-05-25T06:11:35.662 INFO:tasks.workunit.client.0.smithi160.stdout:3/262: mkdir d19/d2f/d38/d60 0 2024-05-25T06:11:35.663 INFO:tasks.workunit.client.0.smithi160.stdout:7/369: creat d6/d3e/d6f/f72 x:0 0 0 2024-05-25T06:11:35.664 INFO:tasks.workunit.client.0.smithi160.stdout:7/370: stat d6/d2b/d5d/f6a 0 2024-05-25T06:11:35.665 INFO:tasks.workunit.client.0.smithi160.stdout:1/238: dread f7 [0,4194304] 0 2024-05-25T06:11:35.665 INFO:tasks.workunit.client.0.smithi160.stdout:8/383: chown d5/c32 112449 1 2024-05-25T06:11:35.665 INFO:tasks.workunit.client.0.smithi160.stdout:6/289: link d5/d30/d36/c48 d5/de/d32/d1b/c68 0 2024-05-25T06:11:35.665 INFO:tasks.workunit.client.0.smithi160.stdout:8/384: chown d5/fb 6615884 1 2024-05-25T06:11:35.667 INFO:tasks.workunit.client.0.smithi160.stdout:9/292: mknod d4/d11/d1e/d3e/c55 0 2024-05-25T06:11:35.668 INFO:tasks.workunit.client.0.smithi160.stdout:7/371: mkdir d6/d3e/d6f/d73 0 2024-05-25T06:11:35.669 INFO:tasks.workunit.client.0.smithi160.stdout:3/263: creat d19/d5d/f61 x:0 0 0 2024-05-25T06:11:35.671 INFO:tasks.workunit.client.0.smithi160.stdout:7/372: chown d6/dc/d64/d5a/l6d 9346180 1 2024-05-25T06:11:35.671 INFO:tasks.workunit.client.0.smithi160.stdout:9/293: symlink d4/d11/d14/l56 0 2024-05-25T06:11:35.672 INFO:tasks.workunit.client.0.smithi160.stdout:1/239: dwrite fb [0,4194304] 0 2024-05-25T06:11:35.674 INFO:tasks.workunit.client.0.smithi160.stdout:8/385: mknod d5/d39/d7a/c7c 0 2024-05-25T06:11:35.674 INFO:tasks.workunit.client.0.smithi160.stdout:6/290: link d5/l19 d5/d30/l69 0 2024-05-25T06:11:35.675 INFO:tasks.workunit.client.0.smithi160.stdout:2/321: truncate d12/d1c/f2b 305351 0 2024-05-25T06:11:35.676 INFO:tasks.workunit.client.0.smithi160.stdout:5/405: truncate db/dc/d13/f65 3001847 0 2024-05-25T06:11:35.678 INFO:tasks.workunit.client.0.smithi160.stdout:7/373: symlink d6/d3e/d6f/d73/l74 0 2024-05-25T06:11:35.678 INFO:tasks.workunit.client.0.smithi160.stdout:7/374: dread - d6/d3e/d6f/f6c zero size 2024-05-25T06:11:35.679 INFO:tasks.workunit.client.0.smithi160.stdout:8/386: creat d5/d39/f7d x:0 0 0 2024-05-25T06:11:35.684 INFO:tasks.workunit.client.0.smithi160.stdout:2/322: rename d12/d1c/d37/d4b/f57 to d12/d1c/d37/f71 0 2024-05-25T06:11:35.686 INFO:tasks.workunit.client.0.smithi160.stdout:9/294: link d4/d11/d1e/d3e/d25/f50 d4/d30/f57 0 2024-05-25T06:11:35.686 INFO:tasks.workunit.client.0.smithi160.stdout:9/295: write d4/f41 [269695,53382] 0 2024-05-25T06:11:35.688 INFO:tasks.workunit.client.0.smithi160.stdout:9/296: readlink d4/d11/d1e/d3e/d2f/l40 0 2024-05-25T06:11:35.688 INFO:tasks.workunit.client.0.smithi160.stdout:8/387: mkdir d5/d44/d7e 0 2024-05-25T06:11:35.689 INFO:tasks.workunit.client.0.smithi160.stdout:8/388: chown d5/l64 55637546 1 2024-05-25T06:11:35.691 INFO:tasks.workunit.client.0.smithi160.stdout:7/375: rename d6/dc/d21/l30 to d6/d2b/d5d/l75 0 2024-05-25T06:11:35.693 INFO:tasks.workunit.client.0.smithi160.stdout:2/323: creat d12/d5c/f72 x:0 0 0 2024-05-25T06:11:35.693 INFO:tasks.workunit.client.0.smithi160.stdout:3/264: rmdir d19/d5d 39 2024-05-25T06:11:35.693 INFO:tasks.workunit.client.0.smithi160.stdout:5/406: dwrite db/dc/d13/d1d/d23/f59 [4194304,4194304] 0 2024-05-25T06:11:35.693 INFO:tasks.workunit.client.0.smithi160.stdout:9/297: fdatasync d4/d11/f38 0 2024-05-25T06:11:35.695 INFO:tasks.workunit.client.0.smithi160.stdout:1/240: getdents de/d21/d49/d28 0 2024-05-25T06:11:35.695 INFO:tasks.workunit.client.0.smithi160.stdout:1/241: chown de/l2c 2 1 2024-05-25T06:11:35.696 INFO:tasks.workunit.client.0.smithi160.stdout:7/376: creat d6/d3e/d6f/f76 x:0 0 0 2024-05-25T06:11:35.697 INFO:tasks.workunit.client.0.smithi160.stdout:3/265: rmdir d19/d2f 39 2024-05-25T06:11:35.698 INFO:tasks.workunit.client.0.smithi160.stdout:6/291: getdents d5/de 0 2024-05-25T06:11:35.699 INFO:tasks.workunit.client.0.smithi160.stdout:8/389: symlink d5/d44/d1f/d6b/l7f 0 2024-05-25T06:11:35.702 INFO:tasks.workunit.client.0.smithi160.stdout:5/407: creat db/dc/d13/d1d/d31/d3b/d69/f79 x:0 0 0 2024-05-25T06:11:35.704 INFO:tasks.workunit.client.0.smithi160.stdout:9/298: dwrite d4/f4e [0,4194304] 0 2024-05-25T06:11:35.706 INFO:tasks.workunit.client.0.smithi160.stdout:7/377: unlink d6/d2b/l5f 0 2024-05-25T06:11:35.708 INFO:tasks.workunit.client.0.smithi160.stdout:6/292: dread d5/f34 [0,4194304] 0 2024-05-25T06:11:35.715 INFO:tasks.workunit.client.0.smithi160.stdout:9/299: mkdir d4/d11/d12/d58 0 2024-05-25T06:11:35.717 INFO:tasks.workunit.client.0.smithi160.stdout:7/378: symlink d6/d3e/l77 0 2024-05-25T06:11:35.717 INFO:tasks.workunit.client.0.smithi160.stdout:9/300: truncate d4/d11/d1e/f4d 264449 0 2024-05-25T06:11:35.717 INFO:tasks.workunit.client.0.smithi160.stdout:7/379: chown d6/d32/f42 123373 1 2024-05-25T06:11:35.719 INFO:tasks.workunit.client.0.smithi160.stdout:1/242: link de/l18 de/d43/l4a 0 2024-05-25T06:11:35.722 INFO:tasks.workunit.client.0.smithi160.stdout:2/324: getdents d12 0 2024-05-25T06:11:35.723 INFO:tasks.workunit.client.0.smithi160.stdout:3/266: mknod d19/d2f/c62 0 2024-05-25T06:11:35.727 INFO:tasks.workunit.client.0.smithi160.stdout:6/293: unlink d5/de/d32/c25 0 2024-05-25T06:11:35.727 INFO:tasks.workunit.client.0.smithi160.stdout:3/267: dread d19/d2f/d34/f55 [0,4194304] 0 2024-05-25T06:11:35.728 INFO:tasks.workunit.client.0.smithi160.stdout:9/301: dread d4/d11/d1e/d24/f4f [0,4194304] 0 2024-05-25T06:11:35.728 INFO:tasks.workunit.client.0.smithi160.stdout:7/380: creat d6/dc/d64/f78 x:0 0 0 2024-05-25T06:11:35.735 INFO:tasks.workunit.client.0.smithi160.stdout:6/294: unlink d5/f7 0 2024-05-25T06:11:35.736 INFO:tasks.workunit.client.0.smithi160.stdout:3/268: rmdir d19/d22 39 2024-05-25T06:11:35.739 INFO:tasks.workunit.client.0.smithi160.stdout:7/381: rmdir d6 39 2024-05-25T06:11:35.740 INFO:tasks.workunit.client.0.smithi160.stdout:9/302: unlink d4/d11/d1e/d24/f39 0 2024-05-25T06:11:35.741 INFO:tasks.workunit.client.0.smithi160.stdout:9/303: dread - d4/d11/d14/f53 zero size 2024-05-25T06:11:35.741 INFO:tasks.workunit.client.0.smithi160.stdout:2/325: write d12/f25 [4011037,84838] 0 2024-05-25T06:11:35.742 INFO:tasks.workunit.client.0.smithi160.stdout:9/304: chown d4/d11/c19 11046 1 2024-05-25T06:11:35.742 INFO:tasks.workunit.client.0.smithi160.stdout:8/390: getdents d5/d44/d1f/d1e/d28/d4a/d4e 0 2024-05-25T06:11:35.744 INFO:tasks.workunit.client.0.smithi160.stdout:6/295: fdatasync d5/f1e 0 2024-05-25T06:11:35.747 INFO:tasks.workunit.client.0.smithi160.stdout:2/326: dread - d12/d3c/f61 zero size 2024-05-25T06:11:35.752 INFO:tasks.workunit.client.0.smithi160.stdout:3/269: truncate f11 6084263 0 2024-05-25T06:11:35.753 INFO:tasks.workunit.client.0.smithi160.stdout:7/382: rename d6/d32/c3d to d6/d32/c79 0 2024-05-25T06:11:35.754 INFO:tasks.workunit.client.0.smithi160.stdout:9/305: dwrite d4/d11/d1e/d24/f1b [0,4194304] 0 2024-05-25T06:11:35.754 INFO:tasks.workunit.client.0.smithi160.stdout:1/243: getdents de 0 2024-05-25T06:11:35.756 INFO:tasks.workunit.client.0.smithi160.stdout:7/383: fdatasync d6/d3e/f70 0 2024-05-25T06:11:35.761 INFO:tasks.workunit.client.0.smithi160.stdout:6/296: chown d5/de/d32/d35/f4e 1157 1 2024-05-25T06:11:35.761 INFO:tasks.workunit.client.0.smithi160.stdout:3/270: unlink d19/l1d 0 2024-05-25T06:11:35.762 INFO:tasks.workunit.client.0.smithi160.stdout:8/391: truncate d5/d44/d1f/d1e/d28/f3f 585575 0 2024-05-25T06:11:35.762 INFO:tasks.workunit.client.0.smithi160.stdout:6/297: chown d5/de/d32/d35/f5f 291 1 2024-05-25T06:11:35.764 INFO:tasks.workunit.client.0.smithi160.stdout:8/392: chown d5/d44/d1f/d1e/l56 235029938 1 2024-05-25T06:11:35.765 INFO:tasks.workunit.client.0.smithi160.stdout:1/244: link de/d21/c36 de/d21/d49/d35/c4b 0 2024-05-25T06:11:35.767 INFO:tasks.workunit.client.0.smithi160.stdout:7/384: creat d6/d2b/f7a x:0 0 0 2024-05-25T06:11:35.767 INFO:tasks.workunit.client.0.smithi160.stdout:2/327: truncate d12/d13/f55 1800836 0 2024-05-25T06:11:35.770 INFO:tasks.workunit.client.0.smithi160.stdout:1/245: truncate de/d43/f46 743548 0 2024-05-25T06:11:35.770 INFO:tasks.workunit.client.0.smithi160.stdout:7/385: fdatasync d6/d3e/f70 0 2024-05-25T06:11:35.772 INFO:tasks.workunit.client.0.smithi160.stdout:8/393: fdatasync d5/d44/d1f/d1e/d28/d4a/f5c 0 2024-05-25T06:11:35.775 INFO:tasks.workunit.client.0.smithi160.stdout:2/328: mknod d12/d3e/d5f/c73 0 2024-05-25T06:11:35.777 INFO:tasks.workunit.client.0.smithi160.stdout:3/271: dwrite d19/d2f/d34/f5e [0,4194304] 0 2024-05-25T06:11:35.777 INFO:tasks.workunit.client.0.smithi160.stdout:7/386: write d6/dc/d21/f71 [330666,77515] 0 2024-05-25T06:11:35.779 INFO:tasks.workunit.client.0.smithi160.stdout:1/246: read de/f15 [98374,125359] 0 2024-05-25T06:11:35.782 INFO:tasks.workunit.client.0.smithi160.stdout:8/394: symlink d5/d39/l80 0 2024-05-25T06:11:35.782 INFO:tasks.workunit.client.0.smithi160.stdout:1/247: fdatasync de/f3d 0 2024-05-25T06:11:35.782 INFO:tasks.workunit.client.0.smithi160.stdout:2/329: write d12/d13/f60 [650973,30389] 0 2024-05-25T06:11:35.783 INFO:tasks.workunit.client.0.smithi160.stdout:4/352: sync 2024-05-25T06:11:35.785 INFO:tasks.workunit.client.0.smithi160.stdout:6/298: dwrite d5/d30/d36/f49 [0,4194304] 0 2024-05-25T06:11:35.786 INFO:tasks.workunit.client.0.smithi160.stdout:7/387: rmdir d6/d32/d5b 39 2024-05-25T06:11:35.787 INFO:tasks.workunit.client.0.smithi160.stdout:9/306: dwrite d4/d11/d12/f35 [0,4194304] 0 2024-05-25T06:11:35.790 INFO:tasks.workunit.client.0.smithi160.stdout:7/388: write d6/dc/d64/f45 [892730,70079] 0 2024-05-25T06:11:35.795 INFO:tasks.workunit.client.0.smithi160.stdout:3/272: fsync f11 0 2024-05-25T06:11:35.798 INFO:tasks.workunit.client.0.smithi160.stdout:1/248: mknod de/d21/d49/d1c/c4c 0 2024-05-25T06:11:35.799 INFO:tasks.workunit.client.0.smithi160.stdout:8/395: creat d5/d44/d1f/d6b/f81 x:0 0 0 2024-05-25T06:11:35.799 INFO:tasks.workunit.client.0.smithi160.stdout:2/330: mkdir d12/d3e/d5f/d74 0 2024-05-25T06:11:35.800 INFO:tasks.workunit.client.0.smithi160.stdout:9/307: read d4/d11/d12/d2e/f32 [1262253,70702] 0 2024-05-25T06:11:35.801 INFO:tasks.workunit.client.0.smithi160.stdout:4/353: rename d4/cf to d4/d1a/c6d 0 2024-05-25T06:11:35.802 INFO:tasks.workunit.client.0.smithi160.stdout:1/249: creat de/f4d x:0 0 0 2024-05-25T06:11:35.803 INFO:tasks.workunit.client.0.smithi160.stdout:3/273: rmdir d19/d5d 39 2024-05-25T06:11:35.803 INFO:tasks.workunit.client.0.smithi160.stdout:4/354: fsync d4/d1c/f3e 0 2024-05-25T06:11:35.803 INFO:tasks.workunit.client.0.smithi160.stdout:8/396: chown d5/d44/d1f/d1e/d28/d49/f70 1777633 1 2024-05-25T06:11:35.804 INFO:tasks.workunit.client.0.smithi160.stdout:7/389: chown d6/d2b/d5d/c61 1 1 2024-05-25T06:11:35.804 INFO:tasks.workunit.client.0.smithi160.stdout:2/331: truncate d12/d5c/f72 322258 0 2024-05-25T06:11:35.805 INFO:tasks.workunit.client.0.smithi160.stdout:9/308: dread d4/d11/d1e/f4d [0,4194304] 0 2024-05-25T06:11:35.806 INFO:tasks.workunit.client.0.smithi160.stdout:6/299: dwrite d5/f1e [0,4194304] 0 2024-05-25T06:11:35.807 INFO:tasks.workunit.client.0.smithi160.stdout:4/355: read d4/d6/d21/f48 [131044,119246] 0 2024-05-25T06:11:35.807 INFO:tasks.workunit.client.0.smithi160.stdout:7/390: read d6/dc/d64/f37 [2979524,35818] 0 2024-05-25T06:11:35.808 INFO:tasks.workunit.client.0.smithi160.stdout:9/309: dread - d4/d11/d1e/d24/f13 zero size 2024-05-25T06:11:35.809 INFO:tasks.workunit.client.0.smithi160.stdout:2/332: mkdir d12/d75 0 2024-05-25T06:11:35.813 INFO:tasks.workunit.client.0.smithi160.stdout:8/397: read d5/d44/d1f/f55 [256391,88709] 0 2024-05-25T06:11:35.815 INFO:tasks.workunit.client.0.smithi160.stdout:9/310: mknod d4/d11/d1e/d3e/d25/d27/c59 0 2024-05-25T06:11:35.820 INFO:tasks.workunit.client.0.smithi160.stdout:7/391: dwrite d6/d3e/d59/f66 [0,4194304] 0 2024-05-25T06:11:35.822 INFO:tasks.workunit.client.0.smithi160.stdout:2/333: dwrite fe [0,4194304] 0 2024-05-25T06:11:35.823 INFO:tasks.workunit.client.0.smithi160.stdout:4/356: dwrite d4/d6/d35/f53 [8388608,4194304] 0 2024-05-25T06:11:35.823 INFO:tasks.workunit.client.0.smithi160.stdout:3/274: truncate f13 1019943 0 2024-05-25T06:11:35.828 INFO:tasks.workunit.client.0.smithi160.stdout:8/398: creat d5/f82 x:0 0 0 2024-05-25T06:11:35.830 INFO:tasks.workunit.client.0.smithi160.stdout:6/300: getdents d5/de/d18 0 2024-05-25T06:11:35.830 INFO:tasks.workunit.client.0.smithi160.stdout:3/275: truncate d19/d2f/d38/d47/f57 792877 0 2024-05-25T06:11:35.833 INFO:tasks.workunit.client.0.smithi160.stdout:7/392: symlink d6/d2b/d5d/l7b 0 2024-05-25T06:11:35.835 INFO:tasks.workunit.client.0.smithi160.stdout:4/357: unlink d4/d6/d5b/l5f 0 2024-05-25T06:11:35.835 INFO:tasks.workunit.client.0.smithi160.stdout:1/250: readlink de/d43/l4a 0 2024-05-25T06:11:35.838 INFO:tasks.workunit.client.0.smithi160.stdout:2/334: chown d12/d1c/d37 34950 1 2024-05-25T06:11:35.838 INFO:tasks.workunit.client.0.smithi160.stdout:7/393: unlink d6/l22 0 2024-05-25T06:11:35.840 INFO:tasks.workunit.client.0.smithi160.stdout:4/358: mknod d4/c6e 0 2024-05-25T06:11:35.843 INFO:tasks.workunit.client.0.smithi160.stdout:1/251: mknod de/d21/d49/d28/c4e 0 2024-05-25T06:11:35.845 INFO:tasks.workunit.client.0.smithi160.stdout:2/335: mkdir d12/d1c/d37/d4b/d76 0 2024-05-25T06:11:35.846 INFO:tasks.workunit.client.0.smithi160.stdout:3/276: dwrite d19/f27 [0,4194304] 0 2024-05-25T06:11:35.847 INFO:tasks.workunit.client.0.smithi160.stdout:8/399: dwrite d5/d9/f40 [4194304,4194304] 0 2024-05-25T06:11:35.847 INFO:tasks.workunit.client.0.smithi160.stdout:7/394: truncate d6/dc/d21/f4f 4548037 0 2024-05-25T06:11:35.848 INFO:tasks.workunit.client.0.smithi160.stdout:6/301: dwrite d5/de/d32/d1b/f55 [0,4194304] 0 2024-05-25T06:11:35.848 INFO:tasks.workunit.client.0.smithi160.stdout:4/359: mknod d4/d6/d5b/d66/c6f 0 2024-05-25T06:11:35.853 INFO:tasks.workunit.client.0.smithi160.stdout:3/277: fsync d19/d2f/d38/d47/f57 0 2024-05-25T06:11:35.853 INFO:tasks.workunit.client.0.smithi160.stdout:6/302: stat d5/de/d32/d1b/c68 0 2024-05-25T06:11:35.860 INFO:tasks.workunit.client.0.smithi160.stdout:7/395: rename d6/d32/d46 to d6/dc/d64/d5a/d7c 0 2024-05-25T06:11:35.860 INFO:tasks.workunit.client.0.smithi160.stdout:0/384: sync 2024-05-25T06:11:35.861 INFO:tasks.workunit.client.0.smithi160.stdout:8/400: symlink d5/d9/d72/l83 0 2024-05-25T06:11:35.861 INFO:tasks.workunit.client.0.smithi160.stdout:9/311: dread d4/d11/d12/d2e/f32 [0,4194304] 0 2024-05-25T06:11:35.865 INFO:tasks.workunit.client.0.smithi160.stdout:9/312: dread d4/d11/d12/d2e/f32 [0,4194304] 0 2024-05-25T06:11:35.870 INFO:tasks.workunit.client.0.smithi160.stdout:0/385: symlink d2b/d58/l7c 0 2024-05-25T06:11:35.871 INFO:tasks.workunit.client.0.smithi160.stdout:0/386: write f5 [3717300,18013] 0 2024-05-25T06:11:35.871 INFO:tasks.workunit.client.0.smithi160.stdout:7/396: mknod d6/d3e/d6f/d73/c7d 0 2024-05-25T06:11:35.872 INFO:tasks.workunit.client.0.smithi160.stdout:8/401: symlink d5/d44/d1f/d1e/d28/d4a/l84 0 2024-05-25T06:11:35.873 INFO:tasks.workunit.client.0.smithi160.stdout:2/336: link f9 d12/d3e/d5f/d74/f77 0 2024-05-25T06:11:35.874 INFO:tasks.workunit.client.0.smithi160.stdout:6/303: symlink d5/de/d58/l6a 0 2024-05-25T06:11:35.875 INFO:tasks.workunit.client.0.smithi160.stdout:9/313: mknod d4/d11/d1e/d3e/d25/d27/c5a 0 2024-05-25T06:11:35.878 INFO:tasks.workunit.client.0.smithi160.stdout:6/304: truncate d5/d39/f41 553536 0 2024-05-25T06:11:35.878 INFO:tasks.workunit.client.0.smithi160.stdout:0/387: creat d2b/d31/f7d x:0 0 0 2024-05-25T06:11:35.900 INFO:tasks.workunit.client.1.smithi203.stdout:2/704: sync 2024-05-25T06:11:35.901 INFO:tasks.workunit.client.1.smithi203.stdout:9/585: sync 2024-05-25T06:11:35.901 INFO:tasks.workunit.client.1.smithi203.stdout:7/641: sync 2024-05-25T06:11:35.901 INFO:tasks.workunit.client.1.smithi203.stdout:9/586: truncate d0/fa6 1039706 0 2024-05-25T06:11:35.901 INFO:tasks.workunit.client.1.smithi203.stdout:7/642: chown d1/d11/d20/f23 489380702 1 2024-05-25T06:11:35.902 INFO:tasks.workunit.client.1.smithi203.stdout:7/643: chown d1/d9/fb 0 1 2024-05-25T06:11:35.903 INFO:tasks.workunit.client.1.smithi203.stdout:2/705: mkdir d2/d1a/d89/daa/def 0 2024-05-25T06:11:35.905 INFO:tasks.workunit.client.1.smithi203.stdout:2/706: creat d2/d38/d2f/ff0 x:0 0 0 2024-05-25T06:11:35.907 INFO:tasks.workunit.client.1.smithi203.stdout:9/587: creat d0/d17/d20/fc8 x:0 0 0 2024-05-25T06:11:35.908 INFO:tasks.workunit.client.1.smithi203.stdout:9/588: dread - d0/df/d73/fa7 zero size 2024-05-25T06:11:35.909 INFO:tasks.workunit.client.1.smithi203.stdout:2/707: symlink d2/d4/d10/d32/lf1 0 2024-05-25T06:11:35.910 INFO:tasks.workunit.client.1.smithi203.stdout:2/708: truncate d2/d38/fe2 763821 0 2024-05-25T06:11:35.912 INFO:tasks.workunit.client.1.smithi203.stdout:2/709: creat d2/d42/ff2 x:0 0 0 2024-05-25T06:11:35.914 INFO:tasks.workunit.client.1.smithi203.stdout:7/644: dread d1/d9/d2c/d3a/f4b [0,4194304] 0 2024-05-25T06:11:35.914 INFO:tasks.workunit.client.1.smithi203.stdout:2/710: fdatasync d2/d4/d10/f5e 0 2024-05-25T06:11:35.920 INFO:tasks.workunit.client.1.smithi203.stdout:9/589: link d0/d17/d20/d3c/l3d d0/d17/d20/d3c/db2/d7a/lc9 0 2024-05-25T06:11:35.926 INFO:tasks.workunit.client.1.smithi203.stdout:7/645: dread d1/d9/d2c/d36/f9d [0,4194304] 0 2024-05-25T06:11:35.929 INFO:tasks.workunit.client.1.smithi203.stdout:2/711: write d2/d1a/d95/fa1 [1182151,87394] 0 2024-05-25T06:11:35.930 INFO:tasks.workunit.client.1.smithi203.stdout:7/646: rmdir d1/d9/d2c/d3a/d4e/db5 0 2024-05-25T06:11:35.932 INFO:tasks.workunit.client.1.smithi203.stdout:2/712: mknod d2/d38/d43/cf3 0 2024-05-25T06:11:35.933 INFO:tasks.workunit.client.1.smithi203.stdout:2/713: fsync d2/d38/d43/d60/d6b/dae/f51 0 2024-05-25T06:11:35.933 INFO:tasks.workunit.client.1.smithi203.stdout:7/647: mkdir d1/d9/d2c/d3a/dde 0 2024-05-25T06:11:35.933 INFO:tasks.workunit.client.1.smithi203.stdout:2/714: stat d2/cb7 0 2024-05-25T06:11:35.935 INFO:tasks.workunit.client.1.smithi203.stdout:2/715: mknod d2/d42/d8e/cf4 0 2024-05-25T06:11:35.936 INFO:tasks.workunit.client.1.smithi203.stdout:9/590: dread d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:35.937 INFO:tasks.workunit.client.1.smithi203.stdout:7/648: link d1/d9/c1e d1/d9/d2c/d36/d54/cdf 0 2024-05-25T06:11:35.938 INFO:tasks.workunit.client.1.smithi203.stdout:2/716: creat d2/d1a/dd3/ff5 x:0 0 0 2024-05-25T06:11:35.941 INFO:tasks.workunit.client.1.smithi203.stdout:7/649: fsync d1/d11/d20/f23 0 2024-05-25T06:11:35.943 INFO:tasks.workunit.client.1.smithi203.stdout:7/650: write d1/d9/d2c/f5d [1049644,14601] 0 2024-05-25T06:11:35.944 INFO:tasks.workunit.client.1.smithi203.stdout:2/717: dwrite d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:35.944 INFO:tasks.workunit.client.0.smithi160.stdout:7/397: mkdir d6/d32/d7e 0 2024-05-25T06:11:35.946 INFO:tasks.workunit.client.1.smithi203.stdout:9/591: write d0/d17/d20/d6e/f65 [706382,127247] 0 2024-05-25T06:11:35.947 INFO:tasks.workunit.client.0.smithi160.stdout:9/314: mknod d4/d11/d1e/d3e/d25/d27/c5b 0 2024-05-25T06:11:35.948 INFO:tasks.workunit.client.0.smithi160.stdout:6/305: write d5/d39/f41 [471637,84811] 0 2024-05-25T06:11:35.949 INFO:tasks.workunit.client.1.smithi203.stdout:2/718: unlink d2/d38/d43/lc2 0 2024-05-25T06:11:35.950 INFO:tasks.workunit.client.0.smithi160.stdout:1/252: truncate de/d21/d49/d1c/f3f 304252 0 2024-05-25T06:11:35.950 INFO:tasks.workunit.client.1.smithi203.stdout:7/651: mknod d1/d9/d60/dac/ce0 0 2024-05-25T06:11:35.950 INFO:tasks.workunit.client.1.smithi203.stdout:7/652: chown d1/dbd/dc1/lc4 1139389 1 2024-05-25T06:11:35.951 INFO:tasks.workunit.client.1.smithi203.stdout:9/592: symlink d0/d17/d20/d6e/d75/lca 0 2024-05-25T06:11:35.952 INFO:tasks.workunit.client.0.smithi160.stdout:4/360: dread d4/d6/d21/f24 [0,4194304] 0 2024-05-25T06:11:35.953 INFO:tasks.workunit.client.0.smithi160.stdout:6/306: dread - d5/de/d32/f67 zero size 2024-05-25T06:11:35.954 INFO:tasks.workunit.client.0.smithi160.stdout:7/398: chown d6/d3e/d6f/f40 50108 1 2024-05-25T06:11:35.954 INFO:tasks.workunit.client.0.smithi160.stdout:9/315: chown d4/d11/d1e/d3e/d25 116 1 2024-05-25T06:11:35.955 INFO:tasks.workunit.client.1.smithi203.stdout:2/719: unlink d2/d38/d2f/fab 0 2024-05-25T06:11:35.955 INFO:tasks.workunit.client.1.smithi203.stdout:7/653: mkdir d1/d11/d3d/d4c/de1 0 2024-05-25T06:11:35.956 INFO:tasks.workunit.client.0.smithi160.stdout:2/337: dwrite d12/d1c/d37/f4f [0,4194304] 0 2024-05-25T06:11:35.956 INFO:tasks.workunit.client.1.smithi203.stdout:7/654: chown d1/d11/d3d/d9a/fab 25554 1 2024-05-25T06:11:35.957 INFO:tasks.workunit.client.1.smithi203.stdout:9/593: symlink d0/d17/d20/d3c/lcb 0 2024-05-25T06:11:35.957 INFO:tasks.workunit.client.1.smithi203.stdout:7/655: chown d1/d11/d20/f28 8 1 2024-05-25T06:11:35.958 INFO:tasks.workunit.client.1.smithi203.stdout:2/720: fsync d2/fe 0 2024-05-25T06:11:35.959 INFO:tasks.workunit.client.0.smithi160.stdout:1/253: unlink de/c12 0 2024-05-25T06:11:35.959 INFO:tasks.workunit.client.0.smithi160.stdout:4/361: mkdir d4/d63/d70 0 2024-05-25T06:11:35.960 INFO:tasks.workunit.client.1.smithi203.stdout:9/594: rmdir d0/df/d2d/d92 39 2024-05-25T06:11:35.961 INFO:tasks.workunit.client.0.smithi160.stdout:3/278: dwrite d19/d2f/d34/f55 [0,4194304] 0 2024-05-25T06:11:35.961 INFO:tasks.workunit.client.0.smithi160.stdout:0/388: getdents d2b/d31/d35/d36/d52 0 2024-05-25T06:11:35.962 INFO:tasks.workunit.client.0.smithi160.stdout:7/399: symlink d6/d32/d7e/l7f 0 2024-05-25T06:11:35.962 INFO:tasks.workunit.client.0.smithi160.stdout:0/389: stat l7 0 2024-05-25T06:11:35.963 INFO:tasks.workunit.client.0.smithi160.stdout:9/316: truncate d4/d11/d1e/f4d 736194 0 2024-05-25T06:11:35.964 INFO:tasks.workunit.client.1.smithi203.stdout:2/721: mknod d2/d4/d10/d58/da0/cf6 0 2024-05-25T06:11:35.968 INFO:tasks.workunit.client.1.smithi203.stdout:9/595: link d0/d17/d20/d29/f32 d0/df/d37/d4f/fcc 0 2024-05-25T06:11:35.969 INFO:tasks.workunit.client.1.smithi203.stdout:9/596: stat d0/fa6 0 2024-05-25T06:11:35.969 INFO:tasks.workunit.client.0.smithi160.stdout:6/307: dwrite d5/de/d32/f37 [0,4194304] 0 2024-05-25T06:11:35.971 INFO:tasks.workunit.client.1.smithi203.stdout:2/722: dread - d2/d1a/f97 zero size 2024-05-25T06:11:35.972 INFO:tasks.workunit.client.0.smithi160.stdout:3/279: symlink d19/d2f/d38/d43/l63 0 2024-05-25T06:11:35.974 INFO:tasks.workunit.client.1.smithi203.stdout:9/597: fdatasync d0/d17/d20/f44 0 2024-05-25T06:11:35.976 INFO:tasks.workunit.client.1.smithi203.stdout:7/656: rmdir d1/d11 39 2024-05-25T06:11:35.977 INFO:tasks.workunit.client.1.smithi203.stdout:7/657: dread - d1/d9/d2c/d3a/dd1/fda zero size 2024-05-25T06:11:35.977 INFO:tasks.workunit.client.1.smithi203.stdout:2/723: truncate d2/d1a/f97 94283 0 2024-05-25T06:11:35.980 INFO:tasks.workunit.client.0.smithi160.stdout:6/308: chown d5/de/f42 0 1 2024-05-25T06:11:35.980 INFO:tasks.workunit.client.0.smithi160.stdout:2/338: truncate d12/d13/f15 1809191 0 2024-05-25T06:11:35.981 INFO:tasks.workunit.client.0.smithi160.stdout:6/309: chown d5/de/d58/f5b 25794524 1 2024-05-25T06:11:35.981 INFO:tasks.workunit.client.0.smithi160.stdout:9/317: creat d4/f5c x:0 0 0 2024-05-25T06:11:35.983 INFO:tasks.workunit.client.0.smithi160.stdout:6/310: chown d5/de/d32/d1b/f55 50132147 1 2024-05-25T06:11:35.983 INFO:tasks.workunit.client.0.smithi160.stdout:9/318: stat d4/d11/d1e/d3e/d25/l51 0 2024-05-25T06:11:35.983 INFO:tasks.workunit.client.0.smithi160.stdout:7/400: rename d6/d3e/d6f/f6c to d6/d32/f80 0 2024-05-25T06:11:35.984 INFO:tasks.workunit.client.0.smithi160.stdout:7/401: stat d6/d3e/d59 0 2024-05-25T06:11:35.985 INFO:tasks.workunit.client.1.smithi203.stdout:9/598: link d0/df/d2d/d92/caf d0/ccd 0 2024-05-25T06:11:35.986 INFO:tasks.workunit.client.0.smithi160.stdout:6/311: truncate d5/de/f12 577751 0 2024-05-25T06:11:35.987 INFO:tasks.workunit.client.0.smithi160.stdout:9/319: creat d4/d11/d1e/d3e/d25/d27/f5d x:0 0 0 2024-05-25T06:11:35.988 INFO:tasks.workunit.client.0.smithi160.stdout:7/402: mkdir d6/dc/d64/d5a/d7c/d81 0 2024-05-25T06:11:35.990 INFO:tasks.workunit.client.1.smithi203.stdout:9/599: rename d0/df/d73/c9f to d0/d57/d8f/cce 0 2024-05-25T06:11:35.990 INFO:tasks.workunit.client.0.smithi160.stdout:7/403: creat d6/dc/f82 x:0 0 0 2024-05-25T06:11:35.991 INFO:tasks.workunit.client.1.smithi203.stdout:9/600: readlink d0/d17/d20/d29/l99 0 2024-05-25T06:11:35.992 INFO:tasks.workunit.client.1.smithi203.stdout:9/601: chown d0/d57/c68 991061892 1 2024-05-25T06:11:35.993 INFO:tasks.workunit.client.0.smithi160.stdout:9/320: unlink d4/f47 0 2024-05-25T06:11:35.993 INFO:tasks.workunit.client.0.smithi160.stdout:7/404: write d6/d3e/d6f/f6e [1363150,91461] 0 2024-05-25T06:11:35.994 INFO:tasks.workunit.client.0.smithi160.stdout:0/390: write d2b/d31/d35/d36/d37/f44 [322162,97020] 0 2024-05-25T06:11:35.995 INFO:tasks.workunit.client.1.smithi203.stdout:9/602: truncate d0/f39 1297483 0 2024-05-25T06:11:35.996 INFO:tasks.workunit.client.0.smithi160.stdout:1/254: write de/d21/d49/f27 [1046664,90274] 0 2024-05-25T06:11:35.996 INFO:tasks.workunit.client.0.smithi160.stdout:4/362: truncate d4/d6/f7 1407982 0 2024-05-25T06:11:35.998 INFO:tasks.workunit.client.0.smithi160.stdout:0/391: write f5 [344989,123430] 0 2024-05-25T06:11:35.999 INFO:tasks.workunit.client.1.smithi203.stdout:9/603: creat d0/df/d2d/d92/fcf x:0 0 0 2024-05-25T06:11:36.001 INFO:tasks.workunit.client.0.smithi160.stdout:6/312: dwrite d5/de/ff [0,4194304] 0 2024-05-25T06:11:36.002 INFO:tasks.workunit.client.0.smithi160.stdout:3/280: truncate d19/f1e 699692 0 2024-05-25T06:11:36.004 INFO:tasks.workunit.client.0.smithi160.stdout:6/313: stat l4 0 2024-05-25T06:11:36.007 INFO:tasks.workunit.client.1.smithi203.stdout:9/604: rename d0/df/d2b/d96 to d0/dd0 0 2024-05-25T06:11:36.008 INFO:tasks.workunit.client.0.smithi160.stdout:0/392: creat d2b/d46/d63/f7e x:0 0 0 2024-05-25T06:11:36.008 INFO:tasks.workunit.client.0.smithi160.stdout:6/314: fsync d5/de/d32/f67 0 2024-05-25T06:11:36.009 INFO:tasks.workunit.client.1.smithi203.stdout:9/605: readlink d0/lb0 0 2024-05-25T06:11:36.009 INFO:tasks.workunit.client.0.smithi160.stdout:5/408: sync 2024-05-25T06:11:36.011 INFO:tasks.workunit.client.1.smithi203.stdout:2/724: dread d2/d38/d2f/d69/fd7 [0,4194304] 0 2024-05-25T06:11:36.012 INFO:tasks.workunit.client.0.smithi160.stdout:3/281: creat d19/d2f/d38/d47/f64 x:0 0 0 2024-05-25T06:11:36.013 INFO:tasks.workunit.client.1.smithi203.stdout:7/658: dread d1/d9/d2c/d3a/d4e/f6c [0,4194304] 0 2024-05-25T06:11:36.013 INFO:tasks.workunit.client.0.smithi160.stdout:2/339: dwrite d12/d1c/f31 [0,4194304] 0 2024-05-25T06:11:36.013 INFO:tasks.workunit.client.0.smithi160.stdout:5/409: chown db/dc/d13/c3c 0 1 2024-05-25T06:11:36.014 INFO:tasks.workunit.client.1.smithi203.stdout:2/725: chown d2/d42/d4e/d68/faf 2163054 1 2024-05-25T06:11:36.014 INFO:tasks.workunit.client.1.smithi203.stdout:2/726: stat d2/d4/d10/l40 0 2024-05-25T06:11:36.016 INFO:tasks.workunit.client.0.smithi160.stdout:4/363: getdents d4/d6/d5b 0 2024-05-25T06:11:36.016 INFO:tasks.workunit.client.0.smithi160.stdout:2/340: write d12/d1c/f56 [361670,126905] 0 2024-05-25T06:11:36.016 INFO:tasks.workunit.client.1.smithi203.stdout:2/727: fdatasync d2/d1a/d87/fbe 0 2024-05-25T06:11:36.016 INFO:tasks.workunit.client.1.smithi203.stdout:7/659: mknod d1/d9/d2c/d3a/d4e/d50/ce2 0 2024-05-25T06:11:36.017 INFO:tasks.workunit.client.0.smithi160.stdout:1/255: dwrite de/d21/d49/d35/f24 [0,4194304] 0 2024-05-25T06:11:36.019 INFO:tasks.workunit.client.0.smithi160.stdout:0/393: rmdir d2b/d31/d35/d36 39 2024-05-25T06:11:36.021 INFO:tasks.workunit.client.0.smithi160.stdout:5/410: write db/dc/d13/d1d/d31/d56/f78 [160186,91713] 0 2024-05-25T06:11:36.021 INFO:tasks.workunit.client.0.smithi160.stdout:2/341: write d12/d1c/d2a/f3a [1228836,2303] 0 2024-05-25T06:11:36.022 INFO:tasks.workunit.client.0.smithi160.stdout:9/321: truncate d4/d11/d14/f3a 589856 0 2024-05-25T06:11:36.023 INFO:tasks.workunit.client.0.smithi160.stdout:5/411: chown db/dc/d13/d1d/d3f/c6c 5 1 2024-05-25T06:11:36.024 INFO:tasks.workunit.client.0.smithi160.stdout:6/315: unlink d5/de/d32/d35/l47 0 2024-05-25T06:11:36.025 INFO:tasks.workunit.client.0.smithi160.stdout:5/412: write db/dc/d13/d1d/d23/d24/d2e/f5b [2318824,107901] 0 2024-05-25T06:11:36.025 INFO:tasks.workunit.client.0.smithi160.stdout:6/316: dread - d5/d45/d4a/f53 zero size 2024-05-25T06:11:36.026 INFO:tasks.workunit.client.0.smithi160.stdout:3/282: stat d19/d22/f2c 0 2024-05-25T06:11:36.026 INFO:tasks.workunit.client.0.smithi160.stdout:3/283: chown d19/d2f/c3f 5882 1 2024-05-25T06:11:36.027 INFO:tasks.workunit.client.1.smithi203.stdout:2/728: dwrite d2/d4/d10/f21 [0,4194304] 0 2024-05-25T06:11:36.029 INFO:tasks.workunit.client.1.smithi203.stdout:2/729: fdatasync d2/d4/d10/d5f/fc0 0 2024-05-25T06:11:36.032 INFO:tasks.workunit.client.0.smithi160.stdout:3/284: fsync d19/d2f/d38/d47/f64 0 2024-05-25T06:11:36.034 INFO:tasks.workunit.client.1.smithi203.stdout:2/730: truncate d2/d4/d10/d5f/f85 4749162 0 2024-05-25T06:11:36.035 INFO:tasks.workunit.client.1.smithi203.stdout:9/606: dread d0/f6f [0,4194304] 0 2024-05-25T06:11:36.037 INFO:tasks.workunit.client.0.smithi160.stdout:4/364: rename d4/d1c/d54/f61 to d4/d6/d5b/d66/f71 0 2024-05-25T06:11:36.038 INFO:tasks.workunit.client.1.smithi203.stdout:2/731: creat d2/d1a/d87/d9e/ff7 x:0 0 0 2024-05-25T06:11:36.039 INFO:tasks.workunit.client.0.smithi160.stdout:2/342: symlink d12/d3e/l78 0 2024-05-25T06:11:36.040 INFO:tasks.workunit.client.0.smithi160.stdout:6/317: unlink d5/f2c 0 2024-05-25T06:11:36.041 INFO:tasks.workunit.client.0.smithi160.stdout:5/413: symlink db/dc/d13/d1d/d23/d24/d2e/l7a 0 2024-05-25T06:11:36.041 INFO:tasks.workunit.client.0.smithi160.stdout:7/405: getdents d6/dc/d64/d5a/d7c 0 2024-05-25T06:11:36.042 INFO:tasks.workunit.client.1.smithi203.stdout:2/732: creat d2/d1a/d89/daa/ff8 x:0 0 0 2024-05-25T06:11:36.042 INFO:tasks.workunit.client.1.smithi203.stdout:2/733: chown d2 72 1 2024-05-25T06:11:36.043 INFO:tasks.workunit.client.0.smithi160.stdout:5/414: fsync db/dc/d13/d1d/d31/d3b/d69/f79 0 2024-05-25T06:11:36.044 INFO:tasks.workunit.client.0.smithi160.stdout:5/415: dread - db/dc/d13/d1d/d31/d56/f61 zero size 2024-05-25T06:11:36.044 INFO:tasks.workunit.client.1.smithi203.stdout:2/734: mknod d2/d1a/d89/cf9 0 2024-05-25T06:11:36.045 INFO:tasks.workunit.client.0.smithi160.stdout:5/416: fsync db/dc/d13/d1d/f20 0 2024-05-25T06:11:36.046 INFO:tasks.workunit.client.0.smithi160.stdout:0/394: creat d2b/d31/d35/d36/f7f x:0 0 0 2024-05-25T06:11:36.047 INFO:tasks.workunit.client.0.smithi160.stdout:0/395: chown d2b/f42 1 1 2024-05-25T06:11:36.048 INFO:tasks.workunit.client.0.smithi160.stdout:7/406: dwrite d6/dc/d21/f4f [0,4194304] 0 2024-05-25T06:11:36.051 INFO:tasks.workunit.client.1.smithi203.stdout:2/735: dwrite d2/d38/d2f/ff0 [0,4194304] 0 2024-05-25T06:11:36.051 INFO:tasks.workunit.client.0.smithi160.stdout:3/285: fsync d19/d2f/d38/d43/f4e 0 2024-05-25T06:11:36.053 INFO:tasks.workunit.client.1.smithi203.stdout:2/736: chown d2/d4/d10/d5f/f81 88197125 1 2024-05-25T06:11:36.054 INFO:tasks.workunit.client.0.smithi160.stdout:7/407: symlink d6/d3e/d6f/d73/l83 0 2024-05-25T06:11:36.054 INFO:tasks.workunit.client.0.smithi160.stdout:3/286: fdatasync d19/d2f/d38/d47/f54 0 2024-05-25T06:11:36.056 INFO:tasks.workunit.client.1.smithi203.stdout:2/737: rmdir d2/d1a/dd3 39 2024-05-25T06:11:36.056 INFO:tasks.workunit.client.0.smithi160.stdout:0/396: dwrite f12 [0,4194304] 0 2024-05-25T06:11:36.057 INFO:tasks.workunit.client.1.smithi203.stdout:2/738: readlink d2/d38/lf 0 2024-05-25T06:11:36.057 INFO:tasks.workunit.client.0.smithi160.stdout:3/287: readlink d19/d2f/d34/l59 0 2024-05-25T06:11:36.059 INFO:tasks.workunit.client.0.smithi160.stdout:6/318: getdents d5/d39 0 2024-05-25T06:11:36.061 INFO:tasks.workunit.client.0.smithi160.stdout:3/288: chown d19/d2f/d38/d47/f54 5279 1 2024-05-25T06:11:36.063 INFO:tasks.workunit.client.0.smithi160.stdout:6/319: fdatasync d5/de/d32/d35/f5f 0 2024-05-25T06:11:36.064 INFO:tasks.workunit.client.0.smithi160.stdout:0/397: rename d2b/d31/d6d/l77 to d2b/d46/l80 0 2024-05-25T06:11:36.069 INFO:tasks.workunit.client.1.smithi203.stdout:7/660: dwrite d1/d11/f41 [0,4194304] 0 2024-05-25T06:11:36.073 INFO:tasks.workunit.client.0.smithi160.stdout:2/343: dread d12/f1d [0,4194304] 0 2024-05-25T06:11:36.075 INFO:tasks.workunit.client.0.smithi160.stdout:0/398: creat d2b/d31/d35/d41/f81 x:0 0 0 2024-05-25T06:11:36.076 INFO:tasks.workunit.client.0.smithi160.stdout:0/399: write d2b/d58/f5e [912717,116008] 0 2024-05-25T06:11:36.077 INFO:tasks.workunit.client.1.smithi203.stdout:7/661: rename d1/d9/d2c/d36/d3b/f47 to d1/d9/d2c/d62/db3/fe3 0 2024-05-25T06:11:36.078 INFO:tasks.workunit.client.0.smithi160.stdout:9/322: mkdir d4/d11/d1e/d3e/d25/d5e 0 2024-05-25T06:11:36.078 INFO:tasks.workunit.client.1.smithi203.stdout:9/607: dread d0/d57/d8f/f9e [4194304,4194304] 0 2024-05-25T06:11:36.078 INFO:tasks.workunit.client.0.smithi160.stdout:1/256: write de/f10 [481151,69116] 0 2024-05-25T06:11:36.080 INFO:tasks.workunit.client.0.smithi160.stdout:3/289: mkdir d19/d2f/d38/d43/d5b/d65 0 2024-05-25T06:11:36.080 INFO:tasks.workunit.client.1.smithi203.stdout:9/608: rmdir d0/df/d2d 39 2024-05-25T06:11:36.082 INFO:tasks.workunit.client.0.smithi160.stdout:2/344: creat d12/d1c/d37/d4b/f79 x:0 0 0 2024-05-25T06:11:36.084 INFO:tasks.workunit.client.1.smithi203.stdout:9/609: creat d0/df/fd1 x:0 0 0 2024-05-25T06:11:36.085 INFO:tasks.workunit.client.0.smithi160.stdout:1/257: write de/d21/d49/f2b [1342911,27308] 0 2024-05-25T06:11:36.087 INFO:tasks.workunit.client.0.smithi160.stdout:6/320: link d5/de/d32/d35/f5f d5/de/d32/d1b/f6b 0 2024-05-25T06:11:36.088 INFO:tasks.workunit.client.1.smithi203.stdout:9/610: link d0/d17/d20/d3c/lcb d0/d17/d20/ld2 0 2024-05-25T06:11:36.088 INFO:tasks.workunit.client.0.smithi160.stdout:3/290: creat d19/d2f/d38/d43/f66 x:0 0 0 2024-05-25T06:11:36.090 INFO:tasks.workunit.client.0.smithi160.stdout:0/400: mknod d2b/d31/d35/d7b/c82 0 2024-05-25T06:11:36.092 INFO:tasks.workunit.client.0.smithi160.stdout:0/401: chown d2b/d31/f7d 1 1 2024-05-25T06:11:36.092 INFO:tasks.workunit.client.0.smithi160.stdout:9/323: write d4/d11/f38 [113079,37938] 0 2024-05-25T06:11:36.093 INFO:tasks.workunit.client.0.smithi160.stdout:1/258: dwrite de/d21/d49/d35/f24 [0,4194304] 0 2024-05-25T06:11:36.093 INFO:tasks.workunit.client.1.smithi203.stdout:2/739: write d2/f6c [52833,1464] 0 2024-05-25T06:11:36.096 INFO:tasks.workunit.client.0.smithi160.stdout:3/291: mkdir d19/d2f/d34/d67 0 2024-05-25T06:11:36.096 INFO:tasks.workunit.client.1.smithi203.stdout:7/662: truncate d1/d11/f21 1538066 0 2024-05-25T06:11:36.097 INFO:tasks.workunit.client.0.smithi160.stdout:9/324: dread - d4/d11/d1e/f1f zero size 2024-05-25T06:11:36.098 INFO:tasks.workunit.client.1.smithi203.stdout:9/611: write d0/df/d37/f58 [3801395,126878] 0 2024-05-25T06:11:36.100 INFO:tasks.workunit.client.1.smithi203.stdout:7/663: readlink d1/d9/d2c/d3a/la2 0 2024-05-25T06:11:36.100 INFO:tasks.workunit.client.0.smithi160.stdout:3/292: dread - d19/d2f/d38/d47/f64 zero size 2024-05-25T06:11:36.100 INFO:tasks.workunit.client.0.smithi160.stdout:9/325: write d4/f4e [1900313,92089] 0 2024-05-25T06:11:36.102 INFO:tasks.workunit.client.0.smithi160.stdout:2/345: getdents d12/d1c/d59 0 2024-05-25T06:11:36.102 INFO:tasks.workunit.client.0.smithi160.stdout:0/402: read d2b/d31/d35/d43/f4c [209983,101834] 0 2024-05-25T06:11:36.102 INFO:tasks.workunit.client.1.smithi203.stdout:7/664: fdatasync d1/d9/dc2/fcb 0 2024-05-25T06:11:36.103 INFO:tasks.workunit.client.1.smithi203.stdout:2/740: dwrite d2/d42/f83 [0,4194304] 0 2024-05-25T06:11:36.106 INFO:tasks.workunit.client.1.smithi203.stdout:9/612: symlink d0/df/d2d/ld3 0 2024-05-25T06:11:36.106 INFO:tasks.workunit.client.0.smithi160.stdout:9/326: dread - d4/d11/d12/f48 zero size 2024-05-25T06:11:36.107 INFO:tasks.workunit.client.1.smithi203.stdout:9/613: readlink d0/df/d37/d4f/lab 0 2024-05-25T06:11:36.108 INFO:tasks.workunit.client.1.smithi203.stdout:2/741: mknod d2/d4/d10/d32/dca/cfa 0 2024-05-25T06:11:36.111 INFO:tasks.workunit.client.1.smithi203.stdout:9/614: creat d0/df/d37/da8/db8/fd4 x:0 0 0 2024-05-25T06:11:36.111 INFO:tasks.workunit.client.1.smithi203.stdout:2/742: creat d2/d38/ffb x:0 0 0 2024-05-25T06:11:36.114 INFO:tasks.workunit.client.0.smithi160.stdout:6/321: dwrite d5/de/d32/d35/f5f [0,4194304] 0 2024-05-25T06:11:36.114 INFO:tasks.workunit.client.0.smithi160.stdout:0/403: symlink d2b/d31/d35/d36/d37/l83 0 2024-05-25T06:11:36.116 INFO:tasks.workunit.client.1.smithi203.stdout:7/665: write d1/d9/d2c/d36/d54/f8f [1037395,110870] 0 2024-05-25T06:11:36.116 INFO:tasks.workunit.client.0.smithi160.stdout:3/293: rename d19/d2f/d38/l56 to d19/d2f/d38/d43/l68 0 2024-05-25T06:11:36.118 INFO:tasks.workunit.client.0.smithi160.stdout:2/346: unlink d12/l1b 0 2024-05-25T06:11:36.118 INFO:tasks.workunit.client.1.smithi203.stdout:7/666: mkdir d1/d11/d20/de4 0 2024-05-25T06:11:36.120 INFO:tasks.workunit.client.1.smithi203.stdout:7/667: write d1/d9/d2c/d62/d9b/fa0 [2692636,34942] 0 2024-05-25T06:11:36.123 INFO:tasks.workunit.client.0.smithi160.stdout:9/327: rename d4/c6 to d4/d11/d1e/d24/c5f 0 2024-05-25T06:11:36.124 INFO:tasks.workunit.client.0.smithi160.stdout:1/259: truncate de/d21/d49/d35/f42 699301 0 2024-05-25T06:11:36.126 INFO:tasks.workunit.client.0.smithi160.stdout:6/322: dread d5/de/d32/f37 [0,4194304] 0 2024-05-25T06:11:36.126 INFO:tasks.workunit.client.0.smithi160.stdout:3/294: symlink d19/d2f/d38/d60/l69 0 2024-05-25T06:11:36.126 INFO:tasks.workunit.client.0.smithi160.stdout:1/260: write de/f10 [260756,46078] 0 2024-05-25T06:11:36.127 INFO:tasks.workunit.client.0.smithi160.stdout:1/261: stat cc 0 2024-05-25T06:11:36.128 INFO:tasks.workunit.client.1.smithi203.stdout:7/668: dwrite d1/f24 [0,4194304] 0 2024-05-25T06:11:36.129 INFO:tasks.workunit.client.1.smithi203.stdout:1/584: sync 2024-05-25T06:11:36.129 INFO:tasks.workunit.client.1.smithi203.stdout:8/808: sync 2024-05-25T06:11:36.130 INFO:tasks.workunit.client.1.smithi203.stdout:7/669: write d1/d11/d3d/d4c/f93 [3382564,96197] 0 2024-05-25T06:11:36.131 INFO:tasks.workunit.client.0.smithi160.stdout:2/347: dwrite d12/d13/f4c [0,4194304] 0 2024-05-25T06:11:36.131 INFO:tasks.workunit.client.0.smithi160.stdout:9/328: symlink d4/d11/d14/l60 0 2024-05-25T06:11:36.134 INFO:tasks.workunit.client.0.smithi160.stdout:1/262: mkdir de/d21/d49/d1c/d4f 0 2024-05-25T06:11:36.134 INFO:tasks.workunit.client.0.smithi160.stdout:9/329: truncate d4/f41 931314 0 2024-05-25T06:11:36.137 INFO:tasks.workunit.client.1.smithi203.stdout:7/670: mkdir d1/d9/d2c/d62/db3/de5 0 2024-05-25T06:11:36.139 INFO:tasks.workunit.client.0.smithi160.stdout:3/295: dread d19/f44 [0,4194304] 0 2024-05-25T06:11:36.139 INFO:tasks.workunit.client.0.smithi160.stdout:9/330: truncate d4/d11/d14/f37 1215733 0 2024-05-25T06:11:36.140 INFO:tasks.workunit.client.1.smithi203.stdout:1/585: dwrite d14/d23/f78 [4194304,4194304] 0 2024-05-25T06:11:36.140 INFO:tasks.workunit.client.0.smithi160.stdout:6/323: dwrite d5/d30/f3d [0,4194304] 0 2024-05-25T06:11:36.145 INFO:tasks.workunit.client.0.smithi160.stdout:1/263: symlink de/d43/l50 0 2024-05-25T06:11:36.146 INFO:tasks.workunit.client.0.smithi160.stdout:1/264: stat f5 0 2024-05-25T06:11:36.146 INFO:tasks.workunit.client.1.smithi203.stdout:1/586: dread - d14/d57/d29/d94/db8/dc2/d36/db3/d5e/fb1 zero size 2024-05-25T06:11:36.147 INFO:tasks.workunit.client.0.smithi160.stdout:6/324: dwrite d5/d45/d4a/f53 [0,4194304] 0 2024-05-25T06:11:36.148 INFO:tasks.workunit.client.1.smithi203.stdout:7/671: unlink d1/fa 0 2024-05-25T06:11:36.151 INFO:tasks.workunit.client.0.smithi160.stdout:2/348: mkdir d12/d3e/d5f/d74/d7a 0 2024-05-25T06:11:36.154 INFO:tasks.workunit.client.1.smithi203.stdout:7/672: creat d1/dbd/dc1/fe6 x:0 0 0 2024-05-25T06:11:36.155 INFO:tasks.workunit.client.0.smithi160.stdout:9/331: stat d4/d11/d1e/d24/c5f 0 2024-05-25T06:11:36.155 INFO:tasks.workunit.client.1.smithi203.stdout:9/615: dwrite d0/df/d73/fa7 [0,4194304] 0 2024-05-25T06:11:36.155 INFO:tasks.workunit.client.0.smithi160.stdout:9/332: dread - d4/d11/d1e/d24/f13 zero size 2024-05-25T06:11:36.159 INFO:tasks.workunit.client.0.smithi160.stdout:0/404: truncate d2b/f42 4387847 0 2024-05-25T06:11:36.159 INFO:tasks.workunit.client.0.smithi160.stdout:1/265: chown de/d21/d49/c2e 4154 1 2024-05-25T06:11:36.160 INFO:tasks.workunit.client.1.smithi203.stdout:1/587: rename d14/d7c/c9e to d14/d57/d97/da4/cc3 0 2024-05-25T06:11:36.160 INFO:tasks.workunit.client.0.smithi160.stdout:1/266: write de/f10 [1219294,31015] 0 2024-05-25T06:11:36.162 INFO:tasks.workunit.client.0.smithi160.stdout:3/296: mkdir d19/d5d/d6a 0 2024-05-25T06:11:36.163 INFO:tasks.workunit.client.0.smithi160.stdout:3/297: write d19/d2f/d38/d43/f46 [151124,91793] 0 2024-05-25T06:11:36.165 INFO:tasks.workunit.client.1.smithi203.stdout:2/743: dwrite d2/d38/d43/d60/d6b/dae/f67 [0,4194304] 0 2024-05-25T06:11:36.165 INFO:tasks.workunit.client.0.smithi160.stdout:6/325: creat d5/d39/f6c x:0 0 0 2024-05-25T06:11:36.168 INFO:tasks.workunit.client.0.smithi160.stdout:0/405: creat d2b/d31/d35/d36/f84 x:0 0 0 2024-05-25T06:11:36.169 INFO:tasks.workunit.client.1.smithi203.stdout:7/673: creat d1/d9/d2c/fe7 x:0 0 0 2024-05-25T06:11:36.169 INFO:tasks.workunit.client.0.smithi160.stdout:3/298: creat d19/d2f/d38/f6b x:0 0 0 2024-05-25T06:11:36.171 INFO:tasks.workunit.client.0.smithi160.stdout:6/326: fsync d5/de/f22 0 2024-05-25T06:11:36.172 INFO:tasks.workunit.client.1.smithi203.stdout:1/588: getdents d14/d57/d1d 0 2024-05-25T06:11:36.172 INFO:tasks.workunit.client.0.smithi160.stdout:0/406: creat d2b/d58/f85 x:0 0 0 2024-05-25T06:11:36.173 INFO:tasks.workunit.client.0.smithi160.stdout:1/267: dwrite de/d21/d49/d28/f3c [0,4194304] 0 2024-05-25T06:11:36.175 INFO:tasks.workunit.client.1.smithi203.stdout:2/744: read d2/d4/f74 [3097384,39353] 0 2024-05-25T06:11:36.176 INFO:tasks.workunit.client.0.smithi160.stdout:3/299: unlink d19/d2f/c3f 0 2024-05-25T06:11:36.177 INFO:tasks.workunit.client.0.smithi160.stdout:8/402: sync 2024-05-25T06:11:36.179 INFO:tasks.workunit.client.1.smithi203.stdout:7/674: dread d1/d9/d2c/d3a/f5c [0,4194304] 0 2024-05-25T06:11:36.181 INFO:tasks.workunit.client.0.smithi160.stdout:1/268: mknod de/c51 0 2024-05-25T06:11:36.182 INFO:tasks.workunit.client.0.smithi160.stdout:8/403: creat d5/d39/d7a/f85 x:0 0 0 2024-05-25T06:11:36.182 INFO:tasks.workunit.client.0.smithi160.stdout:8/404: stat d5/fb 0 2024-05-25T06:11:36.184 INFO:tasks.workunit.client.0.smithi160.stdout:2/349: getdents d12/d3c 0 2024-05-25T06:11:36.185 INFO:tasks.workunit.client.1.smithi203.stdout:3/801: sync 2024-05-25T06:11:36.185 INFO:tasks.workunit.client.1.smithi203.stdout:0/631: sync 2024-05-25T06:11:36.185 INFO:tasks.workunit.client.1.smithi203.stdout:5/902: sync 2024-05-25T06:11:36.186 INFO:tasks.workunit.client.1.smithi203.stdout:4/821: sync 2024-05-25T06:11:36.186 INFO:tasks.workunit.client.0.smithi160.stdout:8/405: write d5/d39/d75/f76 [635599,27330] 0 2024-05-25T06:11:36.186 INFO:tasks.workunit.client.0.smithi160.stdout:0/407: dwrite d2b/f78 [0,4194304] 0 2024-05-25T06:11:36.186 INFO:tasks.workunit.client.0.smithi160.stdout:6/327: link d5/de/d32/d1b/c4b d5/de/d32/d35/d40/c6d 0 2024-05-25T06:11:36.191 INFO:tasks.workunit.client.1.smithi203.stdout:8/809: dwrite d2/d5/d28/d38/d4e/d97/f110 [0,4194304] 0 2024-05-25T06:11:36.192 INFO:tasks.workunit.client.0.smithi160.stdout:1/269: dread de/f29 [0,4194304] 0 2024-05-25T06:11:36.192 INFO:tasks.workunit.client.0.smithi160.stdout:2/350: dwrite d12/d1c/f2f [0,4194304] 0 2024-05-25T06:11:36.195 INFO:tasks.workunit.client.1.smithi203.stdout:0/632: creat dd/d3d/d75/fe1 x:0 0 0 2024-05-25T06:11:36.199 INFO:tasks.workunit.client.0.smithi160.stdout:0/408: rename d2b/d31/d35/d41 to d2b/d31/d35/d36/d86 0 2024-05-25T06:11:36.202 INFO:tasks.workunit.client.1.smithi203.stdout:3/802: dread d10/d16/d23/d4a/d88/fd8 [0,4194304] 0 2024-05-25T06:11:36.202 INFO:tasks.workunit.client.0.smithi160.stdout:1/270: write de/f3d [570554,64800] 0 2024-05-25T06:11:36.202 INFO:tasks.workunit.client.0.smithi160.stdout:0/409: mknod d2b/d31/d35/d36/d37/c87 0 2024-05-25T06:11:36.203 INFO:tasks.workunit.client.1.smithi203.stdout:4/822: truncate d0/d8/d3a/d81/d93/d98/d99/fcc 1129126 0 2024-05-25T06:11:36.208 INFO:tasks.workunit.client.1.smithi203.stdout:0/633: unlink dd/d3d/d4f/d62/d78/f7d 0 2024-05-25T06:11:36.208 INFO:tasks.workunit.client.1.smithi203.stdout:4/823: chown d0/d9/d50/dce/fa1 6603 1 2024-05-25T06:11:36.208 INFO:tasks.workunit.client.0.smithi160.stdout:2/351: rmdir d12/d1c/d37/d4b/d76 0 2024-05-25T06:11:36.209 INFO:tasks.workunit.client.0.smithi160.stdout:1/271: mkdir de/d21/d49/d1c/d52 0 2024-05-25T06:11:36.210 INFO:tasks.workunit.client.1.smithi203.stdout:5/903: dwrite dc/d11/d17/d1a/d2d/d4d/d101/f133 [0,4194304] 0 2024-05-25T06:11:36.210 INFO:tasks.workunit.client.0.smithi160.stdout:0/410: creat d2b/d31/d35/d7b/f88 x:0 0 0 2024-05-25T06:11:36.212 INFO:tasks.workunit.client.1.smithi203.stdout:5/904: chown dc/d11/d18 48992855 1 2024-05-25T06:11:36.213 INFO:tasks.workunit.client.0.smithi160.stdout:8/406: dread d5/d44/f25 [0,4194304] 0 2024-05-25T06:11:36.213 INFO:tasks.workunit.client.0.smithi160.stdout:2/352: chown d12/l14 144568108 1 2024-05-25T06:11:36.213 INFO:tasks.workunit.client.1.smithi203.stdout:3/803: creat d10/d16/d23/d4a/d88/deb/f100 x:0 0 0 2024-05-25T06:11:36.214 INFO:tasks.workunit.client.1.smithi203.stdout:9/616: dread d0/df/d73/f49 [0,4194304] 0 2024-05-25T06:11:36.215 INFO:tasks.workunit.client.1.smithi203.stdout:0/634: mkdir dd/d3a/de2 0 2024-05-25T06:11:36.216 INFO:tasks.workunit.client.0.smithi160.stdout:0/411: unlink d2b/d31/d35/d36/f61 0 2024-05-25T06:11:36.216 INFO:tasks.workunit.client.0.smithi160.stdout:8/407: write d5/d44/f37 [4898897,112068] 0 2024-05-25T06:11:36.216 INFO:tasks.workunit.client.1.smithi203.stdout:8/810: getdents d2/d5/d18/d31/dac/dc2/d88 0 2024-05-25T06:11:36.217 INFO:tasks.workunit.client.1.smithi203.stdout:3/804: stat d10/d16/c1e 0 2024-05-25T06:11:36.217 INFO:tasks.workunit.client.0.smithi160.stdout:1/272: creat de/d21/d49/d1c/f53 x:0 0 0 2024-05-25T06:11:36.218 INFO:tasks.workunit.client.1.smithi203.stdout:3/805: chown lf 806853 1 2024-05-25T06:11:36.218 INFO:tasks.workunit.client.1.smithi203.stdout:9/617: fdatasync d0/d17/f4b 0 2024-05-25T06:11:36.219 INFO:tasks.workunit.client.0.smithi160.stdout:9/333: dread d4/f1c [0,4194304] 0 2024-05-25T06:11:36.222 INFO:tasks.workunit.client.1.smithi203.stdout:5/905: creat dc/d11/d17/d1a/d2d/d53/da2/f140 x:0 0 0 2024-05-25T06:11:36.222 INFO:tasks.workunit.client.1.smithi203.stdout:8/811: rmdir d2/d5/d18/d31/dac/dc2 39 2024-05-25T06:11:36.223 INFO:tasks.workunit.client.0.smithi160.stdout:0/412: creat d2b/d31/d35/d36/f89 x:0 0 0 2024-05-25T06:11:36.223 INFO:tasks.workunit.client.0.smithi160.stdout:1/273: write de/f3a [828709,107697] 0 2024-05-25T06:11:36.224 INFO:tasks.workunit.client.1.smithi203.stdout:8/812: chown d2/d5/d10/d11/d1f/c67 250199337 1 2024-05-25T06:11:36.224 INFO:tasks.workunit.client.0.smithi160.stdout:0/413: chown d2b/d31/f7a 522 1 2024-05-25T06:11:36.224 INFO:tasks.workunit.client.0.smithi160.stdout:1/274: chown de/f29 1888 1 2024-05-25T06:11:36.225 INFO:tasks.workunit.client.1.smithi203.stdout:3/806: creat d10/d16/d23/d4a/d88/dca/f101 x:0 0 0 2024-05-25T06:11:36.225 INFO:tasks.workunit.client.0.smithi160.stdout:8/408: rename d5/d52 to d5/d44/d1f/d1e/d28/d4a/d4e/d86 0 2024-05-25T06:11:36.225 INFO:tasks.workunit.client.0.smithi160.stdout:9/334: creat d4/d11/d14/f61 x:0 0 0 2024-05-25T06:11:36.226 INFO:tasks.workunit.client.0.smithi160.stdout:8/409: truncate d5/f48 2014766 0 2024-05-25T06:11:36.227 INFO:tasks.workunit.client.0.smithi160.stdout:0/414: creat d2b/d46/f8a x:0 0 0 2024-05-25T06:11:36.228 INFO:tasks.workunit.client.1.smithi203.stdout:7/675: dread d1/d11/d20/f27 [0,4194304] 0 2024-05-25T06:11:36.228 INFO:tasks.workunit.client.0.smithi160.stdout:2/353: getdents d12/d1c/d2a 0 2024-05-25T06:11:36.228 INFO:tasks.workunit.client.1.smithi203.stdout:3/807: creat d10/d16/d23/d6c/f102 x:0 0 0 2024-05-25T06:11:36.229 INFO:tasks.workunit.client.0.smithi160.stdout:2/354: fdatasync d12/d13/f60 0 2024-05-25T06:11:36.229 INFO:tasks.workunit.client.1.smithi203.stdout:5/906: mknod dc/d11/d42/d94/d10a/c141 0 2024-05-25T06:11:36.230 INFO:tasks.workunit.client.0.smithi160.stdout:9/335: creat d4/d11/d12/f62 x:0 0 0 2024-05-25T06:11:36.232 INFO:tasks.workunit.client.0.smithi160.stdout:8/410: symlink d5/d44/d1f/d58/d65/l87 0 2024-05-25T06:11:36.235 INFO:tasks.workunit.client.1.smithi203.stdout:8/813: rename d2/d5/d28/d38/d4e/d97/f110 to d2/d5/d10/d45/d5d/f117 0 2024-05-25T06:11:36.236 INFO:tasks.workunit.client.1.smithi203.stdout:8/814: stat d2/l41 0 2024-05-25T06:11:36.237 INFO:tasks.workunit.client.0.smithi160.stdout:0/415: rename d2b/d46/f8a to d2b/d31/d35/d36/f8b 0 2024-05-25T06:11:36.239 INFO:tasks.workunit.client.1.smithi203.stdout:5/907: creat dc/d11/d17/d1a/d2d/d4d/ded/d13f/f142 x:0 0 0 2024-05-25T06:11:36.241 INFO:tasks.workunit.client.1.smithi203.stdout:2/745: write d2/d1a/f27 [900627,104976] 0 2024-05-25T06:11:36.241 INFO:tasks.workunit.client.1.smithi203.stdout:1/589: truncate d14/d57/d97/fa3 4197409 0 2024-05-25T06:11:36.242 INFO:tasks.workunit.client.0.smithi160.stdout:1/275: creat de/f54 x:0 0 0 2024-05-25T06:11:36.242 INFO:tasks.workunit.client.1.smithi203.stdout:3/808: mknod d10/d16/d71/dfa/c103 0 2024-05-25T06:11:36.243 INFO:tasks.workunit.client.1.smithi203.stdout:2/746: chown d2/d4/d10/d58/da0/lba 2325 1 2024-05-25T06:11:36.244 INFO:tasks.workunit.client.0.smithi160.stdout:3/300: dread d19/d22/f39 [0,4194304] 0 2024-05-25T06:11:36.244 INFO:tasks.workunit.client.0.smithi160.stdout:9/336: mknod d4/d11/d14/c63 0 2024-05-25T06:11:36.244 INFO:tasks.workunit.client.1.smithi203.stdout:8/815: unlink d2/d5/d10/d11/de/c115 0 2024-05-25T06:11:36.245 INFO:tasks.workunit.client.0.smithi160.stdout:3/301: fsync d19/d22/f48 0 2024-05-25T06:11:36.245 INFO:tasks.workunit.client.0.smithi160.stdout:9/337: write d4/d11/d14/f53 [91097,3053] 0 2024-05-25T06:11:36.245 INFO:tasks.workunit.client.0.smithi160.stdout:8/411: mknod d5/d39/d75/c88 0 2024-05-25T06:11:36.247 INFO:tasks.workunit.client.0.smithi160.stdout:8/412: write d5/d44/d1f/d1e/d28/d4a/f34 [631910,56218] 0 2024-05-25T06:11:36.249 INFO:tasks.workunit.client.0.smithi160.stdout:0/416: creat d2b/d58/f8c x:0 0 0 2024-05-25T06:11:36.251 INFO:tasks.workunit.client.1.smithi203.stdout:3/809: unlink d10/d16/d23/d67/db8/cbc 0 2024-05-25T06:11:36.251 INFO:tasks.workunit.client.0.smithi160.stdout:1/276: symlink de/d21/l55 0 2024-05-25T06:11:36.252 INFO:tasks.workunit.client.0.smithi160.stdout:1/277: dread - de/f4d zero size 2024-05-25T06:11:36.252 INFO:tasks.workunit.client.1.smithi203.stdout:8/816: write d2/d5/d10/d45/d5d/fe9 [563868,37192] 0 2024-05-25T06:11:36.253 INFO:tasks.workunit.client.0.smithi160.stdout:3/302: creat d19/d2f/d38/f6c x:0 0 0 2024-05-25T06:11:36.255 INFO:tasks.workunit.client.1.smithi203.stdout:5/908: dwrite dc/d11/fab [0,4194304] 0 2024-05-25T06:11:36.256 INFO:tasks.workunit.client.1.smithi203.stdout:4/824: dread d0/d8/d1f/d26/f1e [4194304,4194304] 0 2024-05-25T06:11:36.257 INFO:tasks.workunit.client.0.smithi160.stdout:8/413: symlink d5/d44/d1f/d1e/l89 0 2024-05-25T06:11:36.257 INFO:tasks.workunit.client.1.smithi203.stdout:0/635: dread dd/d10/d34/d4c/d99/fbe [0,4194304] 0 2024-05-25T06:11:36.262 INFO:tasks.workunit.client.1.smithi203.stdout:7/676: write d1/d9/d2c/d79/f81 [5438727,78117] 0 2024-05-25T06:11:36.266 INFO:tasks.workunit.client.0.smithi160.stdout:3/303: rename d19/d2f/f35 to d19/d2f/d38/d43/d5b/f6d 0 2024-05-25T06:11:36.267 INFO:tasks.workunit.client.1.smithi203.stdout:9/618: dread d0/f19 [0,4194304] 0 2024-05-25T06:11:36.267 INFO:tasks.workunit.client.1.smithi203.stdout:2/747: dread d2/d4/d10/d5f/fa2 [0,4194304] 0 2024-05-25T06:11:36.267 INFO:tasks.workunit.client.0.smithi160.stdout:3/304: fsync d19/d2f/d38/d43/f53 0 2024-05-25T06:11:36.267 INFO:tasks.workunit.client.0.smithi160.stdout:6/328: truncate d5/de/ff 3729131 0 2024-05-25T06:11:36.268 INFO:tasks.workunit.client.0.smithi160.stdout:9/338: truncate d4/d11/d1e/f4d 333342 0 2024-05-25T06:11:36.268 INFO:tasks.workunit.client.1.smithi203.stdout:1/590: dread d14/d57/d6b/f5d [0,4194304] 0 2024-05-25T06:11:36.269 INFO:tasks.workunit.client.0.smithi160.stdout:6/329: fdatasync d5/de/f22 0 2024-05-25T06:11:36.269 INFO:tasks.workunit.client.1.smithi203.stdout:8/817: mkdir d2/d5/d10/d45/d5d/d68/d118 0 2024-05-25T06:11:36.270 INFO:tasks.workunit.client.0.smithi160.stdout:6/330: chown d5/d45/d4a/f5d 59238462 1 2024-05-25T06:11:36.270 INFO:tasks.workunit.client.1.smithi203.stdout:4/825: link d0/d8/d10/da9/f106 d0/d8/d3a/d81/d93/d98/d99/ded/dfa/def/f113 0 2024-05-25T06:11:36.271 INFO:tasks.workunit.client.0.smithi160.stdout:6/331: fdatasync d5/de/d32/d35/f5f 0 2024-05-25T06:11:36.271 INFO:tasks.workunit.client.1.smithi203.stdout:9/619: dread - d0/d17/d20/d3c/d5c/fbd zero size 2024-05-25T06:11:36.272 INFO:tasks.workunit.client.1.smithi203.stdout:2/748: creat d2/d1a/d89/ffc x:0 0 0 2024-05-25T06:11:36.272 INFO:tasks.workunit.client.0.smithi160.stdout:2/355: link ff d12/f7b 0 2024-05-25T06:11:36.272 INFO:tasks.workunit.client.1.smithi203.stdout:7/677: symlink d1/d9/d2c/d3a/le8 0 2024-05-25T06:11:36.273 INFO:tasks.workunit.client.0.smithi160.stdout:9/339: dread d4/d30/f57 [0,4194304] 0 2024-05-25T06:11:36.274 INFO:tasks.workunit.client.1.smithi203.stdout:0/636: symlink dd/d3d/d4f/db7/le3 0 2024-05-25T06:11:36.274 INFO:tasks.workunit.client.1.smithi203.stdout:8/818: read - d2/d5/d10/d45/faa zero size 2024-05-25T06:11:36.275 INFO:tasks.workunit.client.0.smithi160.stdout:3/305: chown d19/c20 42519 1 2024-05-25T06:11:36.275 INFO:tasks.workunit.client.1.smithi203.stdout:7/678: fdatasync d1/fd9 0 2024-05-25T06:11:36.276 INFO:tasks.workunit.client.1.smithi203.stdout:4/826: fsync d0/d8/d1f/d26/d3b/fac 0 2024-05-25T06:11:36.276 INFO:tasks.workunit.client.1.smithi203.stdout:1/591: mknod d14/d57/d29/d94/db8/dc2/d36/db3/d5e/cc4 0 2024-05-25T06:11:36.278 INFO:tasks.workunit.client.0.smithi160.stdout:8/414: symlink d5/d39/d6a/l8a 0 2024-05-25T06:11:36.279 INFO:tasks.workunit.client.1.smithi203.stdout:3/810: dwrite d10/d16/d23/d38/f3e [0,4194304] 0 2024-05-25T06:11:36.279 INFO:tasks.workunit.client.1.smithi203.stdout:2/749: mkdir d2/d1a/d89/dfd 0 2024-05-25T06:11:36.279 INFO:tasks.workunit.client.0.smithi160.stdout:1/278: chown de/d21/d49/d35/f42 1 1 2024-05-25T06:11:36.281 INFO:tasks.workunit.client.0.smithi160.stdout:6/332: rmdir d5/de/d18 39 2024-05-25T06:11:36.283 INFO:tasks.workunit.client.1.smithi203.stdout:0/637: mknod dd/d3a/de2/ce4 0 2024-05-25T06:11:36.284 INFO:tasks.workunit.client.0.smithi160.stdout:1/279: dread f7 [0,4194304] 0 2024-05-25T06:11:36.284 INFO:tasks.workunit.client.1.smithi203.stdout:1/592: symlink d14/d57/d97/lc5 0 2024-05-25T06:11:36.285 INFO:tasks.workunit.client.1.smithi203.stdout:4/827: mkdir d0/d8/d3a/d81/dd4/d114 0 2024-05-25T06:11:36.285 INFO:tasks.workunit.client.0.smithi160.stdout:3/306: creat d19/d2f/d38/d47/f6e x:0 0 0 2024-05-25T06:11:36.287 INFO:tasks.workunit.client.1.smithi203.stdout:2/750: unlink d2/d38/d43/d60/c7c 0 2024-05-25T06:11:36.288 INFO:tasks.workunit.client.1.smithi203.stdout:7/679: truncate d1/d11/d20/f23 973908 0 2024-05-25T06:11:36.289 INFO:tasks.workunit.client.0.smithi160.stdout:8/415: creat d5/d44/d1f/d1e/d28/d4a/d4e/d86/f8b x:0 0 0 2024-05-25T06:11:36.290 INFO:tasks.workunit.client.1.smithi203.stdout:3/811: symlink d10/d22/l104 0 2024-05-25T06:11:36.291 INFO:tasks.workunit.client.0.smithi160.stdout:6/333: dwrite d5/de/d32/d1b/f55 [0,4194304] 0 2024-05-25T06:11:36.292 INFO:tasks.workunit.client.0.smithi160.stdout:6/334: write d5/de/d32/f67 [442373,5726] 0 2024-05-25T06:11:36.293 INFO:tasks.workunit.client.0.smithi160.stdout:2/356: rename d12/c1a to d12/d1c/d59/c7c 0 2024-05-25T06:11:36.293 INFO:tasks.workunit.client.1.smithi203.stdout:9/620: dwrite d0/d17/d20/d29/f8a [0,4194304] 0 2024-05-25T06:11:36.295 INFO:tasks.workunit.client.0.smithi160.stdout:9/340: creat d4/d11/d12/d58/f64 x:0 0 0 2024-05-25T06:11:36.295 INFO:tasks.workunit.client.1.smithi203.stdout:8/819: dwrite d2/d5/d10/d11/d1f/d75/f8d [0,4194304] 0 2024-05-25T06:11:36.296 INFO:tasks.workunit.client.1.smithi203.stdout:4/828: write d0/d8/d52/f5e [4618019,14771] 0 2024-05-25T06:11:36.296 INFO:tasks.workunit.client.0.smithi160.stdout:9/341: read - d4/d11/d12/d58/f64 zero size 2024-05-25T06:11:36.297 INFO:tasks.workunit.client.1.smithi203.stdout:5/909: dread dc/d11/f105 [4194304,4194304] 0 2024-05-25T06:11:36.300 INFO:tasks.workunit.client.1.smithi203.stdout:0/638: rename dd/d3a/d66/d88 to dd/d32/de5 0 2024-05-25T06:11:36.303 INFO:tasks.workunit.client.0.smithi160.stdout:3/307: write d19/d22/f2c [1773893,121866] 0 2024-05-25T06:11:36.304 INFO:tasks.workunit.client.1.smithi203.stdout:4/829: read d0/d9/d50/dce/f75 [938901,64518] 0 2024-05-25T06:11:36.304 INFO:tasks.workunit.client.1.smithi203.stdout:7/680: mknod d1/d11/d20/de4/ce9 0 2024-05-25T06:11:36.307 INFO:tasks.workunit.client.1.smithi203.stdout:9/621: mknod d0/df/d37/da8/cd5 0 2024-05-25T06:11:36.308 INFO:tasks.workunit.client.0.smithi160.stdout:6/335: mkdir d5/d45/d63/d61/d6e 0 2024-05-25T06:11:36.309 INFO:tasks.workunit.client.0.smithi160.stdout:6/336: read - d5/d30/f5a zero size 2024-05-25T06:11:36.310 INFO:tasks.workunit.client.1.smithi203.stdout:0/639: mkdir dd/d3d/d48/de6 0 2024-05-25T06:11:36.310 INFO:tasks.workunit.client.0.smithi160.stdout:6/337: chown d5/d45/d4a 5 1 2024-05-25T06:11:36.311 INFO:tasks.workunit.client.1.smithi203.stdout:3/812: dwrite d10/f51 [4194304,4194304] 0 2024-05-25T06:11:36.311 INFO:tasks.workunit.client.0.smithi160.stdout:8/416: getdents d5/d44/d7e 0 2024-05-25T06:11:36.311 INFO:tasks.workunit.client.1.smithi203.stdout:8/820: creat d2/d5/d10/d11/d1f/d75/f119 x:0 0 0 2024-05-25T06:11:36.313 INFO:tasks.workunit.client.0.smithi160.stdout:1/280: rename de/f16 to de/d21/d49/d1c/d52/f56 0 2024-05-25T06:11:36.314 INFO:tasks.workunit.client.1.smithi203.stdout:2/751: getdents d2/d1a/dd3 0 2024-05-25T06:11:36.314 INFO:tasks.workunit.client.0.smithi160.stdout:8/417: write d5/d44/d1f/d1e/d28/f3f [1100283,122011] 0 2024-05-25T06:11:36.315 INFO:tasks.workunit.client.1.smithi203.stdout:5/910: symlink dc/d11/d42/d94/l143 0 2024-05-25T06:11:36.315 INFO:tasks.workunit.client.0.smithi160.stdout:9/342: dread f2 [0,4194304] 0 2024-05-25T06:11:36.316 INFO:tasks.workunit.client.0.smithi160.stdout:6/338: mknod d5/d45/d63/d61/c6f 0 2024-05-25T06:11:36.318 INFO:tasks.workunit.client.1.smithi203.stdout:3/813: chown d10/d16/d50/d7d/l98 23 1 2024-05-25T06:11:36.321 INFO:tasks.workunit.client.0.smithi160.stdout:5/417: sync 2024-05-25T06:11:36.322 INFO:tasks.workunit.client.0.smithi160.stdout:4/365: sync 2024-05-25T06:11:36.322 INFO:tasks.workunit.client.0.smithi160.stdout:4/366: readlink d4/d6/l23 0 2024-05-25T06:11:36.323 INFO:tasks.workunit.client.0.smithi160.stdout:1/281: mknod de/d21/d49/d35/c57 0 2024-05-25T06:11:36.324 INFO:tasks.workunit.client.1.smithi203.stdout:4/830: dwrite d0/d8/d52/d59/d7a/d83/fd6 [0,4194304] 0 2024-05-25T06:11:36.324 INFO:tasks.workunit.client.1.smithi203.stdout:0/640: rename dd/d10/d34/d4c/cdf to dd/d45/d96/ce7 0 2024-05-25T06:11:36.327 INFO:tasks.workunit.client.0.smithi160.stdout:9/343: unlink d4/d11/d12/d2e/c46 0 2024-05-25T06:11:36.329 INFO:tasks.workunit.client.1.smithi203.stdout:8/821: mkdir d2/d5/d28/d11a 0 2024-05-25T06:11:36.329 INFO:tasks.workunit.client.0.smithi160.stdout:9/344: chown d4/d11/d12/d2e/f32 110800 1 2024-05-25T06:11:36.330 INFO:tasks.workunit.client.0.smithi160.stdout:9/345: stat d4/d30/c33 0 2024-05-25T06:11:36.330 INFO:tasks.workunit.client.1.smithi203.stdout:3/814: mkdir d10/d16/d23/d67/d105 0 2024-05-25T06:11:36.330 INFO:tasks.workunit.client.1.smithi203.stdout:1/593: write d14/d57/d1d/f30 [3702872,126977] 0 2024-05-25T06:11:36.333 INFO:tasks.workunit.client.1.smithi203.stdout:5/911: creat dc/d11/d17/d1a/d2d/d53/da2/f144 x:0 0 0 2024-05-25T06:11:36.333 INFO:tasks.workunit.client.0.smithi160.stdout:1/282: dread de/d21/d49/f27 [0,4194304] 0 2024-05-25T06:11:36.333 INFO:tasks.workunit.client.0.smithi160.stdout:6/339: mknod d5/d45/c70 0 2024-05-25T06:11:36.333 INFO:tasks.workunit.client.0.smithi160.stdout:5/418: rename db/dc/d13/d1d/c76 to db/dc/d13/d1d/d31/c7b 0 2024-05-25T06:11:36.334 INFO:tasks.workunit.client.1.smithi203.stdout:3/815: truncate d10/d16/d23/d4a/fd1 4554812 0 2024-05-25T06:11:36.334 INFO:tasks.workunit.client.0.smithi160.stdout:4/367: symlink d4/d6/d35/d3d/l72 0 2024-05-25T06:11:36.335 INFO:tasks.workunit.client.1.smithi203.stdout:3/816: readlink d10/d16/d71/la9 0 2024-05-25T06:11:36.335 INFO:tasks.workunit.client.0.smithi160.stdout:3/308: truncate d19/d2f/d38/d43/f53 3313675 0 2024-05-25T06:11:36.338 INFO:tasks.workunit.client.1.smithi203.stdout:0/641: creat dd/d3d/d75/fe8 x:0 0 0 2024-05-25T06:11:36.339 INFO:tasks.workunit.client.1.smithi203.stdout:4/831: write d0/d8/d3a/d81/d93/d98/d99/ff4 [440131,68837] 0 2024-05-25T06:11:36.340 INFO:tasks.workunit.client.1.smithi203.stdout:2/752: creat d2/d1a/d87/dd9/ffe x:0 0 0 2024-05-25T06:11:36.340 INFO:tasks.workunit.client.0.smithi160.stdout:9/346: creat d4/d11/d1e/d3e/d25/f65 x:0 0 0 2024-05-25T06:11:36.342 INFO:tasks.workunit.client.1.smithi203.stdout:8/822: unlink d2/f14 0 2024-05-25T06:11:36.344 INFO:tasks.workunit.client.0.smithi160.stdout:3/309: dwrite d19/d2f/d38/d43/f46 [0,4194304] 0 2024-05-25T06:11:36.345 INFO:tasks.workunit.client.1.smithi203.stdout:1/594: symlink d14/d57/d29/d94/db8/dc2/lc6 0 2024-05-25T06:11:36.347 INFO:tasks.workunit.client.1.smithi203.stdout:7/681: dread d1/d9/f18 [0,4194304] 0 2024-05-25T06:11:36.348 INFO:tasks.workunit.client.0.smithi160.stdout:5/419: symlink db/dc/d13/d1d/l7c 0 2024-05-25T06:11:36.350 INFO:tasks.workunit.client.1.smithi203.stdout:0/642: creat dd/d3a/de2/fe9 x:0 0 0 2024-05-25T06:11:36.350 INFO:tasks.workunit.client.0.smithi160.stdout:6/340: unlink d5/de/d32/d35/d40/c6d 0 2024-05-25T06:11:36.350 INFO:tasks.workunit.client.0.smithi160.stdout:6/341: write f2 [867450,14010] 0 2024-05-25T06:11:36.353 INFO:tasks.workunit.client.0.smithi160.stdout:2/357: write d12/f1d [1646474,72919] 0 2024-05-25T06:11:36.355 INFO:tasks.workunit.client.0.smithi160.stdout:9/347: mkdir d4/d11/d1e/d24/d66 0 2024-05-25T06:11:36.355 INFO:tasks.workunit.client.1.smithi203.stdout:8/823: rmdir d2/d5/d18/d31 39 2024-05-25T06:11:36.356 INFO:tasks.workunit.client.0.smithi160.stdout:9/348: fsync d4/d11/d12/d58/f64 0 2024-05-25T06:11:36.356 INFO:tasks.workunit.client.1.smithi203.stdout:9/622: write d0/d17/d20/d3c/db2/f84 [38714,109403] 0 2024-05-25T06:11:36.359 INFO:tasks.workunit.client.0.smithi160.stdout:9/349: dread - d4/d11/d1e/d3e/f44 zero size 2024-05-25T06:11:36.359 INFO:tasks.workunit.client.1.smithi203.stdout:2/753: dread d2/d42/fc3 [0,4194304] 0 2024-05-25T06:11:36.360 INFO:tasks.workunit.client.0.smithi160.stdout:5/420: rename db/dc/d13/d1d/d31/d56/f78 to db/dc/d13/d1d/d31/d56/f7d 0 2024-05-25T06:11:36.360 INFO:tasks.workunit.client.1.smithi203.stdout:1/595: symlink d14/d23/lc7 0 2024-05-25T06:11:36.361 INFO:tasks.workunit.client.0.smithi160.stdout:6/342: symlink d5/d30/l71 0 2024-05-25T06:11:36.361 INFO:tasks.workunit.client.0.smithi160.stdout:9/350: chown d4/d11/d12/d58/f64 1 1 2024-05-25T06:11:36.362 INFO:tasks.workunit.client.0.smithi160.stdout:2/358: dwrite d12/d3e/f68 [0,4194304] 0 2024-05-25T06:11:36.365 INFO:tasks.workunit.client.1.smithi203.stdout:4/832: creat d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f115 x:0 0 0 2024-05-25T06:11:36.366 INFO:tasks.workunit.client.0.smithi160.stdout:9/351: rmdir d4/d30 39 2024-05-25T06:11:36.367 INFO:tasks.workunit.client.1.smithi203.stdout:8/824: rename d2/d5/f29 to d2/d5/d18/dbf/f11b 0 2024-05-25T06:11:36.370 INFO:tasks.workunit.client.1.smithi203.stdout:9/623: unlink d0/d17/d20/d29/f48 0 2024-05-25T06:11:36.373 INFO:tasks.workunit.client.1.smithi203.stdout:9/624: dread d0/fa6 [0,4194304] 0 2024-05-25T06:11:36.373 INFO:tasks.workunit.client.0.smithi160.stdout:8/418: write d5/d44/f68 [2914895,15781] 0 2024-05-25T06:11:36.374 INFO:tasks.workunit.client.0.smithi160.stdout:3/310: rename c14 to d19/c6f 0 2024-05-25T06:11:36.374 INFO:tasks.workunit.client.1.smithi203.stdout:3/817: write d10/d22/f57 [834553,9432] 0 2024-05-25T06:11:36.375 INFO:tasks.workunit.client.0.smithi160.stdout:7/408: sync 2024-05-25T06:11:36.376 INFO:tasks.workunit.client.0.smithi160.stdout:7/409: truncate d6/d32/f80 331565 0 2024-05-25T06:11:36.377 INFO:tasks.workunit.client.1.smithi203.stdout:4/833: mkdir d0/d8/d52/d116 0 2024-05-25T06:11:36.377 INFO:tasks.workunit.client.0.smithi160.stdout:5/421: link f1 db/dc/d13/d1d/d31/d3b/d69/f7e 0 2024-05-25T06:11:36.377 INFO:tasks.workunit.client.1.smithi203.stdout:5/912: truncate dc/f10 3990271 0 2024-05-25T06:11:36.378 INFO:tasks.workunit.client.1.smithi203.stdout:4/834: readlink d0/d9/d50/d79/d9a/dbe/lf7 0 2024-05-25T06:11:36.378 INFO:tasks.workunit.client.0.smithi160.stdout:2/359: mknod d12/d3e/d5f/d74/d7a/c7d 0 2024-05-25T06:11:36.380 INFO:tasks.workunit.client.0.smithi160.stdout:2/360: write d12/d1c/f1e [2815960,84954] 0 2024-05-25T06:11:36.380 INFO:tasks.workunit.client.1.smithi203.stdout:2/754: symlink d2/d4/d10/d8c/lff 0 2024-05-25T06:11:36.381 INFO:tasks.workunit.client.1.smithi203.stdout:7/682: write d1/d9/d2c/d3a/d4e/f61 [29714,47154] 0 2024-05-25T06:11:36.381 INFO:tasks.workunit.client.1.smithi203.stdout:1/596: getdents d14/d57/d6b/da1 0 2024-05-25T06:11:36.382 INFO:tasks.workunit.client.0.smithi160.stdout:7/410: dread - d6/d2b/f51 zero size 2024-05-25T06:11:36.383 INFO:tasks.workunit.client.1.smithi203.stdout:9/625: fdatasync d0/df/d73/f49 0 2024-05-25T06:11:36.383 INFO:tasks.workunit.client.1.smithi203.stdout:3/818: rmdir d10/d16/d23/d46 39 2024-05-25T06:11:36.384 INFO:tasks.workunit.client.1.smithi203.stdout:2/755: truncate d2/d1a/d89/fcc 61642 0 2024-05-25T06:11:36.385 INFO:tasks.workunit.client.0.smithi160.stdout:4/368: dwrite d4/d6/fe [0,4194304] 0 2024-05-25T06:11:36.386 INFO:tasks.workunit.client.0.smithi160.stdout:8/419: symlink d5/d44/d7e/l8c 0 2024-05-25T06:11:36.387 INFO:tasks.workunit.client.1.smithi203.stdout:4/835: creat d0/d8/d3a/db6/f117 x:0 0 0 2024-05-25T06:11:36.388 INFO:tasks.workunit.client.0.smithi160.stdout:1/283: truncate de/d21/d49/f2b 587579 0 2024-05-25T06:11:36.389 INFO:tasks.workunit.client.1.smithi203.stdout:1/597: mkdir d14/d7c/d9d/dc8 0 2024-05-25T06:11:36.390 INFO:tasks.workunit.client.1.smithi203.stdout:2/756: truncate d2/d1a/d89/ffc 315839 0 2024-05-25T06:11:36.390 INFO:tasks.workunit.client.0.smithi160.stdout:6/343: rename d5/de/c29 to d5/d45/c72 0 2024-05-25T06:11:36.391 INFO:tasks.workunit.client.1.smithi203.stdout:9/626: write d0/d17/d20/d6e/d75/fc0 [676665,122014] 0 2024-05-25T06:11:36.393 INFO:tasks.workunit.client.0.smithi160.stdout:2/361: dwrite d12/d3e/f41 [0,4194304] 0 2024-05-25T06:11:36.394 INFO:tasks.workunit.client.1.smithi203.stdout:1/598: write d14/d57/d29/f2b [4710118,94918] 0 2024-05-25T06:11:36.395 INFO:tasks.workunit.client.0.smithi160.stdout:5/422: creat db/dc/d13/d1d/d3f/d72/f7f x:0 0 0 2024-05-25T06:11:36.395 INFO:tasks.workunit.client.1.smithi203.stdout:7/683: dwrite d1/d11/d3d/d4c/f93 [0,4194304] 0 2024-05-25T06:11:36.396 INFO:tasks.workunit.client.0.smithi160.stdout:0/417: dwrite d2b/f42 [0,4194304] 0 2024-05-25T06:11:36.398 INFO:tasks.workunit.client.1.smithi203.stdout:4/836: creat d0/d9/d50/d79/d9a/dbe/f118 x:0 0 0 2024-05-25T06:11:36.399 INFO:tasks.workunit.client.0.smithi160.stdout:7/411: creat d6/d3e/d6f/d73/f84 x:0 0 0 2024-05-25T06:11:36.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:36 smithi203 ceph-mon[36030]: pgmap v18: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 55 MiB/s rd, 199 MiB/s wr, 404 op/s 2024-05-25T06:11:36.403 INFO:tasks.workunit.client.1.smithi203.stdout:5/913: dread dc/d11/d17/d1a/d2d/d4d/f7b [0,4194304] 0 2024-05-25T06:11:36.404 INFO:tasks.workunit.client.0.smithi160.stdout:2/362: mkdir d12/d1c/d2a/d7e 0 2024-05-25T06:11:36.406 INFO:tasks.workunit.client.0.smithi160.stdout:7/412: rmdir d6/d2b/d5d 39 2024-05-25T06:11:36.408 INFO:tasks.workunit.client.0.smithi160.stdout:4/369: symlink d4/d63/d64/l73 0 2024-05-25T06:11:36.409 INFO:tasks.workunit.client.0.smithi160.stdout:4/370: stat d4/d1a/l20 0 2024-05-25T06:11:36.411 INFO:tasks.workunit.client.0.smithi160.stdout:5/423: mknod db/dc/d13/d54/c80 0 2024-05-25T06:11:36.415 INFO:tasks.workunit.client.0.smithi160.stdout:4/371: chown d4/d1a/l5e 100 1 2024-05-25T06:11:36.416 INFO:tasks.workunit.client.1.smithi203.stdout:7/684: mkdir d1/d9/d60/dea 0 2024-05-25T06:11:36.416 INFO:tasks.workunit.client.1.smithi203.stdout:9/627: rename d0/d57/d8f to d0/df/d37/da8/dd6 0 2024-05-25T06:11:36.418 INFO:tasks.workunit.client.0.smithi160.stdout:9/352: rename d4/d11/d1e/d3e/d25/d27/f5d to d4/d11/f67 0 2024-05-25T06:11:36.418 INFO:tasks.workunit.client.1.smithi203.stdout:9/628: write d0/d17/d20/fbc [762579,30707] 0 2024-05-25T06:11:36.420 INFO:tasks.workunit.client.0.smithi160.stdout:9/353: write d4/d11/d1e/f1f [879878,59559] 0 2024-05-25T06:11:36.420 INFO:tasks.workunit.client.0.smithi160.stdout:2/363: link d12/d1c/f56 d12/d3e/d5a/f7f 0 2024-05-25T06:11:36.421 INFO:tasks.workunit.client.1.smithi203.stdout:3/819: dwrite d10/d16/d23/d4a/d88/fdf [0,4194304] 0 2024-05-25T06:11:36.423 INFO:tasks.workunit.client.0.smithi160.stdout:3/311: rename d19/d22/f48 to d19/d2f/d38/d47/f70 0 2024-05-25T06:11:36.424 INFO:tasks.workunit.client.1.smithi203.stdout:3/820: write d10/d22/f35 [3945704,16416] 0 2024-05-25T06:11:36.425 INFO:tasks.workunit.client.0.smithi160.stdout:2/364: chown d12/c29 3 1 2024-05-25T06:11:36.427 INFO:tasks.workunit.client.0.smithi160.stdout:9/354: unlink d4/d11/d12/f62 0 2024-05-25T06:11:36.430 INFO:tasks.workunit.client.0.smithi160.stdout:8/420: rename d5/d44/d1f/d1e/d28/d4a/l84 to d5/d44/d1f/d1e/d28/l8d 0 2024-05-25T06:11:36.432 INFO:tasks.workunit.client.0.smithi160.stdout:9/355: unlink d4/f1c 0 2024-05-25T06:11:36.434 INFO:tasks.workunit.client.0.smithi160.stdout:8/421: readlink d5/d7/l10 0 2024-05-25T06:11:36.437 INFO:tasks.workunit.client.0.smithi160.stdout:8/422: stat d5/d39/d6a/l8a 0 2024-05-25T06:11:36.437 INFO:tasks.workunit.client.0.smithi160.stdout:1/284: rename de/l2c to de/d21/d49/d1c/d41/d45/l58 0 2024-05-25T06:11:36.438 INFO:tasks.workunit.client.0.smithi160.stdout:3/312: dwrite d19/d2f/d34/f5e [0,4194304] 0 2024-05-25T06:11:36.439 INFO:tasks.workunit.client.0.smithi160.stdout:1/285: fdatasync f2 0 2024-05-25T06:11:36.441 INFO:tasks.workunit.client.0.smithi160.stdout:2/365: dwrite f8 [0,4194304] 0 2024-05-25T06:11:36.442 INFO:tasks.workunit.client.0.smithi160.stdout:5/424: dread db/dc/d13/f1f [0,4194304] 0 2024-05-25T06:11:36.443 INFO:tasks.workunit.client.0.smithi160.stdout:2/366: chown d12/d1c/d2a/d7e 4 1 2024-05-25T06:11:36.446 INFO:tasks.workunit.client.1.smithi203.stdout:1/599: dread d14/f17 [0,4194304] 0 2024-05-25T06:11:36.447 INFO:tasks.workunit.client.0.smithi160.stdout:6/344: rename d5/d45/d63/d61 to d5/d30/d36/d73 0 2024-05-25T06:11:36.448 INFO:tasks.workunit.client.1.smithi203.stdout:2/757: dread d2/d1a/d33/f71 [0,4194304] 0 2024-05-25T06:11:36.449 INFO:tasks.workunit.client.0.smithi160.stdout:9/356: dread d4/f4e [0,4194304] 0 2024-05-25T06:11:36.449 INFO:tasks.workunit.client.0.smithi160.stdout:3/313: creat d19/d2f/d38/d43/f71 x:0 0 0 2024-05-25T06:11:36.450 INFO:tasks.workunit.client.1.smithi203.stdout:1/600: write d14/d57/d29/d94/db8/dc2/d36/db3/d5e/fb1 [366501,101093] 0 2024-05-25T06:11:36.451 INFO:tasks.workunit.client.0.smithi160.stdout:6/345: chown d5/de/d32/d1b/d2b/c5e 153 1 2024-05-25T06:11:36.452 INFO:tasks.workunit.client.1.smithi203.stdout:2/758: truncate d2/d42/f83 4298231 0 2024-05-25T06:11:36.452 INFO:tasks.workunit.client.0.smithi160.stdout:1/286: mkdir de/d21/d49/d1c/d59 0 2024-05-25T06:11:36.453 INFO:tasks.workunit.client.1.smithi203.stdout:2/759: chown d2/d1a/c2c 16087 1 2024-05-25T06:11:36.453 INFO:tasks.workunit.client.0.smithi160.stdout:7/413: dread d6/dc/ff [4194304,4194304] 0 2024-05-25T06:11:36.454 INFO:tasks.workunit.client.1.smithi203.stdout:4/837: rename d0/d9/d50/d6b/f51 to d0/d8/d3a/f119 0 2024-05-25T06:11:36.455 INFO:tasks.workunit.client.0.smithi160.stdout:9/357: chown d4/d11/d1e/d3e/c3c 510 1 2024-05-25T06:11:36.456 INFO:tasks.workunit.client.1.smithi203.stdout:8/825: getdents d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/d113 0 2024-05-25T06:11:36.456 INFO:tasks.workunit.client.1.smithi203.stdout:4/838: truncate d0/d8/d10/f58 835019 0 2024-05-25T06:11:36.457 INFO:tasks.workunit.client.1.smithi203.stdout:1/601: fsync d14/d57/d29/d94/db8/dc2/d36/f62 0 2024-05-25T06:11:36.459 INFO:tasks.workunit.client.0.smithi160.stdout:0/418: rename d2b/d31/d35/d36/d37/c51 to d2b/d58/c8d 0 2024-05-25T06:11:36.459 INFO:tasks.workunit.client.0.smithi160.stdout:9/358: write d4/d11/d1e/d24/f13 [245752,8595] 0 2024-05-25T06:11:36.460 INFO:tasks.workunit.client.0.smithi160.stdout:2/367: dread d12/d3e/f41 [0,4194304] 0 2024-05-25T06:11:36.460 INFO:tasks.workunit.client.0.smithi160.stdout:7/414: rename d6/dc to d6/dc/d64/d85 22 2024-05-25T06:11:36.461 INFO:tasks.workunit.client.1.smithi203.stdout:9/629: rename d0/d17/d20/d6e/f8d to d0/d17/fd7 0 2024-05-25T06:11:36.462 INFO:tasks.workunit.client.1.smithi203.stdout:8/826: write d2/d5/d10/d11/d1f/fd2 [756190,13010] 0 2024-05-25T06:11:36.462 INFO:tasks.workunit.client.1.smithi203.stdout:9/630: dread - d0/df/fc1 zero size 2024-05-25T06:11:36.462 INFO:tasks.workunit.client.0.smithi160.stdout:8/423: getdents d5 0 2024-05-25T06:11:36.464 INFO:tasks.workunit.client.0.smithi160.stdout:0/419: fdatasync d2b/f30 0 2024-05-25T06:11:36.464 INFO:tasks.workunit.client.0.smithi160.stdout:5/425: dwrite db/dc/d13/d1d/d23/d24/d2e/f32 [0,4194304] 0 2024-05-25T06:11:36.464 INFO:tasks.workunit.client.0.smithi160.stdout:6/346: mkdir d5/de/d32/d74 0 2024-05-25T06:11:36.465 INFO:tasks.workunit.client.1.smithi203.stdout:4/839: write d0/d8/d1f/d26/d3b/f7b [2129743,111850] 0 2024-05-25T06:11:36.467 INFO:tasks.workunit.client.0.smithi160.stdout:2/368: chown d12/d13/f4c 898404 1 2024-05-25T06:11:36.468 INFO:tasks.workunit.client.1.smithi203.stdout:4/840: dread - d0/d9/d50/dce/fcd zero size 2024-05-25T06:11:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:36 smithi160 ceph-mon[29041]: pgmap v18: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 55 MiB/s rd, 199 MiB/s wr, 404 op/s 2024-05-25T06:11:36.471 INFO:tasks.workunit.client.0.smithi160.stdout:0/420: chown d2b/c2c 3243 1 2024-05-25T06:11:36.473 INFO:tasks.workunit.client.1.smithi203.stdout:8/827: dwrite d2/d5/d28/ff9 [0,4194304] 0 2024-05-25T06:11:36.475 INFO:tasks.workunit.client.0.smithi160.stdout:8/424: creat d5/d39/d75/f8e x:0 0 0 2024-05-25T06:11:36.477 INFO:tasks.workunit.client.0.smithi160.stdout:6/347: mknod d5/d39/c75 0 2024-05-25T06:11:36.481 INFO:tasks.workunit.client.0.smithi160.stdout:2/369: creat d12/d1c/d2a/f80 x:0 0 0 2024-05-25T06:11:36.481 INFO:tasks.workunit.client.0.smithi160.stdout:3/314: dread d19/d2f/d38/f49 [0,4194304] 0 2024-05-25T06:11:36.482 INFO:tasks.workunit.client.0.smithi160.stdout:7/415: creat d6/dc/d64/d5a/f86 x:0 0 0 2024-05-25T06:11:36.484 INFO:tasks.workunit.client.0.smithi160.stdout:0/421: unlink d2b/d31/d35/d36/d86/f81 0 2024-05-25T06:11:36.486 INFO:tasks.workunit.client.0.smithi160.stdout:5/426: dwrite f4 [0,4194304] 0 2024-05-25T06:11:36.487 INFO:tasks.workunit.client.0.smithi160.stdout:8/425: write d5/d9/f24 [716758,110510] 0 2024-05-25T06:11:36.488 INFO:tasks.workunit.client.0.smithi160.stdout:6/348: rename d5/d30/f3d to d5/d45/d4a/f76 0 2024-05-25T06:11:36.489 INFO:tasks.workunit.client.0.smithi160.stdout:1/287: write de/d21/d49/f27 [1180234,65130] 0 2024-05-25T06:11:36.489 INFO:tasks.workunit.client.0.smithi160.stdout:0/422: chown d2b/d31/d35/d36/d37/f4d 79585847 1 2024-05-25T06:11:36.491 INFO:tasks.workunit.client.0.smithi160.stdout:1/288: stat de/d21/d49/d1c/d41 0 2024-05-25T06:11:36.491 INFO:tasks.workunit.client.0.smithi160.stdout:8/426: chown d5/d44/f37 147523520 1 2024-05-25T06:11:36.493 INFO:tasks.workunit.client.0.smithi160.stdout:7/416: symlink d6/d3e/d6f/d73/l87 0 2024-05-25T06:11:36.494 INFO:tasks.workunit.client.0.smithi160.stdout:2/370: dread - d12/d5c/f6b zero size 2024-05-25T06:11:36.495 INFO:tasks.workunit.client.0.smithi160.stdout:7/417: fsync d6/dc/f19 0 2024-05-25T06:11:36.497 INFO:tasks.workunit.client.0.smithi160.stdout:7/418: chown d6/dc/d21 257073 1 2024-05-25T06:11:36.499 INFO:tasks.workunit.client.0.smithi160.stdout:0/423: rename d2b/d58/f5e to d2b/d58/f8e 0 2024-05-25T06:11:36.501 INFO:tasks.workunit.client.0.smithi160.stdout:6/349: dread d5/d30/d36/f5c [0,4194304] 0 2024-05-25T06:11:36.502 INFO:tasks.workunit.client.0.smithi160.stdout:9/359: link d4/d11/d1e/c36 d4/d11/c68 0 2024-05-25T06:11:36.504 INFO:tasks.workunit.client.0.smithi160.stdout:7/419: rmdir d6/dc/d64/d5a/d7c 39 2024-05-25T06:11:36.504 INFO:tasks.workunit.client.0.smithi160.stdout:0/424: dread - d2b/d31/d35/d36/f89 zero size 2024-05-25T06:11:36.505 INFO:tasks.workunit.client.0.smithi160.stdout:3/315: getdents d19/d2f/d38/d60 0 2024-05-25T06:11:36.506 INFO:tasks.workunit.client.0.smithi160.stdout:2/371: fsync d12/d13/f15 0 2024-05-25T06:11:36.509 INFO:tasks.workunit.client.0.smithi160.stdout:7/420: fsync d6/dc/d21/f54 0 2024-05-25T06:11:36.511 INFO:tasks.workunit.client.0.smithi160.stdout:8/427: dwrite d5/d44/d1f/d1e/d28/d4a/f5c [0,4194304] 0 2024-05-25T06:11:36.514 INFO:tasks.workunit.client.0.smithi160.stdout:8/428: write d5/d39/f7d [243740,16217] 0 2024-05-25T06:11:36.515 INFO:tasks.workunit.client.0.smithi160.stdout:9/360: creat d4/f69 x:0 0 0 2024-05-25T06:11:36.516 INFO:tasks.workunit.client.0.smithi160.stdout:6/350: mkdir d5/de/d32/d74/d77 0 2024-05-25T06:11:36.518 INFO:tasks.workunit.client.0.smithi160.stdout:3/316: mknod d19/d2f/d38/d43/c72 0 2024-05-25T06:11:36.519 INFO:tasks.workunit.client.0.smithi160.stdout:5/427: dread f6 [0,4194304] 0 2024-05-25T06:11:36.522 INFO:tasks.workunit.client.0.smithi160.stdout:6/351: creat d5/de/d32/d35/f78 x:0 0 0 2024-05-25T06:11:36.523 INFO:tasks.workunit.client.0.smithi160.stdout:6/352: chown d5/de/f22 3 1 2024-05-25T06:11:36.524 INFO:tasks.workunit.client.0.smithi160.stdout:3/317: symlink d19/d22/l73 0 2024-05-25T06:11:36.525 INFO:tasks.workunit.client.0.smithi160.stdout:3/318: fsync d19/d2f/d34/f55 0 2024-05-25T06:11:36.528 INFO:tasks.workunit.client.0.smithi160.stdout:6/353: symlink d5/de/d32/d1b/d2b/l79 0 2024-05-25T06:11:36.531 INFO:tasks.workunit.client.0.smithi160.stdout:6/354: fdatasync d5/d39/f4d 0 2024-05-25T06:11:36.531 INFO:tasks.workunit.client.0.smithi160.stdout:8/429: rename d5/d44/d1f/d6b/f81 to d5/d44/d1f/f8f 0 2024-05-25T06:11:36.534 INFO:tasks.workunit.client.0.smithi160.stdout:0/425: dread f16 [0,4194304] 0 2024-05-25T06:11:36.535 INFO:tasks.workunit.client.0.smithi160.stdout:6/355: write d5/de/d32/d1b/f55 [4889471,20779] 0 2024-05-25T06:11:36.536 INFO:tasks.workunit.client.0.smithi160.stdout:9/361: rename d4/d30/f57 to d4/d11/d1e/d3e/d2f/f6a 0 2024-05-25T06:11:36.538 INFO:tasks.workunit.client.1.smithi203.stdout:7/685: getdents d1/d9/d2c 0 2024-05-25T06:11:36.539 INFO:tasks.workunit.client.1.smithi203.stdout:7/686: readlink d1/d9/d2c/d3a/d4e/d50/l7b 0 2024-05-25T06:11:36.539 INFO:tasks.workunit.client.0.smithi160.stdout:1/289: dwrite de/d21/d49/d1c/f3f [0,4194304] 0 2024-05-25T06:11:36.539 INFO:tasks.workunit.client.1.smithi203.stdout:7/687: chown d1/d9/d2c/d36/d3b 148109 1 2024-05-25T06:11:36.540 INFO:tasks.workunit.client.0.smithi160.stdout:3/319: getdents d19/d2f 0 2024-05-25T06:11:36.541 INFO:tasks.workunit.client.1.smithi203.stdout:8/828: creat d2/d5/d10/d11/d59/f11c x:0 0 0 2024-05-25T06:11:36.544 INFO:tasks.workunit.client.0.smithi160.stdout:6/356: write d5/d30/d36/d73/f64 [669304,120077] 0 2024-05-25T06:11:36.544 INFO:tasks.workunit.client.0.smithi160.stdout:2/372: dwrite d12/d1c/d37/f71 [0,4194304] 0 2024-05-25T06:11:36.545 INFO:tasks.workunit.client.0.smithi160.stdout:0/426: chown d2b/d31/d35/d7b/c82 28124 1 2024-05-25T06:11:36.547 INFO:tasks.workunit.client.0.smithi160.stdout:5/428: truncate db/dc/d13/d1d/d31/d56/f7d 23883 0 2024-05-25T06:11:36.549 INFO:tasks.workunit.client.0.smithi160.stdout:9/362: mknod d4/d11/d1e/d24/c6b 0 2024-05-25T06:11:36.549 INFO:tasks.workunit.client.0.smithi160.stdout:6/357: fdatasync d5/f34 0 2024-05-25T06:11:36.552 INFO:tasks.workunit.client.1.smithi203.stdout:8/829: creat d2/d5/d10/dfa/f11d x:0 0 0 2024-05-25T06:11:36.554 INFO:tasks.workunit.client.0.smithi160.stdout:0/427: creat d2b/d31/f8f x:0 0 0 2024-05-25T06:11:36.554 INFO:tasks.workunit.client.0.smithi160.stdout:8/430: dread d5/d44/d1f/d1e/d28/d4a/f5c [0,4194304] 0 2024-05-25T06:11:36.555 INFO:tasks.workunit.client.0.smithi160.stdout:6/358: chown d5/d30/c3a 46578196 1 2024-05-25T06:11:36.556 INFO:tasks.workunit.client.0.smithi160.stdout:1/290: creat de/d21/d49/d1c/d4f/f5a x:0 0 0 2024-05-25T06:11:36.559 INFO:tasks.workunit.client.0.smithi160.stdout:8/431: fsync d5/d44/f68 0 2024-05-25T06:11:36.560 INFO:tasks.workunit.client.0.smithi160.stdout:9/363: fsync d4/d11/d14/f3a 0 2024-05-25T06:11:36.560 INFO:tasks.workunit.client.1.smithi203.stdout:8/830: dwrite d2/d5/d18/d31/dac/dc2/d88/ff8 [0,4194304] 0 2024-05-25T06:11:36.561 INFO:tasks.workunit.client.0.smithi160.stdout:6/359: symlink d5/d45/l7a 0 2024-05-25T06:11:36.561 INFO:tasks.workunit.client.0.smithi160.stdout:1/291: read - de/d21/f31 zero size 2024-05-25T06:11:36.561 INFO:tasks.workunit.client.0.smithi160.stdout:3/320: dwrite d19/f44 [0,4194304] 0 2024-05-25T06:11:36.566 INFO:tasks.workunit.client.0.smithi160.stdout:9/364: stat d4/d11/f2c 0 2024-05-25T06:11:36.568 INFO:tasks.workunit.client.0.smithi160.stdout:2/373: dwrite f11 [0,4194304] 0 2024-05-25T06:11:36.571 INFO:tasks.workunit.client.0.smithi160.stdout:8/432: write d5/d44/d1f/d1e/d28/f38 [3416402,119230] 0 2024-05-25T06:11:36.572 INFO:tasks.workunit.client.0.smithi160.stdout:1/292: creat de/d21/d49/d28/f5b x:0 0 0 2024-05-25T06:11:36.575 INFO:tasks.workunit.client.0.smithi160.stdout:6/360: mknod d5/d45/d63/c7b 0 2024-05-25T06:11:36.576 INFO:tasks.workunit.client.0.smithi160.stdout:0/428: dwrite f20 [0,4194304] 0 2024-05-25T06:11:36.577 INFO:tasks.workunit.client.0.smithi160.stdout:6/361: chown d5/de/f42 67367 1 2024-05-25T06:11:36.581 INFO:tasks.workunit.client.0.smithi160.stdout:8/433: write d5/d44/d1f/d1e/d28/d4a/f5c [730054,55489] 0 2024-05-25T06:11:36.583 INFO:tasks.workunit.client.0.smithi160.stdout:2/374: getdents d12/d1c/d37 0 2024-05-25T06:11:36.584 INFO:tasks.workunit.client.0.smithi160.stdout:6/362: dread - d5/d30/f5a zero size 2024-05-25T06:11:36.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/293: creat de/d21/d49/d1c/d4f/f5c x:0 0 0 2024-05-25T06:11:36.585 INFO:tasks.workunit.client.0.smithi160.stdout:0/429: unlink d2b/c2d 0 2024-05-25T06:11:36.586 INFO:tasks.workunit.client.0.smithi160.stdout:6/363: fdatasync d5/f14 0 2024-05-25T06:11:36.587 INFO:tasks.workunit.client.0.smithi160.stdout:2/375: write f7 [1215548,45501] 0 2024-05-25T06:11:36.590 INFO:tasks.workunit.client.0.smithi160.stdout:8/434: symlink d5/d44/d1f/d58/d65/l90 0 2024-05-25T06:11:36.590 INFO:tasks.workunit.client.0.smithi160.stdout:6/364: fdatasync d5/d39/f6c 0 2024-05-25T06:11:36.590 INFO:tasks.workunit.client.0.smithi160.stdout:9/365: dread d4/d11/d12/f35 [0,4194304] 0 2024-05-25T06:11:36.592 INFO:tasks.workunit.client.0.smithi160.stdout:0/430: rename d2b/d31/d35/d7b to d2b/d31/d35/d7b/d90 22 2024-05-25T06:11:36.595 INFO:tasks.workunit.client.0.smithi160.stdout:6/365: symlink d5/de/d32/d1b/d2b/l7c 0 2024-05-25T06:11:36.596 INFO:tasks.workunit.client.0.smithi160.stdout:3/321: dwrite d19/d2f/d38/d47/f6e [0,4194304] 0 2024-05-25T06:11:36.596 INFO:tasks.workunit.client.0.smithi160.stdout:9/366: mkdir d4/d11/d1e/d6c 0 2024-05-25T06:11:36.597 INFO:tasks.workunit.client.0.smithi160.stdout:8/435: truncate d5/d44/d1f/f55 409524 0 2024-05-25T06:11:36.599 INFO:tasks.workunit.client.0.smithi160.stdout:3/322: write d19/f45 [133829,62337] 0 2024-05-25T06:11:36.600 INFO:tasks.workunit.client.0.smithi160.stdout:1/294: dwrite f7 [0,4194304] 0 2024-05-25T06:11:36.603 INFO:tasks.workunit.client.0.smithi160.stdout:9/367: symlink d4/d11/d12/d2e/l6d 0 2024-05-25T06:11:36.603 INFO:tasks.workunit.client.0.smithi160.stdout:8/436: fdatasync d5/d44/d1f/d1e/d28/f67 0 2024-05-25T06:11:36.605 INFO:tasks.workunit.client.0.smithi160.stdout:3/323: stat d19/d2f/d38/d43/f71 0 2024-05-25T06:11:36.607 INFO:tasks.workunit.client.0.smithi160.stdout:0/431: dwrite f24 [4194304,4194304] 0 2024-05-25T06:11:36.608 INFO:tasks.workunit.client.0.smithi160.stdout:3/324: truncate d19/f44 5209178 0 2024-05-25T06:11:36.610 INFO:tasks.workunit.client.0.smithi160.stdout:6/366: dwrite d5/f11 [0,4194304] 0 2024-05-25T06:11:36.610 INFO:tasks.workunit.client.0.smithi160.stdout:1/295: rename de/f29 to de/d21/d49/d1c/d59/f5d 0 2024-05-25T06:11:36.615 INFO:tasks.workunit.client.0.smithi160.stdout:8/437: creat d5/d44/d1f/d1e/d28/d4a/d73/f91 x:0 0 0 2024-05-25T06:11:36.616 INFO:tasks.workunit.client.0.smithi160.stdout:8/438: fdatasync d5/d44/f16 0 2024-05-25T06:11:36.619 INFO:tasks.workunit.client.0.smithi160.stdout:0/432: rename d2b/d31/d35/d36/d52/f62 to d2b/d46/f91 0 2024-05-25T06:11:36.619 INFO:tasks.workunit.client.0.smithi160.stdout:3/325: symlink d19/l74 0 2024-05-25T06:11:36.622 INFO:tasks.workunit.client.0.smithi160.stdout:3/326: dread - d19/d2f/d38/d47/f70 zero size 2024-05-25T06:11:36.622 INFO:tasks.workunit.client.0.smithi160.stdout:1/296: symlink de/d21/d49/l5e 0 2024-05-25T06:11:36.623 INFO:tasks.workunit.client.0.smithi160.stdout:2/376: dwrite d12/d1c/d37/f43 [4194304,4194304] 0 2024-05-25T06:11:36.623 INFO:tasks.workunit.client.0.smithi160.stdout:0/433: mknod d2b/d46/c92 0 2024-05-25T06:11:36.624 INFO:tasks.workunit.client.0.smithi160.stdout:1/297: stat de/d21/d49/d1c/d41 0 2024-05-25T06:11:36.624 INFO:tasks.workunit.client.0.smithi160.stdout:1/298: fdatasync de/f4d 0 2024-05-25T06:11:36.625 INFO:tasks.workunit.client.0.smithi160.stdout:0/434: chown d2b/d58/d67/d6a/l6b 118 1 2024-05-25T06:11:36.628 INFO:tasks.workunit.client.0.smithi160.stdout:8/439: dread d5/d44/d1f/d1e/d28/f5e [0,4194304] 0 2024-05-25T06:11:36.632 INFO:tasks.workunit.client.0.smithi160.stdout:0/435: getdents d2b/d31 0 2024-05-25T06:11:36.633 INFO:tasks.workunit.client.0.smithi160.stdout:8/440: creat d5/d39/d7a/f92 x:0 0 0 2024-05-25T06:11:36.637 INFO:tasks.workunit.client.0.smithi160.stdout:3/327: dwrite d19/d2f/d38/d47/f70 [0,4194304] 0 2024-05-25T06:11:36.637 INFO:tasks.workunit.client.0.smithi160.stdout:2/377: dwrite d12/d5c/f6b [0,4194304] 0 2024-05-25T06:11:36.647 INFO:tasks.workunit.client.0.smithi160.stdout:8/441: mkdir d5/d44/d1f/d1e/d28/d4a/d93 0 2024-05-25T06:11:36.649 INFO:tasks.workunit.client.0.smithi160.stdout:1/299: fsync de/d21/d49/d1c/d59/f5d 0 2024-05-25T06:11:36.651 INFO:tasks.workunit.client.0.smithi160.stdout:3/328: mknod d19/d2f/d38/d47/c75 0 2024-05-25T06:11:36.653 INFO:tasks.workunit.client.0.smithi160.stdout:0/436: link d2b/d31/d35/d36/d86/f66 d2b/d31/d6d/f93 0 2024-05-25T06:11:36.653 INFO:tasks.workunit.client.0.smithi160.stdout:1/300: write de/d21/d49/d1c/d52/f56 [5240486,73283] 0 2024-05-25T06:11:36.657 INFO:tasks.workunit.client.0.smithi160.stdout:8/442: dread d5/d44/f25 [0,4194304] 0 2024-05-25T06:11:36.658 INFO:tasks.workunit.client.0.smithi160.stdout:3/329: truncate d19/d2f/d38/d43/f4e 818025 0 2024-05-25T06:11:36.662 INFO:tasks.workunit.client.0.smithi160.stdout:6/367: dread f2 [0,4194304] 0 2024-05-25T06:11:36.664 INFO:tasks.workunit.client.1.smithi203.stdout:5/914: write dc/d11/d17/d1a/d2d/d4d/ded/d13f/f108 [813731,83068] 0 2024-05-25T06:11:36.665 INFO:tasks.workunit.client.1.smithi203.stdout:5/915: fdatasync dc/d3c/d3f/fd8 0 2024-05-25T06:11:36.666 INFO:tasks.workunit.client.1.smithi203.stdout:5/916: readlink dc/d11/l3a 0 2024-05-25T06:11:36.667 INFO:tasks.workunit.client.1.smithi203.stdout:5/917: chown dc/d11/d18/l2b 73171 1 2024-05-25T06:11:36.667 INFO:tasks.workunit.client.0.smithi160.stdout:0/437: dwrite d2b/d46/f5a [0,4194304] 0 2024-05-25T06:11:36.668 INFO:tasks.workunit.client.0.smithi160.stdout:6/368: dread - d5/d39/f6c zero size 2024-05-25T06:11:36.672 INFO:tasks.workunit.client.0.smithi160.stdout:1/301: dwrite de/d21/d49/d35/f24 [4194304,4194304] 0 2024-05-25T06:11:36.674 INFO:tasks.workunit.client.0.smithi160.stdout:6/369: readlink d5/de/d32/l54 0 2024-05-25T06:11:36.676 INFO:tasks.workunit.client.0.smithi160.stdout:3/330: dwrite d19/f29 [0,4194304] 0 2024-05-25T06:11:36.678 INFO:tasks.workunit.client.0.smithi160.stdout:3/331: stat d19/d2f/d38/d43/f46 0 2024-05-25T06:11:36.678 INFO:tasks.workunit.client.0.smithi160.stdout:6/370: stat d5/f14 0 2024-05-25T06:11:36.678 INFO:tasks.workunit.client.0.smithi160.stdout:3/332: readlink d19/d2f/d38/d43/l63 0 2024-05-25T06:11:36.681 INFO:tasks.workunit.client.0.smithi160.stdout:6/371: write d5/d45/d4a/f5d [326352,108516] 0 2024-05-25T06:11:36.682 INFO:tasks.workunit.client.0.smithi160.stdout:6/372: write d5/d30/f5a [199326,61218] 0 2024-05-25T06:11:36.683 INFO:tasks.workunit.client.0.smithi160.stdout:0/438: symlink d2b/d58/d67/l94 0 2024-05-25T06:11:36.685 INFO:tasks.workunit.client.0.smithi160.stdout:4/372: sync 2024-05-25T06:11:36.692 INFO:tasks.workunit.client.0.smithi160.stdout:4/373: read d4/d6/d28/f4e [719384,103521] 0 2024-05-25T06:11:36.693 INFO:tasks.workunit.client.0.smithi160.stdout:0/439: dread d2b/d46/f5a [0,4194304] 0 2024-05-25T06:11:36.698 INFO:tasks.workunit.client.0.smithi160.stdout:0/440: link d2b/d58/f8c d2b/d31/d35/d36/d37/f95 0 2024-05-25T06:11:36.699 INFO:tasks.workunit.client.0.smithi160.stdout:4/374: link d4/d6/d5b/d66/f71 d4/d6/d5b/d66/f74 0 2024-05-25T06:11:36.700 INFO:tasks.workunit.client.0.smithi160.stdout:0/441: mknod d2b/d46/c96 0 2024-05-25T06:11:36.701 INFO:tasks.workunit.client.0.smithi160.stdout:4/375: write d4/d1a/f50 [829129,91786] 0 2024-05-25T06:11:36.703 INFO:tasks.workunit.client.0.smithi160.stdout:0/442: mknod d2b/c97 0 2024-05-25T06:11:36.708 INFO:tasks.workunit.client.0.smithi160.stdout:4/376: getdents d4/d1c/d54 0 2024-05-25T06:11:36.710 INFO:tasks.workunit.client.0.smithi160.stdout:6/373: dread d5/de/f22 [0,4194304] 0 2024-05-25T06:11:36.711 INFO:tasks.workunit.client.1.smithi203.stdout:9/631: write d0/df/d2d/f7c [279112,110733] 0 2024-05-25T06:11:36.712 INFO:tasks.workunit.client.0.smithi160.stdout:4/377: readlink d4/d1c/d54/l5d 0 2024-05-25T06:11:36.716 INFO:tasks.workunit.client.0.smithi160.stdout:6/374: mkdir d5/de/d7d 0 2024-05-25T06:11:36.717 INFO:tasks.workunit.client.0.smithi160.stdout:4/378: fsync d4/d6/d35/d3d/d55/f56 0 2024-05-25T06:11:36.718 INFO:tasks.workunit.client.1.smithi203.stdout:9/632: creat d0/df/d37/fd8 x:0 0 0 2024-05-25T06:11:36.718 INFO:tasks.workunit.client.0.smithi160.stdout:1/302: dwrite de/d21/d49/f2b [0,4194304] 0 2024-05-25T06:11:36.719 INFO:tasks.workunit.client.1.smithi203.stdout:7/688: write d1/d9/d2c/d3a/d4e/f75 [859645,7973] 0 2024-05-25T06:11:36.719 INFO:tasks.workunit.client.0.smithi160.stdout:0/443: dwrite d2b/d31/d35/d7b/f88 [0,4194304] 0 2024-05-25T06:11:36.720 INFO:tasks.workunit.client.1.smithi203.stdout:9/633: creat d0/d57/fd9 x:0 0 0 2024-05-25T06:11:36.723 INFO:tasks.workunit.client.1.smithi203.stdout:7/689: symlink d1/d9/d2c/d3a/dd1/leb 0 2024-05-25T06:11:36.725 INFO:tasks.workunit.client.0.smithi160.stdout:6/375: truncate d5/de/d32/d35/f78 569952 0 2024-05-25T06:11:36.726 INFO:tasks.workunit.client.1.smithi203.stdout:9/634: unlink d0/d17/d20/d3c/l3d 0 2024-05-25T06:11:36.729 INFO:tasks.workunit.client.0.smithi160.stdout:0/444: dread d2b/f78 [0,4194304] 0 2024-05-25T06:11:36.732 INFO:tasks.workunit.client.0.smithi160.stdout:4/379: truncate d4/f4f 1933797 0 2024-05-25T06:11:36.733 INFO:tasks.workunit.client.0.smithi160.stdout:4/380: readlink d4/l14 0 2024-05-25T06:11:36.734 INFO:tasks.workunit.client.1.smithi203.stdout:7/690: link d1/d9/d2c/d62/d9b/fd2 d1/d9/d2c/d62/d7a/d89/fec 0 2024-05-25T06:11:36.735 INFO:tasks.workunit.client.0.smithi160.stdout:4/381: symlink d4/d6/d21/d42/l75 0 2024-05-25T06:11:36.736 INFO:tasks.workunit.client.0.smithi160.stdout:6/376: dwrite d5/de/f12 [0,4194304] 0 2024-05-25T06:11:36.737 INFO:tasks.workunit.client.0.smithi160.stdout:4/382: chown d4/d6/d21/f3f 8915683 1 2024-05-25T06:11:36.737 INFO:tasks.workunit.client.0.smithi160.stdout:0/445: dread f23 [0,4194304] 0 2024-05-25T06:11:36.737 INFO:tasks.workunit.client.0.smithi160.stdout:6/377: chown d5/d30/f5a 4 1 2024-05-25T06:11:36.738 INFO:tasks.workunit.client.1.smithi203.stdout:7/691: creat d1/d11/d3d/d4c/fed x:0 0 0 2024-05-25T06:11:36.740 INFO:tasks.workunit.client.0.smithi160.stdout:0/446: mkdir d2b/d31/d35/d43/d98 0 2024-05-25T06:11:36.741 INFO:tasks.workunit.client.1.smithi203.stdout:7/692: symlink d1/d9/d2c/d79/dc7/lee 0 2024-05-25T06:11:36.744 INFO:tasks.workunit.client.0.smithi160.stdout:0/447: creat d2b/d31/d35/d43/d98/f99 x:0 0 0 2024-05-25T06:11:36.747 INFO:tasks.workunit.client.1.smithi203.stdout:7/693: truncate d1/d9/d2c/d79/dc7/fd4 4099095 0 2024-05-25T06:11:36.832 INFO:tasks.workunit.client.0.smithi160.stdout:7/421: sync 2024-05-25T06:11:36.842 INFO:tasks.workunit.client.0.smithi160.stdout:7/422: dread d6/dc/f41 [0,4194304] 0 2024-05-25T06:11:36.845 INFO:tasks.workunit.client.0.smithi160.stdout:7/423: creat d6/dc/d64/d5a/f88 x:0 0 0 2024-05-25T06:11:36.860 INFO:tasks.workunit.client.1.smithi203.stdout:9/635: write d0/d17/f1b [7495405,19146] 0 2024-05-25T06:11:36.863 INFO:tasks.workunit.client.0.smithi160.stdout:6/378: truncate d5/d30/d36/d73/f64 616066 0 2024-05-25T06:11:36.864 INFO:tasks.workunit.client.0.smithi160.stdout:6/379: readlink d5/d45/l52 0 2024-05-25T06:11:36.864 INFO:tasks.workunit.client.0.smithi160.stdout:7/424: rmdir d6 39 2024-05-25T06:11:36.866 INFO:tasks.workunit.client.1.smithi203.stdout:9/636: symlink d0/d17/db6/lda 0 2024-05-25T06:11:36.866 INFO:tasks.workunit.client.0.smithi160.stdout:4/383: dread d4/f26 [0,4194304] 0 2024-05-25T06:11:36.867 INFO:tasks.workunit.client.1.smithi203.stdout:9/637: chown d0/df/d2d/f7c 340 1 2024-05-25T06:11:36.871 INFO:tasks.workunit.client.1.smithi203.stdout:9/638: fdatasync d0/d17/d20/f46 0 2024-05-25T06:11:36.871 INFO:tasks.workunit.client.0.smithi160.stdout:0/448: dwrite d2b/d31/d35/d36/d37/f95 [0,4194304] 0 2024-05-25T06:11:36.873 INFO:tasks.workunit.client.0.smithi160.stdout:6/380: rename d5/f34 to d5/de/d32/d35/d40/f7e 0 2024-05-25T06:11:36.873 INFO:tasks.workunit.client.0.smithi160.stdout:6/381: stat d5/d30/c3a 0 2024-05-25T06:11:36.876 INFO:tasks.workunit.client.0.smithi160.stdout:4/384: dread d4/d1a/f50 [0,4194304] 0 2024-05-25T06:11:36.879 INFO:tasks.workunit.client.0.smithi160.stdout:0/449: mknod d2b/d31/d35/d43/c9a 0 2024-05-25T06:11:36.880 INFO:tasks.workunit.client.0.smithi160.stdout:6/382: creat d5/de/d58/f7f x:0 0 0 2024-05-25T06:11:36.882 INFO:tasks.workunit.client.1.smithi203.stdout:1/602: rename d14/d57/fa9 to d14/d57/db7/fc9 0 2024-05-25T06:11:36.886 INFO:tasks.workunit.client.0.smithi160.stdout:7/425: getdents d6/dc/d64/d5a/d7c/d81 0 2024-05-25T06:11:36.887 INFO:tasks.workunit.client.1.smithi203.stdout:3/821: creat d10/d16/f106 x:0 0 0 2024-05-25T06:11:36.887 INFO:tasks.workunit.client.1.smithi203.stdout:9/639: dread d0/d17/d20/d3c/db2/f94 [0,4194304] 0 2024-05-25T06:11:36.888 INFO:tasks.workunit.client.0.smithi160.stdout:4/385: creat d4/f76 x:0 0 0 2024-05-25T06:11:36.889 INFO:tasks.workunit.client.1.smithi203.stdout:9/640: stat d0/d17/d20/d3c/c67 0 2024-05-25T06:11:36.891 INFO:tasks.workunit.client.0.smithi160.stdout:6/383: dwrite d5/de/d32/d35/f78 [0,4194304] 0 2024-05-25T06:11:36.891 INFO:tasks.workunit.client.0.smithi160.stdout:4/386: fdatasync d4/d6/d35/f53 0 2024-05-25T06:11:36.894 INFO:tasks.workunit.client.1.smithi203.stdout:4/841: rename d0/d8/d1f/d26/d3b/d9c/la5 to d0/d8/d3a/d81/dd4/d114/l11a 0 2024-05-25T06:11:36.895 INFO:tasks.workunit.client.0.smithi160.stdout:7/426: mkdir d6/d2b/d5d/d89 0 2024-05-25T06:11:36.897 INFO:tasks.workunit.client.0.smithi160.stdout:6/384: creat d5/de/d58/f80 x:0 0 0 2024-05-25T06:11:36.898 INFO:tasks.workunit.client.0.smithi160.stdout:6/385: write d5/de/f22 [1530582,40207] 0 2024-05-25T06:11:36.901 INFO:tasks.workunit.client.1.smithi203.stdout:9/641: creat d0/dd0/fdb x:0 0 0 2024-05-25T06:11:36.901 INFO:tasks.workunit.client.0.smithi160.stdout:7/427: creat d6/d2b/d5d/d89/f8a x:0 0 0 2024-05-25T06:11:36.903 INFO:tasks.workunit.client.0.smithi160.stdout:6/386: mknod d5/de/d58/c81 0 2024-05-25T06:11:36.903 INFO:tasks.workunit.client.0.smithi160.stdout:7/428: fdatasync d6/dc/f82 0 2024-05-25T06:11:36.904 INFO:tasks.workunit.client.1.smithi203.stdout:5/918: rename dc/d11/f11a to dc/d3c/f145 0 2024-05-25T06:11:36.905 INFO:tasks.workunit.client.1.smithi203.stdout:4/842: mkdir d0/d9/d50/d79/d9a/dbe/de0/d11b 0 2024-05-25T06:11:36.906 INFO:tasks.workunit.client.0.smithi160.stdout:6/387: creat d5/de/d32/d35/f82 x:0 0 0 2024-05-25T06:11:36.910 INFO:tasks.workunit.client.0.smithi160.stdout:4/387: dwrite d4/d1a/f40 [0,4194304] 0 2024-05-25T06:11:36.911 INFO:tasks.workunit.client.0.smithi160.stdout:4/388: readlink d4/d6/d35/d3d/l72 0 2024-05-25T06:11:36.917 INFO:tasks.workunit.client.0.smithi160.stdout:4/389: write d4/d6/f22 [448300,100280] 0 2024-05-25T06:11:36.918 INFO:tasks.workunit.client.0.smithi160.stdout:0/450: dread d2b/d31/d6d/f93 [0,4194304] 0 2024-05-25T06:11:36.919 INFO:tasks.workunit.client.1.smithi203.stdout:7/694: rename d1/d11/d3d/d4c/l58 to d1/d9/d2c/lef 0 2024-05-25T06:11:36.922 INFO:tasks.workunit.client.0.smithi160.stdout:0/451: creat d2b/d31/d35/d36/d86/f9b x:0 0 0 2024-05-25T06:11:36.924 INFO:tasks.workunit.client.0.smithi160.stdout:0/452: chown d2b/d31/d35/d43/l73 2416883 1 2024-05-25T06:11:36.926 INFO:tasks.workunit.client.1.smithi203.stdout:4/843: rename d0/d8/d10/d91/fba to d0/d8/d3a/d81/d93/d98/d99/ded/f11c 0 2024-05-25T06:11:36.929 INFO:tasks.workunit.client.1.smithi203.stdout:7/695: truncate d1/d11/f21 2127188 0 2024-05-25T06:11:36.930 INFO:tasks.workunit.client.1.smithi203.stdout:1/603: write d14/d57/d7e/fb5 [1245128,33600] 0 2024-05-25T06:11:36.930 INFO:tasks.workunit.client.0.smithi160.stdout:0/453: dwrite d2b/d58/f8c [0,4194304] 0 2024-05-25T06:11:36.932 INFO:tasks.workunit.client.1.smithi203.stdout:3/822: write d10/f21 [4904640,17641] 0 2024-05-25T06:11:36.932 INFO:tasks.workunit.client.0.smithi160.stdout:0/454: fdatasync d2b/d58/f8e 0 2024-05-25T06:11:36.934 INFO:tasks.workunit.client.0.smithi160.stdout:4/390: write d4/d6/d28/f25 [1324898,128315] 0 2024-05-25T06:11:36.936 INFO:tasks.workunit.client.0.smithi160.stdout:6/388: dread d5/d39/f4d [4194304,4194304] 0 2024-05-25T06:11:36.936 INFO:tasks.workunit.client.1.smithi203.stdout:1/604: mkdir d14/d57/d29/d94/db8/dc2/d36/dca 0 2024-05-25T06:11:36.937 INFO:tasks.workunit.client.1.smithi203.stdout:3/823: mkdir d10/d16/d71/dfa/d107 0 2024-05-25T06:11:36.938 INFO:tasks.workunit.client.1.smithi203.stdout:5/919: write dc/d11/d17/f5d [4810945,75895] 0 2024-05-25T06:11:36.939 INFO:tasks.workunit.client.1.smithi203.stdout:9/642: truncate d0/df/d37/fb7 1536880 0 2024-05-25T06:11:36.941 INFO:tasks.workunit.client.1.smithi203.stdout:3/824: write d10/d16/d23/d4a/d88/deb/f100 [184053,63307] 0 2024-05-25T06:11:36.941 INFO:tasks.workunit.client.0.smithi160.stdout:0/455: truncate d2b/d58/f8c 2584783 0 2024-05-25T06:11:36.942 INFO:tasks.workunit.client.1.smithi203.stdout:1/605: mknod d14/d7c/d9d/dc8/ccb 0 2024-05-25T06:11:36.943 INFO:tasks.workunit.client.1.smithi203.stdout:3/825: read d10/f4f [77936,118320] 0 2024-05-25T06:11:36.944 INFO:tasks.workunit.client.0.smithi160.stdout:4/391: dwrite d4/d1c/f62 [0,4194304] 0 2024-05-25T06:11:36.945 INFO:tasks.workunit.client.1.smithi203.stdout:7/696: dread - d1/d9/d2c/fa1 zero size 2024-05-25T06:11:36.945 INFO:tasks.workunit.client.0.smithi160.stdout:4/392: stat d4/d1c/d54/c5a 0 2024-05-25T06:11:36.946 INFO:tasks.workunit.client.1.smithi203.stdout:1/606: rmdir d14/d7c/d9d/dc8 39 2024-05-25T06:11:36.947 INFO:tasks.workunit.client.0.smithi160.stdout:4/393: write d4/d1c/f62 [1293157,125128] 0 2024-05-25T06:11:36.947 INFO:tasks.workunit.client.1.smithi203.stdout:3/826: dread d10/d22/f57 [0,4194304] 0 2024-05-25T06:11:36.947 INFO:tasks.workunit.client.0.smithi160.stdout:4/394: readlink d4/d6/d35/d3d/l72 0 2024-05-25T06:11:36.948 INFO:tasks.workunit.client.1.smithi203.stdout:4/844: truncate d0/d8/d1f/d26/d3b/d9c/fe5 553030 0 2024-05-25T06:11:36.949 INFO:tasks.workunit.client.1.smithi203.stdout:5/920: rename dc/d11/d17/d1a/d2d/d4d/f83 to dc/d11/d17/d1a/d75/f146 0 2024-05-25T06:11:36.952 INFO:tasks.workunit.client.1.smithi203.stdout:4/845: creat d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f11d x:0 0 0 2024-05-25T06:11:36.952 INFO:tasks.workunit.client.0.smithi160.stdout:0/456: truncate f1f 820480 0 2024-05-25T06:11:36.954 INFO:tasks.workunit.client.1.smithi203.stdout:4/846: write d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f11d [1045473,2324] 0 2024-05-25T06:11:36.954 INFO:tasks.workunit.client.1.smithi203.stdout:1/607: rename d14/c2d to d14/d57/d97/ccc 0 2024-05-25T06:11:36.955 INFO:tasks.workunit.client.1.smithi203.stdout:3/827: mknod d10/d16/d23/d67/d105/c108 0 2024-05-25T06:11:36.956 INFO:tasks.workunit.client.1.smithi203.stdout:7/697: mkdir d1/d9/d2c/d79/df0 0 2024-05-25T06:11:36.956 INFO:tasks.workunit.client.0.smithi160.stdout:0/457: unlink d2b/d31/d35/d36/d37/c50 0 2024-05-25T06:11:36.957 INFO:tasks.workunit.client.1.smithi203.stdout:4/847: chown d0/d8/d1f/d26/d3b/df5/f10c 0 1 2024-05-25T06:11:36.958 INFO:tasks.workunit.client.1.smithi203.stdout:3/828: unlink d10/d22/cb1 0 2024-05-25T06:11:36.958 INFO:tasks.workunit.client.0.smithi160.stdout:0/458: truncate d2b/d31/d35/f49 799083 0 2024-05-25T06:11:36.959 INFO:tasks.workunit.client.0.smithi160.stdout:0/459: chown d2b/d31/d35/d7b 9 1 2024-05-25T06:11:36.959 INFO:tasks.workunit.client.1.smithi203.stdout:1/608: creat d14/d57/d29/d94/db8/dc2/d36/fcd x:0 0 0 2024-05-25T06:11:36.961 INFO:tasks.workunit.client.0.smithi160.stdout:0/460: fsync d2b/d46/f5a 0 2024-05-25T06:11:36.961 INFO:tasks.workunit.client.0.smithi160.stdout:0/461: chown f15 65402 1 2024-05-25T06:11:36.965 INFO:tasks.workunit.client.0.smithi160.stdout:0/462: rmdir d2b/d31/d35/d36/d86 39 2024-05-25T06:11:36.966 INFO:tasks.workunit.client.1.smithi203.stdout:9/643: dread d0/d17/f1b [4194304,4194304] 0 2024-05-25T06:11:36.975 INFO:tasks.workunit.client.1.smithi203.stdout:9/644: chown d0/d17/d20/d3c/db2/d7a/fb4 90250029 1 2024-05-25T06:11:36.975 INFO:tasks.workunit.client.0.smithi160.stdout:0/463: rename d2b/d31/d35/d7b/f88 to d2b/d31/d35/d36/d86/f9c 0 2024-05-25T06:11:36.976 INFO:tasks.workunit.client.1.smithi203.stdout:4/848: dwrite d0/d8/d3a/d81/d93/d98/d99/fa7 [0,4194304] 0 2024-05-25T06:11:36.976 INFO:tasks.workunit.client.1.smithi203.stdout:7/698: dread d1/f3 [4194304,4194304] 0 2024-05-25T06:11:36.976 INFO:tasks.workunit.client.0.smithi160.stdout:0/464: chown d2b/d31/f8f 5 1 2024-05-25T06:11:36.976 INFO:tasks.workunit.client.1.smithi203.stdout:1/609: dwrite d14/d23/f78 [0,4194304] 0 2024-05-25T06:11:36.977 INFO:tasks.workunit.client.0.smithi160.stdout:4/395: dread d4/f3b [4194304,4194304] 0 2024-05-25T06:11:36.977 INFO:tasks.workunit.client.0.smithi160.stdout:0/465: write d2b/d58/f8e [836500,56781] 0 2024-05-25T06:11:36.978 INFO:tasks.workunit.client.1.smithi203.stdout:5/921: dread dc/d11/d42/d94/fc7 [0,4194304] 0 2024-05-25T06:11:36.982 INFO:tasks.workunit.client.0.smithi160.stdout:4/396: creat d4/d6/d21/f77 x:0 0 0 2024-05-25T06:11:36.982 INFO:tasks.workunit.client.1.smithi203.stdout:4/849: rename d0/d8/d1f/d26/d3b/df5 to d0/d8/d1f/d26/d3b/df5/d11e 22 2024-05-25T06:11:36.982 INFO:tasks.workunit.client.1.smithi203.stdout:5/922: dread - dc/d11/d17/d1a/d72/f109 zero size 2024-05-25T06:11:36.983 INFO:tasks.workunit.client.1.smithi203.stdout:4/850: chown d0/d8/d3a/f119 23 1 2024-05-25T06:11:36.983 INFO:tasks.workunit.client.0.smithi160.stdout:4/397: stat d4/d6/fe 0 2024-05-25T06:11:36.983 INFO:tasks.workunit.client.1.smithi203.stdout:7/699: truncate d1/d9/d2c/d62/d9b/fc9 951398 0 2024-05-25T06:11:36.984 INFO:tasks.workunit.client.1.smithi203.stdout:1/610: creat d14/d57/d29/d94/db8/dc2/d36/fce x:0 0 0 2024-05-25T06:11:36.984 INFO:tasks.workunit.client.1.smithi203.stdout:0/643: sync 2024-05-25T06:11:36.987 INFO:tasks.workunit.client.1.smithi203.stdout:3/829: dread d10/d22/d2a/d99/fb6 [0,4194304] 0 2024-05-25T06:11:36.990 INFO:tasks.workunit.client.0.smithi160.stdout:0/466: dwrite d2b/d58/f72 [0,4194304] 0 2024-05-25T06:11:36.991 INFO:tasks.workunit.client.1.smithi203.stdout:9/645: mkdir d0/df/d37/d4f/d93/ddc 0 2024-05-25T06:11:36.992 INFO:tasks.workunit.client.0.smithi160.stdout:0/467: chown f23 25490 1 2024-05-25T06:11:36.992 INFO:tasks.workunit.client.0.smithi160.stdout:5/429: sync 2024-05-25T06:11:36.994 INFO:tasks.workunit.client.1.smithi203.stdout:5/923: creat dc/d11/d17/d1a/d2d/d4d/ded/d13f/dd4/f147 x:0 0 0 2024-05-25T06:11:36.995 INFO:tasks.workunit.client.1.smithi203.stdout:4/851: mknod d0/d8/d10/d91/da2/c11f 0 2024-05-25T06:11:36.996 INFO:tasks.workunit.client.0.smithi160.stdout:5/430: rmdir db/dc/d13/d1d/d23/d45 39 2024-05-25T06:11:36.996 INFO:tasks.workunit.client.1.smithi203.stdout:0/644: unlink dd/d10/d34/d4c/d5d/lc1 0 2024-05-25T06:11:36.996 INFO:tasks.workunit.client.0.smithi160.stdout:4/398: dwrite d4/d6/d35/d3d/d55/f56 [8388608,4194304] 0 2024-05-25T06:11:37.000 INFO:tasks.workunit.client.1.smithi203.stdout:3/830: mkdir d10/d16/d23/d67/d105/d109 0 2024-05-25T06:11:37.002 INFO:tasks.workunit.client.1.smithi203.stdout:5/924: creat dc/d11/d17/d1a/d75/f148 x:0 0 0 2024-05-25T06:11:37.002 INFO:tasks.workunit.client.0.smithi160.stdout:5/431: read db/dc/f15 [2334767,109725] 0 2024-05-25T06:11:37.004 INFO:tasks.workunit.client.1.smithi203.stdout:4/852: readlink d0/d9/d50/lc3 0 2024-05-25T06:11:37.004 INFO:tasks.workunit.client.0.smithi160.stdout:4/399: mkdir d4/d6/d5b/d66/d78 0 2024-05-25T06:11:37.005 INFO:tasks.workunit.client.0.smithi160.stdout:4/400: dread - d4/f76 zero size 2024-05-25T06:11:37.005 INFO:tasks.workunit.client.1.smithi203.stdout:7/700: link d1/d9/d2c/d3a/d4e/f61 d1/d11/d3d/d4c/ff1 0 2024-05-25T06:11:37.005 INFO:tasks.workunit.client.0.smithi160.stdout:0/468: read f2 [2041598,118068] 0 2024-05-25T06:11:37.007 INFO:tasks.workunit.client.1.smithi203.stdout:4/853: unlink d0/d8/cfb 0 2024-05-25T06:11:37.008 INFO:tasks.workunit.client.1.smithi203.stdout:0/645: dwrite dd/d3d/d4f/f8a [0,4194304] 0 2024-05-25T06:11:37.011 INFO:tasks.workunit.client.1.smithi203.stdout:3/831: symlink d10/d16/d23/l10a 0 2024-05-25T06:11:37.013 INFO:tasks.workunit.client.1.smithi203.stdout:9/646: rename d0/df/d2d/d97 to d0/df/ddd 0 2024-05-25T06:11:37.015 INFO:tasks.workunit.client.1.smithi203.stdout:7/701: chown d1/f1c 3487 1 2024-05-25T06:11:37.017 INFO:tasks.workunit.client.1.smithi203.stdout:4/854: dread d0/d8/d3a/d81/d93/d98/d99/fcc [0,4194304] 0 2024-05-25T06:11:37.020 INFO:tasks.workunit.client.1.smithi203.stdout:7/702: chown d1/d9/d2c/d62/c6f 2312 1 2024-05-25T06:11:37.020 INFO:tasks.workunit.client.1.smithi203.stdout:0/646: link dd/d3d/d4f/c57 dd/d3a/d66/d86/cea 0 2024-05-25T06:11:37.022 INFO:tasks.workunit.client.1.smithi203.stdout:0/647: read - dd/d3d/d75/fcf zero size 2024-05-25T06:11:37.022 INFO:tasks.workunit.client.1.smithi203.stdout:7/703: symlink d1/dbd/dc1/lf2 0 2024-05-25T06:11:37.023 INFO:tasks.workunit.client.1.smithi203.stdout:9/647: link d0/df/d2d/l40 d0/df/d2d/d92/lde 0 2024-05-25T06:11:37.024 INFO:tasks.workunit.client.1.smithi203.stdout:4/855: dwrite d0/d8/d3a/f109 [0,4194304] 0 2024-05-25T06:11:37.027 INFO:tasks.workunit.client.1.smithi203.stdout:9/648: creat d0/df/d2d/d92/fdf x:0 0 0 2024-05-25T06:11:37.027 INFO:tasks.workunit.client.1.smithi203.stdout:1/611: truncate d14/d57/d1d/f2f 8136329 0 2024-05-25T06:11:37.034 INFO:tasks.workunit.client.1.smithi203.stdout:0/648: dread dd/d3d/d4f/f87 [0,4194304] 0 2024-05-25T06:11:37.035 INFO:tasks.workunit.client.1.smithi203.stdout:9/649: dread - d0/df/d37/da8/dd6/fb5 zero size 2024-05-25T06:11:37.035 INFO:tasks.workunit.client.1.smithi203.stdout:1/612: truncate fc 1845331 0 2024-05-25T06:11:37.039 INFO:tasks.workunit.client.1.smithi203.stdout:1/613: fsync f10 0 2024-05-25T06:11:37.039 INFO:tasks.workunit.client.1.smithi203.stdout:0/649: chown dd/f64 24 1 2024-05-25T06:11:37.042 INFO:tasks.workunit.client.1.smithi203.stdout:4/856: dwrite d0/d8/d52/d59/d7a/d83/fd6 [0,4194304] 0 2024-05-25T06:11:37.045 INFO:tasks.workunit.client.1.smithi203.stdout:0/650: fdatasync dd/d3a/fae 0 2024-05-25T06:11:37.046 INFO:tasks.workunit.client.1.smithi203.stdout:5/925: dwrite dc/d11/d17/d1a/d2d/d4d/dcc/ff4 [0,4194304] 0 2024-05-25T06:11:37.046 INFO:tasks.workunit.client.1.smithi203.stdout:1/614: rmdir d14/d57/d97/da4 39 2024-05-25T06:11:37.048 INFO:tasks.workunit.client.1.smithi203.stdout:7/704: dwrite d1/d9/f29 [0,4194304] 0 2024-05-25T06:11:37.049 INFO:tasks.workunit.client.0.smithi160.stdout:4/401: write d4/d1a/f40 [305559,76579] 0 2024-05-25T06:11:37.050 INFO:tasks.workunit.client.0.smithi160.stdout:5/432: mkdir db/d47/d81 0 2024-05-25T06:11:37.050 INFO:tasks.workunit.client.1.smithi203.stdout:3/832: dread d10/d16/d23/fdd [0,4194304] 0 2024-05-25T06:11:37.052 INFO:tasks.workunit.client.1.smithi203.stdout:7/705: chown d1/d9/d2c/d3a/d4e/f61 25 1 2024-05-25T06:11:37.053 INFO:tasks.workunit.client.0.smithi160.stdout:5/433: write db/dc/d13/d1d/d31/d3b/d69/f79 [202791,84765] 0 2024-05-25T06:11:37.055 INFO:tasks.workunit.client.0.smithi160.stdout:5/434: fsync db/dc/d13/d1d/d31/d56/f61 0 2024-05-25T06:11:37.060 INFO:tasks.workunit.client.1.smithi203.stdout:1/615: dwrite d14/d57/d7e/fb5 [0,4194304] 0 2024-05-25T06:11:37.060 INFO:tasks.workunit.client.0.smithi160.stdout:5/435: creat db/dc/d13/d1d/d31/d56/f82 x:0 0 0 2024-05-25T06:11:37.060 INFO:tasks.workunit.client.0.smithi160.stdout:4/402: dwrite d4/d6/d21/f3f [0,4194304] 0 2024-05-25T06:11:37.062 INFO:tasks.workunit.client.0.smithi160.stdout:4/403: chown d4/d6/l3a 50 1 2024-05-25T06:11:37.062 INFO:tasks.workunit.client.1.smithi203.stdout:4/857: write d0/d8/d3a/d81/d93/d98/d99/ded/f11c [116389,19258] 0 2024-05-25T06:11:37.066 INFO:tasks.workunit.client.1.smithi203.stdout:5/926: truncate dc/d3c/f78 1036819 0 2024-05-25T06:11:37.066 INFO:tasks.workunit.client.0.smithi160.stdout:4/404: creat d4/d6/d35/d3d/d55/f79 x:0 0 0 2024-05-25T06:11:37.067 INFO:tasks.workunit.client.1.smithi203.stdout:1/616: write d14/d57/d29/d94/db8/dc2/d36/f39 [1884909,97161] 0 2024-05-25T06:11:37.067 INFO:tasks.workunit.client.1.smithi203.stdout:0/651: dwrite dd/f72 [0,4194304] 0 2024-05-25T06:11:37.067 INFO:tasks.workunit.client.0.smithi160.stdout:5/436: creat db/dc/f83 x:0 0 0 2024-05-25T06:11:37.068 INFO:tasks.workunit.client.1.smithi203.stdout:0/652: chown dd/c30 120214 1 2024-05-25T06:11:37.070 INFO:tasks.workunit.client.0.smithi160.stdout:4/405: mknod d4/d6/d21/c7a 0 2024-05-25T06:11:37.078 INFO:tasks.workunit.client.1.smithi203.stdout:7/706: write d1/d11/d20/f28 [4235269,72777] 0 2024-05-25T06:11:37.078 INFO:tasks.workunit.client.1.smithi203.stdout:3/833: creat d10/d16/d23/f10b x:0 0 0 2024-05-25T06:11:37.080 INFO:tasks.workunit.client.1.smithi203.stdout:1/617: mkdir d14/d57/d29/d94/db8/dc2/d36/dca/dcf 0 2024-05-25T06:11:37.080 INFO:tasks.workunit.client.1.smithi203.stdout:0/653: creat dd/d3d/d48/de6/feb x:0 0 0 2024-05-25T06:11:37.080 INFO:tasks.workunit.client.0.smithi160.stdout:5/437: truncate db/dc/d13/d1d/f20 1027166 0 2024-05-25T06:11:37.081 INFO:tasks.workunit.client.1.smithi203.stdout:0/654: fdatasync dd/d3a/f4b 0 2024-05-25T06:11:37.081 INFO:tasks.workunit.client.0.smithi160.stdout:0/469: truncate f5 2703204 0 2024-05-25T06:11:37.081 INFO:tasks.workunit.client.1.smithi203.stdout:4/858: creat d0/d8/d52/d59/f120 x:0 0 0 2024-05-25T06:11:37.082 INFO:tasks.workunit.client.0.smithi160.stdout:4/406: mkdir d4/d6/d7b 0 2024-05-25T06:11:37.083 INFO:tasks.workunit.client.0.smithi160.stdout:4/407: fsync d4/f2a 0 2024-05-25T06:11:37.084 INFO:tasks.workunit.client.1.smithi203.stdout:1/618: fdatasync d14/d57/d29/d94/db8/dc2/d36/fce 0 2024-05-25T06:11:37.084 INFO:tasks.workunit.client.1.smithi203.stdout:4/859: mkdir d0/d8/d3a/d81/d121 0 2024-05-25T06:11:37.085 INFO:tasks.workunit.client.1.smithi203.stdout:0/655: mknod dd/d3a/cec 0 2024-05-25T06:11:37.087 INFO:tasks.workunit.client.1.smithi203.stdout:1/619: truncate d14/f17 1009675 0 2024-05-25T06:11:37.089 INFO:tasks.workunit.client.0.smithi160.stdout:0/470: truncate d2b/d31/d35/d36/d37/f3b 701417 0 2024-05-25T06:11:37.089 INFO:tasks.workunit.client.0.smithi160.stdout:0/471: readlink l3 0 2024-05-25T06:11:37.091 INFO:tasks.workunit.client.1.smithi203.stdout:1/620: fsync d14/d57/d29/d94/db8/dc2/d36/f39 0 2024-05-25T06:11:37.092 INFO:tasks.workunit.client.0.smithi160.stdout:2/378: sync 2024-05-25T06:11:37.092 INFO:tasks.workunit.client.0.smithi160.stdout:9/368: sync 2024-05-25T06:11:37.092 INFO:tasks.workunit.client.1.smithi203.stdout:4/860: dwrite d0/d8/f23 [0,4194304] 0 2024-05-25T06:11:37.093 INFO:tasks.workunit.client.0.smithi160.stdout:0/472: creat d2b/d31/d35/d7b/f9d x:0 0 0 2024-05-25T06:11:37.093 INFO:tasks.workunit.client.0.smithi160.stdout:5/438: dwrite db/dc/d13/d1d/d23/f46 [0,4194304] 0 2024-05-25T06:11:37.094 INFO:tasks.workunit.client.1.smithi203.stdout:1/621: creat d14/d57/d7e/fd0 x:0 0 0 2024-05-25T06:11:37.097 INFO:tasks.workunit.client.1.smithi203.stdout:4/861: symlink d0/d9/d50/d79/d9a/dbe/de0/l122 0 2024-05-25T06:11:37.099 INFO:tasks.workunit.client.1.smithi203.stdout:1/622: creat d14/d57/db7/fd1 x:0 0 0 2024-05-25T06:11:37.102 INFO:tasks.workunit.client.0.smithi160.stdout:0/473: mknod d2b/d31/d6d/c9e 0 2024-05-25T06:11:37.103 INFO:tasks.workunit.client.0.smithi160.stdout:5/439: readlink db/dc/d13/d1d/d31/d3b/d69/l71 0 2024-05-25T06:11:37.108 INFO:tasks.workunit.client.0.smithi160.stdout:9/369: rename d4/d11/d1e/d3e/d2f/f6a to d4/f6e 0 2024-05-25T06:11:37.108 INFO:tasks.workunit.client.0.smithi160.stdout:5/440: creat db/dc/d13/d1d/d31/d3b/f84 x:0 0 0 2024-05-25T06:11:37.109 INFO:tasks.workunit.client.0.smithi160.stdout:9/370: stat d4/d11/d12/d58 0 2024-05-25T06:11:37.110 INFO:tasks.workunit.client.1.smithi203.stdout:5/927: write dc/d11/d17/d1a/d2d/d53/da2/fd6 [885450,60706] 0 2024-05-25T06:11:37.110 INFO:tasks.workunit.client.0.smithi160.stdout:0/474: link d2b/d46/d63/f7e d2b/d46/d63/f9f 0 2024-05-25T06:11:37.111 INFO:tasks.workunit.client.0.smithi160.stdout:0/475: truncate f1e 4350914 0 2024-05-25T06:11:37.111 INFO:tasks.workunit.client.1.smithi203.stdout:1/623: rename d14/d57/d29/d94/db8/dc2/d36 to d14/d57/db7/dd2 0 2024-05-25T06:11:37.112 INFO:tasks.workunit.client.1.smithi203.stdout:1/624: chown f10 6 1 2024-05-25T06:11:37.113 INFO:tasks.workunit.client.0.smithi160.stdout:0/476: truncate d2b/d31/f8f 650189 0 2024-05-25T06:11:37.113 INFO:tasks.workunit.client.0.smithi160.stdout:5/441: fdatasync db/dc/d13/d1d/d31/d3b/f38 0 2024-05-25T06:11:37.113 INFO:tasks.workunit.client.1.smithi203.stdout:5/928: chown dc/d11/d42/leb 1709464 1 2024-05-25T06:11:37.114 INFO:tasks.workunit.client.0.smithi160.stdout:9/371: dread d4/f4e [0,4194304] 0 2024-05-25T06:11:37.114 INFO:tasks.workunit.client.1.smithi203.stdout:1/625: creat d14/d7c/fd3 x:0 0 0 2024-05-25T06:11:37.115 INFO:tasks.workunit.client.0.smithi160.stdout:5/442: chown db/dc/d13/d1d/d31/d3b/c29 11314 1 2024-05-25T06:11:37.118 INFO:tasks.workunit.client.1.smithi203.stdout:1/626: rename d14/d23/c38 to d14/d57/dbb/cd4 0 2024-05-25T06:11:37.120 INFO:tasks.workunit.client.1.smithi203.stdout:3/834: dread d10/d16/d23/d46/fce [0,4194304] 0 2024-05-25T06:11:37.123 INFO:tasks.workunit.client.0.smithi160.stdout:0/477: link d2b/d31/f34 d2b/d31/d35/d7b/fa0 0 2024-05-25T06:11:37.126 INFO:tasks.workunit.client.1.smithi203.stdout:3/835: readlink d10/l18 0 2024-05-25T06:11:37.127 INFO:tasks.workunit.client.1.smithi203.stdout:1/627: dwrite d14/d57/d1d/f67 [4194304,4194304] 0 2024-05-25T06:11:37.128 INFO:tasks.workunit.client.0.smithi160.stdout:4/408: dread d4/f4f [0,4194304] 0 2024-05-25T06:11:37.130 INFO:tasks.workunit.client.0.smithi160.stdout:9/372: dread d4/d11/d1e/d24/f1b [0,4194304] 0 2024-05-25T06:11:37.131 INFO:tasks.workunit.client.0.smithi160.stdout:2/379: truncate fe 245211 0 2024-05-25T06:11:37.132 INFO:tasks.workunit.client.1.smithi203.stdout:4/862: truncate d0/d8/d1f/f2b 8796926 0 2024-05-25T06:11:37.135 INFO:tasks.workunit.client.1.smithi203.stdout:7/707: dwrite d1/d9/d2c/d36/fbc [0,4194304] 0 2024-05-25T06:11:37.136 INFO:tasks.workunit.client.0.smithi160.stdout:4/409: unlink d4/d1a/l20 0 2024-05-25T06:11:37.138 INFO:tasks.workunit.client.1.smithi203.stdout:5/929: truncate dc/d11/f105 8149491 0 2024-05-25T06:11:37.142 INFO:tasks.workunit.client.1.smithi203.stdout:4/863: write d0/d9/d50/fc5 [4351012,40441] 0 2024-05-25T06:11:37.144 INFO:tasks.workunit.client.1.smithi203.stdout:3/836: dwrite d10/fb5 [0,4194304] 0 2024-05-25T06:11:37.145 INFO:tasks.workunit.client.1.smithi203.stdout:7/708: mkdir d1/dbd/dc1/df3 0 2024-05-25T06:11:37.146 INFO:tasks.workunit.client.1.smithi203.stdout:7/709: fdatasync d1/d11/fc0 0 2024-05-25T06:11:37.146 INFO:tasks.workunit.client.0.smithi160.stdout:4/410: rmdir d4/d6/d35 39 2024-05-25T06:11:37.147 INFO:tasks.workunit.client.1.smithi203.stdout:7/710: stat d1/d11/d20/c98 0 2024-05-25T06:11:37.147 INFO:tasks.workunit.client.1.smithi203.stdout:5/930: creat dc/d11/d17/d1a/d2d/d4d/ded/f149 x:0 0 0 2024-05-25T06:11:37.150 INFO:tasks.workunit.client.1.smithi203.stdout:4/864: stat d0/d8/d52/l107 0 2024-05-25T06:11:37.151 INFO:tasks.workunit.client.0.smithi160.stdout:2/380: mknod d12/d75/c81 0 2024-05-25T06:11:37.152 INFO:tasks.workunit.client.0.smithi160.stdout:5/443: write db/dc/d13/d1d/d31/d3b/d69/f7e [29879,124283] 0 2024-05-25T06:11:37.153 INFO:tasks.workunit.client.1.smithi203.stdout:3/837: getdents d10/d16/da1/de6 0 2024-05-25T06:11:37.154 INFO:tasks.workunit.client.1.smithi203.stdout:1/628: dwrite d14/d57/d29/d94/db8/dc2/f90 [0,4194304] 0 2024-05-25T06:11:37.155 INFO:tasks.workunit.client.1.smithi203.stdout:1/629: chown d14/d57/d7e 73909137 1 2024-05-25T06:11:37.156 INFO:tasks.workunit.client.1.smithi203.stdout:4/865: rmdir d0/d8/d3a/d81 39 2024-05-25T06:11:37.157 INFO:tasks.workunit.client.0.smithi160.stdout:0/478: dread d2b/d31/d35/f3e [0,4194304] 0 2024-05-25T06:11:37.158 INFO:tasks.workunit.client.0.smithi160.stdout:2/381: unlink fb 0 2024-05-25T06:11:37.158 INFO:tasks.workunit.client.1.smithi203.stdout:7/711: rename d1/d9/d2c/d79/dc7/fd4 to d1/dbd/dc1/df3/ff4 0 2024-05-25T06:11:37.165 INFO:tasks.workunit.client.0.smithi160.stdout:0/479: write f2 [1260162,75471] 0 2024-05-25T06:11:37.166 INFO:tasks.workunit.client.0.smithi160.stdout:0/480: write d2b/d31/d35/d36/d86/f9b [278617,52] 0 2024-05-25T06:11:37.167 INFO:tasks.workunit.client.0.smithi160.stdout:9/373: write d4/f6e [4260341,116606] 0 2024-05-25T06:11:37.167 INFO:tasks.workunit.client.0.smithi160.stdout:5/444: fdatasync db/dc/d13/d1d/d31/d56/f7d 0 2024-05-25T06:11:37.169 INFO:tasks.workunit.client.0.smithi160.stdout:9/374: chown d4/d11/d12/f2b 109 1 2024-05-25T06:11:37.170 INFO:tasks.workunit.client.1.smithi203.stdout:7/712: getdents d1/d11/d3d/d4c 0 2024-05-25T06:11:37.172 INFO:tasks.workunit.client.0.smithi160.stdout:2/382: link d12/d3e/c64 d12/d1c/c82 0 2024-05-25T06:11:37.173 INFO:tasks.workunit.client.0.smithi160.stdout:0/481: write d2b/d58/f8c [2168639,48066] 0 2024-05-25T06:11:37.174 INFO:tasks.workunit.client.0.smithi160.stdout:5/445: write db/dc/d13/d1d/d23/f59 [1608357,116447] 0 2024-05-25T06:11:37.174 INFO:tasks.workunit.client.0.smithi160.stdout:9/375: unlink d4/d11/d1e/d3e/d25/f65 0 2024-05-25T06:11:37.175 INFO:tasks.workunit.client.1.smithi203.stdout:7/713: getdents d1/d9/d2c/d3a/dd1 0 2024-05-25T06:11:37.176 INFO:tasks.workunit.client.0.smithi160.stdout:0/482: chown d2b/d46 0 1 2024-05-25T06:11:37.176 INFO:tasks.workunit.client.0.smithi160.stdout:8/443: sync 2024-05-25T06:11:37.178 INFO:tasks.workunit.client.0.smithi160.stdout:4/411: dwrite d4/d1c/d54/f65 [0,4194304] 0 2024-05-25T06:11:37.180 INFO:tasks.workunit.client.0.smithi160.stdout:8/444: chown d5/d7/l10 5 1 2024-05-25T06:11:37.181 INFO:tasks.workunit.client.1.smithi203.stdout:3/838: write d10/d16/d23/d45/f4e [1138691,7139] 0 2024-05-25T06:11:37.182 INFO:tasks.workunit.client.0.smithi160.stdout:9/376: symlink d4/d11/d1e/d3e/d25/d27/l6f 0 2024-05-25T06:11:37.184 INFO:tasks.workunit.client.1.smithi203.stdout:4/866: write d0/d9/d50/d6b/f4c [979052,52238] 0 2024-05-25T06:11:37.185 INFO:tasks.workunit.client.0.smithi160.stdout:2/383: dwrite d12/d1c/f27 [0,4194304] 0 2024-05-25T06:11:37.187 INFO:tasks.workunit.client.1.smithi203.stdout:7/714: creat d1/d9/d2c/d36/ff5 x:0 0 0 2024-05-25T06:11:37.189 INFO:tasks.workunit.client.0.smithi160.stdout:8/445: mknod d5/d44/d1f/d1e/d28/d49/d61/c94 0 2024-05-25T06:11:37.190 INFO:tasks.workunit.client.1.smithi203.stdout:4/867: mkdir d0/d8/d1f/d26/d3b/d123 0 2024-05-25T06:11:37.191 INFO:tasks.workunit.client.1.smithi203.stdout:4/868: fsync d0/d9/d11/d54/ffd 0 2024-05-25T06:11:37.191 INFO:tasks.workunit.client.0.smithi160.stdout:5/446: dwrite db/dc/d13/d1d/d23/d45/f5f [4194304,4194304] 0 2024-05-25T06:11:37.192 INFO:tasks.workunit.client.0.smithi160.stdout:5/447: fsync db/d47/f68 0 2024-05-25T06:11:37.193 INFO:tasks.workunit.client.1.smithi203.stdout:1/630: dread d14/d57/db7/dd2/db3/f7a [0,4194304] 0 2024-05-25T06:11:37.193 INFO:tasks.workunit.client.1.smithi203.stdout:3/839: creat d10/d16/da1/de6/f10c x:0 0 0 2024-05-25T06:11:37.194 INFO:tasks.workunit.client.0.smithi160.stdout:9/377: mknod d4/d11/d1e/d3e/c70 0 2024-05-25T06:11:37.194 INFO:tasks.workunit.client.0.smithi160.stdout:8/446: dread - d5/d39/d7a/f85 zero size 2024-05-25T06:11:37.194 INFO:tasks.workunit.client.0.smithi160.stdout:5/448: readlink db/dc/d13/l18 0 2024-05-25T06:11:37.196 INFO:tasks.workunit.client.0.smithi160.stdout:8/447: chown d5/d44/d1f/d58/d65/c77 7261 1 2024-05-25T06:11:37.197 INFO:tasks.workunit.client.0.smithi160.stdout:5/449: chown db/dc/d13/d1d/d23/f46 26 1 2024-05-25T06:11:37.197 INFO:tasks.workunit.client.1.smithi203.stdout:1/631: symlink d14/d57/db7/ld5 0 2024-05-25T06:11:37.198 INFO:tasks.workunit.client.0.smithi160.stdout:0/483: truncate f1e 4085620 0 2024-05-25T06:11:37.199 INFO:tasks.workunit.client.1.smithi203.stdout:4/869: unlink d0/d9/d50/d6b/lc0 0 2024-05-25T06:11:37.200 INFO:tasks.workunit.client.1.smithi203.stdout:3/840: mkdir d10/d10d 0 2024-05-25T06:11:37.212 INFO:tasks.workunit.client.0.smithi160.stdout:1/303: sync 2024-05-25T06:11:37.212 INFO:tasks.workunit.client.0.smithi160.stdout:3/333: sync 2024-05-25T06:11:37.212 INFO:tasks.workunit.client.0.smithi160.stdout:6/389: sync 2024-05-25T06:11:37.212 INFO:tasks.workunit.client.0.smithi160.stdout:7/429: sync 2024-05-25T06:11:37.213 INFO:tasks.workunit.client.0.smithi160.stdout:7/430: write d6/d3e/d6f/f56 [4037929,105762] 0 2024-05-25T06:11:37.214 INFO:tasks.workunit.client.0.smithi160.stdout:7/431: stat d6/dc/d21/l48 0 2024-05-25T06:11:37.215 INFO:tasks.workunit.client.0.smithi160.stdout:7/432: write d6/dc/d21/f57 [488368,57099] 0 2024-05-25T06:11:37.218 INFO:tasks.workunit.client.0.smithi160.stdout:3/334: dwrite d19/d2f/d38/d47/f54 [0,4194304] 0 2024-05-25T06:11:37.219 INFO:tasks.workunit.client.0.smithi160.stdout:3/335: stat d19/d2f/d38/d60 0 2024-05-25T06:11:37.226 INFO:tasks.workunit.client.1.smithi203.stdout:2/760: sync 2024-05-25T06:11:37.226 INFO:tasks.workunit.client.1.smithi203.stdout:8/831: sync 2024-05-25T06:11:37.229 INFO:tasks.workunit.client.0.smithi160.stdout:8/448: creat d5/d44/d1f/d6b/f95 x:0 0 0 2024-05-25T06:11:37.230 INFO:tasks.workunit.client.0.smithi160.stdout:5/450: creat db/f85 x:0 0 0 2024-05-25T06:11:37.231 INFO:tasks.workunit.client.1.smithi203.stdout:4/870: truncate d0/d9/d11/f66 237575 0 2024-05-25T06:11:37.231 INFO:tasks.workunit.client.0.smithi160.stdout:9/378: symlink d4/d11/d1e/d6c/l71 0 2024-05-25T06:11:37.231 INFO:tasks.workunit.client.0.smithi160.stdout:8/449: truncate d5/d44/d1f/f55 832671 0 2024-05-25T06:11:37.231 INFO:tasks.workunit.client.0.smithi160.stdout:5/451: chown db/dc/d13/f44 2 1 2024-05-25T06:11:37.232 INFO:tasks.workunit.client.1.smithi203.stdout:1/632: mknod d14/d57/cd6 0 2024-05-25T06:11:37.232 INFO:tasks.workunit.client.1.smithi203.stdout:2/761: write d2/d38/d43/d60/f96 [5387691,125001] 0 2024-05-25T06:11:37.233 INFO:tasks.workunit.client.0.smithi160.stdout:8/450: write d5/f48 [1010210,93493] 0 2024-05-25T06:11:37.236 INFO:tasks.workunit.client.0.smithi160.stdout:0/484: symlink d2b/d31/d35/d36/d37/la1 0 2024-05-25T06:11:37.236 INFO:tasks.workunit.client.1.smithi203.stdout:2/762: read - d2/d1a/dd3/ff5 zero size 2024-05-25T06:11:37.238 INFO:tasks.workunit.client.0.smithi160.stdout:6/390: creat d5/d45/d63/f83 x:0 0 0 2024-05-25T06:11:37.239 INFO:tasks.workunit.client.1.smithi203.stdout:8/832: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/f11e x:0 0 0 2024-05-25T06:11:37.239 INFO:tasks.workunit.client.1.smithi203.stdout:3/841: truncate d10/d16/d23/d3a/dc6/fcc 294808 0 2024-05-25T06:11:37.240 INFO:tasks.workunit.client.1.smithi203.stdout:1/633: creat d14/d57/db7/dd2/db3/d5e/fd7 x:0 0 0 2024-05-25T06:11:37.240 INFO:tasks.workunit.client.0.smithi160.stdout:3/336: creat d19/d5d/f76 x:0 0 0 2024-05-25T06:11:37.245 INFO:tasks.workunit.client.1.smithi203.stdout:3/842: read d10/d22/d2a/f81 [61353,107130] 0 2024-05-25T06:11:37.245 INFO:tasks.workunit.client.0.smithi160.stdout:9/379: dread d4/d11/f4c [0,4194304] 0 2024-05-25T06:11:37.246 INFO:tasks.workunit.client.0.smithi160.stdout:1/304: write de/f15 [3172256,107722] 0 2024-05-25T06:11:37.248 INFO:tasks.workunit.client.0.smithi160.stdout:7/433: mkdir d6/d2b/d8b 0 2024-05-25T06:11:37.249 INFO:tasks.workunit.client.0.smithi160.stdout:7/434: stat d6/dc/d64/f53 0 2024-05-25T06:11:37.249 INFO:tasks.workunit.client.0.smithi160.stdout:6/391: stat d5/de/d32/d1b/d2b/c57 0 2024-05-25T06:11:37.249 INFO:tasks.workunit.client.1.smithi203.stdout:8/833: mkdir d2/d5/d10/d11/de/d11f 0 2024-05-25T06:11:37.250 INFO:tasks.workunit.client.1.smithi203.stdout:3/843: write d10/d16/da1/de6/f10c [939325,107653] 0 2024-05-25T06:11:37.251 INFO:tasks.workunit.client.1.smithi203.stdout:4/871: dwrite d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f11d [0,4194304] 0 2024-05-25T06:11:37.251 INFO:tasks.workunit.client.1.smithi203.stdout:1/634: rename d14/d57/d29/f9b to d14/d57/d29/d94/db8/fd8 0 2024-05-25T06:11:37.252 INFO:tasks.workunit.client.0.smithi160.stdout:9/380: creat d4/d30/f72 x:0 0 0 2024-05-25T06:11:37.254 INFO:tasks.workunit.client.0.smithi160.stdout:3/337: truncate d19/f3a 2031 0 2024-05-25T06:11:37.256 INFO:tasks.workunit.client.0.smithi160.stdout:5/452: getdents db/dc 0 2024-05-25T06:11:37.257 INFO:tasks.workunit.client.0.smithi160.stdout:7/435: write d6/d3e/d6f/f55 [996742,128805] 0 2024-05-25T06:11:37.258 INFO:tasks.workunit.client.1.smithi203.stdout:7/715: dwrite d1/d9/f14 [0,4194304] 0 2024-05-25T06:11:37.260 INFO:tasks.workunit.client.0.smithi160.stdout:6/392: unlink d5/de/d32/d1b/d2b/l79 0 2024-05-25T06:11:37.261 INFO:tasks.workunit.client.1.smithi203.stdout:3/844: creat d10/d22/d2a/d99/f10e x:0 0 0 2024-05-25T06:11:37.261 INFO:tasks.workunit.client.0.smithi160.stdout:5/453: write db/dc/d13/d1d/d31/d3b/f70 [1881623,102121] 0 2024-05-25T06:11:37.262 INFO:tasks.workunit.client.1.smithi203.stdout:8/834: mknod d2/d5/d18/d31/dac/c120 0 2024-05-25T06:11:37.263 INFO:tasks.workunit.client.0.smithi160.stdout:9/381: unlink d4/lb 0 2024-05-25T06:11:37.264 INFO:tasks.workunit.client.0.smithi160.stdout:8/451: getdents d5/d44/d1f/d1e/d28/d49/d61 0 2024-05-25T06:11:37.264 INFO:tasks.workunit.client.0.smithi160.stdout:1/305: dwrite de/d21/d49/f1d [4194304,4194304] 0 2024-05-25T06:11:37.266 INFO:tasks.workunit.client.1.smithi203.stdout:3/845: rmdir d10/dea 39 2024-05-25T06:11:37.268 INFO:tasks.workunit.client.0.smithi160.stdout:1/306: chown de/d21/d49/d1c/d59 0 1 2024-05-25T06:11:37.269 INFO:tasks.workunit.client.0.smithi160.stdout:3/338: mknod d19/d2f/c77 0 2024-05-25T06:11:37.269 INFO:tasks.workunit.client.1.smithi203.stdout:4/872: dwrite d0/d8/d52/d59/f120 [0,4194304] 0 2024-05-25T06:11:37.272 INFO:tasks.workunit.client.0.smithi160.stdout:7/436: mknod d6/dc/d64/d5a/d7c/c8c 0 2024-05-25T06:11:37.274 INFO:tasks.workunit.client.1.smithi203.stdout:2/763: truncate d2/d1a/f27 2969497 0 2024-05-25T06:11:37.275 INFO:tasks.workunit.client.0.smithi160.stdout:4/412: dread d4/d6/d35/f58 [0,4194304] 0 2024-05-25T06:11:37.275 INFO:tasks.workunit.client.0.smithi160.stdout:9/382: fdatasync d4/d11/d1e/d24/f1b 0 2024-05-25T06:11:37.279 INFO:tasks.workunit.client.0.smithi160.stdout:0/485: rmdir d2b/d31/d35/d36/d37 39 2024-05-25T06:11:37.279 INFO:tasks.workunit.client.0.smithi160.stdout:4/413: chown d4/d1c/f62 49173858 1 2024-05-25T06:11:37.279 INFO:tasks.workunit.client.0.smithi160.stdout:2/384: dread d12/d3e/d5f/f63 [0,4194304] 0 2024-05-25T06:11:37.282 INFO:tasks.workunit.client.0.smithi160.stdout:8/452: dwrite d5/d44/d1f/d58/f60 [4194304,4194304] 0 2024-05-25T06:11:37.284 INFO:tasks.workunit.client.0.smithi160.stdout:1/307: readlink de/d39/l44 0 2024-05-25T06:11:37.285 INFO:tasks.workunit.client.0.smithi160.stdout:6/393: creat d5/d30/d36/d73/d6e/f84 x:0 0 0 2024-05-25T06:11:37.287 INFO:tasks.workunit.client.0.smithi160.stdout:7/437: write d6/dc/d64/f3c [3769912,82866] 0 2024-05-25T06:11:37.289 INFO:tasks.workunit.client.0.smithi160.stdout:0/486: creat d2b/d46/d63/fa2 x:0 0 0 2024-05-25T06:11:37.293 INFO:tasks.workunit.client.1.smithi203.stdout:3/846: dwrite d10/d22/f86 [0,4194304] 0 2024-05-25T06:11:37.296 INFO:tasks.workunit.client.0.smithi160.stdout:4/414: rename d4/d6/d21/f24 to d4/d6/d21/d42/f7c 0 2024-05-25T06:11:37.296 INFO:tasks.workunit.client.0.smithi160.stdout:6/394: dwrite d5/de/f22 [0,4194304] 0 2024-05-25T06:11:37.296 INFO:tasks.workunit.client.0.smithi160.stdout:0/487: dread d2b/f78 [0,4194304] 0 2024-05-25T06:11:37.298 INFO:tasks.workunit.client.1.smithi203.stdout:4/873: dwrite d0/d9/d50/f10e [0,4194304] 0 2024-05-25T06:11:37.298 INFO:tasks.workunit.client.1.smithi203.stdout:2/764: dwrite d2/d4/f2e [0,4194304] 0 2024-05-25T06:11:37.299 INFO:tasks.workunit.client.0.smithi160.stdout:0/488: write d2b/d31/d35/d36/f7f [879400,2324] 0 2024-05-25T06:11:37.302 INFO:tasks.workunit.client.1.smithi203.stdout:1/635: dwrite d14/d57/d1d/f40 [0,4194304] 0 2024-05-25T06:11:37.305 INFO:tasks.workunit.client.1.smithi203.stdout:4/874: mkdir d0/d9/d9d/d124 0 2024-05-25T06:11:37.306 INFO:tasks.workunit.client.0.smithi160.stdout:7/438: creat d6/d2b/d5d/f8d x:0 0 0 2024-05-25T06:11:37.306 INFO:tasks.workunit.client.1.smithi203.stdout:7/716: dread d1/f4d [0,4194304] 0 2024-05-25T06:11:37.307 INFO:tasks.workunit.client.1.smithi203.stdout:3/847: fdatasync d10/d16/d23/d46/fd5 0 2024-05-25T06:11:37.313 INFO:tasks.workunit.client.0.smithi160.stdout:2/385: dread f10 [0,4194304] 0 2024-05-25T06:11:37.314 INFO:tasks.workunit.client.1.smithi203.stdout:2/765: rename d2/d4 to d2/d1a/dd3/d100 0 2024-05-25T06:11:37.316 INFO:tasks.workunit.client.1.smithi203.stdout:4/875: creat d0/d9/d11/f125 x:0 0 0 2024-05-25T06:11:37.317 INFO:tasks.workunit.client.0.smithi160.stdout:9/383: creat d4/d11/f73 x:0 0 0 2024-05-25T06:11:37.318 INFO:tasks.workunit.client.1.smithi203.stdout:4/876: write d0/d9/d50/fc5 [401717,38917] 0 2024-05-25T06:11:37.319 INFO:tasks.workunit.client.1.smithi203.stdout:3/848: symlink d10/d16/d50/d8b/l10f 0 2024-05-25T06:11:37.320 INFO:tasks.workunit.client.0.smithi160.stdout:0/489: chown d2b/d31/d35/d36/d37/l83 989242 1 2024-05-25T06:11:37.320 INFO:tasks.workunit.client.0.smithi160.stdout:4/415: dwrite d4/f3b [0,4194304] 0 2024-05-25T06:11:37.321 INFO:tasks.workunit.client.0.smithi160.stdout:3/339: creat d19/d2f/d38/f78 x:0 0 0 2024-05-25T06:11:37.323 INFO:tasks.workunit.client.0.smithi160.stdout:7/439: mknod d6/c8e 0 2024-05-25T06:11:37.323 INFO:tasks.workunit.client.0.smithi160.stdout:0/490: fdatasync d2b/d46/d63/fa2 0 2024-05-25T06:11:37.324 INFO:tasks.workunit.client.1.smithi203.stdout:4/877: creat d0/d8/d1f/d26/d3b/d9c/f126 x:0 0 0 2024-05-25T06:11:37.325 INFO:tasks.workunit.client.0.smithi160.stdout:6/395: rename d5/de/d58/f7f to d5/de/f85 0 2024-05-25T06:11:37.326 INFO:tasks.workunit.client.0.smithi160.stdout:2/386: mknod d12/d1c/d59/c83 0 2024-05-25T06:11:37.327 INFO:tasks.workunit.client.0.smithi160.stdout:8/453: getdents d5/d44/d1f/d58 0 2024-05-25T06:11:37.329 INFO:tasks.workunit.client.1.smithi203.stdout:4/878: fdatasync d0/f3 0 2024-05-25T06:11:37.329 INFO:tasks.workunit.client.1.smithi203.stdout:3/849: link d10/c49 d10/d16/d71/dfa/d107/c110 0 2024-05-25T06:11:37.331 INFO:tasks.workunit.client.0.smithi160.stdout:8/454: read - d5/d44/d1f/d1e/d28/d4a/d4e/d86/f8b zero size 2024-05-25T06:11:37.333 INFO:tasks.workunit.client.1.smithi203.stdout:4/879: creat d0/d8/d1f/d26/d3b/d9c/f127 x:0 0 0 2024-05-25T06:11:37.333 INFO:tasks.workunit.client.0.smithi160.stdout:0/491: dwrite d2b/f30 [0,4194304] 0 2024-05-25T06:11:37.333 INFO:tasks.workunit.client.0.smithi160.stdout:6/396: rename d5/d30/l71 to d5/de/d7d/l86 0 2024-05-25T06:11:37.335 INFO:tasks.workunit.client.1.smithi203.stdout:4/880: mkdir d0/d8/d52/d128 0 2024-05-25T06:11:37.336 INFO:tasks.workunit.client.0.smithi160.stdout:6/397: chown d5/d45/d4a/f53 108628848 1 2024-05-25T06:11:37.338 INFO:tasks.workunit.client.1.smithi203.stdout:4/881: mkdir d0/d8/d10/d91/da2/d129 0 2024-05-25T06:11:37.338 INFO:tasks.workunit.client.0.smithi160.stdout:9/384: creat d4/d11/d1e/d3e/f74 x:0 0 0 2024-05-25T06:11:37.340 INFO:tasks.workunit.client.0.smithi160.stdout:7/440: fsync d6/dc/f15 0 2024-05-25T06:11:37.341 INFO:tasks.workunit.client.0.smithi160.stdout:8/455: creat d5/d44/d1f/d1e/f96 x:0 0 0 2024-05-25T06:11:37.342 INFO:tasks.workunit.client.1.smithi203.stdout:4/882: dread d0/d9/d9d/fbf [0,4194304] 0 2024-05-25T06:11:37.344 INFO:tasks.workunit.client.0.smithi160.stdout:7/441: fdatasync d6/d3e/d6f/f6e 0 2024-05-25T06:11:37.345 INFO:tasks.workunit.client.0.smithi160.stdout:1/308: dread f9 [0,4194304] 0 2024-05-25T06:11:37.346 INFO:tasks.workunit.client.0.smithi160.stdout:0/492: dwrite d2b/d58/f85 [0,4194304] 0 2024-05-25T06:11:37.346 INFO:tasks.workunit.client.0.smithi160.stdout:9/385: chown d4/d11/d1e/d3e/d25/d27/l6f 53 1 2024-05-25T06:11:37.347 INFO:tasks.workunit.client.1.smithi203.stdout:4/883: dread d0/d8/d3a/f77 [0,4194304] 0 2024-05-25T06:11:37.347 INFO:tasks.workunit.client.0.smithi160.stdout:0/493: stat d2b/d46/f91 0 2024-05-25T06:11:37.348 INFO:tasks.workunit.client.0.smithi160.stdout:0/494: stat d2b/d58/d67 0 2024-05-25T06:11:37.351 INFO:tasks.workunit.client.1.smithi203.stdout:4/884: creat d0/d9/d50/d79/d9a/dbe/de0/f12a x:0 0 0 2024-05-25T06:11:37.354 INFO:tasks.workunit.client.1.smithi203.stdout:4/885: mkdir d0/d9/d50/d6b/d12b 0 2024-05-25T06:11:37.355 INFO:tasks.workunit.client.1.smithi203.stdout:4/886: read d0/d9/d9d/fbf [1491489,64883] 0 2024-05-25T06:11:37.356 INFO:tasks.workunit.client.0.smithi160.stdout:9/386: dwrite d4/d11/d1e/f1f [0,4194304] 0 2024-05-25T06:11:37.356 INFO:tasks.workunit.client.1.smithi203.stdout:4/887: dread - d0/d9/d11/f101 zero size 2024-05-25T06:11:37.356 INFO:tasks.workunit.client.1.smithi203.stdout:7/717: truncate d1/d9/f2e 3573802 0 2024-05-25T06:11:37.357 INFO:tasks.workunit.client.0.smithi160.stdout:1/309: dread de/d21/d49/f27 [0,4194304] 0 2024-05-25T06:11:37.358 INFO:tasks.workunit.client.1.smithi203.stdout:1/636: dread f10 [4194304,4194304] 0 2024-05-25T06:11:37.361 INFO:tasks.workunit.client.1.smithi203.stdout:4/888: write d0/d9/d9d/fbf [3912771,121104] 0 2024-05-25T06:11:37.363 INFO:tasks.workunit.client.0.smithi160.stdout:9/387: dwrite d4/d11/d12/f35 [0,4194304] 0 2024-05-25T06:11:37.365 INFO:tasks.workunit.client.1.smithi203.stdout:7/718: rename d1/d9/d2c/d3a/d4e/d50/fbe to d1/d9/d2c/d62/db3/ff6 0 2024-05-25T06:11:37.365 INFO:tasks.workunit.client.0.smithi160.stdout:9/388: write d4/d11/d1e/d24/f13 [30843,124943] 0 2024-05-25T06:11:37.367 INFO:tasks.workunit.client.1.smithi203.stdout:4/889: creat d0/d9/d50/d79/d9a/dbe/de0/f12c x:0 0 0 2024-05-25T06:11:37.368 INFO:tasks.workunit.client.1.smithi203.stdout:1/637: rmdir d14/d57/d6b/db9 0 2024-05-25T06:11:37.370 INFO:tasks.workunit.client.1.smithi203.stdout:4/890: write d0/d9/d11/f125 [769034,23571] 0 2024-05-25T06:11:37.370 INFO:tasks.workunit.client.1.smithi203.stdout:7/719: dread - d1/d11/d3d/fa6 zero size 2024-05-25T06:11:37.373 INFO:tasks.workunit.client.1.smithi203.stdout:3/850: dwrite d10/d16/d71/fc8 [0,4194304] 0 2024-05-25T06:11:37.375 INFO:tasks.workunit.client.1.smithi203.stdout:4/891: chown d0/d8/d3a/d81/d93/d98/d99/ded/f11c 0 1 2024-05-25T06:11:37.379 INFO:tasks.workunit.client.1.smithi203.stdout:3/851: symlink d10/d16/d23/d4a/d88/dca/l111 0 2024-05-25T06:11:37.381 INFO:tasks.workunit.client.1.smithi203.stdout:1/638: mkdir d14/d57/d97/da4/dad/dd9 0 2024-05-25T06:11:37.390 INFO:tasks.workunit.client.1.smithi203.stdout:4/892: rename d0/d8/d3a/c6e to d0/d8/d52/d59/c12d 0 2024-05-25T06:11:37.390 INFO:tasks.workunit.client.1.smithi203.stdout:7/720: write d1/d9/d2c/d3a/d4e/f61 [733424,46913] 0 2024-05-25T06:11:37.391 INFO:tasks.workunit.client.1.smithi203.stdout:1/639: link d14/f32 d14/d57/d1d/fda 0 2024-05-25T06:11:37.393 INFO:tasks.workunit.client.1.smithi203.stdout:1/640: write d14/d57/f58 [3056223,110801] 0 2024-05-25T06:11:37.397 INFO:tasks.workunit.client.1.smithi203.stdout:1/641: symlink d14/d81/ldb 0 2024-05-25T06:11:37.399 INFO:tasks.workunit.client.1.smithi203.stdout:4/893: rmdir d0/d9/d9d/d124 0 2024-05-25T06:11:37.402 INFO:tasks.workunit.client.1.smithi203.stdout:4/894: symlink d0/d8/d3a/db6/l12e 0 2024-05-25T06:11:37.402 INFO:tasks.workunit.client.1.smithi203.stdout:1/642: dwrite d14/f54 [0,4194304] 0 2024-05-25T06:11:37.402 INFO:tasks.workunit.client.1.smithi203.stdout:7/721: link d1/d9/d2c/d36/d54/cdf d1/d9/d60/cf7 0 2024-05-25T06:11:37.404 INFO:tasks.workunit.client.1.smithi203.stdout:4/895: chown d0/d9/c36 23192 1 2024-05-25T06:11:37.404 INFO:tasks.workunit.client.0.smithi160.stdout:5/454: sync 2024-05-25T06:11:37.405 INFO:tasks.workunit.client.1.smithi203.stdout:3/852: dread d10/d16/d23/d3a/f53 [0,4194304] 0 2024-05-25T06:11:37.406 INFO:tasks.workunit.client.1.smithi203.stdout:7/722: write d1/d9/d2c/d36/ff5 [513631,17455] 0 2024-05-25T06:11:37.411 INFO:tasks.workunit.client.1.smithi203.stdout:4/896: dread d0/d8/d52/d59/f120 [0,4194304] 0 2024-05-25T06:11:37.412 INFO:tasks.workunit.client.0.smithi160.stdout:5/455: dwrite db/dc/d13/d1d/d23/d45/f5f [0,4194304] 0 2024-05-25T06:11:37.412 INFO:tasks.workunit.client.1.smithi203.stdout:7/723: chown d1/d9/d2c/d62/c7d 548 1 2024-05-25T06:11:37.421 INFO:tasks.workunit.client.0.smithi160.stdout:4/416: link d4/d6/l3a d4/d1c/l7d 0 2024-05-25T06:11:37.421 INFO:tasks.workunit.client.1.smithi203.stdout:1/643: write d14/d57/d29/d94/db8/dc2/f9a [412735,19058] 0 2024-05-25T06:11:37.422 INFO:tasks.workunit.client.0.smithi160.stdout:4/417: fdatasync d4/d6/d28/f4e 0 2024-05-25T06:11:37.423 INFO:tasks.workunit.client.0.smithi160.stdout:3/340: rename d19/l74 to d19/l79 0 2024-05-25T06:11:37.426 INFO:tasks.workunit.client.0.smithi160.stdout:6/398: readlink d5/de/d32/d1b/l46 0 2024-05-25T06:11:37.429 INFO:tasks.workunit.client.0.smithi160.stdout:8/456: creat d5/d44/d1f/d1e/d28/d49/f97 x:0 0 0 2024-05-25T06:11:37.434 INFO:tasks.workunit.client.0.smithi160.stdout:1/310: mknod de/d21/d49/d35/c5f 0 2024-05-25T06:11:37.436 INFO:tasks.workunit.client.0.smithi160.stdout:9/389: chown d4/d11/d1e/d3e/d25/d27/c2d 96195113 1 2024-05-25T06:11:37.438 INFO:tasks.workunit.client.0.smithi160.stdout:9/390: read d4/d11/d1e/d24/fc [1499034,26693] 0 2024-05-25T06:11:37.438 INFO:tasks.workunit.client.0.smithi160.stdout:5/456: truncate db/dc/d13/d1d/d23/f33 5180602 0 2024-05-25T06:11:37.439 INFO:tasks.workunit.client.0.smithi160.stdout:4/418: mkdir d4/d63/d7e 0 2024-05-25T06:11:37.439 INFO:tasks.workunit.client.0.smithi160.stdout:5/457: read - db/dc/d13/f6a zero size 2024-05-25T06:11:37.441 INFO:tasks.workunit.client.0.smithi160.stdout:6/399: stat d5/de/d32/d1b/c4b 0 2024-05-25T06:11:37.443 INFO:tasks.workunit.client.0.smithi160.stdout:8/457: rename d5/d44/d1f/f42 to d5/d44/d1f/d1e/d28/d49/f98 0 2024-05-25T06:11:37.446 INFO:tasks.workunit.client.0.smithi160.stdout:5/458: truncate f9 2555758 0 2024-05-25T06:11:37.448 INFO:tasks.workunit.client.0.smithi160.stdout:7/442: creat d6/dc/d64/d5a/f8f x:0 0 0 2024-05-25T06:11:37.449 INFO:tasks.workunit.client.1.smithi203.stdout:7/724: write d1/d9/d2c/d36/f40 [873415,35519] 0 2024-05-25T06:11:37.450 INFO:tasks.workunit.client.1.smithi203.stdout:4/897: symlink d0/d8/d52/d59/d7a/l12f 0 2024-05-25T06:11:37.450 INFO:tasks.workunit.client.0.smithi160.stdout:3/341: rename d19/d2f/d34/l59 to d19/d2f/d38/d43/d5b/d65/l7a 0 2024-05-25T06:11:37.451 INFO:tasks.workunit.client.0.smithi160.stdout:2/387: truncate d12/f25 3259809 0 2024-05-25T06:11:37.451 INFO:tasks.workunit.client.1.smithi203.stdout:1/644: mkdir d14/d57/db7/dd2/ddc 0 2024-05-25T06:11:37.451 INFO:tasks.workunit.client.1.smithi203.stdout:3/853: link d10/d16/d50/d8b/f85 d10/d16/d23/d46/dd3/f112 0 2024-05-25T06:11:37.452 INFO:tasks.workunit.client.0.smithi160.stdout:8/458: creat d5/d39/d7a/f99 x:0 0 0 2024-05-25T06:11:37.452 INFO:tasks.workunit.client.0.smithi160.stdout:1/311: dwrite de/f13 [0,4194304] 0 2024-05-25T06:11:37.454 INFO:tasks.workunit.client.1.smithi203.stdout:4/898: unlink d0/d8/d3a/f53 0 2024-05-25T06:11:37.455 INFO:tasks.workunit.client.0.smithi160.stdout:0/495: getdents d2b/d58/d67 0 2024-05-25T06:11:37.456 INFO:tasks.workunit.client.1.smithi203.stdout:3/854: creat d10/d16/da1/de6/f113 x:0 0 0 2024-05-25T06:11:37.456 INFO:tasks.workunit.client.0.smithi160.stdout:1/312: write de/d21/d49/d35/f3b [919830,100101] 0 2024-05-25T06:11:37.457 INFO:tasks.workunit.client.1.smithi203.stdout:4/899: creat d0/d8/d10/d91/da2/f130 x:0 0 0 2024-05-25T06:11:37.458 INFO:tasks.workunit.client.0.smithi160.stdout:6/400: dread d5/f26 [4194304,4194304] 0 2024-05-25T06:11:37.458 INFO:tasks.workunit.client.0.smithi160.stdout:7/443: unlink d6/dc/d21/c60 0 2024-05-25T06:11:37.459 INFO:tasks.workunit.client.0.smithi160.stdout:6/401: stat d5/de/d32/d35/d40/l62 0 2024-05-25T06:11:37.459 INFO:tasks.workunit.client.0.smithi160.stdout:9/391: creat d4/f75 x:0 0 0 2024-05-25T06:11:37.461 INFO:tasks.workunit.client.0.smithi160.stdout:6/402: write d5/de/d32/d35/f82 [972071,79677] 0 2024-05-25T06:11:37.462 INFO:tasks.workunit.client.0.smithi160.stdout:1/313: dread de/d43/f46 [0,4194304] 0 2024-05-25T06:11:37.463 INFO:tasks.workunit.client.0.smithi160.stdout:1/314: readlink de/d21/l55 0 2024-05-25T06:11:37.466 INFO:tasks.workunit.client.0.smithi160.stdout:0/496: rmdir d2b 39 2024-05-25T06:11:37.469 INFO:tasks.workunit.client.0.smithi160.stdout:5/459: rename db/dc/d13/d1d/d23/d45/c74 to db/dc/d13/d1d/d23/d24/d2e/c86 0 2024-05-25T06:11:37.472 INFO:tasks.workunit.client.0.smithi160.stdout:4/419: truncate d4/d1a/f50 243437 0 2024-05-25T06:11:37.473 INFO:tasks.workunit.client.0.smithi160.stdout:3/342: mknod d19/d2f/d34/d67/c7b 0 2024-05-25T06:11:37.474 INFO:tasks.workunit.client.0.smithi160.stdout:8/459: mkdir d5/d7/d9a 0 2024-05-25T06:11:37.477 INFO:tasks.workunit.client.0.smithi160.stdout:6/403: mkdir d5/d30/d36/d73/d87 0 2024-05-25T06:11:37.477 INFO:tasks.workunit.client.0.smithi160.stdout:1/315: rmdir de/d21/d49/d35 39 2024-05-25T06:11:37.479 INFO:tasks.workunit.client.0.smithi160.stdout:5/460: stat db/d47/d5e 0 2024-05-25T06:11:37.480 INFO:tasks.workunit.client.0.smithi160.stdout:6/404: chown d5/d30/d36/d73/d6e 9963 1 2024-05-25T06:11:37.480 INFO:tasks.workunit.client.0.smithi160.stdout:4/420: dread d4/d1a/f59 [0,4194304] 0 2024-05-25T06:11:37.481 INFO:tasks.workunit.client.0.smithi160.stdout:3/343: rmdir d19/d2f/d38/d43/d5b/d65 39 2024-05-25T06:11:37.483 INFO:tasks.workunit.client.0.smithi160.stdout:7/444: write d6/dc/d64/f53 [157445,86661] 0 2024-05-25T06:11:37.484 INFO:tasks.workunit.client.0.smithi160.stdout:8/460: dwrite d5/d39/d7a/f85 [0,4194304] 0 2024-05-25T06:11:37.485 INFO:tasks.workunit.client.0.smithi160.stdout:1/316: rmdir de/d21/d49/d1c/d52 39 2024-05-25T06:11:37.489 INFO:tasks.workunit.client.0.smithi160.stdout:8/461: dread - d5/d44/d1f/d1e/d28/d4a/d73/f91 zero size 2024-05-25T06:11:37.490 INFO:tasks.workunit.client.0.smithi160.stdout:3/344: mknod d19/d2f/d38/c7c 0 2024-05-25T06:11:37.491 INFO:tasks.workunit.client.0.smithi160.stdout:2/388: truncate d12/d13/f15 14602 0 2024-05-25T06:11:37.493 INFO:tasks.workunit.client.0.smithi160.stdout:6/405: dwrite d5/d39/f41 [0,4194304] 0 2024-05-25T06:11:37.493 INFO:tasks.workunit.client.0.smithi160.stdout:4/421: rename d4/d1c/c2f to d4/d6/d5b/c7f 0 2024-05-25T06:11:37.494 INFO:tasks.workunit.client.0.smithi160.stdout:2/389: write d12/d1c/d37/d4b/f5e [432043,40118] 0 2024-05-25T06:11:37.498 INFO:tasks.workunit.client.1.smithi203.stdout:1/645: dwrite d14/f1a [0,4194304] 0 2024-05-25T06:11:37.498 INFO:tasks.workunit.client.1.smithi203.stdout:4/900: chown d0/d8/d10/d91/ld2 7109188 1 2024-05-25T06:11:37.498 INFO:tasks.workunit.client.0.smithi160.stdout:9/392: dread d4/d11/f26 [0,4194304] 0 2024-05-25T06:11:37.500 INFO:tasks.workunit.client.0.smithi160.stdout:0/497: creat d2b/d31/d35/d36/fa3 x:0 0 0 2024-05-25T06:11:37.501 INFO:tasks.workunit.client.1.smithi203.stdout:4/901: chown d0/d8/d52/d128 4034745 1 2024-05-25T06:11:37.502 INFO:tasks.workunit.client.1.smithi203.stdout:4/902: dread - d0/d9/d11/f101 zero size 2024-05-25T06:11:37.503 INFO:tasks.workunit.client.1.smithi203.stdout:3/855: dread d10/d16/d23/d4a/d88/fdf [0,4194304] 0 2024-05-25T06:11:37.507 INFO:tasks.workunit.client.1.smithi203.stdout:4/903: creat d0/f131 x:0 0 0 2024-05-25T06:11:37.508 INFO:tasks.workunit.client.0.smithi160.stdout:8/462: mkdir d5/d44/d1f/d6b/d9b 0 2024-05-25T06:11:37.508 INFO:tasks.workunit.client.0.smithi160.stdout:4/422: rmdir d4/d6/d21 39 2024-05-25T06:11:37.508 INFO:tasks.workunit.client.0.smithi160.stdout:8/463: readlink d5/d44/d1f/d6b/l7f 0 2024-05-25T06:11:37.509 INFO:tasks.workunit.client.0.smithi160.stdout:6/406: creat d5/de/d32/d1b/d2b/f88 x:0 0 0 2024-05-25T06:11:37.510 INFO:tasks.workunit.client.0.smithi160.stdout:4/423: truncate d4/d6/f22 1270208 0 2024-05-25T06:11:37.510 INFO:tasks.workunit.client.0.smithi160.stdout:8/464: chown d5/d39/d7a/f92 1432253935 1 2024-05-25T06:11:37.510 INFO:tasks.workunit.client.0.smithi160.stdout:2/390: mkdir d12/d3e/d84 0 2024-05-25T06:11:37.510 INFO:tasks.workunit.client.1.smithi203.stdout:4/904: mkdir d0/d9/d11/d54/d132 0 2024-05-25T06:11:37.513 INFO:tasks.workunit.client.0.smithi160.stdout:1/317: mknod de/d21/d49/c60 0 2024-05-25T06:11:37.514 INFO:tasks.workunit.client.0.smithi160.stdout:9/393: symlink d4/d11/d14/l76 0 2024-05-25T06:11:37.514 INFO:tasks.workunit.client.0.smithi160.stdout:0/498: mknod d2b/d31/d6d/ca4 0 2024-05-25T06:11:37.516 INFO:tasks.workunit.client.0.smithi160.stdout:5/461: dwrite db/dc/d13/d1d/d23/d24/d2e/f32 [4194304,4194304] 0 2024-05-25T06:11:37.517 INFO:tasks.workunit.client.0.smithi160.stdout:6/407: stat d5/c38 0 2024-05-25T06:11:37.517 INFO:tasks.workunit.client.0.smithi160.stdout:9/394: write d4/d11/d12/d58/f64 [287080,100131] 0 2024-05-25T06:11:37.517 INFO:tasks.workunit.client.1.smithi203.stdout:4/905: dwrite d0/d9/d50/d79/d9a/dbe/de0/f12c [0,4194304] 0 2024-05-25T06:11:37.518 INFO:tasks.workunit.client.0.smithi160.stdout:8/465: mkdir d5/d9c 0 2024-05-25T06:11:37.518 INFO:tasks.workunit.client.0.smithi160.stdout:2/391: mkdir d12/d5c/d85 0 2024-05-25T06:11:37.522 INFO:tasks.workunit.client.0.smithi160.stdout:4/424: creat d4/d63/d64/f80 x:0 0 0 2024-05-25T06:11:37.523 INFO:tasks.workunit.client.1.smithi203.stdout:3/856: truncate d10/f51 1585608 0 2024-05-25T06:11:37.523 INFO:tasks.workunit.client.1.smithi203.stdout:4/906: read d0/d8/d52/f6f [703503,2971] 0 2024-05-25T06:11:37.523 INFO:tasks.workunit.client.0.smithi160.stdout:7/445: rmdir d6/dc/d64/d5a/d7c/d81 0 2024-05-25T06:11:37.524 INFO:tasks.workunit.client.0.smithi160.stdout:7/446: chown d6/dc/d64/f37 3496 1 2024-05-25T06:11:37.525 INFO:tasks.workunit.client.1.smithi203.stdout:3/857: rename d10/d16/d23 to d10/d16/d23/d4a/d88/dca/dda/d114 22 2024-05-25T06:11:37.525 INFO:tasks.workunit.client.0.smithi160.stdout:1/318: write f9 [3872069,95309] 0 2024-05-25T06:11:37.525 INFO:tasks.workunit.client.0.smithi160.stdout:7/447: stat d6/dc/d21/f4f 0 2024-05-25T06:11:37.526 INFO:tasks.workunit.client.0.smithi160.stdout:1/319: chown fb 147870 1 2024-05-25T06:11:37.527 INFO:tasks.workunit.client.0.smithi160.stdout:1/320: read de/f13 [4136086,21071] 0 2024-05-25T06:11:37.528 INFO:tasks.workunit.client.0.smithi160.stdout:1/321: fdatasync de/f3a 0 2024-05-25T06:11:37.529 INFO:tasks.workunit.client.1.smithi203.stdout:3/858: mknod d10/d22/d2a/c115 0 2024-05-25T06:11:37.530 INFO:tasks.workunit.client.0.smithi160.stdout:0/499: creat d2b/d58/d67/fa5 x:0 0 0 2024-05-25T06:11:37.531 INFO:tasks.workunit.client.0.smithi160.stdout:5/462: creat db/dc/d13/f87 x:0 0 0 2024-05-25T06:11:37.531 INFO:tasks.workunit.client.1.smithi203.stdout:3/859: truncate d10/d16/d23/d4a/d88/dca/f101 283993 0 2024-05-25T06:11:37.532 INFO:tasks.workunit.client.0.smithi160.stdout:1/322: dread de/d21/d49/d1c/d59/f5d [0,4194304] 0 2024-05-25T06:11:37.532 INFO:tasks.workunit.client.0.smithi160.stdout:0/500: fsync d2b/d31/d35/d36/d86/f5b 0 2024-05-25T06:11:37.533 INFO:tasks.workunit.client.0.smithi160.stdout:4/425: creat d4/d63/f81 x:0 0 0 2024-05-25T06:11:37.534 INFO:tasks.workunit.client.0.smithi160.stdout:3/345: getdents d19/d2f/d38/d43/d5b 0 2024-05-25T06:11:37.534 INFO:tasks.workunit.client.0.smithi160.stdout:7/448: creat d6/d2b/d5d/d89/f90 x:0 0 0 2024-05-25T06:11:37.534 INFO:tasks.workunit.client.0.smithi160.stdout:4/426: chown d4/d6/d5b/d66/f67 1420 1 2024-05-25T06:11:37.536 INFO:tasks.workunit.client.0.smithi160.stdout:2/392: mkdir d12/d86 0 2024-05-25T06:11:37.537 INFO:tasks.workunit.client.0.smithi160.stdout:2/393: chown d12/d13/c18 229604901 1 2024-05-25T06:11:37.537 INFO:tasks.workunit.client.0.smithi160.stdout:6/408: mkdir d5/de/d32/d74/d77/d89 0 2024-05-25T06:11:37.543 INFO:tasks.workunit.client.0.smithi160.stdout:3/346: creat d19/d2f/d38/d47/f7d x:0 0 0 2024-05-25T06:11:37.544 INFO:tasks.workunit.client.1.smithi203.stdout:3/860: dwrite d10/d16/d23/d4a/d88/dca/ff6 [0,4194304] 0 2024-05-25T06:11:37.544 INFO:tasks.workunit.client.0.smithi160.stdout:7/449: mknod d6/d3e/d59/c91 0 2024-05-25T06:11:37.545 INFO:tasks.workunit.client.1.smithi203.stdout:3/861: chown d10/d16/d23/d6c/f102 4570 1 2024-05-25T06:11:37.546 INFO:tasks.workunit.client.0.smithi160.stdout:3/347: write d19/d22/f2d [336624,109667] 0 2024-05-25T06:11:37.547 INFO:tasks.workunit.client.0.smithi160.stdout:6/409: dwrite d5/d39/f51 [0,4194304] 0 2024-05-25T06:11:37.548 INFO:tasks.workunit.client.1.smithi203.stdout:3/862: creat d10/d16/da1/de6/f116 x:0 0 0 2024-05-25T06:11:37.550 INFO:tasks.workunit.client.0.smithi160.stdout:9/395: dread d4/d11/f2c [0,4194304] 0 2024-05-25T06:11:37.550 INFO:tasks.workunit.client.1.smithi203.stdout:3/863: rmdir d10/d16/d23/d45 39 2024-05-25T06:11:37.552 INFO:tasks.workunit.client.1.smithi203.stdout:3/864: dread - d10/d16/d23/d38/f89 zero size 2024-05-25T06:11:37.556 INFO:tasks.workunit.client.1.smithi203.stdout:3/865: symlink d10/d22/d2a/l117 0 2024-05-25T06:11:37.557 INFO:tasks.workunit.client.0.smithi160.stdout:7/450: write d6/dc/d64/f43 [350094,5827] 0 2024-05-25T06:11:37.559 INFO:tasks.workunit.client.0.smithi160.stdout:3/348: mkdir d19/d2f/d38/d43/d5b/d65/d7e 0 2024-05-25T06:11:37.560 INFO:tasks.workunit.client.0.smithi160.stdout:7/451: chown d6/d3e/d59/c91 847578 1 2024-05-25T06:11:37.561 INFO:tasks.workunit.client.0.smithi160.stdout:9/396: rename d4/f4e to d4/d11/d1e/d3e/d25/d27/f77 0 2024-05-25T06:11:37.561 INFO:tasks.workunit.client.0.smithi160.stdout:0/501: dread f20 [0,4194304] 0 2024-05-25T06:11:37.561 INFO:tasks.workunit.client.1.smithi203.stdout:3/866: rename d10/d16/d23/d3a/ff8 to d10/d16/d23/def/f118 0 2024-05-25T06:11:37.561 INFO:tasks.workunit.client.0.smithi160.stdout:7/452: chown d6/dc/f15 48252024 1 2024-05-25T06:11:37.562 INFO:tasks.workunit.client.0.smithi160.stdout:5/463: link db/dc/d13/c1e db/dc/c88 0 2024-05-25T06:11:37.564 INFO:tasks.workunit.client.1.smithi203.stdout:3/867: creat d10/d16/d23/d67/d105/d109/f119 x:0 0 0 2024-05-25T06:11:37.564 INFO:tasks.workunit.client.0.smithi160.stdout:8/466: write d5/d44/d1f/d1e/d28/f5e [716839,15585] 0 2024-05-25T06:11:37.564 INFO:tasks.workunit.client.0.smithi160.stdout:7/453: read d6/dc/d21/f57 [1111553,17935] 0 2024-05-25T06:11:37.565 INFO:tasks.workunit.client.0.smithi160.stdout:6/410: creat d5/de/d32/d74/d77/d89/f8a x:0 0 0 2024-05-25T06:11:37.566 INFO:tasks.workunit.client.0.smithi160.stdout:3/349: mkdir d19/d2f/d38/d43/d5b/d65/d7f 0 2024-05-25T06:11:37.567 INFO:tasks.workunit.client.1.smithi203.stdout:3/868: mknod d10/d16/d23/d4a/d88/dca/c11a 0 2024-05-25T06:11:37.567 INFO:tasks.workunit.client.0.smithi160.stdout:2/394: rmdir d12 39 2024-05-25T06:11:37.568 INFO:tasks.workunit.client.0.smithi160.stdout:1/323: getdents de 0 2024-05-25T06:11:37.569 INFO:tasks.workunit.client.1.smithi203.stdout:3/869: creat d10/d16/d23/d4a/d88/deb/f11b x:0 0 0 2024-05-25T06:11:37.569 INFO:tasks.workunit.client.0.smithi160.stdout:7/454: truncate d6/dc/f82 810278 0 2024-05-25T06:11:37.569 INFO:tasks.workunit.client.0.smithi160.stdout:0/502: mkdir d2b/d58/da6 0 2024-05-25T06:11:37.570 INFO:tasks.workunit.client.0.smithi160.stdout:9/397: dwrite d4/f69 [0,4194304] 0 2024-05-25T06:11:37.572 INFO:tasks.workunit.client.0.smithi160.stdout:0/503: read d2b/d31/d35/f55 [260803,38373] 0 2024-05-25T06:11:37.572 INFO:tasks.workunit.client.0.smithi160.stdout:7/455: truncate d6/d2b/d5d/f8d 701330 0 2024-05-25T06:11:37.573 INFO:tasks.workunit.client.0.smithi160.stdout:0/504: dread - d2b/d46/f91 zero size 2024-05-25T06:11:37.574 INFO:tasks.workunit.client.0.smithi160.stdout:7/456: truncate d6/d2b/d5d/f8d 1036261 0 2024-05-25T06:11:37.576 INFO:tasks.workunit.client.0.smithi160.stdout:6/411: mknod d5/d45/d4a/c8b 0 2024-05-25T06:11:37.579 INFO:tasks.workunit.client.0.smithi160.stdout:3/350: fsync d19/d2f/d38/f49 0 2024-05-25T06:11:37.580 INFO:tasks.workunit.client.0.smithi160.stdout:6/412: read d5/d45/d4a/f53 [2837032,65700] 0 2024-05-25T06:11:37.580 INFO:tasks.workunit.client.0.smithi160.stdout:4/427: truncate d4/f11 7164791 0 2024-05-25T06:11:37.581 INFO:tasks.workunit.client.0.smithi160.stdout:4/428: write d4/d63/d64/f80 [224041,115827] 0 2024-05-25T06:11:37.585 INFO:tasks.workunit.client.0.smithi160.stdout:9/398: unlink d4/f41 0 2024-05-25T06:11:37.585 INFO:tasks.workunit.client.0.smithi160.stdout:3/351: dread d19/f45 [0,4194304] 0 2024-05-25T06:11:37.586 INFO:tasks.workunit.client.0.smithi160.stdout:8/467: symlink d5/d39/l9d 0 2024-05-25T06:11:37.587 INFO:tasks.workunit.client.1.smithi203.stdout:3/870: dread d10/d16/d23/d45/f76 [0,4194304] 0 2024-05-25T06:11:37.589 INFO:tasks.workunit.client.1.smithi203.stdout:3/871: read d10/d16/d23/d3a/f5d [3602563,95658] 0 2024-05-25T06:11:37.591 INFO:tasks.workunit.client.1.smithi203.stdout:3/872: truncate d10/d22/d2a/f4d 2651495 0 2024-05-25T06:11:37.591 INFO:tasks.workunit.client.0.smithi160.stdout:8/468: dwrite d5/d44/d1f/d1e/d28/f3f [4194304,4194304] 0 2024-05-25T06:11:37.593 INFO:tasks.workunit.client.1.smithi203.stdout:3/873: mknod d10/d16/d23/d67/d105/c11c 0 2024-05-25T06:11:37.595 INFO:tasks.workunit.client.1.smithi203.stdout:3/874: symlink d10/l11d 0 2024-05-25T06:11:37.597 INFO:tasks.workunit.client.1.smithi203.stdout:3/875: mkdir d10/d16/d23/d38/d11e 0 2024-05-25T06:11:37.600 INFO:tasks.workunit.client.1.smithi203.stdout:3/876: symlink d10/dea/l11f 0 2024-05-25T06:11:37.620 INFO:tasks.workunit.client.1.smithi203.stdout:0/656: sync 2024-05-25T06:11:37.621 INFO:tasks.workunit.client.1.smithi203.stdout:9/650: sync 2024-05-25T06:11:37.621 INFO:tasks.workunit.client.1.smithi203.stdout:9/651: stat d0/d74 0 2024-05-25T06:11:37.622 INFO:tasks.workunit.client.1.smithi203.stdout:9/652: chown d0/df/d2b 1405 1 2024-05-25T06:11:37.624 INFO:tasks.workunit.client.1.smithi203.stdout:9/653: truncate d0/dd0/fa0 924043 0 2024-05-25T06:11:37.626 INFO:tasks.workunit.client.0.smithi160.stdout:0/505: rename d2b/d58 to d2b/d46/d63/da7 0 2024-05-25T06:11:37.626 INFO:tasks.workunit.client.1.smithi203.stdout:0/657: rename dd/d10/d34/d4c/c80 to dd/d3d/d4f/ced 0 2024-05-25T06:11:37.626 INFO:tasks.workunit.client.0.smithi160.stdout:0/506: fdatasync d2b/d46/f5a 0 2024-05-25T06:11:37.627 INFO:tasks.workunit.client.0.smithi160.stdout:0/507: chown d2b/d31/d35/d36/d86 2497 1 2024-05-25T06:11:37.628 INFO:tasks.workunit.client.1.smithi203.stdout:9/654: mkdir d0/d57/de0 0 2024-05-25T06:11:37.628 INFO:tasks.workunit.client.1.smithi203.stdout:0/658: write dd/d45/d96/fcd [736173,5798] 0 2024-05-25T06:11:37.628 INFO:tasks.workunit.client.0.smithi160.stdout:0/508: dread - d2b/d31/f7d zero size 2024-05-25T06:11:37.630 INFO:tasks.workunit.client.0.smithi160.stdout:6/413: symlink d5/d30/d36/d73/d6e/l8c 0 2024-05-25T06:11:37.630 INFO:tasks.workunit.client.1.smithi203.stdout:9/655: chown d0/d17/d20/d29 16726231 1 2024-05-25T06:11:37.631 INFO:tasks.workunit.client.1.smithi203.stdout:0/659: fdatasync dd/d3a/f7f 0 2024-05-25T06:11:37.631 INFO:tasks.workunit.client.0.smithi160.stdout:4/429: rmdir d4/d63 39 2024-05-25T06:11:37.632 INFO:tasks.workunit.client.1.smithi203.stdout:0/660: chown dd/d10/d34/d4c/l9e 3 1 2024-05-25T06:11:37.633 INFO:tasks.workunit.client.1.smithi203.stdout:9/656: getdents d0/df/d37/d4f/d93/ddc 0 2024-05-25T06:11:37.633 INFO:tasks.workunit.client.1.smithi203.stdout:0/661: write dd/d3d/da0/fc4 [585862,123177] 0 2024-05-25T06:11:37.633 INFO:tasks.workunit.client.1.smithi203.stdout:9/657: chown d0/d57 834 1 2024-05-25T06:11:37.638 INFO:tasks.workunit.client.1.smithi203.stdout:0/662: rename dd/d10/d34/f58 to dd/d3d/d59/fee 0 2024-05-25T06:11:37.638 INFO:tasks.workunit.client.0.smithi160.stdout:9/399: truncate d4/d11/d14/f3a 1593376 0 2024-05-25T06:11:37.643 INFO:tasks.workunit.client.1.smithi203.stdout:9/658: dwrite d0/d17/d20/d3c/db2/f84 [0,4194304] 0 2024-05-25T06:11:37.644 INFO:tasks.workunit.client.1.smithi203.stdout:3/877: dread d10/d16/d23/d3a/f80 [0,4194304] 0 2024-05-25T06:11:37.644 INFO:tasks.workunit.client.0.smithi160.stdout:5/464: symlink db/dc/d13/d1d/d23/d24/l89 0 2024-05-25T06:11:37.647 INFO:tasks.workunit.client.1.smithi203.stdout:9/659: symlink d0/df/d2b/le1 0 2024-05-25T06:11:37.649 INFO:tasks.workunit.client.0.smithi160.stdout:1/324: rename de/d21/d49/d35/c5f to de/d21/c61 0 2024-05-25T06:11:37.650 INFO:tasks.workunit.client.0.smithi160.stdout:7/457: mkdir d6/d32/d92 0 2024-05-25T06:11:37.651 INFO:tasks.workunit.client.0.smithi160.stdout:8/469: dwrite d5/d44/d1f/d1e/d28/d49/f98 [0,4194304] 0 2024-05-25T06:11:37.653 INFO:tasks.workunit.client.1.smithi203.stdout:3/878: dwrite d10/d22/f86 [4194304,4194304] 0 2024-05-25T06:11:37.654 INFO:tasks.workunit.client.0.smithi160.stdout:2/395: unlink d12/l4d 0 2024-05-25T06:11:37.654 INFO:tasks.workunit.client.0.smithi160.stdout:2/396: readlink d12/d1c/l23 0 2024-05-25T06:11:37.655 INFO:tasks.workunit.client.1.smithi203.stdout:0/663: dwrite dd/d3d/d4f/f50 [0,4194304] 0 2024-05-25T06:11:37.659 INFO:tasks.workunit.client.0.smithi160.stdout:6/414: unlink l4 0 2024-05-25T06:11:37.663 INFO:tasks.workunit.client.1.smithi203.stdout:9/660: dwrite d0/f51 [0,4194304] 0 2024-05-25T06:11:37.663 INFO:tasks.workunit.client.1.smithi203.stdout:3/879: truncate d10/d16/d23/d46/fbb 1446944 0 2024-05-25T06:11:37.667 INFO:tasks.workunit.client.0.smithi160.stdout:3/352: mkdir d19/d2f/d38/d43/d5b/d65/d7f/d80 0 2024-05-25T06:11:37.668 INFO:tasks.workunit.client.1.smithi203.stdout:9/661: chown d0/d17/d20/d3c/db2/cc7 56398702 1 2024-05-25T06:11:37.669 INFO:tasks.workunit.client.0.smithi160.stdout:3/353: write d19/f52 [482511,45049] 0 2024-05-25T06:11:37.669 INFO:tasks.workunit.client.0.smithi160.stdout:3/354: stat cc 0 2024-05-25T06:11:37.672 INFO:tasks.workunit.client.1.smithi203.stdout:0/664: dread dd/d3d/d4f/d62/f9c [0,4194304] 0 2024-05-25T06:11:37.673 INFO:tasks.workunit.client.1.smithi203.stdout:0/665: chown dd/d3d/d75 83 1 2024-05-25T06:11:37.673 INFO:tasks.workunit.client.0.smithi160.stdout:5/465: rmdir db/dc/d13/d1d 39 2024-05-25T06:11:37.673 INFO:tasks.workunit.client.0.smithi160.stdout:0/509: rename d2b/d31/d35/d36/d52 to d2b/da8 0 2024-05-25T06:11:37.674 INFO:tasks.workunit.client.0.smithi160.stdout:0/510: write d2b/d31/d35/d36/d37/f75 [182885,126118] 0 2024-05-25T06:11:37.675 INFO:tasks.workunit.client.0.smithi160.stdout:1/325: creat de/d21/d49/d1c/d59/f62 x:0 0 0 2024-05-25T06:11:37.676 INFO:tasks.workunit.client.1.smithi203.stdout:0/666: chown dd/d3a/d66/d9d/cbb 272 1 2024-05-25T06:11:37.676 INFO:tasks.workunit.client.1.smithi203.stdout:3/880: dread d10/d16/d23/d4a/f58 [0,4194304] 0 2024-05-25T06:11:37.676 INFO:tasks.workunit.client.0.smithi160.stdout:1/326: chown de/f3d 104809898 1 2024-05-25T06:11:37.678 INFO:tasks.workunit.client.1.smithi203.stdout:3/881: truncate d10/d16/d50/f84 5602321 0 2024-05-25T06:11:37.680 INFO:tasks.workunit.client.1.smithi203.stdout:3/882: mkdir d10/d16/d23/d4a/d88/dca/d120 0 2024-05-25T06:11:37.681 INFO:tasks.workunit.client.1.smithi203.stdout:0/667: link dd/d3d/d4f/d62/f6a dd/d3a/d66/d9d/fef 0 2024-05-25T06:11:37.681 INFO:tasks.workunit.client.0.smithi160.stdout:0/511: dwrite d2b/d46/d63/da7/f72 [0,4194304] 0 2024-05-25T06:11:37.682 INFO:tasks.workunit.client.1.smithi203.stdout:3/883: dread - d10/d16/d23/d4a/d88/f9f zero size 2024-05-25T06:11:37.682 INFO:tasks.workunit.client.1.smithi203.stdout:3/884: chown d10/d16/d23/d3a/f80 4328125 1 2024-05-25T06:11:37.683 INFO:tasks.workunit.client.1.smithi203.stdout:0/668: mkdir dd/d3d/d4f/db7/df0 0 2024-05-25T06:11:37.686 INFO:tasks.workunit.client.1.smithi203.stdout:3/885: truncate d10/d16/d23/d4a/d88/fdb 2843834 0 2024-05-25T06:11:37.690 INFO:tasks.workunit.client.1.smithi203.stdout:3/886: creat d10/d16/d23/d4a/d88/dca/d120/f121 x:0 0 0 2024-05-25T06:11:37.691 INFO:tasks.workunit.client.1.smithi203.stdout:3/887: dread - d10/d16/d23/d67/d105/d109/f119 zero size 2024-05-25T06:11:37.694 INFO:tasks.workunit.client.1.smithi203.stdout:3/888: dread d10/d16/d23/d45/fdc [0,4194304] 0 2024-05-25T06:11:37.697 INFO:tasks.workunit.client.1.smithi203.stdout:3/889: unlink d10/d16/d23/c82 0 2024-05-25T06:11:37.699 INFO:tasks.workunit.client.1.smithi203.stdout:3/890: fsync d10/d16/d23/d4a/d88/fa7 0 2024-05-25T06:11:37.702 INFO:tasks.workunit.client.1.smithi203.stdout:0/669: write dd/d3a/f3b [453536,92020] 0 2024-05-25T06:11:37.702 INFO:tasks.workunit.client.1.smithi203.stdout:0/670: chown cb 49 1 2024-05-25T06:11:37.705 INFO:tasks.workunit.client.0.smithi160.stdout:3/355: dread d19/f27 [0,4194304] 0 2024-05-25T06:11:37.709 INFO:tasks.workunit.client.1.smithi203.stdout:0/671: write dd/d32/d37/f52 [2472467,109834] 0 2024-05-25T06:11:37.710 INFO:tasks.workunit.client.1.smithi203.stdout:0/672: chown dd/d5c/c6f 3950 1 2024-05-25T06:11:37.712 INFO:tasks.workunit.client.1.smithi203.stdout:0/673: creat dd/d5c/d63/ff1 x:0 0 0 2024-05-25T06:11:37.713 INFO:tasks.workunit.client.1.smithi203.stdout:0/674: chown dd/d3a/fae 0 1 2024-05-25T06:11:37.715 INFO:tasks.workunit.client.1.smithi203.stdout:3/891: dread d10/d16/d23/d3a/f5d [0,4194304] 0 2024-05-25T06:11:37.718 INFO:tasks.workunit.client.1.smithi203.stdout:0/675: dwrite dd/d45/f5b [0,4194304] 0 2024-05-25T06:11:37.720 INFO:tasks.workunit.client.1.smithi203.stdout:3/892: write d10/d16/d23/d4a/d88/fa7 [1010433,117258] 0 2024-05-25T06:11:37.722 INFO:tasks.workunit.client.0.smithi160.stdout:6/415: mknod d5/de/d32/d35/d40/c8d 0 2024-05-25T06:11:37.723 INFO:tasks.workunit.client.1.smithi203.stdout:0/676: mknod dd/d45/d96/cf2 0 2024-05-25T06:11:37.724 INFO:tasks.workunit.client.1.smithi203.stdout:0/677: chown dd/d3d/d4f/d62/f9c 87737834 1 2024-05-25T06:11:37.725 INFO:tasks.workunit.client.1.smithi203.stdout:3/893: rmdir d10/d16/d23/d6c 39 2024-05-25T06:11:37.729 INFO:tasks.workunit.client.1.smithi203.stdout:0/678: rename dd/d3a to dd/d3d/df3 0 2024-05-25T06:11:37.730 INFO:tasks.workunit.client.1.smithi203.stdout:3/894: mkdir d10/d16/d23/d4a/d88/dca/d120/d122 0 2024-05-25T06:11:37.733 INFO:tasks.workunit.client.1.smithi203.stdout:3/895: rename d10/cbe to d10/dea/c123 0 2024-05-25T06:11:37.734 INFO:tasks.workunit.client.1.smithi203.stdout:0/679: truncate dd/d5c/f90 2581800 0 2024-05-25T06:11:37.734 INFO:tasks.workunit.client.1.smithi203.stdout:3/896: stat d10/d16/d23/d45/c7b 0 2024-05-25T06:11:37.735 INFO:tasks.workunit.client.1.smithi203.stdout:0/680: chown dd/c1d 4657 1 2024-05-25T06:11:37.737 INFO:tasks.workunit.client.0.smithi160.stdout:3/356: creat d19/d2f/d38/d43/d5b/d65/d7f/f81 x:0 0 0 2024-05-25T06:11:37.738 INFO:tasks.workunit.client.0.smithi160.stdout:4/430: dwrite d4/f4f [0,4194304] 0 2024-05-25T06:11:37.740 INFO:tasks.workunit.client.1.smithi203.stdout:3/897: dwrite d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:37.741 INFO:tasks.workunit.client.0.smithi160.stdout:6/416: rename d5/de/d32/d1b/f6b to d5/d45/f8e 0 2024-05-25T06:11:37.741 INFO:tasks.workunit.client.1.smithi203.stdout:0/681: rename dd/d10/f6d to dd/d3d/d4f/ff4 0 2024-05-25T06:11:37.742 INFO:tasks.workunit.client.0.smithi160.stdout:5/466: symlink db/dc/d13/l8a 0 2024-05-25T06:11:37.743 INFO:tasks.workunit.client.0.smithi160.stdout:1/327: mkdir de/d21/d49/d1c/d63 0 2024-05-25T06:11:37.744 INFO:tasks.workunit.client.1.smithi203.stdout:3/898: write d10/d16/da1/de6/f116 [107268,92524] 0 2024-05-25T06:11:37.745 INFO:tasks.workunit.client.1.smithi203.stdout:0/682: mkdir dd/d10/df5 0 2024-05-25T06:11:37.746 INFO:tasks.workunit.client.0.smithi160.stdout:2/397: rmdir d12/d86 0 2024-05-25T06:11:37.746 INFO:tasks.workunit.client.1.smithi203.stdout:0/683: readlink dd/d3d/d48/lda 0 2024-05-25T06:11:37.750 INFO:tasks.workunit.client.1.smithi203.stdout:3/899: mknod d10/d16/da1/c124 0 2024-05-25T06:11:37.750 INFO:tasks.workunit.client.0.smithi160.stdout:8/470: truncate d5/d44/f25 2043020 0 2024-05-25T06:11:37.750 INFO:tasks.workunit.client.0.smithi160.stdout:7/458: truncate d6/dc/d64/f53 840253 0 2024-05-25T06:11:37.751 INFO:tasks.workunit.client.1.smithi203.stdout:0/684: fsync dd/d5c/f98 0 2024-05-25T06:11:37.751 INFO:tasks.workunit.client.1.smithi203.stdout:3/900: stat d10/d16/d23/d4a/d88/f95 0 2024-05-25T06:11:37.756 INFO:tasks.workunit.client.0.smithi160.stdout:1/328: rename de/d21/d49/d1c/l1f to de/d21/d49/d1c/d41/d45/l64 0 2024-05-25T06:11:37.757 INFO:tasks.workunit.client.0.smithi160.stdout:2/398: mknod d12/d3e/c87 0 2024-05-25T06:11:37.758 INFO:tasks.workunit.client.1.smithi203.stdout:0/685: rename dd/d32/c4d to dd/d3d/df3/cf6 0 2024-05-25T06:11:37.758 INFO:tasks.workunit.client.0.smithi160.stdout:4/431: dread d4/d6/d35/d3d/d55/f56 [4194304,4194304] 0 2024-05-25T06:11:37.758 INFO:tasks.workunit.client.0.smithi160.stdout:8/471: mknod d5/d44/d1f/d1e/d28/d49/c9e 0 2024-05-25T06:11:37.759 INFO:tasks.workunit.client.1.smithi203.stdout:0/686: chown dd/d3d/d59/f9b 15084947 1 2024-05-25T06:11:37.759 INFO:tasks.workunit.client.0.smithi160.stdout:2/399: write d12/d1c/d37/f4f [1293660,22661] 0 2024-05-25T06:11:37.761 INFO:tasks.workunit.client.0.smithi160.stdout:5/467: unlink db/dc/d13/d1d/d31/d3b/f41 0 2024-05-25T06:11:37.762 INFO:tasks.workunit.client.0.smithi160.stdout:0/512: dwrite d2b/d31/d6d/f93 [0,4194304] 0 2024-05-25T06:11:37.763 INFO:tasks.workunit.client.0.smithi160.stdout:1/329: creat de/d21/d49/d28/f65 x:0 0 0 2024-05-25T06:11:37.764 INFO:tasks.workunit.client.0.smithi160.stdout:0/513: dread - d2b/d46/d63/f7e zero size 2024-05-25T06:11:37.767 INFO:tasks.workunit.client.0.smithi160.stdout:0/514: fsync d2b/d31/d35/d36/d86/f5b 0 2024-05-25T06:11:37.770 INFO:tasks.workunit.client.1.smithi203.stdout:3/901: write d10/d16/d23/d46/dd3/f112 [757511,119797] 0 2024-05-25T06:11:37.771 INFO:tasks.workunit.client.0.smithi160.stdout:5/468: dwrite db/dc/d13/d1d/d23/d24/d2e/f32 [4194304,4194304] 0 2024-05-25T06:11:37.774 INFO:tasks.workunit.client.1.smithi203.stdout:5/931: sync 2024-05-25T06:11:37.777 INFO:tasks.workunit.client.1.smithi203.stdout:3/902: symlink d10/d10d/l125 0 2024-05-25T06:11:37.780 INFO:tasks.workunit.client.1.smithi203.stdout:3/903: creat d10/d16/d23/d45/f126 x:0 0 0 2024-05-25T06:11:37.781 INFO:tasks.workunit.client.1.smithi203.stdout:5/932: link dc/d11/d42/fce dc/d11/d17/d1a/d72/de4/f14a 0 2024-05-25T06:11:37.783 INFO:tasks.workunit.client.0.smithi160.stdout:4/432: rename d4/d63/d7e to d4/d6/d5b/d66/d82 0 2024-05-25T06:11:37.785 INFO:tasks.workunit.client.1.smithi203.stdout:3/904: rename d10/d16/d23/d4a/d88/fdb to d10/dea/f127 0 2024-05-25T06:11:37.786 INFO:tasks.workunit.client.1.smithi203.stdout:3/905: write d10/d16/d23/d45/f126 [1047794,130283] 0 2024-05-25T06:11:37.787 INFO:tasks.workunit.client.1.smithi203.stdout:5/933: read dc/d11/f1c [768117,41685] 0 2024-05-25T06:11:37.787 INFO:tasks.workunit.client.0.smithi160.stdout:8/472: getdents d5/d44/d1f/d1e/d28/d49/d61 0 2024-05-25T06:11:37.789 INFO:tasks.workunit.client.0.smithi160.stdout:5/469: creat db/dc/d13/d1d/d23/d45/f8b x:0 0 0 2024-05-25T06:11:37.792 INFO:tasks.workunit.client.1.smithi203.stdout:3/906: dwrite d10/f21 [4194304,4194304] 0 2024-05-25T06:11:37.793 INFO:tasks.workunit.client.0.smithi160.stdout:6/417: rename d5/de/d32/d1b to d5/d39/d8f 0 2024-05-25T06:11:37.793 INFO:tasks.workunit.client.1.smithi203.stdout:3/907: write d10/d16/d23/d67/ffd [693300,119061] 0 2024-05-25T06:11:37.795 INFO:tasks.workunit.client.0.smithi160.stdout:7/459: getdents d6/dc/d64/d5a/d7c 0 2024-05-25T06:11:37.797 INFO:tasks.workunit.client.0.smithi160.stdout:0/515: mkdir d2b/d46/d63/da7/da6/da9 0 2024-05-25T06:11:37.798 INFO:tasks.workunit.client.0.smithi160.stdout:0/516: fsync d2b/f30 0 2024-05-25T06:11:37.802 INFO:tasks.workunit.client.0.smithi160.stdout:8/473: unlink d5/d44/d1f/d1e/d28/d49/c57 0 2024-05-25T06:11:37.803 INFO:tasks.workunit.client.0.smithi160.stdout:0/517: read f2 [1650251,119342] 0 2024-05-25T06:11:37.806 INFO:tasks.workunit.client.0.smithi160.stdout:6/418: creat d5/d45/d4a/f90 x:0 0 0 2024-05-25T06:11:37.806 INFO:tasks.workunit.client.0.smithi160.stdout:5/470: rmdir db/dc/d13 39 2024-05-25T06:11:37.806 INFO:tasks.workunit.client.0.smithi160.stdout:6/419: fdatasync d5/d45/d63/f83 0 2024-05-25T06:11:37.807 INFO:tasks.workunit.client.0.smithi160.stdout:4/433: write d4/d6/d5b/d66/f74 [276810,87346] 0 2024-05-25T06:11:37.807 INFO:tasks.workunit.client.0.smithi160.stdout:7/460: mknod d6/d3e/c93 0 2024-05-25T06:11:37.808 INFO:tasks.workunit.client.0.smithi160.stdout:7/461: dread - d6/d3e/d6f/f76 zero size 2024-05-25T06:11:37.809 INFO:tasks.workunit.client.0.smithi160.stdout:7/462: fsync d6/d3e/d6f/f72 0 2024-05-25T06:11:37.809 INFO:tasks.workunit.client.0.smithi160.stdout:8/474: rmdir d5/d44/d1f/d58 39 2024-05-25T06:11:37.811 INFO:tasks.workunit.client.0.smithi160.stdout:2/400: link d12/d1c/f2b d12/d75/f88 0 2024-05-25T06:11:37.811 INFO:tasks.workunit.client.0.smithi160.stdout:0/518: write d2b/d31/d35/f3e [1808517,65433] 0 2024-05-25T06:11:37.812 INFO:tasks.workunit.client.0.smithi160.stdout:4/434: symlink d4/d1c/d54/l83 0 2024-05-25T06:11:37.813 INFO:tasks.workunit.client.0.smithi160.stdout:7/463: symlink d6/d3e/d59/l94 0 2024-05-25T06:11:37.815 INFO:tasks.workunit.client.0.smithi160.stdout:8/475: creat d5/d44/d1f/d1e/d28/d49/d61/f9f x:0 0 0 2024-05-25T06:11:37.816 INFO:tasks.workunit.client.0.smithi160.stdout:6/420: link d5/d45/f8e d5/d30/f91 0 2024-05-25T06:11:37.863 INFO:tasks.workunit.client.1.smithi203.stdout:0/687: dread dd/d10/f31 [0,4194304] 0 2024-05-25T06:11:37.866 INFO:tasks.workunit.client.1.smithi203.stdout:0/688: creat dd/d3d/d48/d91/ff7 x:0 0 0 2024-05-25T06:11:37.866 INFO:tasks.workunit.client.1.smithi203.stdout:5/934: dwrite dc/d11/d17/d44/fbe [0,4194304] 0 2024-05-25T06:11:37.866 INFO:tasks.workunit.client.1.smithi203.stdout:3/908: truncate d10/d16/d23/fdd 714078 0 2024-05-25T06:11:37.870 INFO:tasks.workunit.client.1.smithi203.stdout:5/935: write dc/d11/d42/d94/fc7 [1551692,2259] 0 2024-05-25T06:11:37.874 INFO:tasks.workunit.client.1.smithi203.stdout:3/909: creat d10/d16/f128 x:0 0 0 2024-05-25T06:11:37.876 INFO:tasks.workunit.client.1.smithi203.stdout:5/936: chown dc/d11/d17/d1a/d2d/d4d/ded/d13f/dd4 18154301 1 2024-05-25T06:11:37.881 INFO:tasks.workunit.client.1.smithi203.stdout:3/910: dwrite d10/d16/d23/d3a/f80 [0,4194304] 0 2024-05-25T06:11:37.885 INFO:tasks.workunit.client.1.smithi203.stdout:0/689: dwrite dd/d32/d37/f42 [0,4194304] 0 2024-05-25T06:11:37.887 INFO:tasks.workunit.client.1.smithi203.stdout:0/690: readlink dd/d3d/d75/d93/la5 0 2024-05-25T06:11:37.887 INFO:tasks.workunit.client.1.smithi203.stdout:0/691: write dd/d10/fdd [956960,111080] 0 2024-05-25T06:11:37.888 INFO:tasks.workunit.client.1.smithi203.stdout:8/835: sync 2024-05-25T06:11:37.894 INFO:tasks.workunit.client.1.smithi203.stdout:5/937: dwrite dc/d11/f1c [4194304,4194304] 0 2024-05-25T06:11:37.896 INFO:tasks.workunit.client.1.smithi203.stdout:0/692: link dd/d3d/d4f/f8a dd/d3d/d4f/d62/d78/ff8 0 2024-05-25T06:11:37.897 INFO:tasks.workunit.client.1.smithi203.stdout:0/693: write dd/d45/f5b [887300,34408] 0 2024-05-25T06:11:37.897 INFO:tasks.workunit.client.0.smithi160.stdout:4/435: rename d4/d6/d5b/d66/f67 to d4/d6/d5b/d66/d82/f84 0 2024-05-25T06:11:37.899 INFO:tasks.workunit.client.0.smithi160.stdout:5/471: symlink db/dc/l8c 0 2024-05-25T06:11:37.901 INFO:tasks.workunit.client.0.smithi160.stdout:6/421: write d5/d45/d4a/f5d [500912,60522] 0 2024-05-25T06:11:37.903 INFO:tasks.workunit.client.0.smithi160.stdout:7/464: dwrite d6/d3e/f4b [4194304,4194304] 0 2024-05-25T06:11:37.906 INFO:tasks.workunit.client.1.smithi203.stdout:8/836: dwrite d2/d5/d10/d11/de/f1e [0,4194304] 0 2024-05-25T06:11:37.909 INFO:tasks.workunit.client.0.smithi160.stdout:6/422: creat d5/de/d32/d74/f92 x:0 0 0 2024-05-25T06:11:37.909 INFO:tasks.workunit.client.0.smithi160.stdout:8/476: dread d5/d39/d75/f76 [0,4194304] 0 2024-05-25T06:11:37.910 INFO:tasks.workunit.client.0.smithi160.stdout:5/472: mknod db/d47/c8d 0 2024-05-25T06:11:37.910 INFO:tasks.workunit.client.0.smithi160.stdout:7/465: write d6/dc/ff [1175313,37527] 0 2024-05-25T06:11:37.912 INFO:tasks.workunit.client.0.smithi160.stdout:4/436: rename d4/f11 to d4/d1a/f85 0 2024-05-25T06:11:37.916 INFO:tasks.workunit.client.1.smithi203.stdout:0/694: write dd/d5c/d63/faf [274228,72869] 0 2024-05-25T06:11:37.916 INFO:tasks.workunit.client.1.smithi203.stdout:0/695: chown dd/d5c 29 1 2024-05-25T06:11:37.916 INFO:tasks.workunit.client.0.smithi160.stdout:6/423: chown d5/l19 255333 1 2024-05-25T06:11:37.917 INFO:tasks.workunit.client.0.smithi160.stdout:7/466: truncate d6/d3e/d6f/f76 997284 0 2024-05-25T06:11:37.922 INFO:tasks.workunit.client.1.smithi203.stdout:5/938: dwrite dc/d11/d42/d94/dad/db1/fe8 [0,4194304] 0 2024-05-25T06:11:37.922 INFO:tasks.workunit.client.0.smithi160.stdout:8/477: dwrite d5/d44/d1f/f22 [0,4194304] 0 2024-05-25T06:11:37.924 INFO:tasks.workunit.client.1.smithi203.stdout:5/939: chown dc/d3c/df5/l11f 402649 1 2024-05-25T06:11:37.924 INFO:tasks.workunit.client.0.smithi160.stdout:5/473: dwrite f1 [0,4194304] 0 2024-05-25T06:11:37.925 INFO:tasks.workunit.client.0.smithi160.stdout:7/467: mknod d6/dc/d64/d5a/c95 0 2024-05-25T06:11:37.925 INFO:tasks.workunit.client.1.smithi203.stdout:0/696: rename dd/d3d/df3/d66 to dd/d3d/da0/db9/df9 0 2024-05-25T06:11:37.926 INFO:tasks.workunit.client.0.smithi160.stdout:4/437: getdents d4/d6/d21 0 2024-05-25T06:11:37.927 INFO:tasks.workunit.client.1.smithi203.stdout:5/940: write dc/d11/d17/f5d [5660363,16624] 0 2024-05-25T06:11:37.927 INFO:tasks.workunit.client.0.smithi160.stdout:4/438: chown d4/d6/d28 47 1 2024-05-25T06:11:37.927 INFO:tasks.workunit.client.0.smithi160.stdout:7/468: chown d6/dc/l1f 477706845 1 2024-05-25T06:11:37.927 INFO:tasks.workunit.client.0.smithi160.stdout:7/469: stat d6/d3e/f70 0 2024-05-25T06:11:37.930 INFO:tasks.workunit.client.1.smithi203.stdout:8/837: dread d2/d5/d18/d31/f50 [0,4194304] 0 2024-05-25T06:11:37.933 INFO:tasks.workunit.client.0.smithi160.stdout:2/401: dread d12/d3e/d5f/d74/f77 [4194304,4194304] 0 2024-05-25T06:11:37.934 INFO:tasks.workunit.client.1.smithi203.stdout:0/697: creat dd/d45/ffa x:0 0 0 2024-05-25T06:11:37.934 INFO:tasks.workunit.client.0.smithi160.stdout:9/400: sync 2024-05-25T06:11:37.934 INFO:tasks.workunit.client.1.smithi203.stdout:0/698: read - dd/d45/ffa zero size 2024-05-25T06:11:37.935 INFO:tasks.workunit.client.0.smithi160.stdout:7/470: creat d6/d32/d5b/f96 x:0 0 0 2024-05-25T06:11:37.938 INFO:tasks.workunit.client.0.smithi160.stdout:2/402: creat d12/d1c/d37/d4b/f89 x:0 0 0 2024-05-25T06:11:37.939 INFO:tasks.workunit.client.0.smithi160.stdout:2/403: chown d12/d1c/c1f 98507 1 2024-05-25T06:11:37.940 INFO:tasks.workunit.client.0.smithi160.stdout:8/478: getdents d5/d44/d1f/d58 0 2024-05-25T06:11:37.940 INFO:tasks.workunit.client.1.smithi203.stdout:0/699: mknod dd/d10/cfb 0 2024-05-25T06:11:37.943 INFO:tasks.workunit.client.0.smithi160.stdout:2/404: dread d12/d1c/f27 [0,4194304] 0 2024-05-25T06:11:37.943 INFO:tasks.workunit.client.0.smithi160.stdout:9/401: link d4/d11/d1e/f1f d4/d11/d1e/f78 0 2024-05-25T06:11:37.947 INFO:tasks.workunit.client.0.smithi160.stdout:9/402: getdents d4/d11/d1e/d24/d66 0 2024-05-25T06:11:37.948 INFO:tasks.workunit.client.0.smithi160.stdout:8/479: rmdir d5/d44/d1f/d6b/d9b 0 2024-05-25T06:11:37.948 INFO:tasks.workunit.client.0.smithi160.stdout:2/405: dwrite d12/d5c/f6b [0,4194304] 0 2024-05-25T06:11:37.950 INFO:tasks.workunit.client.0.smithi160.stdout:8/480: stat d5/d44/d1f/d1e/d28/f4d 0 2024-05-25T06:11:37.950 INFO:tasks.workunit.client.0.smithi160.stdout:2/406: truncate d12/d1c/d37/f3b 503377 0 2024-05-25T06:11:37.952 INFO:tasks.workunit.client.0.smithi160.stdout:8/481: chown d5/d9/f26 661201035 1 2024-05-25T06:11:37.952 INFO:tasks.workunit.client.0.smithi160.stdout:8/482: readlink d5/d44/d7e/l8c 0 2024-05-25T06:11:37.952 INFO:tasks.workunit.client.0.smithi160.stdout:2/407: mknod d12/c8a 0 2024-05-25T06:11:37.953 INFO:tasks.workunit.client.0.smithi160.stdout:8/483: truncate d5/d44/d1f/f55 1394537 0 2024-05-25T06:11:37.956 INFO:tasks.workunit.client.0.smithi160.stdout:8/484: creat d5/d39/fa0 x:0 0 0 2024-05-25T06:11:37.958 INFO:tasks.workunit.client.0.smithi160.stdout:8/485: creat d5/d44/d1f/d1e/d28/d4a/d4e/fa1 x:0 0 0 2024-05-25T06:11:37.959 INFO:tasks.workunit.client.0.smithi160.stdout:2/408: link d12/d13/l65 d12/d5c/d85/l8b 0 2024-05-25T06:11:37.960 INFO:tasks.workunit.client.0.smithi160.stdout:8/486: fsync d5/d44/d1f/d1e/d28/d4a/f3a 0 2024-05-25T06:11:37.960 INFO:tasks.workunit.client.0.smithi160.stdout:2/409: chown d12/d3e/d5f/d74 0 1 2024-05-25T06:11:37.960 INFO:tasks.workunit.client.0.smithi160.stdout:8/487: readlink d5/d39/l9d 0 2024-05-25T06:11:37.961 INFO:tasks.workunit.client.0.smithi160.stdout:2/410: fdatasync d12/d3e/f41 0 2024-05-25T06:11:37.963 INFO:tasks.workunit.client.0.smithi160.stdout:8/488: mkdir d5/d44/d1f/d58/d65/da2 0 2024-05-25T06:11:37.963 INFO:tasks.workunit.client.0.smithi160.stdout:5/474: dread db/dc/d13/d1d/d31/d3b/d69/f79 [0,4194304] 0 2024-05-25T06:11:37.965 INFO:tasks.workunit.client.0.smithi160.stdout:2/411: symlink d12/d1c/d37/d49/l8c 0 2024-05-25T06:11:37.967 INFO:tasks.workunit.client.1.smithi203.stdout:5/941: write dc/d11/d17/d1a/d22/f24 [489732,20441] 0 2024-05-25T06:11:37.967 INFO:tasks.workunit.client.0.smithi160.stdout:6/424: truncate d5/de/f42 940549 0 2024-05-25T06:11:37.968 INFO:tasks.workunit.client.1.smithi203.stdout:0/700: write dd/f23 [4160930,119276] 0 2024-05-25T06:11:37.969 INFO:tasks.workunit.client.0.smithi160.stdout:6/425: truncate d5/d39/d8f/d2b/f88 871401 0 2024-05-25T06:11:37.970 INFO:tasks.workunit.client.0.smithi160.stdout:2/412: mkdir d12/d3e/d5a/d8d 0 2024-05-25T06:11:37.970 INFO:tasks.workunit.client.0.smithi160.stdout:7/471: getdents d6/d32/d5b 0 2024-05-25T06:11:37.971 INFO:tasks.workunit.client.1.smithi203.stdout:5/942: unlink dc/c99 0 2024-05-25T06:11:37.971 INFO:tasks.workunit.client.0.smithi160.stdout:4/439: truncate d4/d1a/f59 1052190 0 2024-05-25T06:11:37.972 INFO:tasks.workunit.client.1.smithi203.stdout:0/701: symlink dd/d3d/d48/d91/dad/lfc 0 2024-05-25T06:11:37.972 INFO:tasks.workunit.client.0.smithi160.stdout:5/475: symlink db/dc/d13/d1d/l8e 0 2024-05-25T06:11:37.974 INFO:tasks.workunit.client.0.smithi160.stdout:5/476: chown db/dc/d13/f6a 7 1 2024-05-25T06:11:37.974 INFO:tasks.workunit.client.0.smithi160.stdout:6/426: rmdir d5/de/d32/d74/d77 39 2024-05-25T06:11:37.974 INFO:tasks.workunit.client.1.smithi203.stdout:0/702: fdatasync dd/d3d/df3/f46 0 2024-05-25T06:11:37.975 INFO:tasks.workunit.client.0.smithi160.stdout:2/413: truncate ff 903597 0 2024-05-25T06:11:37.976 INFO:tasks.workunit.client.0.smithi160.stdout:6/427: read d5/d39/d8f/d2b/f88 [562353,67116] 0 2024-05-25T06:11:37.977 INFO:tasks.workunit.client.1.smithi203.stdout:5/943: getdents dc/d11/d17/d1a/d103/d98 0 2024-05-25T06:11:37.978 INFO:tasks.workunit.client.1.smithi203.stdout:0/703: truncate dd/d3d/d4f/d62/f9c 968458 0 2024-05-25T06:11:37.979 INFO:tasks.workunit.client.1.smithi203.stdout:5/944: dread - dc/d11/d17/d44/f11c zero size 2024-05-25T06:11:37.979 INFO:tasks.workunit.client.0.smithi160.stdout:3/357: sync 2024-05-25T06:11:37.979 INFO:tasks.workunit.client.0.smithi160.stdout:1/330: sync 2024-05-25T06:11:37.981 INFO:tasks.workunit.client.0.smithi160.stdout:9/403: write f2 [207311,40337] 0 2024-05-25T06:11:37.982 INFO:tasks.workunit.client.1.smithi203.stdout:0/704: symlink dd/d3d/d48/de6/lfd 0 2024-05-25T06:11:37.983 INFO:tasks.workunit.client.0.smithi160.stdout:5/477: dwrite db/dc/d13/d1d/d23/d24/d2e/f2f [0,4194304] 0 2024-05-25T06:11:37.986 INFO:tasks.workunit.client.1.smithi203.stdout:0/705: creat dd/d3d/df3/ffe x:0 0 0 2024-05-25T06:11:37.988 INFO:tasks.workunit.client.1.smithi203.stdout:5/945: getdents dc/d11/d17/d1a/d2d/d53 0 2024-05-25T06:11:37.988 INFO:tasks.workunit.client.0.smithi160.stdout:8/489: rename d5/d44/d1f/d1e to d5/d44/d1f/da3 0 2024-05-25T06:11:37.990 INFO:tasks.workunit.client.0.smithi160.stdout:2/414: symlink d12/d3e/d5f/d74/d7a/l8e 0 2024-05-25T06:11:37.992 INFO:tasks.workunit.client.0.smithi160.stdout:3/358: creat d19/d2f/d38/d60/f82 x:0 0 0 2024-05-25T06:11:37.992 INFO:tasks.workunit.client.0.smithi160.stdout:9/404: dwrite d4/d11/d12/f48 [0,4194304] 0 2024-05-25T06:11:37.994 INFO:tasks.workunit.client.0.smithi160.stdout:5/478: dwrite db/f53 [0,4194304] 0 2024-05-25T06:11:37.995 INFO:tasks.workunit.client.0.smithi160.stdout:3/359: chown d19/d2f/d38/c5a 427868378 1 2024-05-25T06:11:37.996 INFO:tasks.workunit.client.0.smithi160.stdout:5/479: readlink db/dc/d13/l18 0 2024-05-25T06:11:37.997 INFO:tasks.workunit.client.0.smithi160.stdout:1/331: rename de/d39/l44 to de/d21/d49/d1c/d41/d45/l66 0 2024-05-25T06:11:38.002 INFO:tasks.workunit.client.1.smithi203.stdout:5/946: dwrite dc/d11/d42/d94/fb7 [0,4194304] 0 2024-05-25T06:11:38.002 INFO:tasks.workunit.client.0.smithi160.stdout:4/440: link d4/d6/d21/d42/c44 d4/d1c/d54/c86 0 2024-05-25T06:11:38.005 INFO:tasks.workunit.client.0.smithi160.stdout:5/480: dwrite db/dc/d13/d1d/d23/d24/d2e/f2f [0,4194304] 0 2024-05-25T06:11:38.007 INFO:tasks.workunit.client.0.smithi160.stdout:5/481: chown db/dc/d13/l8a 1 1 2024-05-25T06:11:38.010 INFO:tasks.workunit.client.0.smithi160.stdout:9/405: mkdir d4/d11/d1e/d3e/d2f/d79 0 2024-05-25T06:11:38.011 INFO:tasks.workunit.client.0.smithi160.stdout:7/472: getdents d6/d32 0 2024-05-25T06:11:38.011 INFO:tasks.workunit.client.0.smithi160.stdout:5/482: chown db/dc/d13/d1d/d23/d24/d2e/f5b 241687 1 2024-05-25T06:11:38.012 INFO:tasks.workunit.client.0.smithi160.stdout:5/483: fdatasync db/d47/f51 0 2024-05-25T06:11:38.013 INFO:tasks.workunit.client.0.smithi160.stdout:3/360: mkdir d19/d2f/d38/d43/d5b/d65/d83 0 2024-05-25T06:11:38.015 INFO:tasks.workunit.client.0.smithi160.stdout:1/332: unlink de/d21/d49/l34 0 2024-05-25T06:11:38.018 INFO:tasks.workunit.client.0.smithi160.stdout:8/490: dwrite d5/d44/d1f/da3/d28/d4a/f3d [0,4194304] 0 2024-05-25T06:11:38.020 INFO:tasks.workunit.client.0.smithi160.stdout:9/406: dread d4/d11/d1e/d24/fc [0,4194304] 0 2024-05-25T06:11:38.021 INFO:tasks.workunit.client.0.smithi160.stdout:8/491: dread - d5/d44/d1f/da3/d28/d4a/d4e/d86/f66 zero size 2024-05-25T06:11:38.024 INFO:tasks.workunit.client.0.smithi160.stdout:5/484: mknod db/dc/d13/d1d/d23/d24/d2e/c8f 0 2024-05-25T06:11:38.025 INFO:tasks.workunit.client.0.smithi160.stdout:5/485: truncate db/dc/d13/f87 635971 0 2024-05-25T06:11:38.026 INFO:tasks.workunit.client.0.smithi160.stdout:1/333: mkdir de/d21/d49/d1c/d59/d67 0 2024-05-25T06:11:38.027 INFO:tasks.workunit.client.0.smithi160.stdout:1/334: write de/d21/f31 [322876,1615] 0 2024-05-25T06:11:38.027 INFO:tasks.workunit.client.0.smithi160.stdout:0/519: sync 2024-05-25T06:11:38.028 INFO:tasks.workunit.client.0.smithi160.stdout:2/415: creat d12/f8f x:0 0 0 2024-05-25T06:11:38.030 INFO:tasks.workunit.client.0.smithi160.stdout:3/361: unlink d19/c6f 0 2024-05-25T06:11:38.049 INFO:tasks.workunit.client.1.smithi203.stdout:5/947: write dc/d11/d17/d1a/d72/f109 [731142,112788] 0 2024-05-25T06:11:38.050 INFO:tasks.workunit.client.1.smithi203.stdout:5/948: dread - dc/d11/d106/f13d zero size 2024-05-25T06:11:38.052 INFO:tasks.workunit.client.1.smithi203.stdout:5/949: creat dc/d11/d42/f14b x:0 0 0 2024-05-25T06:11:38.053 INFO:tasks.workunit.client.1.smithi203.stdout:5/950: write dc/d11/d17/d1a/d72/f109 [983731,41930] 0 2024-05-25T06:11:38.055 INFO:tasks.workunit.client.1.smithi203.stdout:5/951: unlink dc/d11/d17/d1a/d72/f109 0 2024-05-25T06:11:38.057 INFO:tasks.workunit.client.1.smithi203.stdout:5/952: creat dc/d11/d18/dc0/d107/f14c x:0 0 0 2024-05-25T06:11:38.059 INFO:tasks.workunit.client.1.smithi203.stdout:5/953: read dc/d11/d17/d1a/d72/f8c [3863111,92199] 0 2024-05-25T06:11:38.060 INFO:tasks.workunit.client.1.smithi203.stdout:5/954: dread - dc/d11/d42/f14b zero size 2024-05-25T06:11:38.061 INFO:tasks.workunit.client.0.smithi160.stdout:7/473: link d6/d32/d5b/f96 d6/dc/d21/f97 0 2024-05-25T06:11:38.061 INFO:tasks.workunit.client.0.smithi160.stdout:3/362: read d19/d2f/d38/d47/f54 [1435004,57681] 0 2024-05-25T06:11:38.062 INFO:tasks.workunit.client.0.smithi160.stdout:5/486: unlink db/dc/d13/c3c 0 2024-05-25T06:11:38.062 INFO:tasks.workunit.client.0.smithi160.stdout:0/520: rmdir d2b/d46 39 2024-05-25T06:11:38.064 INFO:tasks.workunit.client.1.smithi203.stdout:5/955: getdents dc/d11/d17/d1a/d2d/d53 0 2024-05-25T06:11:38.064 INFO:tasks.workunit.client.0.smithi160.stdout:1/335: creat de/d21/d49/d1c/d4f/f68 x:0 0 0 2024-05-25T06:11:38.065 INFO:tasks.workunit.client.0.smithi160.stdout:6/428: truncate d5/de/f12 1912308 0 2024-05-25T06:11:38.065 INFO:tasks.workunit.client.0.smithi160.stdout:5/487: chown db/dc/d13/d1d/d23/d24/d2e/l52 179276523 1 2024-05-25T06:11:38.067 INFO:tasks.workunit.client.1.smithi203.stdout:2/766: sync 2024-05-25T06:11:38.068 INFO:tasks.workunit.client.1.smithi203.stdout:7/725: sync 2024-05-25T06:11:38.069 INFO:tasks.workunit.client.0.smithi160.stdout:5/488: dread - db/dc/d13/d1d/d23/d45/f8b zero size 2024-05-25T06:11:38.069 INFO:tasks.workunit.client.0.smithi160.stdout:2/416: mkdir d12/d1c/d2a/d7e/d90 0 2024-05-25T06:11:38.069 INFO:tasks.workunit.client.0.smithi160.stdout:3/363: mknod d19/d2f/d38/d43/d5b/c84 0 2024-05-25T06:11:38.069 INFO:tasks.workunit.client.0.smithi160.stdout:2/417: write f11 [399207,15173] 0 2024-05-25T06:11:38.070 INFO:tasks.workunit.client.0.smithi160.stdout:3/364: chown d19/f52 93 1 2024-05-25T06:11:38.071 INFO:tasks.workunit.client.0.smithi160.stdout:1/336: creat de/d39/f69 x:0 0 0 2024-05-25T06:11:38.072 INFO:tasks.workunit.client.0.smithi160.stdout:3/365: write d19/d2f/d38/f3e [3884791,30293] 0 2024-05-25T06:11:38.073 INFO:tasks.workunit.client.1.smithi203.stdout:2/767: rename d2/d1a/d33/d80/fb5 to d2/d1a/d87/dd9/f101 0 2024-05-25T06:11:38.073 INFO:tasks.workunit.client.0.smithi160.stdout:7/474: symlink d6/l98 0 2024-05-25T06:11:38.073 INFO:tasks.workunit.client.0.smithi160.stdout:8/492: dwrite d5/d9/fc [0,4194304] 0 2024-05-25T06:11:38.074 INFO:tasks.workunit.client.0.smithi160.stdout:7/475: readlink d6/dc/d21/l48 0 2024-05-25T06:11:38.074 INFO:tasks.workunit.client.0.smithi160.stdout:8/493: chown d5/d39/d7a/f99 8 1 2024-05-25T06:11:38.075 INFO:tasks.workunit.client.1.smithi203.stdout:7/726: dwrite d1/d9/fdb [0,4194304] 0 2024-05-25T06:11:38.075 INFO:tasks.workunit.client.0.smithi160.stdout:6/429: unlink d5/d30/d36/c48 0 2024-05-25T06:11:38.076 INFO:tasks.workunit.client.0.smithi160.stdout:4/441: write d4/d1c/d54/f65 [1722005,97055] 0 2024-05-25T06:11:38.076 INFO:tasks.workunit.client.1.smithi203.stdout:2/768: truncate d2/d1a/dd3/d100/d10/d32/fe7 988940 0 2024-05-25T06:11:38.079 INFO:tasks.workunit.client.1.smithi203.stdout:7/727: chown d1/d9/l19 626479 1 2024-05-25T06:11:38.079 INFO:tasks.workunit.client.0.smithi160.stdout:2/418: symlink d12/d3e/d5f/d74/d7a/l91 0 2024-05-25T06:11:38.081 INFO:tasks.workunit.client.1.smithi203.stdout:2/769: fsync d2/d38/f7d 0 2024-05-25T06:11:38.082 INFO:tasks.workunit.client.0.smithi160.stdout:8/494: chown d5/l64 42 1 2024-05-25T06:11:38.083 INFO:tasks.workunit.client.1.smithi203.stdout:2/770: readlink d2/d38/l4a 0 2024-05-25T06:11:38.083 INFO:tasks.workunit.client.1.smithi203.stdout:5/956: dwrite dc/d11/d17/d1a/d72/f8c [0,4194304] 0 2024-05-25T06:11:38.084 INFO:tasks.workunit.client.1.smithi203.stdout:7/728: mkdir d1/d9/d2c/d3a/dd1/df8 0 2024-05-25T06:11:38.084 INFO:tasks.workunit.client.0.smithi160.stdout:7/476: rename d6/dc/f4a to d6/d32/f99 0 2024-05-25T06:11:38.084 INFO:tasks.workunit.client.0.smithi160.stdout:3/366: dwrite d19/d2f/d38/d47/f4b [0,4194304] 0 2024-05-25T06:11:38.085 INFO:tasks.workunit.client.1.smithi203.stdout:7/729: dread - d1/d11/d3d/fa6 zero size 2024-05-25T06:11:38.087 INFO:tasks.workunit.client.0.smithi160.stdout:3/367: write d19/d2f/d38/f49 [1584685,50851] 0 2024-05-25T06:11:38.087 INFO:tasks.workunit.client.0.smithi160.stdout:5/489: unlink db/dc/l1b 0 2024-05-25T06:11:38.089 INFO:tasks.workunit.client.1.smithi203.stdout:7/730: mknod d1/d9/d60/cf9 0 2024-05-25T06:11:38.089 INFO:tasks.workunit.client.0.smithi160.stdout:6/430: rmdir d5/d30/d36/d73/d6e 39 2024-05-25T06:11:38.090 INFO:tasks.workunit.client.0.smithi160.stdout:0/521: getdents d2b/d31/d35/d43/d98 0 2024-05-25T06:11:38.091 INFO:tasks.workunit.client.1.smithi203.stdout:7/731: read - d1/d9/d2c/d36/d54/fb8 zero size 2024-05-25T06:11:38.101 INFO:tasks.workunit.client.1.smithi203.stdout:7/732: dread d1/d9/d2c/d62/d7a/d89/f8c [0,4194304] 0 2024-05-25T06:11:38.101 INFO:tasks.workunit.client.0.smithi160.stdout:5/490: creat db/dc/d13/d54/f90 x:0 0 0 2024-05-25T06:11:38.102 INFO:tasks.workunit.client.0.smithi160.stdout:8/495: dwrite d5/d44/d1f/f8f [0,4194304] 0 2024-05-25T06:11:38.102 INFO:tasks.workunit.client.0.smithi160.stdout:6/431: symlink d5/d30/d36/d73/d6e/l93 0 2024-05-25T06:11:38.103 INFO:tasks.workunit.client.1.smithi203.stdout:7/733: getdents d1/d9/d2c/d62/db3/de5 0 2024-05-25T06:11:38.105 INFO:tasks.workunit.client.0.smithi160.stdout:6/432: stat d5/de/d58/l6a 0 2024-05-25T06:11:38.106 INFO:tasks.workunit.client.1.smithi203.stdout:7/734: getdents d1/d9/d2c/d79/df0 0 2024-05-25T06:11:38.108 INFO:tasks.workunit.client.0.smithi160.stdout:8/496: creat d5/d44/d1f/da3/d28/d4a/d4e/d86/fa4 x:0 0 0 2024-05-25T06:11:38.109 INFO:tasks.workunit.client.1.smithi203.stdout:7/735: truncate d1/d11/d3d/d9a/fab 1040420 0 2024-05-25T06:11:38.110 INFO:tasks.workunit.client.0.smithi160.stdout:0/522: dwrite f16 [0,4194304] 0 2024-05-25T06:11:38.110 INFO:tasks.workunit.client.0.smithi160.stdout:6/433: write d5/de/d32/f37 [599205,126752] 0 2024-05-25T06:11:38.110 INFO:tasks.workunit.client.0.smithi160.stdout:8/497: fsync d5/d39/d75/f8e 0 2024-05-25T06:11:38.112 INFO:tasks.workunit.client.0.smithi160.stdout:8/498: chown d5/d44/d1f/da3/d28/f5e 33284824 1 2024-05-25T06:11:38.116 INFO:tasks.workunit.client.0.smithi160.stdout:6/434: read d5/de/f22 [879407,64656] 0 2024-05-25T06:11:38.119 INFO:tasks.workunit.client.0.smithi160.stdout:8/499: creat d5/d39/d75/fa5 x:0 0 0 2024-05-25T06:11:38.121 INFO:tasks.workunit.client.0.smithi160.stdout:0/523: creat d2b/d46/faa x:0 0 0 2024-05-25T06:11:38.123 INFO:tasks.workunit.client.1.smithi203.stdout:2/771: dwrite d2/fe [0,4194304] 0 2024-05-25T06:11:38.123 INFO:tasks.workunit.client.0.smithi160.stdout:0/524: write d2b/d31/d35/f49 [1258885,110084] 0 2024-05-25T06:11:38.126 INFO:tasks.workunit.client.0.smithi160.stdout:6/435: dwrite d5/d30/d36/d73/d6e/f84 [0,4194304] 0 2024-05-25T06:11:38.128 INFO:tasks.workunit.client.0.smithi160.stdout:4/442: dread d4/d6/d35/f53 [8388608,4194304] 0 2024-05-25T06:11:38.129 INFO:tasks.workunit.client.0.smithi160.stdout:8/500: symlink d5/d44/d1f/d6b/la6 0 2024-05-25T06:11:38.130 INFO:tasks.workunit.client.0.smithi160.stdout:6/436: fdatasync d5/de/d32/d35/f82 0 2024-05-25T06:11:38.130 INFO:tasks.workunit.client.0.smithi160.stdout:4/443: chown d4/d1c/d54 0 1 2024-05-25T06:11:38.130 INFO:tasks.workunit.client.1.smithi203.stdout:5/957: dwrite dc/d11/d17/d1a/f27 [4194304,4194304] 0 2024-05-25T06:11:38.131 INFO:tasks.workunit.client.0.smithi160.stdout:8/501: chown d5/d44/d1f/da3/d28/d49/f70 1097231 1 2024-05-25T06:11:38.132 INFO:tasks.workunit.client.0.smithi160.stdout:6/437: write d5/de/d32/f67 [373629,4418] 0 2024-05-25T06:11:38.132 INFO:tasks.workunit.client.1.smithi203.stdout:7/736: dwrite d1/d9/d2c/d3a/f3e [0,4194304] 0 2024-05-25T06:11:38.138 INFO:tasks.workunit.client.0.smithi160.stdout:0/525: mkdir d2b/d31/d35/d43/dab 0 2024-05-25T06:11:38.143 INFO:tasks.workunit.client.0.smithi160.stdout:7/477: write d6/dc/f15 [913529,56803] 0 2024-05-25T06:11:38.144 INFO:tasks.workunit.client.0.smithi160.stdout:2/419: truncate d12/d1c/f31 3683987 0 2024-05-25T06:11:38.144 INFO:tasks.workunit.client.0.smithi160.stdout:5/491: dread db/f2a [4194304,4194304] 0 2024-05-25T06:11:38.151 INFO:tasks.workunit.client.0.smithi160.stdout:5/492: dwrite f8 [0,4194304] 0 2024-05-25T06:11:38.152 INFO:tasks.workunit.client.0.smithi160.stdout:3/368: rename d19/d2f/c32 to d19/d2f/d38/d43/d5b/d65/d7e/c85 0 2024-05-25T06:11:38.154 INFO:tasks.workunit.client.0.smithi160.stdout:2/420: chown f8 1901397 1 2024-05-25T06:11:38.155 INFO:tasks.workunit.client.0.smithi160.stdout:7/478: symlink d6/dc/d64/d5a/d7c/l9a 0 2024-05-25T06:11:38.157 INFO:tasks.workunit.client.0.smithi160.stdout:0/526: dread f1f [0,4194304] 0 2024-05-25T06:11:38.158 INFO:tasks.workunit.client.0.smithi160.stdout:4/444: link d4/f4f d4/d1a/f87 0 2024-05-25T06:11:38.160 INFO:tasks.workunit.client.0.smithi160.stdout:8/502: rename d5/d44/d1f/da3/c3c to d5/d39/d7a/ca7 0 2024-05-25T06:11:38.161 INFO:tasks.workunit.client.0.smithi160.stdout:2/421: symlink d12/d75/l92 0 2024-05-25T06:11:38.162 INFO:tasks.workunit.client.1.smithi203.stdout:5/958: dwrite dc/d11/d17/d45/f5c [0,4194304] 0 2024-05-25T06:11:38.162 INFO:tasks.workunit.client.0.smithi160.stdout:4/445: mkdir d4/d63/d64/d88 0 2024-05-25T06:11:38.166 INFO:tasks.workunit.client.0.smithi160.stdout:6/438: rename d5/d45/d63/c7b to d5/d30/c94 0 2024-05-25T06:11:38.167 INFO:tasks.workunit.client.0.smithi160.stdout:3/369: dread d19/d22/f2a [0,4194304] 0 2024-05-25T06:11:38.168 INFO:tasks.workunit.client.0.smithi160.stdout:8/503: creat d5/d44/d1f/da3/fa8 x:0 0 0 2024-05-25T06:11:38.169 INFO:tasks.workunit.client.1.smithi203.stdout:5/959: dwrite dc/d11/d106/f125 [0,4194304] 0 2024-05-25T06:11:38.169 INFO:tasks.workunit.client.0.smithi160.stdout:0/527: link d2b/d46/d63/da7/d67/fa5 d2b/d31/d35/d43/dab/fac 0 2024-05-25T06:11:38.170 INFO:tasks.workunit.client.0.smithi160.stdout:5/493: creat db/dc/d13/d1d/d23/f91 x:0 0 0 2024-05-25T06:11:38.171 INFO:tasks.workunit.client.0.smithi160.stdout:6/439: readlink d5/de/d32/l24 0 2024-05-25T06:11:38.172 INFO:tasks.workunit.client.0.smithi160.stdout:5/494: dread - db/dc/d13/d1d/d31/d56/f82 zero size 2024-05-25T06:11:38.174 INFO:tasks.workunit.client.0.smithi160.stdout:8/504: mknod d5/d44/d1f/da3/d28/d4a/ca9 0 2024-05-25T06:11:38.174 INFO:tasks.workunit.client.1.smithi203.stdout:2/772: dread d2/d1a/dd3/d100/d10/f5e [0,4194304] 0 2024-05-25T06:11:38.175 INFO:tasks.workunit.client.0.smithi160.stdout:8/505: read d5/d39/d75/f76 [484715,103031] 0 2024-05-25T06:11:38.179 INFO:tasks.workunit.client.0.smithi160.stdout:0/528: truncate d2b/d31/d35/d7b/fa0 3087409 0 2024-05-25T06:11:38.180 INFO:tasks.workunit.client.0.smithi160.stdout:0/529: chown d2b/d31/d6d/ca4 1493 1 2024-05-25T06:11:38.182 INFO:tasks.workunit.client.0.smithi160.stdout:8/506: chown d5/d7/d9a 3906621 1 2024-05-25T06:11:38.185 INFO:tasks.workunit.client.0.smithi160.stdout:6/440: creat d5/d30/d36/d73/f95 x:0 0 0 2024-05-25T06:11:38.185 INFO:tasks.workunit.client.0.smithi160.stdout:6/441: fdatasync d5/d45/d63/f83 0 2024-05-25T06:11:38.188 INFO:tasks.workunit.client.0.smithi160.stdout:8/507: mknod d5/d39/d7a/caa 0 2024-05-25T06:11:38.192 INFO:tasks.workunit.client.0.smithi160.stdout:6/442: creat d5/d30/d36/d73/d6e/f96 x:0 0 0 2024-05-25T06:11:38.196 INFO:tasks.workunit.client.0.smithi160.stdout:8/508: dwrite d5/d9/f40 [8388608,4194304] 0 2024-05-25T06:11:38.196 INFO:tasks.workunit.client.0.smithi160.stdout:3/370: mknod d19/d2f/d34/d67/c86 0 2024-05-25T06:11:38.199 INFO:tasks.workunit.client.0.smithi160.stdout:6/443: read d5/de/ff [1277286,37640] 0 2024-05-25T06:11:38.200 INFO:tasks.workunit.client.0.smithi160.stdout:3/371: write d19/d2f/d38/d43/f46 [2971932,115272] 0 2024-05-25T06:11:38.203 INFO:tasks.workunit.client.1.smithi203.stdout:2/773: dread d2/d1a/d33/d80/fc5 [4194304,4194304] 0 2024-05-25T06:11:38.205 INFO:tasks.workunit.client.0.smithi160.stdout:5/495: write db/dc/d13/d1d/d31/d56/f7d [964133,109415] 0 2024-05-25T06:11:38.206 INFO:tasks.workunit.client.1.smithi203.stdout:2/774: mknod d2/d38/d43/d60/da3/c102 0 2024-05-25T06:11:38.209 INFO:tasks.workunit.client.0.smithi160.stdout:6/444: mknod d5/d45/d63/c97 0 2024-05-25T06:11:38.211 INFO:tasks.workunit.client.1.smithi203.stdout:2/775: link d2/d1a/d89/daa/ff8 d2/d1a/dd3/d100/d10/d32/dca/de8/f103 0 2024-05-25T06:11:38.213 INFO:tasks.workunit.client.1.smithi203.stdout:2/776: creat d2/d1a/d33/d80/dd1/f104 x:0 0 0 2024-05-25T06:11:38.214 INFO:tasks.workunit.client.0.smithi160.stdout:6/445: write d5/de/d32/d35/d40/f7e [1265888,22588] 0 2024-05-25T06:11:38.215 INFO:tasks.workunit.client.0.smithi160.stdout:0/530: truncate d2b/d31/d35/d36/d86/f9c 967840 0 2024-05-25T06:11:38.216 INFO:tasks.workunit.client.1.smithi203.stdout:2/777: creat d2/d38/d43/d60/da3/f105 x:0 0 0 2024-05-25T06:11:38.217 INFO:tasks.workunit.client.1.smithi203.stdout:2/778: stat d2/d1a/d33/d80/fc5 0 2024-05-25T06:11:38.217 INFO:tasks.workunit.client.0.smithi160.stdout:5/496: rename f7 to db/dc/d13/d1d/d31/d56/f92 0 2024-05-25T06:11:38.219 INFO:tasks.workunit.client.1.smithi203.stdout:2/779: mknod d2/d42/de5/c106 0 2024-05-25T06:11:38.219 INFO:tasks.workunit.client.0.smithi160.stdout:6/446: mkdir d5/d45/d63/d98 0 2024-05-25T06:11:38.222 INFO:tasks.workunit.client.0.smithi160.stdout:3/372: link d19/c23 d19/c87 0 2024-05-25T06:11:38.222 INFO:tasks.workunit.client.0.smithi160.stdout:8/509: truncate d5/d44/d1f/d58/f60 8261572 0 2024-05-25T06:11:38.223 INFO:tasks.workunit.client.1.smithi203.stdout:2/780: getdents d2/d1a/dd3/d100/d10/d58/da0 0 2024-05-25T06:11:38.224 INFO:tasks.workunit.client.0.smithi160.stdout:6/447: chown d5/de/l27 13496538 1 2024-05-25T06:11:38.224 INFO:tasks.workunit.client.1.smithi203.stdout:2/781: readlink d2/d1a/d33/l76 0 2024-05-25T06:11:38.224 INFO:tasks.workunit.client.0.smithi160.stdout:5/497: dwrite db/dc/d13/d1d/d23/d45/f8b [0,4194304] 0 2024-05-25T06:11:38.226 INFO:tasks.workunit.client.0.smithi160.stdout:9/407: sync 2024-05-25T06:11:38.229 INFO:tasks.workunit.client.0.smithi160.stdout:6/448: mkdir d5/d45/d4a/d99 0 2024-05-25T06:11:38.232 INFO:tasks.workunit.client.0.smithi160.stdout:8/510: rename d5/d44/d1f/da3/f96 to d5/d44/d1f/da3/d28/d4a/d93/fab 0 2024-05-25T06:11:38.232 INFO:tasks.workunit.client.0.smithi160.stdout:8/511: fsync d5/d44/f37 0 2024-05-25T06:11:38.234 INFO:tasks.workunit.client.0.smithi160.stdout:8/512: stat d5/d9/fc 0 2024-05-25T06:11:38.236 INFO:tasks.workunit.client.1.smithi203.stdout:2/782: rename d2/d38/d43/d60/da3 to d2/d1a/d89/daa/def/d107 0 2024-05-25T06:11:38.237 INFO:tasks.workunit.client.0.smithi160.stdout:3/373: creat d19/f88 x:0 0 0 2024-05-25T06:11:38.238 INFO:tasks.workunit.client.0.smithi160.stdout:0/531: dwrite d2b/d31/f38 [0,4194304] 0 2024-05-25T06:11:38.239 INFO:tasks.workunit.client.0.smithi160.stdout:5/498: creat db/dc/f93 x:0 0 0 2024-05-25T06:11:38.241 INFO:tasks.workunit.client.0.smithi160.stdout:6/449: link d5/d30/d36/f5c d5/d45/f9a 0 2024-05-25T06:11:38.241 INFO:tasks.workunit.client.1.smithi203.stdout:2/783: getdents d2/d38 0 2024-05-25T06:11:38.241 INFO:tasks.workunit.client.0.smithi160.stdout:0/532: truncate d2b/d46/d63/da7/f60 453674 0 2024-05-25T06:11:38.243 INFO:tasks.workunit.client.1.smithi203.stdout:2/784: fdatasync d2/d1a/dd3/d100/d10/d32/f1f 0 2024-05-25T06:11:38.245 INFO:tasks.workunit.client.1.smithi203.stdout:2/785: write d2/d38/d43/d60/d6b/dae/f67 [738619,56585] 0 2024-05-25T06:11:38.247 INFO:tasks.workunit.client.1.smithi203.stdout:2/786: chown d2/d1a/d33/d80/fed 8079 1 2024-05-25T06:11:38.247 INFO:tasks.workunit.client.1.smithi203.stdout:4/907: sync 2024-05-25T06:11:38.247 INFO:tasks.workunit.client.1.smithi203.stdout:1/646: sync 2024-05-25T06:11:38.249 INFO:tasks.workunit.client.1.smithi203.stdout:4/908: stat d0/d8/c78 0 2024-05-25T06:11:38.251 INFO:tasks.workunit.client.1.smithi203.stdout:1/647: creat d14/dac/fdd x:0 0 0 2024-05-25T06:11:38.252 INFO:tasks.workunit.client.1.smithi203.stdout:1/648: read f10 [12188891,32811] 0 2024-05-25T06:11:38.253 INFO:tasks.workunit.client.0.smithi160.stdout:3/374: creat d19/d2f/d38/d43/d5b/d65/f89 x:0 0 0 2024-05-25T06:11:38.253 INFO:tasks.workunit.client.1.smithi203.stdout:2/787: link d2/d1a/d87/dd9/ffe d2/d38/d43/d60/f108 0 2024-05-25T06:11:38.254 INFO:tasks.workunit.client.0.smithi160.stdout:3/375: stat d19/d2f 0 2024-05-25T06:11:38.255 INFO:tasks.workunit.client.1.smithi203.stdout:2/788: mknod d2/d42/d8e/c109 0 2024-05-25T06:11:38.256 INFO:tasks.workunit.client.0.smithi160.stdout:5/499: dread db/dc/d13/f44 [0,4194304] 0 2024-05-25T06:11:38.257 INFO:tasks.workunit.client.1.smithi203.stdout:1/649: rename d14/d57/db7/dd2/db3/d5e/cc4 to d14/d57/d6b/da1/cde 0 2024-05-25T06:11:38.258 INFO:tasks.workunit.client.0.smithi160.stdout:9/408: rename d4/d30 to d4/d11/d7a 0 2024-05-25T06:11:38.259 INFO:tasks.workunit.client.1.smithi203.stdout:1/650: mknod d14/d81/cdf 0 2024-05-25T06:11:38.260 INFO:tasks.workunit.client.1.smithi203.stdout:1/651: rmdir d14/d23 39 2024-05-25T06:11:38.261 INFO:tasks.workunit.client.0.smithi160.stdout:3/376: creat d19/d5d/f8a x:0 0 0 2024-05-25T06:11:38.262 INFO:tasks.workunit.client.1.smithi203.stdout:1/652: write d14/d57/db7/dd2/db3/d5e/fd7 [599009,35900] 0 2024-05-25T06:11:38.262 INFO:tasks.workunit.client.1.smithi203.stdout:4/909: truncate d0/d8/d1f/d26/f1e 9027202 0 2024-05-25T06:11:38.263 INFO:tasks.workunit.client.0.smithi160.stdout:6/450: creat d5/de/d7d/f9b x:0 0 0 2024-05-25T06:11:38.264 INFO:tasks.workunit.client.1.smithi203.stdout:1/653: write d14/d57/d6b/f5d [106838,126393] 0 2024-05-25T06:11:38.266 INFO:tasks.workunit.client.0.smithi160.stdout:0/533: truncate d2b/f30 4728156 0 2024-05-25T06:11:38.267 INFO:tasks.workunit.client.0.smithi160.stdout:8/513: write d5/d44/f25 [1189244,85866] 0 2024-05-25T06:11:38.267 INFO:tasks.workunit.client.0.smithi160.stdout:0/534: chown f15 317 1 2024-05-25T06:11:38.268 INFO:tasks.workunit.client.0.smithi160.stdout:8/514: readlink d5/l64 0 2024-05-25T06:11:38.268 INFO:tasks.workunit.client.0.smithi160.stdout:5/500: mkdir db/dc/d13/d1d/d31/d3b/d69/d94 0 2024-05-25T06:11:38.268 INFO:tasks.workunit.client.1.smithi203.stdout:1/654: truncate d14/f24 7016770 0 2024-05-25T06:11:38.269 INFO:tasks.workunit.client.1.smithi203.stdout:4/910: link d0/d8/d3a/c87 d0/d8/d3a/d81/dd4/d114/c133 0 2024-05-25T06:11:38.270 INFO:tasks.workunit.client.1.smithi203.stdout:1/655: chown d14/d57/db7/fd1 0 1 2024-05-25T06:11:38.270 INFO:tasks.workunit.client.0.smithi160.stdout:9/409: read d4/d11/d14/f3a [1186933,107588] 0 2024-05-25T06:11:38.271 INFO:tasks.workunit.client.0.smithi160.stdout:7/479: sync 2024-05-25T06:11:38.271 INFO:tasks.workunit.client.0.smithi160.stdout:1/337: sync 2024-05-25T06:11:38.272 INFO:tasks.workunit.client.0.smithi160.stdout:3/377: rename d19/d2f/d38/f49 to d19/d2f/d38/d47/f8b 0 2024-05-25T06:11:38.273 INFO:tasks.workunit.client.1.smithi203.stdout:2/789: dread d2/d1a/dd3/d100/d10/d32/f1b [0,4194304] 0 2024-05-25T06:11:38.274 INFO:tasks.workunit.client.0.smithi160.stdout:0/535: write f23 [1267737,3551] 0 2024-05-25T06:11:38.275 INFO:tasks.workunit.client.0.smithi160.stdout:8/515: unlink d5/d39/d75/f8e 0 2024-05-25T06:11:38.275 INFO:tasks.workunit.client.1.smithi203.stdout:1/656: rename d14/d57/db7/dd2/c6c to d14/d57/d29/d94/ce0 0 2024-05-25T06:11:38.277 INFO:tasks.workunit.client.0.smithi160.stdout:5/501: creat db/dc/d13/d54/f95 x:0 0 0 2024-05-25T06:11:38.277 INFO:tasks.workunit.client.1.smithi203.stdout:4/911: link d0/d8/d3a/f77 d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f134 0 2024-05-25T06:11:38.281 INFO:tasks.workunit.client.0.smithi160.stdout:9/410: creat d4/d11/d7a/f7b x:0 0 0 2024-05-25T06:11:38.281 INFO:tasks.workunit.client.0.smithi160.stdout:1/338: dwrite de/f4d [0,4194304] 0 2024-05-25T06:11:38.282 INFO:tasks.workunit.client.1.smithi203.stdout:4/912: unlink d0/d9/d50/dce/fa1 0 2024-05-25T06:11:38.284 INFO:tasks.workunit.client.1.smithi203.stdout:4/913: truncate d0/d9/d50/dce/fcd 193002 0 2024-05-25T06:11:38.285 INFO:tasks.workunit.client.1.smithi203.stdout:1/657: mknod d14/d57/d1d/da8/ce1 0 2024-05-25T06:11:38.287 INFO:tasks.workunit.client.0.smithi160.stdout:8/516: dwrite d5/d44/d1f/f8f [0,4194304] 0 2024-05-25T06:11:38.288 INFO:tasks.workunit.client.0.smithi160.stdout:0/536: creat d2b/d46/d63/da7/d67/fad x:0 0 0 2024-05-25T06:11:38.292 INFO:tasks.workunit.client.1.smithi203.stdout:2/790: dread d2/d1a/dd3/d100/f74 [0,4194304] 0 2024-05-25T06:11:38.294 INFO:tasks.workunit.client.0.smithi160.stdout:3/378: symlink d19/d2f/l8c 0 2024-05-25T06:11:38.294 INFO:tasks.workunit.client.0.smithi160.stdout:0/537: fdatasync d2b/d31/d35/d36/d37/f75 0 2024-05-25T06:11:38.296 INFO:tasks.workunit.client.0.smithi160.stdout:6/451: rename d5/de/d32/l54 to d5/de/l9c 0 2024-05-25T06:11:38.296 INFO:tasks.workunit.client.1.smithi203.stdout:2/791: creat d2/d38/d2f/f10a x:0 0 0 2024-05-25T06:11:38.298 INFO:tasks.workunit.client.0.smithi160.stdout:9/411: dwrite d4/d11/d1e/d24/f13 [0,4194304] 0 2024-05-25T06:11:38.298 INFO:tasks.workunit.client.1.smithi203.stdout:1/658: rename d14/d57/db7/dd2/db3/d5e to d14/d57/d6b/da1/de2 0 2024-05-25T06:11:38.300 INFO:tasks.workunit.client.0.smithi160.stdout:0/538: dread f16 [0,4194304] 0 2024-05-25T06:11:38.300 INFO:tasks.workunit.client.0.smithi160.stdout:7/480: dread d6/dc/d64/f37 [0,4194304] 0 2024-05-25T06:11:38.300 INFO:tasks.workunit.client.1.smithi203.stdout:2/792: creat d2/d38/d43/d60/d6b/f10b x:0 0 0 2024-05-25T06:11:38.301 INFO:tasks.workunit.client.0.smithi160.stdout:8/517: creat d5/d44/d1f/da3/d28/d4a/d4e/fac x:0 0 0 2024-05-25T06:11:38.304 INFO:tasks.workunit.client.0.smithi160.stdout:7/481: stat d6/dc/d64/d5a/d7c/c8c 0 2024-05-25T06:11:38.305 INFO:tasks.workunit.client.0.smithi160.stdout:6/452: fsync f2 0 2024-05-25T06:11:38.306 INFO:tasks.workunit.client.1.smithi203.stdout:1/659: symlink d14/le3 0 2024-05-25T06:11:38.307 INFO:tasks.workunit.client.0.smithi160.stdout:8/518: fdatasync d5/d44/d1f/da3/d28/d4a/d73/f91 0 2024-05-25T06:11:38.310 INFO:tasks.workunit.client.0.smithi160.stdout:9/412: unlink d4/d11/d1e/d24/f13 0 2024-05-25T06:11:38.311 INFO:tasks.workunit.client.0.smithi160.stdout:0/539: dwrite d2b/d31/f7a [0,4194304] 0 2024-05-25T06:11:38.311 INFO:tasks.workunit.client.0.smithi160.stdout:3/379: symlink d19/d2f/d38/d47/l8d 0 2024-05-25T06:11:38.312 INFO:tasks.workunit.client.0.smithi160.stdout:9/413: chown d4/d11/d1e/d3e/d25/l54 22023 1 2024-05-25T06:11:38.313 INFO:tasks.workunit.client.1.smithi203.stdout:4/914: write d0/d9/d50/fd1 [11282,79373] 0 2024-05-25T06:11:38.314 INFO:tasks.workunit.client.0.smithi160.stdout:9/414: chown d4/d11/d1e/d3e/d25/d5e 999473 1 2024-05-25T06:11:38.314 INFO:tasks.workunit.client.0.smithi160.stdout:0/540: truncate d2b/d46/d63/fa2 550697 0 2024-05-25T06:11:38.315 INFO:tasks.workunit.client.0.smithi160.stdout:0/541: truncate f23 2034103 0 2024-05-25T06:11:38.321 INFO:tasks.workunit.client.1.smithi203.stdout:2/793: write d2/d6/f46 [536811,44798] 0 2024-05-25T06:11:38.321 INFO:tasks.workunit.client.0.smithi160.stdout:8/519: creat d5/d44/d1f/da3/d28/d49/d61/fad x:0 0 0 2024-05-25T06:11:38.323 INFO:tasks.workunit.client.0.smithi160.stdout:7/482: chown d6/d32/d5b 2800 1 2024-05-25T06:11:38.324 INFO:tasks.workunit.client.0.smithi160.stdout:9/415: dwrite d4/d11/f38 [0,4194304] 0 2024-05-25T06:11:38.324 INFO:tasks.workunit.client.0.smithi160.stdout:1/339: link de/d43/l4a de/d21/d49/d1c/l6a 0 2024-05-25T06:11:38.325 INFO:tasks.workunit.client.1.smithi203.stdout:2/794: mkdir d2/d1a/d33/d80/dd1/ddf/d10c 0 2024-05-25T06:11:38.326 INFO:tasks.workunit.client.0.smithi160.stdout:8/520: write d5/d44/d1f/da3/d28/f3f [2374958,53832] 0 2024-05-25T06:11:38.327 INFO:tasks.workunit.client.0.smithi160.stdout:7/483: write d6/d3e/d6f/f76 [1938295,99684] 0 2024-05-25T06:11:38.331 INFO:tasks.workunit.client.0.smithi160.stdout:3/380: dread d19/f27 [0,4194304] 0 2024-05-25T06:11:38.331 INFO:tasks.workunit.client.0.smithi160.stdout:6/453: creat d5/d39/d8f/f9d x:0 0 0 2024-05-25T06:11:38.333 INFO:tasks.workunit.client.1.smithi203.stdout:2/795: dwrite d2/d1a/d33/d80/fed [0,4194304] 0 2024-05-25T06:11:38.334 INFO:tasks.workunit.client.0.smithi160.stdout:3/381: write d19/f52 [595582,26704] 0 2024-05-25T06:11:38.334 INFO:tasks.workunit.client.1.smithi203.stdout:0/706: sync 2024-05-25T06:11:38.334 INFO:tasks.workunit.client.1.smithi203.stdout:7/737: sync 2024-05-25T06:11:38.335 INFO:tasks.workunit.client.1.smithi203.stdout:3/911: sync 2024-05-25T06:11:38.335 INFO:tasks.workunit.client.1.smithi203.stdout:8/838: sync 2024-05-25T06:11:38.335 INFO:tasks.workunit.client.1.smithi203.stdout:9/662: sync 2024-05-25T06:11:38.335 INFO:tasks.workunit.client.0.smithi160.stdout:9/416: symlink d4/d11/d1e/d3e/d25/d27/l7c 0 2024-05-25T06:11:38.336 INFO:tasks.workunit.client.0.smithi160.stdout:3/382: dread - d19/d5d/f61 zero size 2024-05-25T06:11:38.337 INFO:tasks.workunit.client.0.smithi160.stdout:1/340: dread de/f4d [0,4194304] 0 2024-05-25T06:11:38.338 INFO:tasks.workunit.client.0.smithi160.stdout:3/383: chown d19 764 1 2024-05-25T06:11:38.339 INFO:tasks.workunit.client.0.smithi160.stdout:9/417: fsync d4/d11/d1e/d3e/f44 0 2024-05-25T06:11:38.342 INFO:tasks.workunit.client.1.smithi203.stdout:4/915: dwrite d0/d8/d3a/d81/d93/d98/d99/ded/dfa/fc2 [0,4194304] 0 2024-05-25T06:11:38.343 INFO:tasks.workunit.client.1.smithi203.stdout:2/796: readlink d2/d1a/dd3/d100/l12 0 2024-05-25T06:11:38.343 INFO:tasks.workunit.client.0.smithi160.stdout:7/484: dwrite d6/dc/d21/f63 [0,4194304] 0 2024-05-25T06:11:38.343 INFO:tasks.workunit.client.0.smithi160.stdout:8/521: dwrite d5/d39/d7a/f92 [0,4194304] 0 2024-05-25T06:11:38.343 INFO:tasks.workunit.client.0.smithi160.stdout:3/384: write d19/d2f/d38/d47/f4c [1022030,123570] 0 2024-05-25T06:11:38.346 INFO:tasks.workunit.client.1.smithi203.stdout:8/839: write d2/d5/d10/d4a/fed [3360369,89460] 0 2024-05-25T06:11:38.346 INFO:tasks.workunit.client.1.smithi203.stdout:1/660: dread d14/d57/d29/d94/db8/fd8 [0,4194304] 0 2024-05-25T06:11:38.350 INFO:tasks.workunit.client.1.smithi203.stdout:7/738: dread d1/d9/d2c/d3a/d4e/f61 [0,4194304] 0 2024-05-25T06:11:38.351 INFO:tasks.workunit.client.1.smithi203.stdout:4/916: creat d0/d8/d3a/d81/d93/d98/f135 x:0 0 0 2024-05-25T06:11:38.351 INFO:tasks.workunit.client.0.smithi160.stdout:6/454: mkdir d5/de/d9e 0 2024-05-25T06:11:38.352 INFO:tasks.workunit.client.1.smithi203.stdout:4/917: dread - d0/d9/d11/f101 zero size 2024-05-25T06:11:38.352 INFO:tasks.workunit.client.0.smithi160.stdout:7/485: creat d6/dc/d21/f9b x:0 0 0 2024-05-25T06:11:38.354 INFO:tasks.workunit.client.0.smithi160.stdout:3/385: rmdir d19/d2f/d38/d47 39 2024-05-25T06:11:38.354 INFO:tasks.workunit.client.0.smithi160.stdout:7/486: chown d6/dc/c34 2 1 2024-05-25T06:11:38.355 INFO:tasks.workunit.client.0.smithi160.stdout:6/455: write d5/de/d7d/f9b [472225,55475] 0 2024-05-25T06:11:38.355 INFO:tasks.workunit.client.0.smithi160.stdout:7/487: chown d6/dc/d64/c44 1 1 2024-05-25T06:11:38.356 INFO:tasks.workunit.client.1.smithi203.stdout:2/797: rename d2/d1a/dd3/d100/l8f to d2/d1a/d33/d80/dd1/ddf/d10c/l10d 0 2024-05-25T06:11:38.356 INFO:tasks.workunit.client.0.smithi160.stdout:9/418: rename d4/d11/f73 to d4/d11/d1e/d3e/d25/d27/f7d 0 2024-05-25T06:11:38.358 INFO:tasks.workunit.client.0.smithi160.stdout:7/488: truncate d6/dc/d64/f78 397706 0 2024-05-25T06:11:38.358 INFO:tasks.workunit.client.0.smithi160.stdout:9/419: dread - d4/d11/d14/f61 zero size 2024-05-25T06:11:38.358 INFO:tasks.workunit.client.0.smithi160.stdout:8/522: unlink d5/d44/d1f/da3/d28/d4a/f3d 0 2024-05-25T06:11:38.359 INFO:tasks.workunit.client.1.smithi203.stdout:0/707: mkdir dd/d10/d34/d4c/d99/d70/dff 0 2024-05-25T06:11:38.360 INFO:tasks.workunit.client.1.smithi203.stdout:0/708: chown dd/d3d/l95 1989 1 2024-05-25T06:11:38.362 INFO:tasks.workunit.client.0.smithi160.stdout:8/523: dread - d5/d44/d1f/da3/d28/d49/d61/fad zero size 2024-05-25T06:11:38.364 INFO:tasks.workunit.client.1.smithi203.stdout:0/709: fdatasync dd/d3d/d75/f79 0 2024-05-25T06:11:38.366 INFO:tasks.workunit.client.0.smithi160.stdout:6/456: mkdir d5/de/d32/d9f 0 2024-05-25T06:11:38.367 INFO:tasks.workunit.client.0.smithi160.stdout:9/420: dread d4/d11/f26 [0,4194304] 0 2024-05-25T06:11:38.367 INFO:tasks.workunit.client.1.smithi203.stdout:4/918: dread d0/d8/d3a/d81/d93/d98/d99/fcc [0,4194304] 0 2024-05-25T06:11:38.368 INFO:tasks.workunit.client.1.smithi203.stdout:0/710: mkdir dd/d3d/d4f/d62/d78/d100 0 2024-05-25T06:11:38.368 INFO:tasks.workunit.client.1.smithi203.stdout:9/663: write d0/df/d37/da8/dd6/f9b [1002812,60149] 0 2024-05-25T06:11:38.369 INFO:tasks.workunit.client.0.smithi160.stdout:7/489: fsync d6/d32/f99 0 2024-05-25T06:11:38.370 INFO:tasks.workunit.client.1.smithi203.stdout:9/664: write d0/df/d37/f58 [1358703,25157] 0 2024-05-25T06:11:38.371 INFO:tasks.workunit.client.0.smithi160.stdout:8/524: mknod d5/d44/cae 0 2024-05-25T06:11:38.372 INFO:tasks.workunit.client.0.smithi160.stdout:1/341: link de/d21/d49/d28/l48 de/d21/d49/l6b 0 2024-05-25T06:11:38.372 INFO:tasks.workunit.client.0.smithi160.stdout:7/490: truncate d6/d2b/d5d/d89/f90 340438 0 2024-05-25T06:11:38.373 INFO:tasks.workunit.client.1.smithi203.stdout:4/919: mkdir d0/d9/d11/d136 0 2024-05-25T06:11:38.373 INFO:tasks.workunit.client.0.smithi160.stdout:8/525: stat d5/d44/d1f/da3/l56 0 2024-05-25T06:11:38.373 INFO:tasks.workunit.client.0.smithi160.stdout:1/342: chown de/d21/d49/c60 1224100 1 2024-05-25T06:11:38.373 INFO:tasks.workunit.client.0.smithi160.stdout:3/386: rename d19/d2f/d38/d43/f46 to d19/f8e 0 2024-05-25T06:11:38.374 INFO:tasks.workunit.client.0.smithi160.stdout:8/526: truncate d5/d44/d1f/da3/d28/d4a/d73/f91 929059 0 2024-05-25T06:11:38.376 INFO:tasks.workunit.client.0.smithi160.stdout:9/421: truncate d4/d11/f4c 824515 0 2024-05-25T06:11:38.376 INFO:tasks.workunit.client.1.smithi203.stdout:4/920: unlink d0/d9/d50/f10e 0 2024-05-25T06:11:38.376 INFO:tasks.workunit.client.1.smithi203.stdout:8/840: write d2/d5/d10/f53 [1921420,23074] 0 2024-05-25T06:11:38.379 INFO:tasks.workunit.client.0.smithi160.stdout:1/343: creat de/d21/d49/d1c/d41/f6c x:0 0 0 2024-05-25T06:11:38.379 INFO:tasks.workunit.client.1.smithi203.stdout:3/912: dread d10/d16/d23/f6e [0,4194304] 0 2024-05-25T06:11:38.380 INFO:tasks.workunit.client.1.smithi203.stdout:2/798: write d2/d1a/dd3/d100/d10/d32/f1f [1303287,52242] 0 2024-05-25T06:11:38.381 INFO:tasks.workunit.client.0.smithi160.stdout:3/387: creat d19/d22/f8f x:0 0 0 2024-05-25T06:11:38.383 INFO:tasks.workunit.client.1.smithi203.stdout:1/661: write d14/f24 [6316684,104328] 0 2024-05-25T06:11:38.384 INFO:tasks.workunit.client.0.smithi160.stdout:7/491: rename d6/dc/f15 to d6/d32/d92/f9c 0 2024-05-25T06:11:38.385 INFO:tasks.workunit.client.1.smithi203.stdout:9/665: link d0/d17/d20/d29/l50 d0/dd0/le2 0 2024-05-25T06:11:38.385 INFO:tasks.workunit.client.0.smithi160.stdout:7/492: readlink d6/dc/d21/l3b 0 2024-05-25T06:11:38.386 INFO:tasks.workunit.client.0.smithi160.stdout:7/493: read - d6/d32/d5b/f96 zero size 2024-05-25T06:11:38.387 INFO:tasks.workunit.client.1.smithi203.stdout:0/711: write dd/f24 [902994,128268] 0 2024-05-25T06:11:38.387 INFO:tasks.workunit.client.1.smithi203.stdout:2/799: symlink d2/d1a/d87/dd9/l10e 0 2024-05-25T06:11:38.387 INFO:tasks.workunit.client.1.smithi203.stdout:3/913: dread d10/d16/da1/de6/f10c [0,4194304] 0 2024-05-25T06:11:38.388 INFO:tasks.workunit.client.1.smithi203.stdout:0/712: dread - dd/d3d/d75/fe8 zero size 2024-05-25T06:11:38.389 INFO:tasks.workunit.client.0.smithi160.stdout:6/457: symlink d5/d39/d8f/la0 0 2024-05-25T06:11:38.389 INFO:tasks.workunit.client.0.smithi160.stdout:6/458: stat d5/d30/d36/d73/c65 0 2024-05-25T06:11:38.390 INFO:tasks.workunit.client.1.smithi203.stdout:1/662: mkdir d14/d57/d29/de4 0 2024-05-25T06:11:38.390 INFO:tasks.workunit.client.1.smithi203.stdout:3/914: fdatasync d10/d16/d23/d38/fe2 0 2024-05-25T06:11:38.391 INFO:tasks.workunit.client.1.smithi203.stdout:9/666: mkdir d0/df/d37/da8/de3 0 2024-05-25T06:11:38.393 INFO:tasks.workunit.client.1.smithi203.stdout:2/800: rename d2/d1a/d33/l76 to d2/d42/de5/l10f 0 2024-05-25T06:11:38.393 INFO:tasks.workunit.client.1.smithi203.stdout:8/841: dread d2/d5/d18/d31/f3d [0,4194304] 0 2024-05-25T06:11:38.394 INFO:tasks.workunit.client.0.smithi160.stdout:9/422: mknod d4/d11/d12/d2e/d45/c7e 0 2024-05-25T06:11:38.395 INFO:tasks.workunit.client.0.smithi160.stdout:9/423: truncate d4/f75 299836 0 2024-05-25T06:11:38.396 INFO:tasks.workunit.client.0.smithi160.stdout:3/388: mknod d19/d5d/c90 0 2024-05-25T06:11:38.396 INFO:tasks.workunit.client.1.smithi203.stdout:1/663: write d14/d57/d6b/f8e [2758140,107172] 0 2024-05-25T06:11:38.396 INFO:tasks.workunit.client.0.smithi160.stdout:9/424: readlink d4/d11/d1e/d3e/d25/d27/l6f 0 2024-05-25T06:11:38.397 INFO:tasks.workunit.client.1.smithi203.stdout:3/915: mkdir d10/d22/d2a/d129 0 2024-05-25T06:11:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:38 smithi203 ceph-mon[36030]: pgmap v19: 65 pgs: 65 active+clean; 2.5 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 108 MiB/s rd, 352 MiB/s wr, 672 op/s 2024-05-25T06:11:38.400 INFO:tasks.workunit.client.1.smithi203.stdout:9/667: dwrite d0/d57/fd9 [0,4194304] 0 2024-05-25T06:11:38.400 INFO:tasks.workunit.client.1.smithi203.stdout:0/713: creat dd/d32/d37/f101 x:0 0 0 2024-05-25T06:11:38.401 INFO:tasks.workunit.client.0.smithi160.stdout:1/344: dread de/d21/d49/d1c/f3f [0,4194304] 0 2024-05-25T06:11:38.402 INFO:tasks.workunit.client.1.smithi203.stdout:4/921: truncate d0/d8/d3a/d81/dd4/ff1 671841 0 2024-05-25T06:11:38.404 INFO:tasks.workunit.client.1.smithi203.stdout:0/714: write dd/d3d/d48/d91/ff7 [552987,77462] 0 2024-05-25T06:11:38.404 INFO:tasks.workunit.client.1.smithi203.stdout:3/916: truncate d10/d16/d23/d67/fb7 1911182 0 2024-05-25T06:11:38.406 INFO:tasks.workunit.client.0.smithi160.stdout:4/446: sync 2024-05-25T06:11:38.406 INFO:tasks.workunit.client.0.smithi160.stdout:2/422: sync 2024-05-25T06:11:38.407 INFO:tasks.workunit.client.1.smithi203.stdout:3/917: write d10/d16/d23/d3a/f55 [4633152,72070] 0 2024-05-25T06:11:38.408 INFO:tasks.workunit.client.0.smithi160.stdout:4/447: readlink d4/d1a/l5e 0 2024-05-25T06:11:38.409 INFO:tasks.workunit.client.1.smithi203.stdout:0/715: dread dd/d3d/d4f/fa2 [0,4194304] 0 2024-05-25T06:11:38.409 INFO:tasks.workunit.client.0.smithi160.stdout:3/389: dread - d19/d2f/d38/d47/f7d zero size 2024-05-25T06:11:38.411 INFO:tasks.workunit.client.1.smithi203.stdout:8/842: write d2/d5/d10/fc9 [33655,109819] 0 2024-05-25T06:11:38.413 INFO:tasks.workunit.client.0.smithi160.stdout:8/527: truncate d5/d39/d7a/f85 420647 0 2024-05-25T06:11:38.414 INFO:tasks.workunit.client.0.smithi160.stdout:2/423: creat d12/d3e/d5a/f93 x:0 0 0 2024-05-25T06:11:38.414 INFO:tasks.workunit.client.0.smithi160.stdout:6/459: symlink d5/d45/d4a/d99/la1 0 2024-05-25T06:11:38.415 INFO:tasks.workunit.client.1.smithi203.stdout:4/922: mkdir d0/d8/d52/d59/d7a/d137 0 2024-05-25T06:11:38.417 INFO:tasks.workunit.client.0.smithi160.stdout:1/345: creat de/d21/d49/d1c/d52/f6d x:0 0 0 2024-05-25T06:11:38.418 INFO:tasks.workunit.client.0.smithi160.stdout:9/425: dread d4/d11/d1e/d3e/d25/f50 [0,4194304] 0 2024-05-25T06:11:38.422 INFO:tasks.workunit.client.0.smithi160.stdout:3/390: symlink d19/l91 0 2024-05-25T06:11:38.423 INFO:tasks.workunit.client.0.smithi160.stdout:7/494: write d6/d32/f99 [136985,38082] 0 2024-05-25T06:11:38.425 INFO:tasks.workunit.client.1.smithi203.stdout:1/664: write d14/d57/d6b/da1/de2/f76 [1029582,65380] 0 2024-05-25T06:11:38.425 INFO:tasks.workunit.client.0.smithi160.stdout:2/424: dread d12/f45 [0,4194304] 0 2024-05-25T06:11:38.427 INFO:tasks.workunit.client.1.smithi203.stdout:9/668: creat d0/d17/d20/d29/fe4 x:0 0 0 2024-05-25T06:11:38.429 INFO:tasks.workunit.client.1.smithi203.stdout:4/923: dread d0/d8/d1f/d26/d3b/fac [0,4194304] 0 2024-05-25T06:11:38.430 INFO:tasks.workunit.client.0.smithi160.stdout:1/346: dwrite de/d21/d49/d28/f5b [0,4194304] 0 2024-05-25T06:11:38.430 INFO:tasks.workunit.client.0.smithi160.stdout:3/391: fsync d19/d22/f2a 0 2024-05-25T06:11:38.433 INFO:tasks.workunit.client.0.smithi160.stdout:9/426: symlink d4/d11/d1e/l7f 0 2024-05-25T06:11:38.433 INFO:tasks.workunit.client.0.smithi160.stdout:6/460: write d5/de/f22 [12087,105033] 0 2024-05-25T06:11:38.434 INFO:tasks.workunit.client.0.smithi160.stdout:3/392: write d19/d2f/d38/f6c [35835,57745] 0 2024-05-25T06:11:38.437 INFO:tasks.workunit.client.1.smithi203.stdout:0/716: dwrite dd/d32/f38 [0,4194304] 0 2024-05-25T06:11:38.438 INFO:tasks.workunit.client.0.smithi160.stdout:3/393: dread - d19/d2f/d38/d43/d5b/d65/d7f/f81 zero size 2024-05-25T06:11:38.440 INFO:tasks.workunit.client.0.smithi160.stdout:4/448: dread d4/d1a/f40 [4194304,4194304] 0 2024-05-25T06:11:38.441 INFO:tasks.workunit.client.0.smithi160.stdout:7/495: link d6/d2b/f51 d6/d3e/f9d 0 2024-05-25T06:11:38.441 INFO:tasks.workunit.client.0.smithi160.stdout:3/394: read d19/d2f/d38/d47/f54 [1157383,97073] 0 2024-05-25T06:11:38.442 INFO:tasks.workunit.client.0.smithi160.stdout:7/496: dread - d6/d3e/d6f/d73/f84 zero size 2024-05-25T06:11:38.442 INFO:tasks.workunit.client.0.smithi160.stdout:3/395: write d19/d22/f2c [4324991,30278] 0 2024-05-25T06:11:38.443 INFO:tasks.workunit.client.1.smithi203.stdout:4/924: dwrite d0/d8/d10/d91/da2/f130 [0,4194304] 0 2024-05-25T06:11:38.444 INFO:tasks.workunit.client.0.smithi160.stdout:7/497: chown d6/d3e/d6f/f40 1146 1 2024-05-25T06:11:38.445 INFO:tasks.workunit.client.0.smithi160.stdout:1/347: rename de/d21/d49/d1c/d41/f6c to de/d21/d49/d35/f6e 0 2024-05-25T06:11:38.445 INFO:tasks.workunit.client.0.smithi160.stdout:8/528: dread d5/fb [0,4194304] 0 2024-05-25T06:11:38.446 INFO:tasks.workunit.client.1.smithi203.stdout:4/925: write d0/d8/d3a/d81/d93/d98/d99/f10d [844165,81866] 0 2024-05-25T06:11:38.447 INFO:tasks.workunit.client.0.smithi160.stdout:8/529: fsync d5/d44/d1f/da3/d28/d49/f97 0 2024-05-25T06:11:38.448 INFO:tasks.workunit.client.0.smithi160.stdout:8/530: fdatasync d5/d44/d1f/f8f 0 2024-05-25T06:11:38.449 INFO:tasks.workunit.client.1.smithi203.stdout:3/918: dwrite d10/d16/d23/d67/fb7 [0,4194304] 0 2024-05-25T06:11:38.450 INFO:tasks.workunit.client.0.smithi160.stdout:6/461: link d5/de/d32/d74/f92 d5/de/d9e/fa2 0 2024-05-25T06:11:38.451 INFO:tasks.workunit.client.1.smithi203.stdout:3/919: stat d10/d16/d50/d7d/l98 0 2024-05-25T06:11:38.451 INFO:tasks.workunit.client.1.smithi203.stdout:3/920: chown d10/d16/d23/d67/d105/c108 3311 1 2024-05-25T06:11:38.452 INFO:tasks.workunit.client.0.smithi160.stdout:9/427: creat d4/d11/f80 x:0 0 0 2024-05-25T06:11:38.453 INFO:tasks.workunit.client.0.smithi160.stdout:7/498: fdatasync d6/d32/d5b/f96 0 2024-05-25T06:11:38.458 INFO:tasks.workunit.client.0.smithi160.stdout:2/425: write d12/d1c/d37/f71 [620853,66532] 0 2024-05-25T06:11:38.459 INFO:tasks.workunit.client.0.smithi160.stdout:1/348: dwrite de/d21/d49/d1c/d52/f6d [0,4194304] 0 2024-05-25T06:11:38.459 INFO:tasks.workunit.client.0.smithi160.stdout:6/462: symlink d5/d45/d4a/la3 0 2024-05-25T06:11:38.466 INFO:tasks.workunit.client.0.smithi160.stdout:7/499: chown d6/dc/d21/f57 0 1 2024-05-25T06:11:38.468 INFO:tasks.workunit.client.0.smithi160.stdout:2/426: readlink d12/d3e/d5f/d74/d7a/l91 0 2024-05-25T06:11:38.468 INFO:tasks.workunit.client.0.smithi160.stdout:6/463: read - d5/de/d32/d35/f4e zero size 2024-05-25T06:11:38.468 INFO:tasks.workunit.client.0.smithi160.stdout:7/500: dread - d6/d2b/f51 zero size 2024-05-25T06:11:38.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:38 smithi160 ceph-mon[29041]: pgmap v19: 65 pgs: 65 active+clean; 2.5 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 108 MiB/s rd, 352 MiB/s wr, 672 op/s 2024-05-25T06:11:38.470 INFO:tasks.workunit.client.0.smithi160.stdout:8/531: dwrite d5/d44/d1f/da3/d28/d49/f70 [0,4194304] 0 2024-05-25T06:11:38.497 INFO:tasks.workunit.client.0.smithi160.stdout:9/428: unlink d4/d11/d1e/f78 0 2024-05-25T06:11:38.498 INFO:tasks.workunit.client.1.smithi203.stdout:5/960: sync 2024-05-25T06:11:38.499 INFO:tasks.workunit.client.0.smithi160.stdout:3/396: write d19/d22/f39 [5691702,89774] 0 2024-05-25T06:11:38.501 INFO:tasks.workunit.client.0.smithi160.stdout:2/427: unlink d12/d75/c81 0 2024-05-25T06:11:38.504 INFO:tasks.workunit.client.0.smithi160.stdout:8/532: mkdir d5/d9/d72/daf 0 2024-05-25T06:11:38.505 INFO:tasks.workunit.client.0.smithi160.stdout:9/429: chown d4/d11/d1e/f1f 28 1 2024-05-25T06:11:38.509 INFO:tasks.workunit.client.0.smithi160.stdout:6/464: mkdir d5/d30/d36/d73/d87/da4 0 2024-05-25T06:11:38.511 INFO:tasks.workunit.client.0.smithi160.stdout:9/430: creat d4/d11/d1e/d6c/f81 x:0 0 0 2024-05-25T06:11:38.512 INFO:tasks.workunit.client.0.smithi160.stdout:2/428: rmdir d12 39 2024-05-25T06:11:38.514 INFO:tasks.workunit.client.0.smithi160.stdout:7/501: getdents d6/dc/d64/d5a 0 2024-05-25T06:11:38.516 INFO:tasks.workunit.client.0.smithi160.stdout:8/533: dwrite d5/d44/d1f/da3/d28/d4a/f34 [0,4194304] 0 2024-05-25T06:11:38.518 INFO:tasks.workunit.client.0.smithi160.stdout:3/397: dwrite d19/d2f/d34/f55 [0,4194304] 0 2024-05-25T06:11:38.521 INFO:tasks.workunit.client.0.smithi160.stdout:1/349: dread de/d21/d49/d35/f24 [0,4194304] 0 2024-05-25T06:11:38.521 INFO:tasks.workunit.client.0.smithi160.stdout:7/502: mknod d6/d32/d92/c9e 0 2024-05-25T06:11:38.522 INFO:tasks.workunit.client.0.smithi160.stdout:9/431: write d4/d11/d1e/d3e/d25/d27/f77 [103234,51339] 0 2024-05-25T06:11:38.523 INFO:tasks.workunit.client.0.smithi160.stdout:2/429: write d12/d1c/d37/f3b [834115,97458] 0 2024-05-25T06:11:38.524 INFO:tasks.workunit.client.0.smithi160.stdout:6/465: getdents d5/d30/d36/d73/d87 0 2024-05-25T06:11:38.524 INFO:tasks.workunit.client.0.smithi160.stdout:8/534: rename d5/d39/l4f to d5/d44/d1f/da3/d28/d49/lb0 0 2024-05-25T06:11:38.525 INFO:tasks.workunit.client.0.smithi160.stdout:9/432: chown d4/d11/d1e/d3e/d25/d27/f77 22 1 2024-05-25T06:11:38.527 INFO:tasks.workunit.client.0.smithi160.stdout:7/503: creat d6/dc/d64/f9f x:0 0 0 2024-05-25T06:11:38.529 INFO:tasks.workunit.client.0.smithi160.stdout:3/398: read f0 [1985426,114093] 0 2024-05-25T06:11:38.530 INFO:tasks.workunit.client.1.smithi203.stdout:8/843: mknod d2/d5/d18/d31/dac/c121 0 2024-05-25T06:11:38.530 INFO:tasks.workunit.client.0.smithi160.stdout:1/350: creat de/d21/d49/f6f x:0 0 0 2024-05-25T06:11:38.531 INFO:tasks.workunit.client.1.smithi203.stdout:1/665: write d14/d57/d1d/f2a [6768590,26309] 0 2024-05-25T06:11:38.533 INFO:tasks.workunit.client.0.smithi160.stdout:6/466: link d5/d45/f8e d5/d45/d4a/fa5 0 2024-05-25T06:11:38.533 INFO:tasks.workunit.client.0.smithi160.stdout:3/399: readlink d19/l79 0 2024-05-25T06:11:38.534 INFO:tasks.workunit.client.1.smithi203.stdout:0/717: mknod dd/d45/d96/c102 0 2024-05-25T06:11:38.534 INFO:tasks.workunit.client.0.smithi160.stdout:1/351: symlink de/d21/l70 0 2024-05-25T06:11:38.535 INFO:tasks.workunit.client.0.smithi160.stdout:7/504: mkdir d6/d2b/da0 0 2024-05-25T06:11:38.535 INFO:tasks.workunit.client.0.smithi160.stdout:8/535: dwrite d5/d44/f37 [4194304,4194304] 0 2024-05-25T06:11:38.537 INFO:tasks.workunit.client.0.smithi160.stdout:1/352: write de/f3d [272967,26390] 0 2024-05-25T06:11:38.538 INFO:tasks.workunit.client.0.smithi160.stdout:8/536: write d5/d44/d1f/da3/d28/d4a/d4e/d86/fa4 [324497,2969] 0 2024-05-25T06:11:38.540 INFO:tasks.workunit.client.0.smithi160.stdout:7/505: unlink d6/c33 0 2024-05-25T06:11:38.541 INFO:tasks.workunit.client.0.smithi160.stdout:8/537: write d5/d39/fa0 [180938,30685] 0 2024-05-25T06:11:38.542 INFO:tasks.workunit.client.1.smithi203.stdout:4/926: rename d0/d9/d50/d79/f5f to d0/d8/d1f/d26/d3b/df5/f138 0 2024-05-25T06:11:38.544 INFO:tasks.workunit.client.1.smithi203.stdout:8/844: dwrite d2/d5/d28/d38/fa6 [0,4194304] 0 2024-05-25T06:11:38.545 INFO:tasks.workunit.client.0.smithi160.stdout:8/538: write d5/d44/f16 [939251,124404] 0 2024-05-25T06:11:38.545 INFO:tasks.workunit.client.0.smithi160.stdout:5/502: sync 2024-05-25T06:11:38.545 INFO:tasks.workunit.client.1.smithi203.stdout:3/921: symlink d10/d16/d23/d67/l12a 0 2024-05-25T06:11:38.546 INFO:tasks.workunit.client.1.smithi203.stdout:5/961: creat dc/d11/d17/d1a/d2d/d53/da0/f14d x:0 0 0 2024-05-25T06:11:38.547 INFO:tasks.workunit.client.1.smithi203.stdout:1/666: creat d14/d81/fe5 x:0 0 0 2024-05-25T06:11:38.548 INFO:tasks.workunit.client.0.smithi160.stdout:1/353: dwrite de/f15 [0,4194304] 0 2024-05-25T06:11:38.548 INFO:tasks.workunit.client.0.smithi160.stdout:8/539: write d5/d44/d1f/da3/d28/d4a/d73/f91 [138376,105106] 0 2024-05-25T06:11:38.549 INFO:tasks.workunit.client.1.smithi203.stdout:0/718: rename dd/d3d/d4f/db7/le3 to dd/d3d/d4f/db7/df0/l103 0 2024-05-25T06:11:38.549 INFO:tasks.workunit.client.1.smithi203.stdout:3/922: write d10/d16/d23/d67/fae [2140697,74440] 0 2024-05-25T06:11:38.550 INFO:tasks.workunit.client.0.smithi160.stdout:7/506: mkdir d6/d3e/da1 0 2024-05-25T06:11:38.551 INFO:tasks.workunit.client.1.smithi203.stdout:4/927: rmdir d0/d8/d3a/d81/d121 0 2024-05-25T06:11:38.553 INFO:tasks.workunit.client.0.smithi160.stdout:3/400: link d19/l79 d19/d2f/d34/d67/l92 0 2024-05-25T06:11:38.555 INFO:tasks.workunit.client.1.smithi203.stdout:4/928: stat d0/d9/d11/d54/cdb 0 2024-05-25T06:11:38.555 INFO:tasks.workunit.client.0.smithi160.stdout:9/433: dread d4/d11/d1e/d24/f4f [0,4194304] 0 2024-05-25T06:11:38.556 INFO:tasks.workunit.client.1.smithi203.stdout:8/845: creat d2/d5/d18/d31/dac/dc2/d88/d93/f122 x:0 0 0 2024-05-25T06:11:38.556 INFO:tasks.workunit.client.1.smithi203.stdout:1/667: read d14/d57/d1d/f89 [3487525,11094] 0 2024-05-25T06:11:38.557 INFO:tasks.workunit.client.1.smithi203.stdout:4/929: readlink d0/d8/d52/l55 0 2024-05-25T06:11:38.558 INFO:tasks.workunit.client.1.smithi203.stdout:0/719: unlink dd/d3d/da0/db9/df9/d9d/la9 0 2024-05-25T06:11:38.558 INFO:tasks.workunit.client.0.smithi160.stdout:7/507: read d6/dc/f41 [1118235,74618] 0 2024-05-25T06:11:38.558 INFO:tasks.workunit.client.0.smithi160.stdout:5/503: dwrite db/d47/f68 [0,4194304] 0 2024-05-25T06:11:38.558 INFO:tasks.workunit.client.0.smithi160.stdout:3/401: chown d19/d2f/d38/c7c 448 1 2024-05-25T06:11:38.559 INFO:tasks.workunit.client.0.smithi160.stdout:7/508: stat d6/dc/d64/d5a/d7c 0 2024-05-25T06:11:38.559 INFO:tasks.workunit.client.1.smithi203.stdout:0/720: chown dd/d10/fdd 11995498 1 2024-05-25T06:11:38.562 INFO:tasks.workunit.client.1.smithi203.stdout:1/668: mknod d14/d57/d97/da4/dad/ce6 0 2024-05-25T06:11:38.563 INFO:tasks.workunit.client.1.smithi203.stdout:4/930: write d0/d9/d50/d79/d9a/dbe/f118 [729332,19438] 0 2024-05-25T06:11:38.563 INFO:tasks.workunit.client.1.smithi203.stdout:0/721: mknod dd/d3d/d48/de6/c104 0 2024-05-25T06:11:38.566 INFO:tasks.workunit.client.0.smithi160.stdout:9/434: creat d4/d11/d1e/d24/f82 x:0 0 0 2024-05-25T06:11:38.566 INFO:tasks.workunit.client.0.smithi160.stdout:3/402: rmdir d19/d2f/d38/d43 39 2024-05-25T06:11:38.570 INFO:tasks.workunit.client.0.smithi160.stdout:9/435: write f2 [2075634,40696] 0 2024-05-25T06:11:38.572 INFO:tasks.workunit.client.1.smithi203.stdout:1/669: dwrite d14/d57/db7/dd2/fcd [0,4194304] 0 2024-05-25T06:11:38.572 INFO:tasks.workunit.client.1.smithi203.stdout:4/931: mkdir d0/d139 0 2024-05-25T06:11:38.576 INFO:tasks.workunit.client.0.smithi160.stdout:3/403: rename d19/d2f/d38/d47/f7d to d19/d2f/d38/d43/d5b/d65/f93 0 2024-05-25T06:11:38.577 INFO:tasks.workunit.client.0.smithi160.stdout:9/436: dwrite d4/d11/d12/f48 [0,4194304] 0 2024-05-25T06:11:38.578 INFO:tasks.workunit.client.0.smithi160.stdout:3/404: chown d19/d22/f24 7 1 2024-05-25T06:11:38.579 INFO:tasks.workunit.client.1.smithi203.stdout:4/932: mkdir d0/d9/d9d/d13a 0 2024-05-25T06:11:38.579 INFO:tasks.workunit.client.0.smithi160.stdout:9/437: dread - d4/d11/f67 zero size 2024-05-25T06:11:38.581 INFO:tasks.workunit.client.1.smithi203.stdout:1/670: truncate f10 6915493 0 2024-05-25T06:11:38.584 INFO:tasks.workunit.client.0.smithi160.stdout:6/467: rmdir d5 39 2024-05-25T06:11:38.585 INFO:tasks.workunit.client.1.smithi203.stdout:4/933: rename d0/d9/d11/d136 to d0/d9/d50/d6b/d13b 0 2024-05-25T06:11:38.586 INFO:tasks.workunit.client.0.smithi160.stdout:9/438: mkdir d4/d11/d12/d2e/d45/d83 0 2024-05-25T06:11:38.586 INFO:tasks.workunit.client.1.smithi203.stdout:1/671: creat d14/d57/d1d/da8/fe7 x:0 0 0 2024-05-25T06:11:38.587 INFO:tasks.workunit.client.1.smithi203.stdout:4/934: write d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f11d [3784654,100709] 0 2024-05-25T06:11:38.588 INFO:tasks.workunit.client.1.smithi203.stdout:4/935: chown d0/d8/d10/da9/cf0 76678903 1 2024-05-25T06:11:38.591 INFO:tasks.workunit.client.0.smithi160.stdout:3/405: rmdir d19/d2f/d38/d43/d5b/d65/d7f/d80 0 2024-05-25T06:11:38.593 INFO:tasks.workunit.client.0.smithi160.stdout:9/439: rename d4/d11/d14/l76 to d4/d11/d1e/d3e/d2f/d79/l84 0 2024-05-25T06:11:38.594 INFO:tasks.workunit.client.0.smithi160.stdout:6/468: rmdir d5/de/d9e 39 2024-05-25T06:11:38.594 INFO:tasks.workunit.client.1.smithi203.stdout:1/672: link d14/d57/d6b/f5d d14/d57/d97/da4/fe8 0 2024-05-25T06:11:38.596 INFO:tasks.workunit.client.0.smithi160.stdout:3/406: unlink d19/f3a 0 2024-05-25T06:11:38.597 INFO:tasks.workunit.client.0.smithi160.stdout:9/440: fsync d4/d11/d14/f31 0 2024-05-25T06:11:38.598 INFO:tasks.workunit.client.1.smithi203.stdout:1/673: fsync d14/d57/d6b/da1/de2/f55 0 2024-05-25T06:11:38.598 INFO:tasks.workunit.client.0.smithi160.stdout:9/441: chown d4/d11/d1e/d3e/c55 19311616 1 2024-05-25T06:11:38.599 INFO:tasks.workunit.client.0.smithi160.stdout:8/540: truncate d5/d44/f16 2904870 0 2024-05-25T06:11:38.600 INFO:tasks.workunit.client.0.smithi160.stdout:1/354: truncate de/d21/d49/d35/f40 7183320 0 2024-05-25T06:11:38.600 INFO:tasks.workunit.client.1.smithi203.stdout:1/674: read d14/d57/d6b/da1/faa [3735,5700] 0 2024-05-25T06:11:38.601 INFO:tasks.workunit.client.1.smithi203.stdout:9/669: write d0/df/f15 [3733918,59227] 0 2024-05-25T06:11:38.603 INFO:tasks.workunit.client.1.smithi203.stdout:1/675: chown d14/d57/db7/ld5 173364159 1 2024-05-25T06:11:38.604 INFO:tasks.workunit.client.0.smithi160.stdout:8/541: mknod d5/d44/d1f/da3/d28/d4a/d93/cb1 0 2024-05-25T06:11:38.604 INFO:tasks.workunit.client.1.smithi203.stdout:9/670: truncate d0/f16 743290 0 2024-05-25T06:11:38.607 INFO:tasks.workunit.client.0.smithi160.stdout:9/442: dwrite d4/f75 [0,4194304] 0 2024-05-25T06:11:38.608 INFO:tasks.workunit.client.0.smithi160.stdout:5/504: dread db/dc/d13/d1d/d23/d24/d2e/f5b [0,4194304] 0 2024-05-25T06:11:38.609 INFO:tasks.workunit.client.0.smithi160.stdout:8/542: mkdir d5/d44/d1f/da3/d28/d49/db2 0 2024-05-25T06:11:38.609 INFO:tasks.workunit.client.0.smithi160.stdout:9/443: write d4/d11/d1e/d24/f82 [301401,126376] 0 2024-05-25T06:11:38.613 INFO:tasks.workunit.client.0.smithi160.stdout:6/469: getdents d5/d45/d63 0 2024-05-25T06:11:38.613 INFO:tasks.workunit.client.0.smithi160.stdout:7/509: truncate d6/d3e/d6f/f56 3629103 0 2024-05-25T06:11:38.614 INFO:tasks.workunit.client.0.smithi160.stdout:5/505: mkdir db/d96 0 2024-05-25T06:11:38.615 INFO:tasks.workunit.client.0.smithi160.stdout:7/510: fdatasync d6/d2b/f6b 0 2024-05-25T06:11:38.617 INFO:tasks.workunit.client.0.smithi160.stdout:7/511: write d6/dc/d21/f54 [6456552,90042] 0 2024-05-25T06:11:38.617 INFO:tasks.workunit.client.0.smithi160.stdout:5/506: truncate db/dc/d13/d1d/d31/d3b/f84 493053 0 2024-05-25T06:11:38.630 INFO:tasks.workunit.client.1.smithi203.stdout:4/936: write d0/d8/d10/f5c [904961,7138] 0 2024-05-25T06:11:38.634 INFO:tasks.workunit.client.1.smithi203.stdout:8/846: dwrite d2/f61 [0,4194304] 0 2024-05-25T06:11:38.636 INFO:tasks.workunit.client.1.smithi203.stdout:1/676: write d14/d57/db7/dd2/db3/f7a [611150,24594] 0 2024-05-25T06:11:38.637 INFO:tasks.workunit.client.1.smithi203.stdout:3/923: dwrite d10/d16/d50/d8b/fb0 [0,4194304] 0 2024-05-25T06:11:38.637 INFO:tasks.workunit.client.0.smithi160.stdout:0/542: sync 2024-05-25T06:11:38.639 INFO:tasks.workunit.client.1.smithi203.stdout:5/962: dwrite dc/d11/d17/d1a/f32 [0,4194304] 0 2024-05-25T06:11:38.649 INFO:tasks.workunit.client.1.smithi203.stdout:5/963: stat dc/d11/d17/d1a/f32 0 2024-05-25T06:11:38.651 INFO:tasks.workunit.client.1.smithi203.stdout:4/937: unlink d0/d8/d10/d91/da2/c11f 0 2024-05-25T06:11:38.652 INFO:tasks.workunit.client.1.smithi203.stdout:8/847: mkdir d2/d5/d10/d11/de/d11f/d123 0 2024-05-25T06:11:38.653 INFO:tasks.workunit.client.1.smithi203.stdout:0/722: dwrite dd/d3d/da0/db9/df9/d9d/fef [0,4194304] 0 2024-05-25T06:11:38.654 INFO:tasks.workunit.client.0.smithi160.stdout:9/444: write d4/d11/d1e/d24/fc [3466038,3734] 0 2024-05-25T06:11:38.659 INFO:tasks.workunit.client.1.smithi203.stdout:3/924: rename d10/d16/d23/d4a/d88/dca/c11a to d10/d16/d71/dfa/c12b 0 2024-05-25T06:11:38.663 INFO:tasks.workunit.client.1.smithi203.stdout:9/671: dwrite d0/dd0/fa0 [0,4194304] 0 2024-05-25T06:11:38.665 INFO:tasks.workunit.client.0.smithi160.stdout:2/430: sync 2024-05-25T06:11:38.665 INFO:tasks.workunit.client.0.smithi160.stdout:4/449: sync 2024-05-25T06:11:38.666 INFO:tasks.workunit.client.0.smithi160.stdout:2/431: chown d12/d1c/d59 877658 1 2024-05-25T06:11:38.666 INFO:tasks.workunit.client.1.smithi203.stdout:4/938: rename d0/d8/d52/f5e to d0/d139/f13c 0 2024-05-25T06:11:38.666 INFO:tasks.workunit.client.1.smithi203.stdout:9/672: write d0/d17/d20/d6e/d75/fc0 [1803391,114934] 0 2024-05-25T06:11:38.669 INFO:tasks.workunit.client.0.smithi160.stdout:7/512: mknod d6/d3e/d59/ca2 0 2024-05-25T06:11:38.669 INFO:tasks.workunit.client.1.smithi203.stdout:5/964: symlink dc/d11/d17/d1a/d2d/d53/da2/d116/l14e 0 2024-05-25T06:11:38.670 INFO:tasks.workunit.client.0.smithi160.stdout:7/513: write d6/d3e/d59/f66 [3784909,60665] 0 2024-05-25T06:11:38.672 INFO:tasks.workunit.client.0.smithi160.stdout:1/355: write de/d21/d49/f2b [3777077,49649] 0 2024-05-25T06:11:38.673 INFO:tasks.workunit.client.1.smithi203.stdout:9/673: dread d0/f51 [0,4194304] 0 2024-05-25T06:11:38.673 INFO:tasks.workunit.client.0.smithi160.stdout:3/407: truncate d19/f52 397636 0 2024-05-25T06:11:38.674 INFO:tasks.workunit.client.1.smithi203.stdout:1/677: truncate d14/d57/db7/dd2/db3/f85 3286271 0 2024-05-25T06:11:38.676 INFO:tasks.workunit.client.0.smithi160.stdout:1/356: dread de/f3a [0,4194304] 0 2024-05-25T06:11:38.678 INFO:tasks.workunit.client.0.smithi160.stdout:0/543: creat d2b/d31/d35/d36/d86/fae x:0 0 0 2024-05-25T06:11:38.679 INFO:tasks.workunit.client.1.smithi203.stdout:4/939: mknod d0/d8/c13d 0 2024-05-25T06:11:38.683 INFO:tasks.workunit.client.1.smithi203.stdout:8/848: getdents d2/d5/d28/d38/d4e/d97/da1 0 2024-05-25T06:11:38.683 INFO:tasks.workunit.client.1.smithi203.stdout:0/723: getdents dd/d10 0 2024-05-25T06:11:38.683 INFO:tasks.workunit.client.1.smithi203.stdout:3/925: link d10/d16/d50/d8b/l10f d10/d22/d2a/d129/l12c 0 2024-05-25T06:11:38.684 INFO:tasks.workunit.client.1.smithi203.stdout:5/965: symlink dc/d11/d17/d1a/d2d/d4d/ded/d13f/dd4/l14f 0 2024-05-25T06:11:38.689 INFO:tasks.workunit.client.1.smithi203.stdout:4/940: creat d0/d9/d9d/d13a/f13e x:0 0 0 2024-05-25T06:11:38.690 INFO:tasks.workunit.client.1.smithi203.stdout:1/678: dwrite d14/d57/d1d/da8/fe7 [0,4194304] 0 2024-05-25T06:11:38.691 INFO:tasks.workunit.client.1.smithi203.stdout:4/941: write d0/d9/f9e [2628319,107267] 0 2024-05-25T06:11:38.691 INFO:tasks.workunit.client.1.smithi203.stdout:5/966: fsync dc/d11/d17/d1a/d75/ddb/f132 0 2024-05-25T06:11:38.697 INFO:tasks.workunit.client.0.smithi160.stdout:4/450: unlink d4/d6/l23 0 2024-05-25T06:11:38.698 INFO:tasks.workunit.client.0.smithi160.stdout:4/451: chown d4/d6/d5b/d66 0 1 2024-05-25T06:11:38.699 INFO:tasks.workunit.client.0.smithi160.stdout:4/452: readlink d4/d1c/d54/l83 0 2024-05-25T06:11:38.699 INFO:tasks.workunit.client.0.smithi160.stdout:2/432: symlink d12/d1c/d59/l94 0 2024-05-25T06:11:38.701 INFO:tasks.workunit.client.1.smithi203.stdout:9/674: dread d0/d57/f86 [0,4194304] 0 2024-05-25T06:11:38.701 INFO:tasks.workunit.client.1.smithi203.stdout:1/679: creat d14/d57/fe9 x:0 0 0 2024-05-25T06:11:38.702 INFO:tasks.workunit.client.1.smithi203.stdout:4/942: mknod d0/d8/d3a/d81/d93/d98/d99/c13f 0 2024-05-25T06:11:38.704 INFO:tasks.workunit.client.0.smithi160.stdout:7/514: unlink d6/d3e/d6f/c4e 0 2024-05-25T06:11:38.705 INFO:tasks.workunit.client.1.smithi203.stdout:8/849: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/f124 x:0 0 0 2024-05-25T06:11:38.706 INFO:tasks.workunit.client.1.smithi203.stdout:8/850: chown d2/d5/d18/d31/dac/dc2/d88/d93/d9d/cb2 0 1 2024-05-25T06:11:38.707 INFO:tasks.workunit.client.0.smithi160.stdout:3/408: chown d19/d2f/d38/d43/d5b/d65/l7a 0 1 2024-05-25T06:11:38.708 INFO:tasks.workunit.client.1.smithi203.stdout:8/851: read d2/f61 [791048,33120] 0 2024-05-25T06:11:38.708 INFO:tasks.workunit.client.1.smithi203.stdout:4/943: mkdir d0/d8/d3a/d81/d140 0 2024-05-25T06:11:38.710 INFO:tasks.workunit.client.1.smithi203.stdout:8/852: chown d2/d5/d18/d31/c95 10906 1 2024-05-25T06:11:38.711 INFO:tasks.workunit.client.1.smithi203.stdout:1/680: link d14/d57/d7e/fa7 d14/d81/fea 0 2024-05-25T06:11:38.711 INFO:tasks.workunit.client.0.smithi160.stdout:1/357: unlink de/d21/d49/d1c/c33 0 2024-05-25T06:11:38.712 INFO:tasks.workunit.client.1.smithi203.stdout:9/675: creat d0/fe5 x:0 0 0 2024-05-25T06:11:38.714 INFO:tasks.workunit.client.0.smithi160.stdout:9/445: getdents d4/d11/d1e/d4b 0 2024-05-25T06:11:38.715 INFO:tasks.workunit.client.0.smithi160.stdout:9/446: write d4/d11/f38 [655509,128469] 0 2024-05-25T06:11:38.718 INFO:tasks.workunit.client.0.smithi160.stdout:7/515: fdatasync d6/dc/f41 0 2024-05-25T06:11:38.719 INFO:tasks.workunit.client.0.smithi160.stdout:9/447: write d4/d11/d1e/d24/f1b [5589399,125298] 0 2024-05-25T06:11:38.719 INFO:tasks.workunit.client.0.smithi160.stdout:7/516: fdatasync f1 0 2024-05-25T06:11:38.719 INFO:tasks.workunit.client.0.smithi160.stdout:1/358: dwrite de/d21/d49/d1c/d4f/f68 [0,4194304] 0 2024-05-25T06:11:38.721 INFO:tasks.workunit.client.0.smithi160.stdout:7/517: fdatasync d6/d2b/f7a 0 2024-05-25T06:11:38.722 INFO:tasks.workunit.client.1.smithi203.stdout:4/944: mknod d0/d9/d50/d79/d9a/dbe/de0/d11b/c141 0 2024-05-25T06:11:38.724 INFO:tasks.workunit.client.0.smithi160.stdout:9/448: write d4/d11/d1e/d24/fc [2675334,53256] 0 2024-05-25T06:11:38.724 INFO:tasks.workunit.client.0.smithi160.stdout:4/453: symlink d4/d6/d5b/d66/d78/l89 0 2024-05-25T06:11:38.725 INFO:tasks.workunit.client.1.smithi203.stdout:8/853: mkdir d2/d5/d10/d11/d125 0 2024-05-25T06:11:38.729 INFO:tasks.workunit.client.1.smithi203.stdout:8/854: chown d2/d5/d28/d38/d4e/d97/f77 175617 1 2024-05-25T06:11:38.729 INFO:tasks.workunit.client.1.smithi203.stdout:1/681: mknod d14/d23/ceb 0 2024-05-25T06:11:38.731 INFO:tasks.workunit.client.1.smithi203.stdout:0/724: write dd/d3d/d4f/f56 [1034926,129236] 0 2024-05-25T06:11:38.732 INFO:tasks.workunit.client.0.smithi160.stdout:9/449: dwrite d4/d11/d1e/d3e/d25/d27/f77 [0,4194304] 0 2024-05-25T06:11:38.734 INFO:tasks.workunit.client.0.smithi160.stdout:1/359: read de/d21/d49/f27 [915929,17417] 0 2024-05-25T06:11:38.736 INFO:tasks.workunit.client.0.smithi160.stdout:9/450: chown d4/d11/d1e/d3e/d2f/d79 749987 1 2024-05-25T06:11:38.740 INFO:tasks.workunit.client.1.smithi203.stdout:8/855: rename d2/d5/d10/d45/d5d/f82 to d2/d5/d10/d4a/db6/f126 0 2024-05-25T06:11:38.741 INFO:tasks.workunit.client.0.smithi160.stdout:7/518: mknod d6/d32/d7e/ca3 0 2024-05-25T06:11:38.743 INFO:tasks.workunit.client.1.smithi203.stdout:3/926: dwrite d10/d16/d50/f64 [4194304,4194304] 0 2024-05-25T06:11:38.745 INFO:tasks.workunit.client.0.smithi160.stdout:7/519: mknod d6/d2b/d5d/d89/ca4 0 2024-05-25T06:11:38.746 INFO:tasks.workunit.client.1.smithi203.stdout:5/967: dwrite dc/d11/d17/d1a/d2d/d53/fec [0,4194304] 0 2024-05-25T06:11:38.747 INFO:tasks.workunit.client.0.smithi160.stdout:1/360: dread de/d21/d49/d1c/d59/f5d [0,4194304] 0 2024-05-25T06:11:38.747 INFO:tasks.workunit.client.1.smithi203.stdout:0/725: creat dd/d3d/da0/db9/f105 x:0 0 0 2024-05-25T06:11:38.752 INFO:tasks.workunit.client.0.smithi160.stdout:9/451: link d4/d11/d1e/d3e/f74 d4/d11/d12/f85 0 2024-05-25T06:11:38.755 INFO:tasks.workunit.client.1.smithi203.stdout:3/927: fdatasync d10/d22/d2a/f81 0 2024-05-25T06:11:38.756 INFO:tasks.workunit.client.1.smithi203.stdout:5/968: readlink dc/d11/d17/d1a/d2d/d53/da2/l7a 0 2024-05-25T06:11:38.760 INFO:tasks.workunit.client.1.smithi203.stdout:4/945: dread d0/d8/d52/d59/d7a/d83/fd6 [0,4194304] 0 2024-05-25T06:11:38.760 INFO:tasks.workunit.client.1.smithi203.stdout:9/676: dwrite d0/d17/d20/d3c/db2/d7a/fb4 [0,4194304] 0 2024-05-25T06:11:38.761 INFO:tasks.workunit.client.1.smithi203.stdout:3/928: mkdir d10/d16/d23/d67/d12d 0 2024-05-25T06:11:38.761 INFO:tasks.workunit.client.0.smithi160.stdout:9/452: getdents d4/d11/d1e/d3e 0 2024-05-25T06:11:38.763 INFO:tasks.workunit.client.1.smithi203.stdout:5/969: creat dc/d11/f150 x:0 0 0 2024-05-25T06:11:38.765 INFO:tasks.workunit.client.0.smithi160.stdout:8/543: rename d5/d9/f24 to d5/d44/d1f/da3/d28/d4a/fb3 0 2024-05-25T06:11:38.766 INFO:tasks.workunit.client.1.smithi203.stdout:4/946: read d0/d8/d52/f6f [3882209,6242] 0 2024-05-25T06:11:38.769 INFO:tasks.workunit.client.1.smithi203.stdout:9/677: creat d0/df/fe6 x:0 0 0 2024-05-25T06:11:38.770 INFO:tasks.workunit.client.0.smithi160.stdout:1/361: rename de/d21/d49/d1c/d4f to de/d21/d49/d1c/d59/d71 0 2024-05-25T06:11:38.771 INFO:tasks.workunit.client.1.smithi203.stdout:5/970: dread - dc/d11/d17/d1a/d2d/d53/da2/f139 zero size 2024-05-25T06:11:38.773 INFO:tasks.workunit.client.0.smithi160.stdout:8/544: dwrite d5/d44/d1f/da3/d28/f4d [4194304,4194304] 0 2024-05-25T06:11:38.773 INFO:tasks.workunit.client.0.smithi160.stdout:1/362: write de/d43/f46 [607001,89209] 0 2024-05-25T06:11:38.773 INFO:tasks.workunit.client.1.smithi203.stdout:3/929: dwrite d10/d16/d50/fc2 [0,4194304] 0 2024-05-25T06:11:38.775 INFO:tasks.workunit.client.1.smithi203.stdout:4/947: creat d0/d8/d3a/d81/d93/d98/d99/f142 x:0 0 0 2024-05-25T06:11:38.778 INFO:tasks.workunit.client.0.smithi160.stdout:9/453: rename d4/d11/d7a/c33 to d4/d11/d1e/d24/c86 0 2024-05-25T06:11:38.782 INFO:tasks.workunit.client.0.smithi160.stdout:1/363: dwrite f9 [4194304,4194304] 0 2024-05-25T06:11:38.782 INFO:tasks.workunit.client.0.smithi160.stdout:1/364: fdatasync f5 0 2024-05-25T06:11:38.784 INFO:tasks.workunit.client.1.smithi203.stdout:4/948: read - d0/d8/d52/d59/d7a/d83/fd0 zero size 2024-05-25T06:11:38.785 INFO:tasks.workunit.client.0.smithi160.stdout:1/365: truncate de/d21/d49/d1c/d59/d71/f5c 836013 0 2024-05-25T06:11:38.785 INFO:tasks.workunit.client.0.smithi160.stdout:1/366: chown de/d21 239615 1 2024-05-25T06:11:38.787 INFO:tasks.workunit.client.1.smithi203.stdout:8/856: write d2/d5/d28/d38/f49 [618104,122964] 0 2024-05-25T06:11:38.790 INFO:tasks.workunit.client.0.smithi160.stdout:9/454: symlink d4/d11/d1e/d3e/d25/d27/l87 0 2024-05-25T06:11:38.790 INFO:tasks.workunit.client.1.smithi203.stdout:1/682: dread d14/f1a [0,4194304] 0 2024-05-25T06:11:38.790 INFO:tasks.workunit.client.1.smithi203.stdout:3/930: link d10/l15 d10/d16/d23/l12e 0 2024-05-25T06:11:38.792 INFO:tasks.workunit.client.0.smithi160.stdout:1/367: creat de/d39/f72 x:0 0 0 2024-05-25T06:11:38.793 INFO:tasks.workunit.client.1.smithi203.stdout:1/683: dread - d14/d7c/fd3 zero size 2024-05-25T06:11:38.793 INFO:tasks.workunit.client.0.smithi160.stdout:1/368: truncate de/d21/d49/d1c/d59/f5d 416948 0 2024-05-25T06:11:38.794 INFO:tasks.workunit.client.1.smithi203.stdout:4/949: dread d0/d9/d9d/fbf [0,4194304] 0 2024-05-25T06:11:38.796 INFO:tasks.workunit.client.0.smithi160.stdout:9/455: write d4/d11/d12/d2e/f32 [3752806,42797] 0 2024-05-25T06:11:38.797 INFO:tasks.workunit.client.1.smithi203.stdout:1/684: mkdir d14/d57/d97/dec 0 2024-05-25T06:11:38.800 INFO:tasks.workunit.client.1.smithi203.stdout:4/950: mkdir d0/d8/d3a/d81/d143 0 2024-05-25T06:11:38.801 INFO:tasks.workunit.client.0.smithi160.stdout:6/470: write d5/de/d32/d35/f5f [2165422,74854] 0 2024-05-25T06:11:38.802 INFO:tasks.workunit.client.1.smithi203.stdout:1/685: fsync d14/d57/d29/d94/db8/dc2/f61 0 2024-05-25T06:11:38.802 INFO:tasks.workunit.client.1.smithi203.stdout:3/931: link d10/d16/d23/d4a/d88/f9f d10/d16/d23/f12f 0 2024-05-25T06:11:38.806 INFO:tasks.workunit.client.0.smithi160.stdout:6/471: fsync d5/d39/d8f/d2b/f88 0 2024-05-25T06:11:38.806 INFO:tasks.workunit.client.1.smithi203.stdout:4/951: rename d0/d9/d50/d6b/ff6 to d0/d9/d50/d79/d9a/dbe/de0/f144 0 2024-05-25T06:11:38.810 INFO:tasks.workunit.client.1.smithi203.stdout:3/932: mknod d10/d16/d23/d38/c130 0 2024-05-25T06:11:38.810 INFO:tasks.workunit.client.1.smithi203.stdout:8/857: link d2/d5/d18/d31/fcf d2/d5/d18/d31/dac/dc2/d8a/d8f/dfc/f127 0 2024-05-25T06:11:38.811 INFO:tasks.workunit.client.1.smithi203.stdout:8/858: chown d2/d5/d18/d31/dac/dc2/d88 10711034 1 2024-05-25T06:11:38.812 INFO:tasks.workunit.client.1.smithi203.stdout:4/952: symlink d0/d8/d10/l145 0 2024-05-25T06:11:38.813 INFO:tasks.workunit.client.0.smithi160.stdout:0/544: truncate d2b/d46/d63/da7/f85 790118 0 2024-05-25T06:11:38.813 INFO:tasks.workunit.client.0.smithi160.stdout:0/545: write d2b/f42 [3256230,11361] 0 2024-05-25T06:11:38.815 INFO:tasks.workunit.client.1.smithi203.stdout:8/859: mknod d2/d5/d18/d31/c128 0 2024-05-25T06:11:38.815 INFO:tasks.workunit.client.0.smithi160.stdout:7/520: rmdir d6/d32 39 2024-05-25T06:11:38.818 INFO:tasks.workunit.client.0.smithi160.stdout:5/507: creat db/dc/f97 x:0 0 0 2024-05-25T06:11:38.818 INFO:tasks.workunit.client.1.smithi203.stdout:3/933: mkdir d10/d131 0 2024-05-25T06:11:38.819 INFO:tasks.workunit.client.0.smithi160.stdout:2/433: truncate d12/d1c/d2a/f3a 1764746 0 2024-05-25T06:11:38.819 INFO:tasks.workunit.client.1.smithi203.stdout:9/678: dwrite d0/d17/d20/f31 [4194304,4194304] 0 2024-05-25T06:11:38.820 INFO:tasks.workunit.client.0.smithi160.stdout:7/521: mknod d6/d32/d92/ca5 0 2024-05-25T06:11:38.820 INFO:tasks.workunit.client.0.smithi160.stdout:5/508: chown db/dc/d13/d1d 17284 1 2024-05-25T06:11:38.820 INFO:tasks.workunit.client.0.smithi160.stdout:2/434: stat d12/d3c/l54 0 2024-05-25T06:11:38.822 INFO:tasks.workunit.client.0.smithi160.stdout:0/546: dwrite d2b/d46/d63/da7/f60 [0,4194304] 0 2024-05-25T06:11:38.823 INFO:tasks.workunit.client.1.smithi203.stdout:5/971: dwrite dc/d11/d17/d45/f62 [0,4194304] 0 2024-05-25T06:11:38.823 INFO:tasks.workunit.client.1.smithi203.stdout:1/686: link d14/f17 d14/fed 0 2024-05-25T06:11:38.824 INFO:tasks.workunit.client.1.smithi203.stdout:8/860: unlink d2/d5/d18/d31/dac/dc2/d8a/lcd 0 2024-05-25T06:11:38.826 INFO:tasks.workunit.client.0.smithi160.stdout:7/522: dread d6/dc/f82 [0,4194304] 0 2024-05-25T06:11:38.827 INFO:tasks.workunit.client.1.smithi203.stdout:4/953: dwrite d0/d9/d50/fd1 [0,4194304] 0 2024-05-25T06:11:38.828 INFO:tasks.workunit.client.1.smithi203.stdout:1/687: chown d14/d57/d29/d94/db8/c71 1435138529 1 2024-05-25T06:11:38.828 INFO:tasks.workunit.client.0.smithi160.stdout:9/456: dread d4/d11/d1e/d24/f1b [0,4194304] 0 2024-05-25T06:11:38.829 INFO:tasks.workunit.client.0.smithi160.stdout:7/523: write d6/dc/ff [5834856,44792] 0 2024-05-25T06:11:38.830 INFO:tasks.workunit.client.0.smithi160.stdout:2/435: creat d12/d1c/d37/f95 x:0 0 0 2024-05-25T06:11:38.835 INFO:tasks.workunit.client.0.smithi160.stdout:5/509: dwrite db/dc/d13/d1d/d31/d3b/f70 [0,4194304] 0 2024-05-25T06:11:38.835 INFO:tasks.workunit.client.1.smithi203.stdout:8/861: creat d2/d5/d18/d31/dac/dc2/f129 x:0 0 0 2024-05-25T06:11:38.835 INFO:tasks.workunit.client.1.smithi203.stdout:5/972: dread dc/d11/f105 [4194304,4194304] 0 2024-05-25T06:11:38.836 INFO:tasks.workunit.client.0.smithi160.stdout:0/547: mkdir d2b/d31/d35/d36/daf 0 2024-05-25T06:11:38.842 INFO:tasks.workunit.client.0.smithi160.stdout:2/436: dwrite d12/d1c/f2c [0,4194304] 0 2024-05-25T06:11:38.843 INFO:tasks.workunit.client.1.smithi203.stdout:9/679: getdents d0 0 2024-05-25T06:11:38.844 INFO:tasks.workunit.client.1.smithi203.stdout:1/688: link d14/d81/fe5 d14/d57/d6b/da1/fee 0 2024-05-25T06:11:38.846 INFO:tasks.workunit.client.0.smithi160.stdout:9/457: rename d4/d11/d12/d2e/l6d to d4/d11/d1e/d3e/l88 0 2024-05-25T06:11:38.846 INFO:tasks.workunit.client.1.smithi203.stdout:9/680: dread d0/fa6 [0,4194304] 0 2024-05-25T06:11:38.847 INFO:tasks.workunit.client.0.smithi160.stdout:8/545: unlink d5/d44/f25 0 2024-05-25T06:11:38.852 INFO:tasks.workunit.client.1.smithi203.stdout:5/973: rename dc/d11/d42/d94/ff0 to dc/d11/d17/d1a/d103/d98/f151 0 2024-05-25T06:11:38.853 INFO:tasks.workunit.client.0.smithi160.stdout:1/369: creat de/d21/d49/d35/f73 x:0 0 0 2024-05-25T06:11:38.854 INFO:tasks.workunit.client.1.smithi203.stdout:5/974: dread - dc/d11/d42/f14b zero size 2024-05-25T06:11:38.854 INFO:tasks.workunit.client.0.smithi160.stdout:5/510: truncate db/dc/f83 827242 0 2024-05-25T06:11:38.855 INFO:tasks.workunit.client.0.smithi160.stdout:1/370: chown fb 943 1 2024-05-25T06:11:38.856 INFO:tasks.workunit.client.1.smithi203.stdout:9/681: dwrite d0/df/d37/da8/dd6/f9b [0,4194304] 0 2024-05-25T06:11:38.856 INFO:tasks.workunit.client.0.smithi160.stdout:2/437: dwrite d12/d1c/f6f [0,4194304] 0 2024-05-25T06:11:38.856 INFO:tasks.workunit.client.0.smithi160.stdout:8/546: write d5/fb [2109908,113511] 0 2024-05-25T06:11:38.860 INFO:tasks.workunit.client.0.smithi160.stdout:0/548: rename d2b/d31/f38 to d2b/da8/fb0 0 2024-05-25T06:11:38.861 INFO:tasks.workunit.client.1.smithi203.stdout:1/689: link d14/d81/cdf d14/d57/db7/cef 0 2024-05-25T06:11:38.863 INFO:tasks.workunit.client.0.smithi160.stdout:1/371: fdatasync de/d21/d49/d1c/d59/d71/f68 0 2024-05-25T06:11:38.864 INFO:tasks.workunit.client.0.smithi160.stdout:9/458: symlink d4/d11/d1e/d24/d66/l89 0 2024-05-25T06:11:38.864 INFO:tasks.workunit.client.0.smithi160.stdout:3/409: rmdir d19/d22 39 2024-05-25T06:11:38.868 INFO:tasks.workunit.client.0.smithi160.stdout:8/547: fdatasync d5/d39/d75/f76 0 2024-05-25T06:11:38.868 INFO:tasks.workunit.client.1.smithi203.stdout:1/690: mknod d14/d57/d97/da4/dad/dd9/cf0 0 2024-05-25T06:11:38.873 INFO:tasks.workunit.client.0.smithi160.stdout:4/454: sync 2024-05-25T06:11:38.874 INFO:tasks.workunit.client.0.smithi160.stdout:1/372: readlink de/d21/d49/d28/l48 0 2024-05-25T06:11:38.875 INFO:tasks.workunit.client.0.smithi160.stdout:4/455: readlink d4/d63/d64/l73 0 2024-05-25T06:11:38.876 INFO:tasks.workunit.client.1.smithi203.stdout:4/954: write d0/d8/d1f/d26/fc9 [434563,38886] 0 2024-05-25T06:11:38.876 INFO:tasks.workunit.client.0.smithi160.stdout:8/548: creat d5/d44/d1f/da3/d28/d49/fb4 x:0 0 0 2024-05-25T06:11:38.877 INFO:tasks.workunit.client.1.smithi203.stdout:8/862: write d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/ff5 [187791,86700] 0 2024-05-25T06:11:38.878 INFO:tasks.workunit.client.0.smithi160.stdout:6/472: write d5/d45/f9a [1148090,28792] 0 2024-05-25T06:11:38.880 INFO:tasks.workunit.client.0.smithi160.stdout:3/410: dwrite d19/d2f/d38/d47/f57 [0,4194304] 0 2024-05-25T06:11:38.881 INFO:tasks.workunit.client.1.smithi203.stdout:1/691: write d14/f32 [1810940,121416] 0 2024-05-25T06:11:38.883 INFO:tasks.workunit.client.0.smithi160.stdout:2/438: dwrite d12/d3e/d5f/d74/f77 [4194304,4194304] 0 2024-05-25T06:11:38.884 INFO:tasks.workunit.client.0.smithi160.stdout:4/456: mknod d4/d6/d5b/d66/c8a 0 2024-05-25T06:11:38.885 INFO:tasks.workunit.client.1.smithi203.stdout:1/692: fdatasync d14/d23/f78 0 2024-05-25T06:11:38.886 INFO:tasks.workunit.client.1.smithi203.stdout:3/934: dread d10/d16/d23/d67/f94 [0,4194304] 0 2024-05-25T06:11:38.888 INFO:tasks.workunit.client.0.smithi160.stdout:6/473: mkdir d5/d39/da6 0 2024-05-25T06:11:38.888 INFO:tasks.workunit.client.0.smithi160.stdout:6/474: chown d5/l19 470625 1 2024-05-25T06:11:38.891 INFO:tasks.workunit.client.0.smithi160.stdout:1/373: creat de/d21/d49/d1c/d59/d71/f74 x:0 0 0 2024-05-25T06:11:38.892 INFO:tasks.workunit.client.0.smithi160.stdout:7/524: dread d6/d32/f42 [0,4194304] 0 2024-05-25T06:11:38.893 INFO:tasks.workunit.client.1.smithi203.stdout:3/935: stat d10/d16/d23/d4a/d88/f95 0 2024-05-25T06:11:38.893 INFO:tasks.workunit.client.1.smithi203.stdout:1/693: symlink d14/d7c/lf1 0 2024-05-25T06:11:38.894 INFO:tasks.workunit.client.0.smithi160.stdout:4/457: dread d4/d63/d64/f80 [0,4194304] 0 2024-05-25T06:11:38.895 INFO:tasks.workunit.client.0.smithi160.stdout:5/511: dread db/dc/d13/f1f [0,4194304] 0 2024-05-25T06:11:38.895 INFO:tasks.workunit.client.0.smithi160.stdout:4/458: chown d4/d1a/c1b 0 1 2024-05-25T06:11:38.896 INFO:tasks.workunit.client.1.smithi203.stdout:9/682: dwrite d0/df/d37/f79 [0,4194304] 0 2024-05-25T06:11:38.896 INFO:tasks.workunit.client.0.smithi160.stdout:4/459: chown d4/d1c/d54/f65 18144 1 2024-05-25T06:11:38.897 INFO:tasks.workunit.client.1.smithi203.stdout:3/936: fsync d10/d16/d23/d67/fb7 0 2024-05-25T06:11:38.901 INFO:tasks.workunit.client.0.smithi160.stdout:8/549: mkdir d5/d7/d9a/db5 0 2024-05-25T06:11:38.901 INFO:tasks.workunit.client.1.smithi203.stdout:1/694: write d14/d57/d6b/da1/de2/f76 [805284,124858] 0 2024-05-25T06:11:38.903 INFO:tasks.workunit.client.1.smithi203.stdout:8/863: dwrite d2/d5/d10/dfa/f11d [0,4194304] 0 2024-05-25T06:11:38.904 INFO:tasks.workunit.client.0.smithi160.stdout:6/475: mkdir d5/d45/d63/da7 0 2024-05-25T06:11:38.905 INFO:tasks.workunit.client.1.smithi203.stdout:4/955: dwrite d0/d8/d3a/d81/d93/d98/d99/f10d [0,4194304] 0 2024-05-25T06:11:38.906 INFO:tasks.workunit.client.1.smithi203.stdout:3/937: rmdir d10/dea 39 2024-05-25T06:11:38.906 INFO:tasks.workunit.client.1.smithi203.stdout:8/864: fsync d2/d5/d10/d11/d59/fd4 0 2024-05-25T06:11:38.908 INFO:tasks.workunit.client.1.smithi203.stdout:4/956: readlink d0/d8/d52/d59/le6 0 2024-05-25T06:11:38.909 INFO:tasks.workunit.client.0.smithi160.stdout:1/374: mknod de/c75 0 2024-05-25T06:11:38.910 INFO:tasks.workunit.client.1.smithi203.stdout:4/957: chown d0/d9/d50/d79/d9a/dbe/de0 126128 1 2024-05-25T06:11:38.910 INFO:tasks.workunit.client.1.smithi203.stdout:9/683: creat d0/d17/d20/d3c/db2/fe7 x:0 0 0 2024-05-25T06:11:38.914 INFO:tasks.workunit.client.1.smithi203.stdout:1/695: creat d14/d57/d97/ff2 x:0 0 0 2024-05-25T06:11:38.916 INFO:tasks.workunit.client.1.smithi203.stdout:8/865: mknod d2/d5/d10/dfa/c12a 0 2024-05-25T06:11:38.917 INFO:tasks.workunit.client.1.smithi203.stdout:9/684: dread d0/df/d37/d4f/f80 [0,4194304] 0 2024-05-25T06:11:38.918 INFO:tasks.workunit.client.0.smithi160.stdout:2/439: dread f11 [0,4194304] 0 2024-05-25T06:11:38.925 INFO:tasks.workunit.client.1.smithi203.stdout:4/958: link d0/d8/d3a/d81/d93/d98/cc8 d0/d8/d3a/c146 0 2024-05-25T06:11:38.926 INFO:tasks.workunit.client.1.smithi203.stdout:1/696: dread d14/f17 [0,4194304] 0 2024-05-25T06:11:38.927 INFO:tasks.workunit.client.0.smithi160.stdout:6/476: fdatasync d5/de/f12 0 2024-05-25T06:11:38.927 INFO:tasks.workunit.client.1.smithi203.stdout:1/697: dread - d14/d7c/fd3 zero size 2024-05-25T06:11:38.928 INFO:tasks.workunit.client.0.smithi160.stdout:6/477: dread - d5/de/d58/f80 zero size 2024-05-25T06:11:38.929 INFO:tasks.workunit.client.0.smithi160.stdout:6/478: write d5/d39/f6c [951684,24185] 0 2024-05-25T06:11:38.929 INFO:tasks.workunit.client.1.smithi203.stdout:1/698: stat d14/d57/db7/dd2/db3/c74 0 2024-05-25T06:11:38.933 INFO:tasks.workunit.client.1.smithi203.stdout:5/975: dread dc/d11/f3e [0,4194304] 0 2024-05-25T06:11:38.933 INFO:tasks.workunit.client.0.smithi160.stdout:3/411: rename d19/f2b to d19/d2f/f94 0 2024-05-25T06:11:38.934 INFO:tasks.workunit.client.1.smithi203.stdout:5/976: rename dc/d11 to dc/d11/d18/dc0/d107/d152 22 2024-05-25T06:11:38.934 INFO:tasks.workunit.client.0.smithi160.stdout:8/550: creat d5/d44/d1f/fb6 x:0 0 0 2024-05-25T06:11:38.935 INFO:tasks.workunit.client.1.smithi203.stdout:5/977: creat dc/d3c/f153 x:0 0 0 2024-05-25T06:11:38.937 INFO:tasks.workunit.client.0.smithi160.stdout:6/479: truncate d5/d39/d8f/f55 5670398 0 2024-05-25T06:11:38.938 INFO:tasks.workunit.client.1.smithi203.stdout:2/801: sync 2024-05-25T06:11:38.938 INFO:tasks.workunit.client.1.smithi203.stdout:7/739: sync 2024-05-25T06:11:38.940 INFO:tasks.workunit.client.0.smithi160.stdout:5/512: rename db/d96 to db/dc/d13/d1d/d3f/d72/d98 0 2024-05-25T06:11:38.941 INFO:tasks.workunit.client.0.smithi160.stdout:0/549: dread d2b/f57 [0,4194304] 0 2024-05-25T06:11:38.942 INFO:tasks.workunit.client.0.smithi160.stdout:3/412: creat d19/d2f/d34/f95 x:0 0 0 2024-05-25T06:11:38.942 INFO:tasks.workunit.client.1.smithi203.stdout:8/866: dread d2/d5/d28/f4c [0,4194304] 0 2024-05-25T06:11:38.943 INFO:tasks.workunit.client.0.smithi160.stdout:0/550: truncate d2b/d31/d35/f49 1886681 0 2024-05-25T06:11:38.943 INFO:tasks.workunit.client.1.smithi203.stdout:7/740: symlink d1/d9/d2c/d3a/lfa 0 2024-05-25T06:11:38.945 INFO:tasks.workunit.client.1.smithi203.stdout:8/867: fsync d2/d5/d28/d38/d4e/fbb 0 2024-05-25T06:11:38.948 INFO:tasks.workunit.client.1.smithi203.stdout:9/685: write d0/df/d73/f87 [882509,9153] 0 2024-05-25T06:11:38.948 INFO:tasks.workunit.client.1.smithi203.stdout:4/959: truncate d0/d8/d52/d59/d7a/d83/fd6 193077 0 2024-05-25T06:11:38.951 INFO:tasks.workunit.client.1.smithi203.stdout:1/699: write d14/d57/d7e/fa7 [2547144,61107] 0 2024-05-25T06:11:38.954 INFO:tasks.workunit.client.1.smithi203.stdout:5/978: write dc/d11/ffc [147023,15925] 0 2024-05-25T06:11:38.956 INFO:tasks.workunit.client.1.smithi203.stdout:4/960: readlink d0/d8/d1f/d26/d3b/ldc 0 2024-05-25T06:11:38.956 INFO:tasks.workunit.client.1.smithi203.stdout:8/868: mkdir d2/d5/d10/d45/d5d/d12b 0 2024-05-25T06:11:38.959 INFO:tasks.workunit.client.1.smithi203.stdout:3/938: dwrite d10/d16/d50/d7d/f9d [0,4194304] 0 2024-05-25T06:11:38.962 INFO:tasks.workunit.client.1.smithi203.stdout:7/741: dwrite d1/d9/f29 [4194304,4194304] 0 2024-05-25T06:11:38.965 INFO:tasks.workunit.client.1.smithi203.stdout:1/700: mkdir d14/d7c/d9d/dc8/df3 0 2024-05-25T06:11:38.966 INFO:tasks.workunit.client.1.smithi203.stdout:2/802: dread d2/d38/d2f/ff0 [0,4194304] 0 2024-05-25T06:11:38.972 INFO:tasks.workunit.client.1.smithi203.stdout:9/686: dwrite d0/df/f4a [0,4194304] 0 2024-05-25T06:11:38.973 INFO:tasks.workunit.client.1.smithi203.stdout:2/803: chown d2/l55 13211 1 2024-05-25T06:11:38.974 INFO:tasks.workunit.client.1.smithi203.stdout:3/939: fsync d10/d16/d23/d4a/d88/f9f 0 2024-05-25T06:11:38.975 INFO:tasks.workunit.client.1.smithi203.stdout:8/869: symlink d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/l12c 0 2024-05-25T06:11:38.979 INFO:tasks.workunit.client.1.smithi203.stdout:1/701: symlink d14/d57/d97/da4/dad/dd9/lf4 0 2024-05-25T06:11:38.980 INFO:tasks.workunit.client.1.smithi203.stdout:7/742: rename d1/d9/d2c/f56 to d1/d9/d2c/d36/d54/ffb 0 2024-05-25T06:11:38.982 INFO:tasks.workunit.client.1.smithi203.stdout:2/804: read d2/d6/f26 [7668206,68604] 0 2024-05-25T06:11:38.985 INFO:tasks.workunit.client.1.smithi203.stdout:8/870: write d2/d5/d10/d11/de/f1e [3470322,98767] 0 2024-05-25T06:11:38.985 INFO:tasks.workunit.client.1.smithi203.stdout:4/961: dwrite d0/d8/f23 [4194304,4194304] 0 2024-05-25T06:11:38.986 INFO:tasks.workunit.client.1.smithi203.stdout:3/940: rename d10/d16/d23/d46 to d10/d16/d50/d7d/d132 0 2024-05-25T06:11:38.990 INFO:tasks.workunit.client.1.smithi203.stdout:5/979: dwrite dc/d11/d17/d1a/d75/ddb/f132 [0,4194304] 0 2024-05-25T06:11:38.990 INFO:tasks.workunit.client.1.smithi203.stdout:1/702: creat d14/d57/d6b/da1/de2/ff5 x:0 0 0 2024-05-25T06:11:38.992 INFO:tasks.workunit.client.1.smithi203.stdout:9/687: read d0/fb [453140,128886] 0 2024-05-25T06:11:38.992 INFO:tasks.workunit.client.1.smithi203.stdout:8/871: creat d2/d5/d18/d31/dac/dc2/d88/d93/f12d x:0 0 0 2024-05-25T06:11:38.992 INFO:tasks.workunit.client.1.smithi203.stdout:2/805: dread d2/d1a/f97 [0,4194304] 0 2024-05-25T06:11:38.994 INFO:tasks.workunit.client.1.smithi203.stdout:3/941: creat d10/dea/f133 x:0 0 0 2024-05-25T06:11:38.995 INFO:tasks.workunit.client.1.smithi203.stdout:5/980: creat dc/d3c/f154 x:0 0 0 2024-05-25T06:11:38.996 INFO:tasks.workunit.client.0.smithi160.stdout:6/480: creat d5/d45/d4a/fa8 x:0 0 0 2024-05-25T06:11:38.996 INFO:tasks.workunit.client.1.smithi203.stdout:1/703: symlink d14/d57/d97/da4/lf6 0 2024-05-25T06:11:38.997 INFO:tasks.workunit.client.0.smithi160.stdout:5/513: creat db/d47/d5e/f99 x:0 0 0 2024-05-25T06:11:38.998 INFO:tasks.workunit.client.1.smithi203.stdout:5/981: fsync dc/d11/d17/d1a/d2d/d4d/ded/d13f/dd4/f147 0 2024-05-25T06:11:38.998 INFO:tasks.workunit.client.1.smithi203.stdout:9/688: dread d0/d17/d20/d3c/db2/d7a/fb4 [0,4194304] 0 2024-05-25T06:11:38.999 INFO:tasks.workunit.client.0.smithi160.stdout:5/514: stat f4 0 2024-05-25T06:11:38.999 INFO:tasks.workunit.client.1.smithi203.stdout:5/982: read - dc/daa/fcf zero size 2024-05-25T06:11:39.000 INFO:tasks.workunit.client.0.smithi160.stdout:7/525: write d6/dc/d64/f38 [670790,68137] 0 2024-05-25T06:11:39.000 INFO:tasks.workunit.client.0.smithi160.stdout:2/440: write d12/d1c/f56 [877474,61617] 0 2024-05-25T06:11:39.001 INFO:tasks.workunit.client.0.smithi160.stdout:6/481: rename d5/de/d32/d35 to d5/de/d32/d35/d40/da9 22 2024-05-25T06:11:39.002 INFO:tasks.workunit.client.0.smithi160.stdout:0/551: link f15 d2b/d46/d63/fb1 0 2024-05-25T06:11:39.003 INFO:tasks.workunit.client.0.smithi160.stdout:9/459: link d4/d11/d1e/d3e/d2f/d79/l84 d4/d11/d1e/d3e/d25/d27/l8a 0 2024-05-25T06:11:39.003 INFO:tasks.workunit.client.0.smithi160.stdout:5/515: dread - db/dc/d13/d54/f90 zero size 2024-05-25T06:11:39.007 INFO:tasks.workunit.client.0.smithi160.stdout:3/413: dwrite d19/d2f/d38/d43/d5b/d65/d7f/f81 [0,4194304] 0 2024-05-25T06:11:39.012 INFO:tasks.workunit.client.0.smithi160.stdout:1/375: dwrite de/d21/d49/d1c/d59/d71/f5c [0,4194304] 0 2024-05-25T06:11:39.013 INFO:tasks.workunit.client.1.smithi203.stdout:2/806: write d2/d1a/f97 [858643,105372] 0 2024-05-25T06:11:39.015 INFO:tasks.workunit.client.1.smithi203.stdout:7/743: creat d1/d9/d2c/d3a/dd1/ffc x:0 0 0 2024-05-25T06:11:39.017 INFO:tasks.workunit.client.1.smithi203.stdout:8/872: write d2/d5/d18/d31/dac/dc2/d8a/d8f/ff4 [510344,72461] 0 2024-05-25T06:11:39.019 INFO:tasks.workunit.client.0.smithi160.stdout:0/552: dwrite d2b/d46/d63/da7/f8e [0,4194304] 0 2024-05-25T06:11:39.019 INFO:tasks.workunit.client.1.smithi203.stdout:8/873: chown d2/d5/d18/d31/dac/dc2/d88/fea 2 1 2024-05-25T06:11:39.025 INFO:tasks.workunit.client.1.smithi203.stdout:5/983: link dc/d11/d17/d1a/d103/cd5 dc/d11/d42/d94/dad/db1/c155 0 2024-05-25T06:11:39.025 INFO:tasks.workunit.client.1.smithi203.stdout:1/704: getdents d14/d57/d6b/da1/de2 0 2024-05-25T06:11:39.027 INFO:tasks.workunit.client.1.smithi203.stdout:3/942: dwrite d10/fa2 [0,4194304] 0 2024-05-25T06:11:39.028 INFO:tasks.workunit.client.1.smithi203.stdout:5/984: fsync dc/d11/d17/d1a/d2d/d53/da2/f118 0 2024-05-25T06:11:39.031 INFO:tasks.workunit.client.0.smithi160.stdout:0/553: dwrite d2b/d46/d63/da7/f60 [0,4194304] 0 2024-05-25T06:11:39.032 INFO:tasks.workunit.client.1.smithi203.stdout:9/689: dwrite d0/d17/d20/d29/f32 [0,4194304] 0 2024-05-25T06:11:39.033 INFO:tasks.workunit.client.1.smithi203.stdout:1/705: unlink d14/d81/fea 0 2024-05-25T06:11:39.035 INFO:tasks.workunit.client.0.smithi160.stdout:6/482: dwrite d5/d45/d4a/f76 [0,4194304] 0 2024-05-25T06:11:39.038 INFO:tasks.workunit.client.1.smithi203.stdout:2/807: link d2/d38/d43/d60/d6b/dae/c94 d2/d42/d4e/d68/c110 0 2024-05-25T06:11:39.038 INFO:tasks.workunit.client.0.smithi160.stdout:4/460: sync 2024-05-25T06:11:39.039 INFO:tasks.workunit.client.1.smithi203.stdout:8/874: rename d2/d5/d28/d38/c6e to d2/d5/d10/d45/c12e 0 2024-05-25T06:11:39.040 INFO:tasks.workunit.client.0.smithi160.stdout:4/461: write d4/f2a [4540612,5526] 0 2024-05-25T06:11:39.045 INFO:tasks.workunit.client.1.smithi203.stdout:3/943: write d10/d16/d23/d45/f93 [1447295,130315] 0 2024-05-25T06:11:39.046 INFO:tasks.workunit.client.1.smithi203.stdout:9/690: fsync d0/d17/d23/f90 0 2024-05-25T06:11:39.047 INFO:tasks.workunit.client.1.smithi203.stdout:1/706: fsync d14/d57/db7/dd2/f8d 0 2024-05-25T06:11:39.048 INFO:tasks.workunit.client.1.smithi203.stdout:3/944: chown d10/d16/d23/d67/fae 1 1 2024-05-25T06:11:39.049 INFO:tasks.workunit.client.1.smithi203.stdout:3/945: fsync d10/fb5 0 2024-05-25T06:11:39.050 INFO:tasks.workunit.client.1.smithi203.stdout:2/808: fdatasync d2/d1a/dd3/d100/d10/d5f/f5b 0 2024-05-25T06:11:39.050 INFO:tasks.workunit.client.1.smithi203.stdout:2/809: readlink d2/d1a/l59 0 2024-05-25T06:11:39.051 INFO:tasks.workunit.client.1.smithi203.stdout:8/875: mknod d2/d5/d10/d11/d1f/d75/c12f 0 2024-05-25T06:11:39.052 INFO:tasks.workunit.client.1.smithi203.stdout:5/985: symlink dc/d11/d17/d1a/d2d/d53/dc9/l156 0 2024-05-25T06:11:39.054 INFO:tasks.workunit.client.1.smithi203.stdout:1/707: readlink d14/d57/d1d/l8b 0 2024-05-25T06:11:39.056 INFO:tasks.workunit.client.1.smithi203.stdout:1/708: write d14/d57/d6b/da1/de2/f76 [1231982,14982] 0 2024-05-25T06:11:39.056 INFO:tasks.workunit.client.1.smithi203.stdout:9/691: unlink d0/df/d37/da8/dd6/f9b 0 2024-05-25T06:11:39.060 INFO:tasks.workunit.client.1.smithi203.stdout:2/810: rename d2/cb7 to d2/d38/d2f/c111 0 2024-05-25T06:11:39.061 INFO:tasks.workunit.client.1.smithi203.stdout:5/986: fsync dc/d11/d17/d44/f51 0 2024-05-25T06:11:39.062 INFO:tasks.workunit.client.1.smithi203.stdout:8/876: truncate d2/d5/d2d/fe5 23145 0 2024-05-25T06:11:39.064 INFO:tasks.workunit.client.1.smithi203.stdout:1/709: chown d14/d57/d1d/l48 1479288 1 2024-05-25T06:11:39.064 INFO:tasks.workunit.client.1.smithi203.stdout:9/692: fdatasync d0/df/d2d/d92/fc2 0 2024-05-25T06:11:39.065 INFO:tasks.workunit.client.1.smithi203.stdout:3/946: symlink d10/d16/d23/d38/d11e/l134 0 2024-05-25T06:11:39.065 INFO:tasks.workunit.client.1.smithi203.stdout:2/811: mkdir d2/d1a/d33/d112 0 2024-05-25T06:11:39.066 INFO:tasks.workunit.client.1.smithi203.stdout:5/987: mkdir dc/d11/d17/d1a/d2d/d4d/dcc/d157 0 2024-05-25T06:11:39.070 INFO:tasks.workunit.client.1.smithi203.stdout:5/988: dread dc/d11/ffc [0,4194304] 0 2024-05-25T06:11:39.070 INFO:tasks.workunit.client.1.smithi203.stdout:1/710: mkdir d14/d57/db7/dd2/dca/dcf/df7 0 2024-05-25T06:11:39.072 INFO:tasks.workunit.client.1.smithi203.stdout:7/744: truncate d1/d9/d2c/d79/f81 3204314 0 2024-05-25T06:11:39.072 INFO:tasks.workunit.client.1.smithi203.stdout:2/812: dwrite d2/d38/f6e [0,4194304] 0 2024-05-25T06:11:39.073 INFO:tasks.workunit.client.1.smithi203.stdout:9/693: creat d0/d17/d20/d29/fe8 x:0 0 0 2024-05-25T06:11:39.073 INFO:tasks.workunit.client.0.smithi160.stdout:8/551: sync 2024-05-25T06:11:39.074 INFO:tasks.workunit.client.1.smithi203.stdout:7/745: write d1/d9/f14 [2366947,39895] 0 2024-05-25T06:11:39.074 INFO:tasks.workunit.client.1.smithi203.stdout:5/989: dread - dc/d11/d42/fa9 zero size 2024-05-25T06:11:39.075 INFO:tasks.workunit.client.1.smithi203.stdout:2/813: chown d2/d1a/dd3/d100/l3b 944118102 1 2024-05-25T06:11:39.077 INFO:tasks.workunit.client.1.smithi203.stdout:9/694: mknod d0/df/ddd/ce9 0 2024-05-25T06:11:39.078 INFO:tasks.workunit.client.1.smithi203.stdout:9/695: readlink d0/l2 0 2024-05-25T06:11:39.078 INFO:tasks.workunit.client.1.smithi203.stdout:1/711: truncate d14/d57/d7e/fa7 2181301 0 2024-05-25T06:11:39.079 INFO:tasks.workunit.client.1.smithi203.stdout:2/814: mkdir d2/d1a/d89/daa/def/d107/d113 0 2024-05-25T06:11:39.081 INFO:tasks.workunit.client.1.smithi203.stdout:9/696: rmdir d0/d17/d20/d6e 39 2024-05-25T06:11:39.082 INFO:tasks.workunit.client.1.smithi203.stdout:1/712: mkdir d14/d57/dbb/df8 0 2024-05-25T06:11:39.083 INFO:tasks.workunit.client.1.smithi203.stdout:2/815: symlink d2/d38/d43/d60/d6b/dae/l114 0 2024-05-25T06:11:39.085 INFO:tasks.workunit.client.1.smithi203.stdout:0/726: sync 2024-05-25T06:11:39.089 INFO:tasks.workunit.client.1.smithi203.stdout:9/697: dwrite d0/d17/d20/f31 [4194304,4194304] 0 2024-05-25T06:11:39.092 INFO:tasks.workunit.client.1.smithi203.stdout:2/816: dwrite d2/d38/f6e [0,4194304] 0 2024-05-25T06:11:39.098 INFO:tasks.workunit.client.1.smithi203.stdout:9/698: creat d0/d17/d20/d3c/fea x:0 0 0 2024-05-25T06:11:39.098 INFO:tasks.workunit.client.1.smithi203.stdout:9/699: chown d0/df/d2d/ld3 3 1 2024-05-25T06:11:39.099 INFO:tasks.workunit.client.1.smithi203.stdout:2/817: creat d2/d38/d2f/f115 x:0 0 0 2024-05-25T06:11:39.108 INFO:tasks.workunit.client.1.smithi203.stdout:8/877: dwrite d2/d5/d2d/f4b [4194304,4194304] 0 2024-05-25T06:11:39.112 INFO:tasks.workunit.client.1.smithi203.stdout:0/727: dwrite dd/d32/d37/f52 [0,4194304] 0 2024-05-25T06:11:39.119 INFO:tasks.workunit.client.1.smithi203.stdout:8/878: mknod d2/d5/d10/d11/d1f/c130 0 2024-05-25T06:11:39.122 INFO:tasks.workunit.client.1.smithi203.stdout:1/713: dwrite d14/fed [0,4194304] 0 2024-05-25T06:11:39.123 INFO:tasks.workunit.client.1.smithi203.stdout:2/818: dwrite d2/d38/d2f/f10a [0,4194304] 0 2024-05-25T06:11:39.128 INFO:tasks.workunit.client.1.smithi203.stdout:1/714: creat d14/d57/db7/dd2/dca/ff9 x:0 0 0 2024-05-25T06:11:39.131 INFO:tasks.workunit.client.1.smithi203.stdout:9/700: dwrite d0/dd0/fb1 [0,4194304] 0 2024-05-25T06:11:39.131 INFO:tasks.workunit.client.1.smithi203.stdout:7/746: dread d1/d9/d2c/d79/f81 [0,4194304] 0 2024-05-25T06:11:39.133 INFO:tasks.workunit.client.0.smithi160.stdout:1/376: rename de/d21/d49/d1c/f47 to de/d21/d49/d28/f76 0 2024-05-25T06:11:39.133 INFO:tasks.workunit.client.1.smithi203.stdout:8/879: rmdir d2/d5/d18/d31/dac/dc2 39 2024-05-25T06:11:39.134 INFO:tasks.workunit.client.1.smithi203.stdout:1/715: chown d14/d57/db7/dd2/dca/dcf 361262181 1 2024-05-25T06:11:39.138 INFO:tasks.workunit.client.1.smithi203.stdout:2/819: mkdir d2/d42/de5/d116 0 2024-05-25T06:11:39.140 INFO:tasks.workunit.client.0.smithi160.stdout:9/460: fsync d4/d11/d1e/f4d 0 2024-05-25T06:11:39.141 INFO:tasks.workunit.client.1.smithi203.stdout:8/880: truncate d2/d5/d10/d11/fc8 977369 0 2024-05-25T06:11:39.145 INFO:tasks.workunit.client.0.smithi160.stdout:5/516: mkdir db/dc/d13/d9a 0 2024-05-25T06:11:39.146 INFO:tasks.workunit.client.1.smithi203.stdout:7/747: link d1/d9/d60/ca8 d1/dbd/dc1/cfd 0 2024-05-25T06:11:39.146 INFO:tasks.workunit.client.1.smithi203.stdout:8/881: write d2/d5/d10/d45/d5d/d68/f73 [2164870,16999] 0 2024-05-25T06:11:39.147 INFO:tasks.workunit.client.1.smithi203.stdout:1/716: getdents d14/d23 0 2024-05-25T06:11:39.147 INFO:tasks.workunit.client.0.smithi160.stdout:0/554: mkdir d2b/da8/db2 0 2024-05-25T06:11:39.148 INFO:tasks.workunit.client.0.smithi160.stdout:0/555: chown d2b/d46/d63/da7/da6/da9 251154 1 2024-05-25T06:11:39.150 INFO:tasks.workunit.client.0.smithi160.stdout:6/483: unlink d5/d30/d36/c2e 0 2024-05-25T06:11:39.151 INFO:tasks.workunit.client.0.smithi160.stdout:4/462: creat d4/d1a/f8b x:0 0 0 2024-05-25T06:11:39.152 INFO:tasks.workunit.client.0.smithi160.stdout:7/526: write d6/dc/d21/f2a [2585724,97239] 0 2024-05-25T06:11:39.152 INFO:tasks.workunit.client.0.smithi160.stdout:4/463: chown d4/f3b 657 1 2024-05-25T06:11:39.153 INFO:tasks.workunit.client.0.smithi160.stdout:4/464: write d4/d6/d5b/d66/f71 [2130517,20058] 0 2024-05-25T06:11:39.154 INFO:tasks.workunit.client.0.smithi160.stdout:1/377: mkdir de/d21/d49/d1c/d52/d77 0 2024-05-25T06:11:39.156 INFO:tasks.workunit.client.0.smithi160.stdout:9/461: creat d4/d11/d1e/d3e/d25/f8b x:0 0 0 2024-05-25T06:11:39.158 INFO:tasks.workunit.client.0.smithi160.stdout:2/441: dwrite d12/f25 [0,4194304] 0 2024-05-25T06:11:39.158 INFO:tasks.workunit.client.0.smithi160.stdout:0/556: mkdir d2b/d46/d63/db3 0 2024-05-25T06:11:39.159 INFO:tasks.workunit.client.0.smithi160.stdout:2/442: chown d12/d1c/c1f 832460987 1 2024-05-25T06:11:39.159 INFO:tasks.workunit.client.0.smithi160.stdout:0/557: fsync d2b/d31/d35/d36/d86/f9b 0 2024-05-25T06:11:39.162 INFO:tasks.workunit.client.0.smithi160.stdout:8/552: creat d5/d44/d1f/d58/d65/da2/fb7 x:0 0 0 2024-05-25T06:11:39.163 INFO:tasks.workunit.client.0.smithi160.stdout:7/527: creat d6/dc/d64/d5a/fa6 x:0 0 0 2024-05-25T06:11:39.163 INFO:tasks.workunit.client.0.smithi160.stdout:0/558: write d2b/f2e [791783,103972] 0 2024-05-25T06:11:39.164 INFO:tasks.workunit.client.0.smithi160.stdout:4/465: mknod d4/d6/d5b/d66/d82/c8c 0 2024-05-25T06:11:39.164 INFO:tasks.workunit.client.0.smithi160.stdout:7/528: fsync d6/dc/d21/f71 0 2024-05-25T06:11:39.167 INFO:tasks.workunit.client.1.smithi203.stdout:8/882: write d2/d5/d28/f7a [2735345,112343] 0 2024-05-25T06:11:39.168 INFO:tasks.workunit.client.1.smithi203.stdout:7/748: write d1/d9/d60/f65 [2823872,119495] 0 2024-05-25T06:11:39.169 INFO:tasks.workunit.client.0.smithi160.stdout:9/462: mknod d4/d11/d7a/c8c 0 2024-05-25T06:11:39.169 INFO:tasks.workunit.client.0.smithi160.stdout:3/414: truncate d19/d2f/f94 896497 0 2024-05-25T06:11:39.171 INFO:tasks.workunit.client.0.smithi160.stdout:9/463: stat d4/d11/d1e/d3e/d25/d27/l87 0 2024-05-25T06:11:39.171 INFO:tasks.workunit.client.1.smithi203.stdout:9/701: dwrite d0/d17/f4b [0,4194304] 0 2024-05-25T06:11:39.172 INFO:tasks.workunit.client.0.smithi160.stdout:9/464: write d4/d11/d7a/f7b [226583,111074] 0 2024-05-25T06:11:39.173 INFO:tasks.workunit.client.0.smithi160.stdout:2/443: mknod d12/d3e/d5f/d74/d7a/c96 0 2024-05-25T06:11:39.174 INFO:tasks.workunit.client.1.smithi203.stdout:2/820: dread d2/fb8 [0,4194304] 0 2024-05-25T06:11:39.175 INFO:tasks.workunit.client.1.smithi203.stdout:7/749: fsync d1/d9/d2c/d36/f40 0 2024-05-25T06:11:39.179 INFO:tasks.workunit.client.0.smithi160.stdout:7/529: creat d6/d2b/d5d/d89/fa7 x:0 0 0 2024-05-25T06:11:39.180 INFO:tasks.workunit.client.1.smithi203.stdout:2/821: creat d2/d38/d2f/d69/f117 x:0 0 0 2024-05-25T06:11:39.181 INFO:tasks.workunit.client.0.smithi160.stdout:9/465: write d4/d11/f67 [220549,76646] 0 2024-05-25T06:11:39.182 INFO:tasks.workunit.client.1.smithi203.stdout:7/750: dread d1/d9/fb4 [0,4194304] 0 2024-05-25T06:11:39.185 INFO:tasks.workunit.client.1.smithi203.stdout:8/883: write d2/d5/d10/d45/d5d/d76/f8b [1138540,68624] 0 2024-05-25T06:11:39.185 INFO:tasks.workunit.client.0.smithi160.stdout:6/484: getdents d5/d45/d4a 0 2024-05-25T06:11:39.186 INFO:tasks.workunit.client.1.smithi203.stdout:9/702: write d0/df/d37/f70 [1126306,33209] 0 2024-05-25T06:11:39.189 INFO:tasks.workunit.client.0.smithi160.stdout:2/444: truncate d12/f7b 505798 0 2024-05-25T06:11:39.189 INFO:tasks.workunit.client.1.smithi203.stdout:9/703: chown d0/df/d73 464637 1 2024-05-25T06:11:39.189 INFO:tasks.workunit.client.1.smithi203.stdout:8/884: truncate d2/d5/f8 1065457 0 2024-05-25T06:11:39.192 INFO:tasks.workunit.client.0.smithi160.stdout:7/530: mkdir d6/da8 0 2024-05-25T06:11:39.193 INFO:tasks.workunit.client.0.smithi160.stdout:4/466: link d4/l34 d4/d1a/l8d 0 2024-05-25T06:11:39.193 INFO:tasks.workunit.client.0.smithi160.stdout:7/531: write d6/d2b/f7a [277226,72511] 0 2024-05-25T06:11:39.194 INFO:tasks.workunit.client.1.smithi203.stdout:9/704: rmdir d0/d17/d20/d6e 39 2024-05-25T06:11:39.194 INFO:tasks.workunit.client.0.smithi160.stdout:2/445: rmdir d12/d3e/d5f/d74 39 2024-05-25T06:11:39.196 INFO:tasks.workunit.client.1.smithi203.stdout:7/751: write d1/d11/d3d/fb9 [351763,98411] 0 2024-05-25T06:11:39.196 INFO:tasks.workunit.client.0.smithi160.stdout:9/466: unlink d4/d11/d1e/d3e/d2f/d79/l84 0 2024-05-25T06:11:39.197 INFO:tasks.workunit.client.0.smithi160.stdout:3/415: dread f11 [0,4194304] 0 2024-05-25T06:11:39.197 INFO:tasks.workunit.client.1.smithi203.stdout:9/705: mknod d0/df/d37/d4f/ceb 0 2024-05-25T06:11:39.198 INFO:tasks.workunit.client.1.smithi203.stdout:2/822: write d2/d1a/d87/dd9/f101 [1817377,86030] 0 2024-05-25T06:11:39.198 INFO:tasks.workunit.client.0.smithi160.stdout:9/467: readlink d4/d11/d1e/d24/d66/l89 0 2024-05-25T06:11:39.201 INFO:tasks.workunit.client.1.smithi203.stdout:7/752: creat d1/d11/d20/ffe x:0 0 0 2024-05-25T06:11:39.202 INFO:tasks.workunit.client.1.smithi203.stdout:2/823: creat d2/d42/f118 x:0 0 0 2024-05-25T06:11:39.203 INFO:tasks.workunit.client.0.smithi160.stdout:8/553: dread d5/d9/f26 [0,4194304] 0 2024-05-25T06:11:39.203 INFO:tasks.workunit.client.1.smithi203.stdout:8/885: link d2/d5/d18/dbf/cd3 d2/d5/d10/d4a/c131 0 2024-05-25T06:11:39.203 INFO:tasks.workunit.client.0.smithi160.stdout:3/416: chown f13 1751 1 2024-05-25T06:11:39.204 INFO:tasks.workunit.client.1.smithi203.stdout:7/753: fdatasync d1/d9/d2c/d36/d54/fb8 0 2024-05-25T06:11:39.206 INFO:tasks.workunit.client.0.smithi160.stdout:9/468: symlink d4/d11/d7a/l8d 0 2024-05-25T06:11:39.207 INFO:tasks.workunit.client.0.smithi160.stdout:4/467: link d4/d6/d5b/d66/c6a d4/d63/d64/d88/c8e 0 2024-05-25T06:11:39.209 INFO:tasks.workunit.client.0.smithi160.stdout:0/559: write f16 [3440548,129191] 0 2024-05-25T06:11:39.209 INFO:tasks.workunit.client.0.smithi160.stdout:3/417: fdatasync d19/d22/f39 0 2024-05-25T06:11:39.209 INFO:tasks.workunit.client.1.smithi203.stdout:7/754: mkdir d1/d9/d2c/d3a/dd1/dff 0 2024-05-25T06:11:39.210 INFO:tasks.workunit.client.1.smithi203.stdout:2/824: dwrite d2/d38/d2f/f31 [0,4194304] 0 2024-05-25T06:11:39.212 INFO:tasks.workunit.client.0.smithi160.stdout:1/378: truncate de/d21/d49/f2b 3431758 0 2024-05-25T06:11:39.212 INFO:tasks.workunit.client.0.smithi160.stdout:5/517: dwrite db/dc/d13/d1d/f20 [0,4194304] 0 2024-05-25T06:11:39.212 INFO:tasks.workunit.client.1.smithi203.stdout:2/825: read d2/d1a/d87/dd9/f101 [583302,70611] 0 2024-05-25T06:11:39.213 INFO:tasks.workunit.client.1.smithi203.stdout:7/755: mknod d1/d9/d60/dac/c100 0 2024-05-25T06:11:39.215 INFO:tasks.workunit.client.1.smithi203.stdout:2/826: write d2/d1a/d95/fa1 [1537293,1680] 0 2024-05-25T06:11:39.221 INFO:tasks.workunit.client.0.smithi160.stdout:6/485: dread d5/d39/f4d [4194304,4194304] 0 2024-05-25T06:11:39.221 INFO:tasks.workunit.client.0.smithi160.stdout:5/518: chown db/dc/d13/d1d/d31/d3b/c29 1831 1 2024-05-25T06:11:39.221 INFO:tasks.workunit.client.0.smithi160.stdout:8/554: unlink d5/d44/d1f/d6b/c79 0 2024-05-25T06:11:39.222 INFO:tasks.workunit.client.0.smithi160.stdout:9/469: dread d4/f75 [0,4194304] 0 2024-05-25T06:11:39.223 INFO:tasks.workunit.client.0.smithi160.stdout:4/468: rename d4/d1c/d54/c5a to d4/d6/d5b/d66/c8f 0 2024-05-25T06:11:39.223 INFO:tasks.workunit.client.1.smithi203.stdout:2/827: symlink d2/d1a/d33/d80/dd1/l119 0 2024-05-25T06:11:39.223 INFO:tasks.workunit.client.1.smithi203.stdout:7/756: read d1/d11/d3d/d9a/fab [279,78453] 0 2024-05-25T06:11:39.223 INFO:tasks.workunit.client.1.smithi203.stdout:8/886: getdents d2/d5/d18/d31/dac/dc2/d8a/d8f/dfc 0 2024-05-25T06:11:39.224 INFO:tasks.workunit.client.0.smithi160.stdout:3/418: chown d19/d2f/c62 48508 1 2024-05-25T06:11:39.224 INFO:tasks.workunit.client.0.smithi160.stdout:6/486: fsync d5/f11 0 2024-05-25T06:11:39.225 INFO:tasks.workunit.client.0.smithi160.stdout:0/560: unlink d2b/d31/d6d/f93 0 2024-05-25T06:11:39.226 INFO:tasks.workunit.client.0.smithi160.stdout:7/532: write d6/dc/d64/f53 [854383,91624] 0 2024-05-25T06:11:39.228 INFO:tasks.workunit.client.1.smithi203.stdout:7/757: symlink d1/d9/d2c/d36/dd3/l101 0 2024-05-25T06:11:39.228 INFO:tasks.workunit.client.0.smithi160.stdout:2/446: dread d12/d1c/d37/f43 [4194304,4194304] 0 2024-05-25T06:11:39.230 INFO:tasks.workunit.client.1.smithi203.stdout:8/887: rename d2/d5/d18/dbf/fe3 to d2/d5/d18/d31/dac/dc2/d88/d93/d9d/f132 0 2024-05-25T06:11:39.231 INFO:tasks.workunit.client.0.smithi160.stdout:2/447: truncate d12/d1c/d37/f4f 4969592 0 2024-05-25T06:11:39.231 INFO:tasks.workunit.client.0.smithi160.stdout:5/519: creat db/dc/d13/d1d/d3f/d72/d98/f9b x:0 0 0 2024-05-25T06:11:39.234 INFO:tasks.workunit.client.0.smithi160.stdout:8/555: fdatasync d5/d44/d1f/da3/d28/d4a/d93/fab 0 2024-05-25T06:11:39.235 INFO:tasks.workunit.client.0.smithi160.stdout:0/561: dwrite d2b/d31/d35/d36/fa3 [0,4194304] 0 2024-05-25T06:11:39.238 INFO:tasks.workunit.client.1.smithi203.stdout:7/758: dread d1/d9/d2c/d3a/d4e/f5b [0,4194304] 0 2024-05-25T06:11:39.240 INFO:tasks.workunit.client.1.smithi203.stdout:7/759: write d1/d9/d2c/d36/f9d [3407946,82512] 0 2024-05-25T06:11:39.244 INFO:tasks.workunit.client.1.smithi203.stdout:2/828: dread d2/d1a/d33/d80/dd1/ddf/fe0 [0,4194304] 0 2024-05-25T06:11:39.244 INFO:tasks.workunit.client.1.smithi203.stdout:7/760: rmdir d1/d9/d2c/d79/dc7 39 2024-05-25T06:11:39.248 INFO:tasks.workunit.client.1.smithi203.stdout:2/829: creat d2/d1a/dd3/d100/d10/f11a x:0 0 0 2024-05-25T06:11:39.253 INFO:tasks.workunit.client.0.smithi160.stdout:6/487: write d5/de/f85 [45185,43057] 0 2024-05-25T06:11:39.253 INFO:tasks.workunit.client.0.smithi160.stdout:5/520: mkdir db/dc/d13/d1d/d3f/d72/d9c 0 2024-05-25T06:11:39.256 INFO:tasks.workunit.client.0.smithi160.stdout:6/488: truncate d5/de/d58/f80 1019017 0 2024-05-25T06:11:39.256 INFO:tasks.workunit.client.1.smithi203.stdout:8/888: dread d2/d5/d10/d45/d5d/d68/f7d [0,4194304] 0 2024-05-25T06:11:39.257 INFO:tasks.workunit.client.0.smithi160.stdout:0/562: dread d2b/d31/d35/d36/d37/f95 [0,4194304] 0 2024-05-25T06:11:39.257 INFO:tasks.workunit.client.0.smithi160.stdout:3/419: read d19/d2f/d34/f40 [154665,97880] 0 2024-05-25T06:11:39.258 INFO:tasks.workunit.client.1.smithi203.stdout:7/761: write d1/d9/fb [3497304,81768] 0 2024-05-25T06:11:39.258 INFO:tasks.workunit.client.0.smithi160.stdout:6/489: truncate d5/d30/d36/d73/d6e/f96 671175 0 2024-05-25T06:11:39.259 INFO:tasks.workunit.client.0.smithi160.stdout:2/448: write d12/d13/f24 [585658,74016] 0 2024-05-25T06:11:39.261 INFO:tasks.workunit.client.1.smithi203.stdout:7/762: truncate d1/d11/d3d/d4c/fed 609594 0 2024-05-25T06:11:39.261 INFO:tasks.workunit.client.0.smithi160.stdout:1/379: link de/d21/d49/l6b de/d21/d49/d1c/d41/d45/l78 0 2024-05-25T06:11:39.262 INFO:tasks.workunit.client.1.smithi203.stdout:8/889: creat d2/d5/d10/d45/d90/f133 x:0 0 0 2024-05-25T06:11:39.263 INFO:tasks.workunit.client.0.smithi160.stdout:1/380: read de/d21/d49/d28/f3c [2398883,83593] 0 2024-05-25T06:11:39.263 INFO:tasks.workunit.client.1.smithi203.stdout:8/890: stat d2/d5/d28/c8c 0 2024-05-25T06:11:39.264 INFO:tasks.workunit.client.1.smithi203.stdout:7/763: dread - d1/d9/d2c/fc5 zero size 2024-05-25T06:11:39.264 INFO:tasks.workunit.client.0.smithi160.stdout:9/470: rmdir d4/d11/d1e/d4b 0 2024-05-25T06:11:39.265 INFO:tasks.workunit.client.0.smithi160.stdout:9/471: chown d4/d11/d1e/d24/c5f 1978012 1 2024-05-25T06:11:39.266 INFO:tasks.workunit.client.1.smithi203.stdout:8/891: fsync d2/d5/d28/f4c 0 2024-05-25T06:11:39.266 INFO:tasks.workunit.client.1.smithi203.stdout:7/764: chown d1/d9/d2c/d79/f81 13 1 2024-05-25T06:11:39.266 INFO:tasks.workunit.client.0.smithi160.stdout:5/521: mknod db/dc/d13/d1d/d31/d3b/c9d 0 2024-05-25T06:11:39.267 INFO:tasks.workunit.client.1.smithi203.stdout:7/765: stat d1/d11/d3d/d4c/de1 0 2024-05-25T06:11:39.268 INFO:tasks.workunit.client.1.smithi203.stdout:7/766: dread - d1/d9/d2c/d36/fdc zero size 2024-05-25T06:11:39.270 INFO:tasks.workunit.client.0.smithi160.stdout:3/420: rmdir d19/d2f/d38/d43/d5b 39 2024-05-25T06:11:39.272 INFO:tasks.workunit.client.0.smithi160.stdout:6/490: mknod d5/d45/d63/caa 0 2024-05-25T06:11:39.273 INFO:tasks.workunit.client.1.smithi203.stdout:8/892: creat d2/d5/d28/d11a/f134 x:0 0 0 2024-05-25T06:11:39.273 INFO:tasks.workunit.client.1.smithi203.stdout:2/830: dread d2/d1a/d33/f71 [0,4194304] 0 2024-05-25T06:11:39.276 INFO:tasks.workunit.client.0.smithi160.stdout:7/533: rename d6/dc/d64/d5a/f88 to d6/d2b/d5d/d89/fa9 0 2024-05-25T06:11:39.276 INFO:tasks.workunit.client.1.smithi203.stdout:2/831: write d2/d38/d2f/f10a [2778924,37925] 0 2024-05-25T06:11:39.276 INFO:tasks.workunit.client.1.smithi203.stdout:8/893: getdents d2/d5/d10/d11/de/d11f/d123 0 2024-05-25T06:11:39.282 INFO:tasks.workunit.client.1.smithi203.stdout:8/894: symlink d2/d5/d18/d31/dac/dc2/d8a/l135 0 2024-05-25T06:11:39.282 INFO:tasks.workunit.client.0.smithi160.stdout:2/449: mkdir d12/d97 0 2024-05-25T06:11:39.283 INFO:tasks.workunit.client.0.smithi160.stdout:3/421: mkdir d19/d2f/d34/d67/d96 0 2024-05-25T06:11:39.285 INFO:tasks.workunit.client.0.smithi160.stdout:1/381: dread f7 [0,4194304] 0 2024-05-25T06:11:39.285 INFO:tasks.workunit.client.0.smithi160.stdout:6/491: mkdir d5/de/d32/d35/dab 0 2024-05-25T06:11:39.286 INFO:tasks.workunit.client.1.smithi203.stdout:8/895: truncate d2/d5/d28/f4c 4144472 0 2024-05-25T06:11:39.286 INFO:tasks.workunit.client.0.smithi160.stdout:2/450: dread d12/d5c/f72 [0,4194304] 0 2024-05-25T06:11:39.286 INFO:tasks.workunit.client.0.smithi160.stdout:5/522: dwrite db/dc/d13/d1d/d23/d24/d2e/f4e [0,4194304] 0 2024-05-25T06:11:39.289 INFO:tasks.workunit.client.0.smithi160.stdout:9/472: getdents d4/d11/d1e/d3e/d25/d5e 0 2024-05-25T06:11:39.289 INFO:tasks.workunit.client.0.smithi160.stdout:0/563: truncate d2b/d46/d63/da7/f8c 3253124 0 2024-05-25T06:11:39.290 INFO:tasks.workunit.client.1.smithi203.stdout:2/832: dwrite d2/d1a/dd3/d100/d10/d5f/f81 [0,4194304] 0 2024-05-25T06:11:39.291 INFO:tasks.workunit.client.0.smithi160.stdout:0/564: chown d2b/d31/f7d 2007706 1 2024-05-25T06:11:39.292 INFO:tasks.workunit.client.0.smithi160.stdout:8/556: dwrite d5/f35 [0,4194304] 0 2024-05-25T06:11:39.293 INFO:tasks.workunit.client.1.smithi203.stdout:7/767: dread d1/d9/d2c/d62/db3/fe3 [0,4194304] 0 2024-05-25T06:11:39.293 INFO:tasks.workunit.client.0.smithi160.stdout:8/557: readlink d5/d7/l10 0 2024-05-25T06:11:39.294 INFO:tasks.workunit.client.0.smithi160.stdout:5/523: symlink db/dc/d13/d1d/d31/d3b/d69/l9e 0 2024-05-25T06:11:39.295 INFO:tasks.workunit.client.0.smithi160.stdout:5/524: write db/dc/d13/d1d/d23/d45/f5f [9199717,35112] 0 2024-05-25T06:11:39.296 INFO:tasks.workunit.client.1.smithi203.stdout:7/768: chown d1/d11/d20/ffe 1916763876 1 2024-05-25T06:11:39.300 INFO:tasks.workunit.client.1.smithi203.stdout:8/896: dwrite d2/d5/d18/d31/dac/fd6 [0,4194304] 0 2024-05-25T06:11:39.302 INFO:tasks.workunit.client.1.smithi203.stdout:7/769: write d1/d9/dc2/fcb [441604,23194] 0 2024-05-25T06:11:39.302 INFO:tasks.workunit.client.1.smithi203.stdout:8/897: chown d2/d5/d10/d45/d90/f133 39 1 2024-05-25T06:11:39.303 INFO:tasks.workunit.client.0.smithi160.stdout:3/422: creat d19/d5d/d6a/f97 x:0 0 0 2024-05-25T06:11:39.303 INFO:tasks.workunit.client.1.smithi203.stdout:2/833: rename d2/d1a/dd3/d100/d10/d32/dca/cd0 to d2/d42/d4e/d68/c11b 0 2024-05-25T06:11:39.304 INFO:tasks.workunit.client.0.smithi160.stdout:9/473: write d4/d11/d1e/d24/f1b [3754131,112952] 0 2024-05-25T06:11:39.305 INFO:tasks.workunit.client.0.smithi160.stdout:3/423: write d19/d2f/d34/f95 [584677,67321] 0 2024-05-25T06:11:39.308 INFO:tasks.workunit.client.0.smithi160.stdout:5/525: dwrite db/dc/d13/f6a [0,4194304] 0 2024-05-25T06:11:39.311 INFO:tasks.workunit.client.0.smithi160.stdout:0/565: rmdir d2b/d31/d35/d43 39 2024-05-25T06:11:39.312 INFO:tasks.workunit.client.0.smithi160.stdout:6/492: mkdir d5/d39/da6/dac 0 2024-05-25T06:11:39.314 INFO:tasks.workunit.client.0.smithi160.stdout:1/382: mknod de/d21/d49/d1c/d59/d67/c79 0 2024-05-25T06:11:39.315 INFO:tasks.workunit.client.1.smithi203.stdout:8/898: mknod d2/d5/d10/d11/de/d11f/d123/c136 0 2024-05-25T06:11:39.316 INFO:tasks.workunit.client.0.smithi160.stdout:5/526: write db/f63 [9418096,76231] 0 2024-05-25T06:11:39.317 INFO:tasks.workunit.client.0.smithi160.stdout:8/558: mkdir d5/d44/d1f/da3/d28/d4a/db8 0 2024-05-25T06:11:39.317 INFO:tasks.workunit.client.1.smithi203.stdout:7/770: rename d1/d11/d3d/cca to d1/c102 0 2024-05-25T06:11:39.319 INFO:tasks.workunit.client.0.smithi160.stdout:9/474: mkdir d4/d11/d14/d8e 0 2024-05-25T06:11:39.319 INFO:tasks.workunit.client.0.smithi160.stdout:0/566: write d2b/d31/d35/d36/d37/f4d [430781,9893] 0 2024-05-25T06:11:39.321 INFO:tasks.workunit.client.0.smithi160.stdout:1/383: creat de/d21/f7a x:0 0 0 2024-05-25T06:11:39.322 INFO:tasks.workunit.client.0.smithi160.stdout:6/493: write d5/d45/d4a/f53 [4150838,61236] 0 2024-05-25T06:11:39.323 INFO:tasks.workunit.client.0.smithi160.stdout:3/424: symlink d19/d2f/d34/d67/d96/l98 0 2024-05-25T06:11:39.323 INFO:tasks.workunit.client.1.smithi203.stdout:7/771: mkdir d1/d9/d2c/d62/d7a/d103 0 2024-05-25T06:11:39.325 INFO:tasks.workunit.client.0.smithi160.stdout:8/559: chown d5/d44/d1f/da3/d28/d4a/d4e/d86/c5d 12 1 2024-05-25T06:11:39.326 INFO:tasks.workunit.client.0.smithi160.stdout:5/527: rename db/d47/f51 to db/dc/d13/d9a/f9f 0 2024-05-25T06:11:39.327 INFO:tasks.workunit.client.1.smithi203.stdout:7/772: dread d1/d9/fb4 [0,4194304] 0 2024-05-25T06:11:39.328 INFO:tasks.workunit.client.0.smithi160.stdout:2/451: dread d12/d3e/f68 [0,4194304] 0 2024-05-25T06:11:39.328 INFO:tasks.workunit.client.0.smithi160.stdout:8/560: fsync d5/fb 0 2024-05-25T06:11:39.329 INFO:tasks.workunit.client.1.smithi203.stdout:7/773: rmdir d1/d9/d2c/d36/dd3 39 2024-05-25T06:11:39.330 INFO:tasks.workunit.client.0.smithi160.stdout:6/494: fdatasync d5/de/fd 0 2024-05-25T06:11:39.333 INFO:tasks.workunit.client.1.smithi203.stdout:7/774: symlink d1/d9/d2c/d79/df0/l104 0 2024-05-25T06:11:39.333 INFO:tasks.workunit.client.0.smithi160.stdout:1/384: creat de/d21/d49/d1c/d63/f7b x:0 0 0 2024-05-25T06:11:39.334 INFO:tasks.workunit.client.1.smithi203.stdout:7/775: rmdir d1/dbd 39 2024-05-25T06:11:39.334 INFO:tasks.workunit.client.0.smithi160.stdout:3/425: creat d19/f99 x:0 0 0 2024-05-25T06:11:39.335 INFO:tasks.workunit.client.1.smithi203.stdout:7/776: write d1/d9/f14 [3406106,99967] 0 2024-05-25T06:11:39.337 INFO:tasks.workunit.client.0.smithi160.stdout:2/452: symlink d12/d3e/l98 0 2024-05-25T06:11:39.338 INFO:tasks.workunit.client.1.smithi203.stdout:2/834: write d2/d1a/dd3/d100/f11 [2045725,46373] 0 2024-05-25T06:11:39.338 INFO:tasks.workunit.client.0.smithi160.stdout:9/475: dwrite d4/d11/d1e/d6c/f81 [0,4194304] 0 2024-05-25T06:11:39.340 INFO:tasks.workunit.client.1.smithi203.stdout:8/899: write d2/d5/d10/d4a/fba [1010729,72472] 0 2024-05-25T06:11:39.343 INFO:tasks.workunit.client.0.smithi160.stdout:7/534: dwrite d6/dc/d64/d5a/f8f [0,4194304] 0 2024-05-25T06:11:39.343 INFO:tasks.workunit.client.1.smithi203.stdout:2/835: creat d2/d1a/dd3/d100/f11c x:0 0 0 2024-05-25T06:11:39.344 INFO:tasks.workunit.client.0.smithi160.stdout:3/426: stat d19/d22/f24 0 2024-05-25T06:11:39.344 INFO:tasks.workunit.client.0.smithi160.stdout:5/528: creat db/dc/d13/d1d/d3f/d72/d98/fa0 x:0 0 0 2024-05-25T06:11:39.344 INFO:tasks.workunit.client.1.smithi203.stdout:8/900: mknod d2/d5/d10/d11/d1f/d75/c137 0 2024-05-25T06:11:39.345 INFO:tasks.workunit.client.0.smithi160.stdout:9/476: fdatasync d4/fa 0 2024-05-25T06:11:39.346 INFO:tasks.workunit.client.0.smithi160.stdout:6/495: symlink d5/d45/d4a/lad 0 2024-05-25T06:11:39.346 INFO:tasks.workunit.client.0.smithi160.stdout:0/567: dwrite d2b/d31/d35/d36/f4a [0,4194304] 0 2024-05-25T06:11:39.346 INFO:tasks.workunit.client.1.smithi203.stdout:7/777: creat d1/d11/f105 x:0 0 0 2024-05-25T06:11:39.349 INFO:tasks.workunit.client.1.smithi203.stdout:8/901: rmdir d2/d5/d10/d4a 39 2024-05-25T06:11:39.349 INFO:tasks.workunit.client.0.smithi160.stdout:0/568: dread - d2b/d46/d63/da7/d67/fa5 zero size 2024-05-25T06:11:39.349 INFO:tasks.workunit.client.0.smithi160.stdout:8/561: unlink d5/d44/d1f/da3/d28/d4a/d4e/d86/f8b 0 2024-05-25T06:11:39.351 INFO:tasks.workunit.client.1.smithi203.stdout:2/836: truncate d2/d1a/dd3/d100/d10/d58/f6d 4390850 0 2024-05-25T06:11:39.351 INFO:tasks.workunit.client.0.smithi160.stdout:0/569: chown d2b/d31/d35/f55 4876040 1 2024-05-25T06:11:39.354 INFO:tasks.workunit.client.0.smithi160.stdout:1/385: truncate de/d21/d49/f27 1278396 0 2024-05-25T06:11:39.357 INFO:tasks.workunit.client.1.smithi203.stdout:2/837: unlink d2/d1a/dd3/d100/d10/d5f/l39 0 2024-05-25T06:11:39.357 INFO:tasks.workunit.client.0.smithi160.stdout:8/562: truncate d5/d44/d1f/da3/d28/d4a/d4e/fa1 367924 0 2024-05-25T06:11:39.358 INFO:tasks.workunit.client.1.smithi203.stdout:7/778: link d1/d9/l16 d1/d9/d2c/d36/d54/l106 0 2024-05-25T06:11:39.358 INFO:tasks.workunit.client.0.smithi160.stdout:0/570: chown d2b/d31/d35/d36/d37/f44 52 1 2024-05-25T06:11:39.359 INFO:tasks.workunit.client.1.smithi203.stdout:2/838: truncate d2/d38/d2f/d69/f117 785967 0 2024-05-25T06:11:39.359 INFO:tasks.workunit.client.0.smithi160.stdout:5/529: dwrite db/dc/d13/d1d/d23/f91 [0,4194304] 0 2024-05-25T06:11:39.360 INFO:tasks.workunit.client.0.smithi160.stdout:9/477: creat d4/d11/d1e/d3e/d25/f8f x:0 0 0 2024-05-25T06:11:39.361 INFO:tasks.workunit.client.0.smithi160.stdout:8/563: fdatasync d5/d39/d7a/f92 0 2024-05-25T06:11:39.363 INFO:tasks.workunit.client.0.smithi160.stdout:2/453: creat d12/d97/f99 x:0 0 0 2024-05-25T06:11:39.363 INFO:tasks.workunit.client.1.smithi203.stdout:7/779: mknod d1/d9/d2c/d62/db3/de5/c107 0 2024-05-25T06:11:39.365 INFO:tasks.workunit.client.1.smithi203.stdout:2/839: rename d2/d1a/dd3/ff5 to d2/d1a/dd3/d100/d10/d32/dca/de8/f11d 0 2024-05-25T06:11:39.366 INFO:tasks.workunit.client.0.smithi160.stdout:7/535: rename d6/d3e/d6f/f40 to d6/dc/d64/d5a/d7c/faa 0 2024-05-25T06:11:39.368 INFO:tasks.workunit.client.1.smithi203.stdout:2/840: truncate d2/d38/f7d 94586 0 2024-05-25T06:11:39.369 INFO:tasks.workunit.client.0.smithi160.stdout:8/564: stat d5/d39/d7a/ca7 0 2024-05-25T06:11:39.369 INFO:tasks.workunit.client.1.smithi203.stdout:8/902: dread d2/d5/f44 [0,4194304] 0 2024-05-25T06:11:39.370 INFO:tasks.workunit.client.0.smithi160.stdout:1/386: getdents de/d21/d49/d1c/d52/d77 0 2024-05-25T06:11:39.371 INFO:tasks.workunit.client.1.smithi203.stdout:2/841: chown d2/d1a/dd3/d100/d10/d32/f52 15 1 2024-05-25T06:11:39.371 INFO:tasks.workunit.client.0.smithi160.stdout:1/387: stat cc 0 2024-05-25T06:11:39.373 INFO:tasks.workunit.client.1.smithi203.stdout:8/903: unlink d2/d5/d28/d38/d4e/f10f 0 2024-05-25T06:11:39.373 INFO:tasks.workunit.client.0.smithi160.stdout:2/454: symlink d12/d3c/l9a 0 2024-05-25T06:11:39.373 INFO:tasks.workunit.client.0.smithi160.stdout:7/536: creat d6/d32/d92/fab x:0 0 0 2024-05-25T06:11:39.375 INFO:tasks.workunit.client.0.smithi160.stdout:5/530: dwrite db/f22 [0,4194304] 0 2024-05-25T06:11:39.375 INFO:tasks.workunit.client.0.smithi160.stdout:2/455: read f11 [2789775,73657] 0 2024-05-25T06:11:39.375 INFO:tasks.workunit.client.0.smithi160.stdout:7/537: fdatasync d6/dc/ff 0 2024-05-25T06:11:39.376 INFO:tasks.workunit.client.1.smithi203.stdout:2/842: symlink d2/d1a/dd3/d100/da6/l11e 0 2024-05-25T06:11:39.377 INFO:tasks.workunit.client.0.smithi160.stdout:8/565: mkdir d5/d9/d72/daf/db9 0 2024-05-25T06:11:39.377 INFO:tasks.workunit.client.1.smithi203.stdout:8/904: mknod d2/d5/d10/d11/de/d11f/c138 0 2024-05-25T06:11:39.378 INFO:tasks.workunit.client.0.smithi160.stdout:9/478: dwrite d4/d11/d12/d58/f64 [0,4194304] 0 2024-05-25T06:11:39.378 INFO:tasks.workunit.client.0.smithi160.stdout:1/388: mkdir de/d21/d49/d1c/d59/d7c 0 2024-05-25T06:11:39.379 INFO:tasks.workunit.client.0.smithi160.stdout:2/456: chown d12/d75 104419367 1 2024-05-25T06:11:39.380 INFO:tasks.workunit.client.0.smithi160.stdout:3/427: write d19/d2f/d34/f40 [1808801,24061] 0 2024-05-25T06:11:39.381 INFO:tasks.workunit.client.0.smithi160.stdout:7/538: mkdir d6/dc/d64/dac 0 2024-05-25T06:11:39.382 INFO:tasks.workunit.client.0.smithi160.stdout:5/531: stat db/dc/d13/f50 0 2024-05-25T06:11:39.383 INFO:tasks.workunit.client.1.smithi203.stdout:2/843: fdatasync d2/d42/fa9 0 2024-05-25T06:11:39.385 INFO:tasks.workunit.client.1.smithi203.stdout:8/905: dwrite d2/d5/d10/d11/f9c [0,4194304] 0 2024-05-25T06:11:39.385 INFO:tasks.workunit.client.0.smithi160.stdout:8/566: mkdir d5/d44/dba 0 2024-05-25T06:11:39.385 INFO:tasks.workunit.client.0.smithi160.stdout:6/496: dread d5/de/d32/d35/f5f [0,4194304] 0 2024-05-25T06:11:39.385 INFO:tasks.workunit.client.0.smithi160.stdout:1/389: write de/d21/d49/d1c/d59/d71/f5c [470546,99634] 0 2024-05-25T06:11:39.392 INFO:tasks.workunit.client.0.smithi160.stdout:2/457: mkdir d12/d5c/d85/d9b 0 2024-05-25T06:11:39.392 INFO:tasks.workunit.client.1.smithi203.stdout:7/780: write d1/d9/d2c/d36/d66/fba [609638,14257] 0 2024-05-25T06:11:39.393 INFO:tasks.workunit.client.0.smithi160.stdout:3/428: read d19/d2f/d38/d47/f6e [3481246,16777] 0 2024-05-25T06:11:39.394 INFO:tasks.workunit.client.0.smithi160.stdout:5/532: rename db/dc/d13/d1d/l7c to db/d47/d5e/d6e/la1 0 2024-05-25T06:11:39.395 INFO:tasks.workunit.client.1.smithi203.stdout:2/844: rmdir d2/d1a/d33/d80/dd1/ddf 39 2024-05-25T06:11:39.395 INFO:tasks.workunit.client.0.smithi160.stdout:5/533: stat db/dc/d13 0 2024-05-25T06:11:39.396 INFO:tasks.workunit.client.0.smithi160.stdout:1/390: unlink de/d21/d49/d35/f3b 0 2024-05-25T06:11:39.398 INFO:tasks.workunit.client.0.smithi160.stdout:7/539: symlink d6/d32/lad 0 2024-05-25T06:11:39.398 INFO:tasks.workunit.client.1.smithi203.stdout:8/906: dwrite d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/f11e [0,4194304] 0 2024-05-25T06:11:39.400 INFO:tasks.workunit.client.0.smithi160.stdout:9/479: getdents d4/d11/d12/d2e 0 2024-05-25T06:11:39.400 INFO:tasks.workunit.client.1.smithi203.stdout:2/845: dread - d2/d38/d43/d60/d6b/f10b zero size 2024-05-25T06:11:39.400 INFO:tasks.workunit.client.1.smithi203.stdout:7/781: rename d1/d9/fc to d1/d9/d2c/d79/dd6/f108 0 2024-05-25T06:11:39.401 INFO:tasks.workunit.client.0.smithi160.stdout:3/429: fsync d19/d2f/d38/d47/f54 0 2024-05-25T06:11:39.402 INFO:tasks.workunit.client.0.smithi160.stdout:3/430: fdatasync d19/d22/f8f 0 2024-05-25T06:11:39.402 INFO:tasks.workunit.client.0.smithi160.stdout:3/431: readlink d19/l91 0 2024-05-25T06:11:39.404 INFO:tasks.workunit.client.0.smithi160.stdout:5/534: creat db/dc/d13/d1d/d3f/d72/fa2 x:0 0 0 2024-05-25T06:11:39.404 INFO:tasks.workunit.client.0.smithi160.stdout:6/497: mknod d5/d39/d8f/cae 0 2024-05-25T06:11:39.404 INFO:tasks.workunit.client.0.smithi160.stdout:3/432: write d19/d2f/d38/d43/f71 [960688,5148] 0 2024-05-25T06:11:39.407 INFO:tasks.workunit.client.1.smithi203.stdout:2/846: fdatasync d2/d1a/d89/daa/fdd 0 2024-05-25T06:11:39.407 INFO:tasks.workunit.client.0.smithi160.stdout:5/535: write db/f57 [409357,52480] 0 2024-05-25T06:11:39.407 INFO:tasks.workunit.client.0.smithi160.stdout:9/480: mkdir d4/d11/d12/d2e/d45/d90 0 2024-05-25T06:11:39.408 INFO:tasks.workunit.client.1.smithi203.stdout:7/782: mknod d1/d9/d2c/d36/d66/c109 0 2024-05-25T06:11:39.409 INFO:tasks.workunit.client.0.smithi160.stdout:6/498: chown d5/d39/d8f/l46 111 1 2024-05-25T06:11:39.409 INFO:tasks.workunit.client.0.smithi160.stdout:2/458: getdents d12/d13 0 2024-05-25T06:11:39.410 INFO:tasks.workunit.client.0.smithi160.stdout:5/536: mknod db/d47/d5e/ca3 0 2024-05-25T06:11:39.411 INFO:tasks.workunit.client.1.smithi203.stdout:7/783: stat d1/d9/d2c/d36/d54/f8f 0 2024-05-25T06:11:39.415 INFO:tasks.workunit.client.1.smithi203.stdout:8/907: rename d2/d5/d10/d11/de/d11f/d123 to d2/d5/d18/d139 0 2024-05-25T06:11:39.415 INFO:tasks.workunit.client.1.smithi203.stdout:2/847: dwrite d2/d1a/d87/d9e/ff7 [0,4194304] 0 2024-05-25T06:11:39.416 INFO:tasks.workunit.client.0.smithi160.stdout:9/481: rename d4/d11/d1e/d24/l15 to d4/d11/d1e/d3e/d25/d5e/l91 0 2024-05-25T06:11:39.417 INFO:tasks.workunit.client.1.smithi203.stdout:2/848: chown d2/d42/fe9 0 1 2024-05-25T06:11:39.418 INFO:tasks.workunit.client.1.smithi203.stdout:7/784: symlink d1/d9/d2c/d79/dd6/l10a 0 2024-05-25T06:11:39.421 INFO:tasks.workunit.client.0.smithi160.stdout:6/499: dwrite d5/de/f22 [4194304,4194304] 0 2024-05-25T06:11:39.422 INFO:tasks.workunit.client.0.smithi160.stdout:6/500: stat d5/f14 0 2024-05-25T06:11:39.423 INFO:tasks.workunit.client.1.smithi203.stdout:7/785: unlink d1/f78 0 2024-05-25T06:11:39.423 INFO:tasks.workunit.client.0.smithi160.stdout:8/567: dread d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:39.425 INFO:tasks.workunit.client.0.smithi160.stdout:8/568: readlink d5/d39/d6a/l8a 0 2024-05-25T06:11:39.426 INFO:tasks.workunit.client.0.smithi160.stdout:2/459: getdents d12/d3e/d5f 0 2024-05-25T06:11:39.427 INFO:tasks.workunit.client.0.smithi160.stdout:9/482: write d4/d11/d1e/d3e/f44 [691175,29545] 0 2024-05-25T06:11:39.433 INFO:tasks.workunit.client.0.smithi160.stdout:8/569: dread d5/d39/f7d [0,4194304] 0 2024-05-25T06:11:39.434 INFO:tasks.workunit.client.0.smithi160.stdout:6/501: rename d5/f14 to d5/de/d32/d35/dab/faf 0 2024-05-25T06:11:39.434 INFO:tasks.workunit.client.0.smithi160.stdout:7/540: dread d6/dc/f19 [0,4194304] 0 2024-05-25T06:11:39.435 INFO:tasks.workunit.client.0.smithi160.stdout:9/483: mknod d4/d11/d1e/c92 0 2024-05-25T06:11:39.436 INFO:tasks.workunit.client.1.smithi203.stdout:8/908: dwrite d2/d5/d18/d31/dac/dc2/d88/d93/fb9 [0,4194304] 0 2024-05-25T06:11:39.436 INFO:tasks.workunit.client.0.smithi160.stdout:6/502: readlink d5/d45/d4a/la3 0 2024-05-25T06:11:39.438 INFO:tasks.workunit.client.0.smithi160.stdout:1/391: dread de/f19 [0,4194304] 0 2024-05-25T06:11:39.439 INFO:tasks.workunit.client.0.smithi160.stdout:8/570: creat d5/d9/fbb x:0 0 0 2024-05-25T06:11:39.440 INFO:tasks.workunit.client.0.smithi160.stdout:7/541: unlink d6/c8 0 2024-05-25T06:11:39.443 INFO:tasks.workunit.client.0.smithi160.stdout:7/542: chown d6/dc/d21/f4f 53601 1 2024-05-25T06:11:39.443 INFO:tasks.workunit.client.0.smithi160.stdout:5/537: dread db/dc/d13/d1d/d23/d24/f3e [0,4194304] 0 2024-05-25T06:11:39.443 INFO:tasks.workunit.client.0.smithi160.stdout:6/503: unlink f2 0 2024-05-25T06:11:39.444 INFO:tasks.workunit.client.1.smithi203.stdout:8/909: dread d2/d5/d18/dbf/f11b [0,4194304] 0 2024-05-25T06:11:39.444 INFO:tasks.workunit.client.0.smithi160.stdout:2/460: link d12/d13/c3f d12/d3e/d5a/c9c 0 2024-05-25T06:11:39.445 INFO:tasks.workunit.client.0.smithi160.stdout:5/538: write db/f85 [244521,92020] 0 2024-05-25T06:11:39.445 INFO:tasks.workunit.client.0.smithi160.stdout:2/461: stat d12/d1c/c38 0 2024-05-25T06:11:39.445 INFO:tasks.workunit.client.0.smithi160.stdout:8/571: symlink d5/d9/lbc 0 2024-05-25T06:11:39.446 INFO:tasks.workunit.client.0.smithi160.stdout:1/392: mkdir de/d21/d49/d1c/d41/d45/d7d 0 2024-05-25T06:11:39.447 INFO:tasks.workunit.client.0.smithi160.stdout:7/543: mkdir d6/dc/d64/dae 0 2024-05-25T06:11:39.447 INFO:tasks.workunit.client.0.smithi160.stdout:1/393: readlink de/d21/d49/d1c/l20 0 2024-05-25T06:11:39.447 INFO:tasks.workunit.client.0.smithi160.stdout:5/539: chown db/dc/d13/d54 22189859 1 2024-05-25T06:11:39.448 INFO:tasks.workunit.client.1.smithi203.stdout:2/849: truncate d2/f6c 560547 0 2024-05-25T06:11:39.451 INFO:tasks.workunit.client.0.smithi160.stdout:6/504: fsync d5/de/f42 0 2024-05-25T06:11:39.452 INFO:tasks.workunit.client.0.smithi160.stdout:2/462: rename d12/f6e to d12/d1c/d37/f9d 0 2024-05-25T06:11:39.453 INFO:tasks.workunit.client.0.smithi160.stdout:5/540: read db/dc/d13/f44 [2511832,17358] 0 2024-05-25T06:11:39.457 INFO:tasks.workunit.client.0.smithi160.stdout:2/463: creat d12/d3c/f9e x:0 0 0 2024-05-25T06:11:39.464 INFO:tasks.workunit.client.0.smithi160.stdout:7/544: dwrite d6/dc/d64/f38 [0,4194304] 0 2024-05-25T06:11:39.466 INFO:tasks.workunit.client.0.smithi160.stdout:9/484: dread d4/d11/f26 [0,4194304] 0 2024-05-25T06:11:39.467 INFO:tasks.workunit.client.0.smithi160.stdout:8/572: dwrite d5/f48 [0,4194304] 0 2024-05-25T06:11:39.468 INFO:tasks.workunit.client.0.smithi160.stdout:7/545: stat d6/dc/d64/d5a/d7c/l9a 0 2024-05-25T06:11:39.470 INFO:tasks.workunit.client.0.smithi160.stdout:7/546: chown d6/c29 207940499 1 2024-05-25T06:11:39.475 INFO:tasks.workunit.client.0.smithi160.stdout:6/505: dwrite d5/d45/d4a/f76 [4194304,4194304] 0 2024-05-25T06:11:39.477 INFO:tasks.workunit.client.1.smithi203.stdout:7/786: write d1/d9/d2c/d79/f81 [3550072,40930] 0 2024-05-25T06:11:39.478 INFO:tasks.workunit.client.0.smithi160.stdout:1/394: dread fb [0,4194304] 0 2024-05-25T06:11:39.479 INFO:tasks.workunit.client.1.smithi203.stdout:7/787: read d1/d11/d3d/d9a/fab [318327,117827] 0 2024-05-25T06:11:39.480 INFO:tasks.workunit.client.0.smithi160.stdout:5/541: mknod db/dc/d13/d1d/d23/ca4 0 2024-05-25T06:11:39.483 INFO:tasks.workunit.client.0.smithi160.stdout:1/395: creat de/d21/d49/d28/f7e x:0 0 0 2024-05-25T06:11:39.483 INFO:tasks.workunit.client.1.smithi203.stdout:7/788: mkdir d1/d11/d20/d10b 0 2024-05-25T06:11:39.484 INFO:tasks.workunit.client.0.smithi160.stdout:6/506: symlink d5/d30/d36/d73/d6e/lb0 0 2024-05-25T06:11:39.486 INFO:tasks.workunit.client.0.smithi160.stdout:8/573: creat d5/d7/d9a/db5/fbd x:0 0 0 2024-05-25T06:11:39.487 INFO:tasks.workunit.client.0.smithi160.stdout:7/547: creat d6/d2b/d8b/faf x:0 0 0 2024-05-25T06:11:39.487 INFO:tasks.workunit.client.1.smithi203.stdout:7/789: creat d1/d9/d2c/d3a/dde/f10c x:0 0 0 2024-05-25T06:11:39.488 INFO:tasks.workunit.client.0.smithi160.stdout:2/464: creat d12/f9f x:0 0 0 2024-05-25T06:11:39.489 INFO:tasks.workunit.client.1.smithi203.stdout:7/790: write d1/d9/d2c/d62/d9b/fc9 [1103386,58555] 0 2024-05-25T06:11:39.490 INFO:tasks.workunit.client.0.smithi160.stdout:9/485: creat d4/d11/f93 x:0 0 0 2024-05-25T06:11:39.491 INFO:tasks.workunit.client.0.smithi160.stdout:6/507: dread d5/d39/f6c [0,4194304] 0 2024-05-25T06:11:39.492 INFO:tasks.workunit.client.0.smithi160.stdout:8/574: unlink d5/d39/d75/c88 0 2024-05-25T06:11:39.492 INFO:tasks.workunit.client.0.smithi160.stdout:1/396: chown de/c51 178 1 2024-05-25T06:11:39.494 INFO:tasks.workunit.client.1.smithi203.stdout:7/791: mknod d1/d9/d2c/d79/dc7/c10d 0 2024-05-25T06:11:39.495 INFO:tasks.workunit.client.0.smithi160.stdout:1/397: chown de/d21/d49/l5e 256077902 1 2024-05-25T06:11:39.495 INFO:tasks.workunit.client.0.smithi160.stdout:7/548: rename d6/dc/d64/d5a/f86 to d6/dc/d64/fb0 0 2024-05-25T06:11:39.496 INFO:tasks.workunit.client.0.smithi160.stdout:2/465: read ff [273990,67596] 0 2024-05-25T06:11:39.497 INFO:tasks.workunit.client.0.smithi160.stdout:1/398: chown de/d21/d49/d1c/d63/f7b 46980 1 2024-05-25T06:11:39.497 INFO:tasks.workunit.client.1.smithi203.stdout:7/792: creat d1/d11/d3d/d9a/da4/f10e x:0 0 0 2024-05-25T06:11:39.499 INFO:tasks.workunit.client.0.smithi160.stdout:9/486: rename d4/d11/d12/f85 to d4/d11/d1e/d24/d66/f94 0 2024-05-25T06:11:39.499 INFO:tasks.workunit.client.0.smithi160.stdout:7/549: symlink d6/d32/d5b/lb1 0 2024-05-25T06:11:39.501 INFO:tasks.workunit.client.1.smithi203.stdout:7/793: stat d1/d9/d60/cf7 0 2024-05-25T06:11:39.502 INFO:tasks.workunit.client.1.smithi203.stdout:7/794: readlink d1/d9/d2c/d79/dd6/l10a 0 2024-05-25T06:11:39.502 INFO:tasks.workunit.client.0.smithi160.stdout:6/508: creat d5/de/d32/d74/d77/fb1 x:0 0 0 2024-05-25T06:11:39.503 INFO:tasks.workunit.client.0.smithi160.stdout:8/575: link d5/d44/d1f/da3/d28/d49/d61/fad d5/d9/d72/fbe 0 2024-05-25T06:11:39.505 INFO:tasks.workunit.client.1.smithi203.stdout:7/795: fdatasync d1/d9/f2e 0 2024-05-25T06:11:39.506 INFO:tasks.workunit.client.0.smithi160.stdout:1/399: unlink de/d21/d49/d1c/d41/d45/l66 0 2024-05-25T06:11:39.507 INFO:tasks.workunit.client.0.smithi160.stdout:2/466: mkdir d12/d47/da0 0 2024-05-25T06:11:39.507 INFO:tasks.workunit.client.0.smithi160.stdout:8/576: write d5/d44/f68 [4308400,61953] 0 2024-05-25T06:11:39.510 INFO:tasks.workunit.client.0.smithi160.stdout:7/550: link d6/d2b/d5d/d89/f90 d6/d2b/fb2 0 2024-05-25T06:11:39.511 INFO:tasks.workunit.client.1.smithi203.stdout:7/796: creat d1/f10f x:0 0 0 2024-05-25T06:11:39.511 INFO:tasks.workunit.client.0.smithi160.stdout:8/577: stat d5/d44/d1f/da3/d28/d4a/d4e/l5b 0 2024-05-25T06:11:39.512 INFO:tasks.workunit.client.0.smithi160.stdout:2/467: symlink d12/d13/la1 0 2024-05-25T06:11:39.512 INFO:tasks.workunit.client.0.smithi160.stdout:5/542: dread db/dc/d13/f65 [0,4194304] 0 2024-05-25T06:11:39.512 INFO:tasks.workunit.client.0.smithi160.stdout:1/400: mkdir de/d21/d49/d1c/d59/d7f 0 2024-05-25T06:11:39.512 INFO:tasks.workunit.client.0.smithi160.stdout:6/509: dwrite d5/d45/d63/f83 [0,4194304] 0 2024-05-25T06:11:39.515 INFO:tasks.workunit.client.0.smithi160.stdout:7/551: symlink d6/d2b/d8b/lb3 0 2024-05-25T06:11:39.516 INFO:tasks.workunit.client.0.smithi160.stdout:5/543: fdatasync db/dc/d13/d1d/d23/f46 0 2024-05-25T06:11:39.518 INFO:tasks.workunit.client.0.smithi160.stdout:2/468: creat d12/d1c/d2a/d7e/fa2 x:0 0 0 2024-05-25T06:11:39.519 INFO:tasks.workunit.client.0.smithi160.stdout:2/469: chown d12/d3c/l54 5095 1 2024-05-25T06:11:39.519 INFO:tasks.workunit.client.0.smithi160.stdout:5/544: read db/f53 [1357172,99640] 0 2024-05-25T06:11:39.521 INFO:tasks.workunit.client.1.smithi203.stdout:2/850: write d2/d38/f29 [58183,84103] 0 2024-05-25T06:11:39.524 INFO:tasks.workunit.client.1.smithi203.stdout:2/851: rmdir d2/d1a/dd3/d100/d10/d58 39 2024-05-25T06:11:39.525 INFO:tasks.workunit.client.1.smithi203.stdout:8/910: dwrite d2/d5/d10/d11/de/f16 [0,4194304] 0 2024-05-25T06:11:39.526 INFO:tasks.workunit.client.1.smithi203.stdout:2/852: write d2/d38/d43/d60/f96 [774891,42851] 0 2024-05-25T06:11:39.528 INFO:tasks.workunit.client.1.smithi203.stdout:7/797: truncate d1/d9/d2c/d36/fbc 1174120 0 2024-05-25T06:11:39.530 INFO:tasks.workunit.client.1.smithi203.stdout:8/911: mknod d2/d5/d28/d38/d4e/d97/ddf/df0/c13a 0 2024-05-25T06:11:39.534 INFO:tasks.workunit.client.1.smithi203.stdout:7/798: rename d1/d9/d2c/d36/d3b/faa to d1/d9/d2c/f110 0 2024-05-25T06:11:39.547 INFO:tasks.workunit.client.0.smithi160.stdout:4/469: sync 2024-05-25T06:11:39.550 INFO:tasks.workunit.client.0.smithi160.stdout:1/401: mknod de/d21/d49/d1c/d59/d71/c80 0 2024-05-25T06:11:39.551 INFO:tasks.workunit.client.1.smithi203.stdout:7/799: creat d1/d9/d2c/d62/db3/f111 x:0 0 0 2024-05-25T06:11:39.552 INFO:tasks.workunit.client.0.smithi160.stdout:2/470: symlink d12/d3c/la3 0 2024-05-25T06:11:39.553 INFO:tasks.workunit.client.0.smithi160.stdout:6/510: dwrite d5/f3f [0,4194304] 0 2024-05-25T06:11:39.554 INFO:tasks.workunit.client.1.smithi203.stdout:7/800: write d1/d11/d3d/f7f [3443941,98511] 0 2024-05-25T06:11:39.554 INFO:tasks.workunit.client.0.smithi160.stdout:5/545: write db/dc/d13/d1d/d31/d56/f7d [1815516,114782] 0 2024-05-25T06:11:39.555 INFO:tasks.workunit.client.0.smithi160.stdout:7/552: symlink d6/dc/d64/dae/lb4 0 2024-05-25T06:11:39.555 INFO:tasks.workunit.client.1.smithi203.stdout:2/853: write d2/d38/d43/d60/f108 [1025459,50407] 0 2024-05-25T06:11:39.558 INFO:tasks.workunit.client.1.smithi203.stdout:2/854: write d2/d1a/d33/d80/fed [2842815,78494] 0 2024-05-25T06:11:39.559 INFO:tasks.workunit.client.0.smithi160.stdout:1/402: write de/f54 [595512,19470] 0 2024-05-25T06:11:39.561 INFO:tasks.workunit.client.0.smithi160.stdout:6/511: read d5/de/f22 [551032,90229] 0 2024-05-25T06:11:39.562 INFO:tasks.workunit.client.0.smithi160.stdout:1/403: mkdir de/d21/d49/d1c/d41/d81 0 2024-05-25T06:11:39.563 INFO:tasks.workunit.client.1.smithi203.stdout:8/912: dwrite d2/d5/d18/d31/fcf [0,4194304] 0 2024-05-25T06:11:39.566 INFO:tasks.workunit.client.0.smithi160.stdout:7/553: fdatasync d6/dc/d64/fb0 0 2024-05-25T06:11:39.567 INFO:tasks.workunit.client.1.smithi203.stdout:7/801: symlink d1/d9/d2c/d3a/d4e/l112 0 2024-05-25T06:11:39.568 INFO:tasks.workunit.client.1.smithi203.stdout:8/913: read d2/d5/d28/f4c [924024,122482] 0 2024-05-25T06:11:39.569 INFO:tasks.workunit.client.1.smithi203.stdout:2/855: mknod d2/d1a/d89/dfd/c11f 0 2024-05-25T06:11:39.571 INFO:tasks.workunit.client.0.smithi160.stdout:5/546: dwrite db/dc/d13/d1d/d31/d56/f61 [0,4194304] 0 2024-05-25T06:11:39.573 INFO:tasks.workunit.client.1.smithi203.stdout:7/802: creat d1/d9/d60/dea/f113 x:0 0 0 2024-05-25T06:11:39.575 INFO:tasks.workunit.client.0.smithi160.stdout:2/471: write d12/f7b [1357986,23840] 0 2024-05-25T06:11:39.576 INFO:tasks.workunit.client.0.smithi160.stdout:4/470: write d4/f4f [2180030,85555] 0 2024-05-25T06:11:39.578 INFO:tasks.workunit.client.0.smithi160.stdout:1/404: creat de/d21/d49/f82 x:0 0 0 2024-05-25T06:11:39.579 INFO:tasks.workunit.client.1.smithi203.stdout:8/914: getdents d2/d5/d18/d139 0 2024-05-25T06:11:39.579 INFO:tasks.workunit.client.0.smithi160.stdout:4/471: chown d4/d6/d5b/d66/c8a 17 1 2024-05-25T06:11:39.582 INFO:tasks.workunit.client.0.smithi160.stdout:1/405: write de/d21/d49/f82 [393257,88765] 0 2024-05-25T06:11:39.583 INFO:tasks.workunit.client.0.smithi160.stdout:6/512: dwrite d5/f1e [0,4194304] 0 2024-05-25T06:11:39.583 INFO:tasks.workunit.client.1.smithi203.stdout:7/803: truncate d1/d9/d2c/d3a/dd1/fdd 1749078 0 2024-05-25T06:11:39.584 INFO:tasks.workunit.client.0.smithi160.stdout:4/472: symlink d4/d6/d28/l90 0 2024-05-25T06:11:39.585 INFO:tasks.workunit.client.0.smithi160.stdout:5/547: dread - db/dc/d13/d1d/d3f/d72/d98/f9b zero size 2024-05-25T06:11:39.586 INFO:tasks.workunit.client.1.smithi203.stdout:4/962: sync 2024-05-25T06:11:39.586 INFO:tasks.workunit.client.0.smithi160.stdout:2/472: creat d12/d3e/d5f/fa4 x:0 0 0 2024-05-25T06:11:39.589 INFO:tasks.workunit.client.0.smithi160.stdout:7/554: dread d6/d3e/d6f/f76 [0,4194304] 0 2024-05-25T06:11:39.590 INFO:tasks.workunit.client.1.smithi203.stdout:2/856: dwrite d2/d1a/dd3/d100/d10/d32/fb6 [0,4194304] 0 2024-05-25T06:11:39.593 INFO:tasks.workunit.client.1.smithi203.stdout:4/963: creat d0/d8/d3a/d81/d93/d98/d99/ded/dfa/f147 x:0 0 0 2024-05-25T06:11:39.594 INFO:tasks.workunit.client.1.smithi203.stdout:4/964: chown d0/d9/d50/d6b/lda 142515 1 2024-05-25T06:11:39.595 INFO:tasks.workunit.client.1.smithi203.stdout:2/857: mknod d2/d1a/d33/d80/dd1/c120 0 2024-05-25T06:11:39.597 INFO:tasks.workunit.client.0.smithi160.stdout:6/513: dwrite d5/d30/d36/d73/f95 [0,4194304] 0 2024-05-25T06:11:39.597 INFO:tasks.workunit.client.0.smithi160.stdout:5/548: mkdir db/dc/d13/d1d/d3f/d72/d98/da5 0 2024-05-25T06:11:39.597 INFO:tasks.workunit.client.0.smithi160.stdout:4/473: dwrite d4/d6/d21/f27 [4194304,4194304] 0 2024-05-25T06:11:39.598 INFO:tasks.workunit.client.0.smithi160.stdout:7/555: mknod d6/d32/d5b/cb5 0 2024-05-25T06:11:39.599 INFO:tasks.workunit.client.1.smithi203.stdout:2/858: truncate d2/d1a/dd3/d100/d10/f11a 35544 0 2024-05-25T06:11:39.600 INFO:tasks.workunit.client.1.smithi203.stdout:7/804: dread d1/d11/d20/f27 [0,4194304] 0 2024-05-25T06:11:39.601 INFO:tasks.workunit.client.1.smithi203.stdout:8/915: truncate d2/d5/d2d/f4b 4851166 0 2024-05-25T06:11:39.602 INFO:tasks.workunit.client.0.smithi160.stdout:5/549: stat db/dc/d13/f6a 0 2024-05-25T06:11:39.605 INFO:tasks.workunit.client.0.smithi160.stdout:1/406: dwrite de/d21/d49/d1c/f3f [8388608,4194304] 0 2024-05-25T06:11:39.606 INFO:tasks.workunit.client.0.smithi160.stdout:7/556: chown d6/dc/d64/d5a/fa6 7 1 2024-05-25T06:11:39.608 INFO:tasks.workunit.client.1.smithi203.stdout:4/965: dwrite d0/d8/d3a/d81/d93/d98/d99/ded/f11c [0,4194304] 0 2024-05-25T06:11:39.611 INFO:tasks.workunit.client.1.smithi203.stdout:2/859: link d2/lbf d2/d38/l121 0 2024-05-25T06:11:39.613 INFO:tasks.workunit.client.0.smithi160.stdout:5/550: mkdir db/dc/d13/d1d/d23/d24/d2e/da6 0 2024-05-25T06:11:39.616 INFO:tasks.workunit.client.0.smithi160.stdout:7/557: symlink d6/lb6 0 2024-05-25T06:11:39.617 INFO:tasks.workunit.client.0.smithi160.stdout:7/558: fsync d6/d2b/f6b 0 2024-05-25T06:11:39.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.617+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/4192932236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fb1e0 msgr2=0x7fbb240fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.617+0000 7fbb2b4c7700 1 --2- 172.21.15.160:0/4192932236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fb1e0 0x7fbb240fb5f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbb20004940 tx=0x7fbb20009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.617+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/4192932236 shutdown_connections 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.617+0000 7fbb2b4c7700 1 --2- 172.21.15.160:0/4192932236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fbb30 0x7fbb24102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.617+0000 7fbb2b4c7700 1 --2- 172.21.15.160:0/4192932236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fb1e0 0x7fbb240fb5f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.617+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/4192932236 >> 172.21.15.160:0/4192932236 conn(0x7fbb240f6b80 msgr2=0x7fbb240f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.618+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/4192932236 shutdown_connections 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.618+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/4192932236 wait complete. 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.618+0000 7fbb2b4c7700 1 Processor -- start 2024-05-25T06:11:39.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.619+0000 7fbb2b4c7700 1 -- start start 2024-05-25T06:11:39.619 INFO:tasks.workunit.client.0.smithi160.stdout:4/474: link d4/d6/d5b/d66/d82/c8c d4/d6/d5b/d66/c91 0 2024-05-25T06:11:39.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.619+0000 7fbb2b4c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 0x7fbb24114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:39.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.619+0000 7fbb2b4c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fbb30 0x7fbb24114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:39.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.620+0000 7fbb2b4c7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24114bd0 con 0x7fbb240fbb30 2024-05-25T06:11:39.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.620+0000 7fbb2b4c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241153c0 con 0x7fbb240fb1e0 2024-05-25T06:11:39.621 INFO:tasks.workunit.client.0.smithi160.stdout:4/475: write d4/d1c/f62 [4607070,74938] 0 2024-05-25T06:11:39.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.620+0000 7fbb29263700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 0x7fbb24114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:39.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.620+0000 7fbb29263700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 0x7fbb24114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34700/0 (socket says 172.21.15.160:34700) 2024-05-25T06:11:39.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.620+0000 7fbb29263700 1 -- 172.21.15.160:0/3845372836 learned_addr learned my addr 172.21.15.160:0/3845372836 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:39.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.620+0000 7fbb28a62700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fbb30 0x7fbb24114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:39.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb29263700 1 -- 172.21.15.160:0/3845372836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fbb30 msgr2=0x7fbb24114540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:39.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb29263700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fbb30 0x7fbb24114540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb29263700 1 -- 172.21.15.160:0/3845372836 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb20005040 con 0x7fbb240fb1e0 2024-05-25T06:11:39.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb29263700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 0x7fbb24114000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000b3b0 tx=0x7fbb20009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:39.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2000e040 con 0x7fbb240fb1e0 2024-05-25T06:11:39.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/3845372836 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb24115640 con 0x7fbb240fb1e0 2024-05-25T06:11:39.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb2000bbf0 con 0x7fbb240fb1e0 2024-05-25T06:11:39.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.621+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2001c630 con 0x7fbb240fb1e0 2024-05-25T06:11:39.623 INFO:tasks.workunit.client.0.smithi160.stdout:4/476: write d4/d63/f81 [893527,86711] 0 2024-05-25T06:11:39.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.622+0000 7fbb2b4c7700 1 -- 172.21.15.160:0/3845372836 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb24115c00 con 0x7fbb240fb1e0 2024-05-25T06:11:39.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.623+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb040052b0 con 0x7fbb240fb1e0 2024-05-25T06:11:39.624 INFO:tasks.workunit.client.0.smithi160.stdout:7/559: creat d6/d2b/da0/fb7 x:0 0 0 2024-05-25T06:11:39.624 INFO:tasks.workunit.client.1.smithi203.stdout:2/860: creat d2/d1a/d89/daa/def/d107/d113/f122 x:0 0 0 2024-05-25T06:11:39.625 INFO:tasks.workunit.client.1.smithi203.stdout:2/861: read - d2/d38/d43/d60/d6b/f10b zero size 2024-05-25T06:11:39.625 INFO:tasks.workunit.client.1.smithi203.stdout:4/966: dread d0/d8/d10/d91/fe7 [0,4194304] 0 2024-05-25T06:11:39.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.625+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fbb20013420 con 0x7fbb240fb1e0 2024-05-25T06:11:39.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.626+0000 7fbb19ffb700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fbb1006b860 0x7fbb1006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:39.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.626+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fbb2008b4c0 con 0x7fbb240fb1e0 2024-05-25T06:11:39.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.626+0000 7fbb28a62700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fbb1006b860 0x7fbb1006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:39.627 INFO:tasks.workunit.client.0.smithi160.stdout:4/477: write d4/d6/d5b/d66/d82/f84 [596879,122016] 0 2024-05-25T06:11:39.627 INFO:tasks.workunit.client.0.smithi160.stdout:5/551: getdents db/dc/d13/d1d/d3f/d72 0 2024-05-25T06:11:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.628+0000 7fbb28a62700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fbb1006b860 0x7fbb1006dd10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009bf0 tx=0x7fbb140091a0 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:39.628 INFO:tasks.workunit.client.1.smithi203.stdout:2/862: dread d2/d38/f7d [0,4194304] 0 2024-05-25T06:11:39.630 INFO:tasks.workunit.client.1.smithi203.stdout:2/863: readlink d2/d1a/dd3/d100/da6/lb3 0 2024-05-25T06:11:39.630 INFO:tasks.workunit.client.0.smithi160.stdout:5/552: creat db/fa7 x:0 0 0 2024-05-25T06:11:39.631 INFO:tasks.workunit.client.1.smithi203.stdout:4/967: dread d0/d9/d11/f125 [0,4194304] 0 2024-05-25T06:11:39.631 INFO:tasks.workunit.client.1.smithi203.stdout:2/864: chown d2/d1a/dd3/d100/d10/f9b 254009150 1 2024-05-25T06:11:39.632 INFO:tasks.workunit.client.1.smithi203.stdout:7/805: dread d1/d9/d2c/d79/dd6/f108 [0,4194304] 0 2024-05-25T06:11:39.632 INFO:tasks.workunit.client.0.smithi160.stdout:7/560: dwrite d6/d32/d5b/f62 [0,4194304] 0 2024-05-25T06:11:39.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.634+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fbb20058c50 con 0x7fbb240fb1e0 2024-05-25T06:11:39.636 INFO:tasks.workunit.client.1.smithi203.stdout:2/865: creat d2/d1a/dd3/d100/d10/d8c/f123 x:0 0 0 2024-05-25T06:11:39.639 INFO:tasks.workunit.client.1.smithi203.stdout:4/968: unlink d0/d8/d1f/d26/d3b/df5/cff 0 2024-05-25T06:11:39.640 INFO:tasks.workunit.client.0.smithi160.stdout:5/553: dwrite db/d47/f68 [4194304,4194304] 0 2024-05-25T06:11:39.645 INFO:tasks.workunit.client.0.smithi160.stdout:5/554: symlink db/dc/d13/d1d/d31/la8 0 2024-05-25T06:11:39.646 INFO:tasks.workunit.client.1.smithi203.stdout:2/866: dread d2/d38/d43/d60/f108 [0,4194304] 0 2024-05-25T06:11:39.647 INFO:tasks.workunit.client.1.smithi203.stdout:8/916: write d2/d5/d10/d11/de/f85 [415270,80774] 0 2024-05-25T06:11:39.647 INFO:tasks.workunit.client.1.smithi203.stdout:2/867: write d2/d1a/f8b [1324888,15571] 0 2024-05-25T06:11:39.652 INFO:tasks.workunit.client.0.smithi160.stdout:5/555: dwrite f8 [0,4194304] 0 2024-05-25T06:11:39.652 INFO:tasks.workunit.client.1.smithi203.stdout:8/917: chown d2/d5/d18/d31/dac/dc2/d88/d93/d9d/f132 23 1 2024-05-25T06:11:39.654 INFO:tasks.workunit.client.1.smithi203.stdout:8/918: dread - d2/d5/d10/d45/d5d/fc5 zero size 2024-05-25T06:11:39.654 INFO:tasks.workunit.client.1.smithi203.stdout:2/868: read d2/f6c [461907,12487] 0 2024-05-25T06:11:39.660 INFO:tasks.workunit.client.1.smithi203.stdout:2/869: dread d2/d42/d8e/fcb [0,4194304] 0 2024-05-25T06:11:39.660 INFO:tasks.workunit.client.0.smithi160.stdout:7/561: truncate d6/d3e/f4b 4084498 0 2024-05-25T06:11:39.662 INFO:tasks.workunit.client.0.smithi160.stdout:7/562: chown d6/d3e/c67 23219544 1 2024-05-25T06:11:39.664 INFO:tasks.workunit.client.1.smithi203.stdout:7/806: dread d1/d11/f21 [0,4194304] 0 2024-05-25T06:11:39.665 INFO:tasks.workunit.client.1.smithi203.stdout:2/870: read d2/d1a/dd3/d100/d10/d32/f52 [4414736,129237] 0 2024-05-25T06:11:39.665 INFO:tasks.workunit.client.1.smithi203.stdout:8/919: dwrite d2/d5/d2d/f3e [4194304,4194304] 0 2024-05-25T06:11:39.666 INFO:tasks.workunit.client.0.smithi160.stdout:1/407: dread de/f15 [0,4194304] 0 2024-05-25T06:11:39.666 INFO:tasks.workunit.client.1.smithi203.stdout:7/807: chown d1/d9/f2e 7940 1 2024-05-25T06:11:39.668 INFO:tasks.workunit.client.0.smithi160.stdout:5/556: dwrite db/d47/f66 [0,4194304] 0 2024-05-25T06:11:39.668 INFO:tasks.workunit.client.0.smithi160.stdout:4/478: dread d4/d6/d28/f17 [8388608,4194304] 0 2024-05-25T06:11:39.669 INFO:tasks.workunit.client.1.smithi203.stdout:2/871: creat d2/d1a/d89/daa/def/d107/d113/f124 x:0 0 0 2024-05-25T06:11:39.670 INFO:tasks.workunit.client.0.smithi160.stdout:1/408: readlink de/d43/l4a 0 2024-05-25T06:11:39.670 INFO:tasks.workunit.client.1.smithi203.stdout:4/969: dread d0/d9/f17 [0,4194304] 0 2024-05-25T06:11:39.671 INFO:tasks.workunit.client.1.smithi203.stdout:4/970: chown d0/d9/d50/d6b/c6c 73 1 2024-05-25T06:11:39.672 INFO:tasks.workunit.client.1.smithi203.stdout:4/971: chown d0/d8/d52/d59/d7a/lec 258292 1 2024-05-25T06:11:39.674 INFO:tasks.workunit.client.1.smithi203.stdout:7/808: symlink d1/d11/d3d/l114 0 2024-05-25T06:11:39.674 INFO:tasks.workunit.client.0.smithi160.stdout:4/479: fsync d4/d6/f7 0 2024-05-25T06:11:39.676 INFO:tasks.workunit.client.1.smithi203.stdout:8/920: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/d113/f13b x:0 0 0 2024-05-25T06:11:39.677 INFO:tasks.workunit.client.0.smithi160.stdout:5/557: rename db/dc/d13/d9a/f9f to db/fa9 0 2024-05-25T06:11:39.677 INFO:tasks.workunit.client.1.smithi203.stdout:2/872: mknod d2/d1a/d87/c125 0 2024-05-25T06:11:39.678 INFO:tasks.workunit.client.0.smithi160.stdout:4/480: symlink d4/d1c/d54/l92 0 2024-05-25T06:11:39.680 INFO:tasks.workunit.client.0.smithi160.stdout:4/481: truncate d4/d6/d28/f25 1591439 0 2024-05-25T06:11:39.682 INFO:tasks.workunit.client.1.smithi203.stdout:2/873: dread - d2/d42/d4e/f92 zero size 2024-05-25T06:11:39.684 INFO:tasks.workunit.client.0.smithi160.stdout:4/482: symlink d4/d6/d28/l93 0 2024-05-25T06:11:39.685 INFO:tasks.workunit.client.1.smithi203.stdout:2/874: write d2/d1a/d87/dd9/ffe [1402063,86450] 0 2024-05-25T06:11:39.687 INFO:tasks.workunit.client.0.smithi160.stdout:1/409: dread de/d21/d49/f1d [0,4194304] 0 2024-05-25T06:11:39.687 INFO:tasks.workunit.client.0.smithi160.stdout:4/483: mkdir d4/d63/d70/d94 0 2024-05-25T06:11:39.690 INFO:tasks.workunit.client.1.smithi203.stdout:2/875: creat d2/d1a/dd3/d100/d10/d58/f126 x:0 0 0 2024-05-25T06:11:39.692 INFO:tasks.workunit.client.1.smithi203.stdout:2/876: stat d2/d1a/d89/daa/def/d107/d113/f124 0 2024-05-25T06:11:39.692 INFO:tasks.workunit.client.0.smithi160.stdout:1/410: truncate f2 5653223 0 2024-05-25T06:11:39.694 INFO:tasks.workunit.client.0.smithi160.stdout:4/484: dwrite d4/d1a/f59 [0,4194304] 0 2024-05-25T06:11:39.695 INFO:tasks.workunit.client.1.smithi203.stdout:2/877: symlink d2/d1a/dd3/d100/da6/l127 0 2024-05-25T06:11:39.697 INFO:tasks.workunit.client.1.smithi203.stdout:7/809: dread d1/d9/d2c/d36/f40 [0,4194304] 0 2024-05-25T06:11:39.697 INFO:tasks.workunit.client.0.smithi160.stdout:4/485: truncate d4/d1a/f8b 223280 0 2024-05-25T06:11:39.700 INFO:tasks.workunit.client.0.smithi160.stdout:4/486: readlink d4/d6/d35/l41 0 2024-05-25T06:11:39.701 INFO:tasks.workunit.client.0.smithi160.stdout:5/558: truncate db/dc/d13/d1d/f20 3780666 0 2024-05-25T06:11:39.706 INFO:tasks.workunit.client.1.smithi203.stdout:7/810: dwrite d1/d11/f105 [0,4194304] 0 2024-05-25T06:11:39.708 INFO:tasks.workunit.client.1.smithi203.stdout:7/811: stat d1/d9/d2c/d36/d54/ffb 0 2024-05-25T06:11:39.709 INFO:tasks.workunit.client.0.smithi160.stdout:4/487: truncate d4/d6/d35/f53 10648258 0 2024-05-25T06:11:39.712 INFO:tasks.workunit.client.0.smithi160.stdout:4/488: mknod d4/d63/d64/c95 0 2024-05-25T06:11:39.713 INFO:tasks.workunit.client.0.smithi160.stdout:4/489: write d4/d1c/f3e [2234120,130513] 0 2024-05-25T06:11:39.714 INFO:tasks.workunit.client.0.smithi160.stdout:5/559: link db/dc/f93 db/dc/d13/d1d/faa 0 2024-05-25T06:11:39.714 INFO:tasks.workunit.client.0.smithi160.stdout:5/560: write db/fa7 [866398,112717] 0 2024-05-25T06:11:39.715 INFO:tasks.workunit.client.0.smithi160.stdout:4/490: symlink d4/d6/d35/d3d/d55/l96 0 2024-05-25T06:11:39.716 INFO:tasks.workunit.client.0.smithi160.stdout:5/561: write db/d47/f66 [1850329,67144] 0 2024-05-25T06:11:39.718 INFO:tasks.workunit.client.1.smithi203.stdout:7/812: write d1/d11/f21 [1534248,119710] 0 2024-05-25T06:11:39.719 INFO:tasks.workunit.client.0.smithi160.stdout:4/491: creat d4/d6/f97 x:0 0 0 2024-05-25T06:11:39.720 INFO:tasks.workunit.client.0.smithi160.stdout:5/562: mknod db/dc/d13/d1d/cab 0 2024-05-25T06:11:39.721 INFO:tasks.workunit.client.1.smithi203.stdout:7/813: creat d1/d9/d2c/d3a/dd1/f115 x:0 0 0 2024-05-25T06:11:39.722 INFO:tasks.workunit.client.0.smithi160.stdout:4/492: unlink d4/d6/d21/d42/c46 0 2024-05-25T06:11:39.722 INFO:tasks.workunit.client.0.smithi160.stdout:4/493: write d4/d1c/d54/f65 [3594892,88060] 0 2024-05-25T06:11:39.723 INFO:tasks.workunit.client.0.smithi160.stdout:4/494: truncate d4/f76 637388 0 2024-05-25T06:11:39.725 INFO:tasks.workunit.client.0.smithi160.stdout:4/495: symlink d4/d1c/d54/l98 0 2024-05-25T06:11:39.726 INFO:tasks.workunit.client.1.smithi203.stdout:7/814: write d1/d9/f76 [1983531,122799] 0 2024-05-25T06:11:39.727 INFO:tasks.workunit.client.0.smithi160.stdout:0/571: sync 2024-05-25T06:11:39.733 INFO:tasks.workunit.client.1.smithi203.stdout:7/815: dwrite d1/d9/d2c/d3a/d4e/f75 [0,4194304] 0 2024-05-25T06:11:39.733 INFO:tasks.workunit.client.0.smithi160.stdout:0/572: symlink d2b/d46/d63/db3/lb4 0 2024-05-25T06:11:39.736 INFO:tasks.workunit.client.0.smithi160.stdout:0/573: mkdir d2b/d46/d63/da7/db5 0 2024-05-25T06:11:39.737 INFO:tasks.workunit.client.0.smithi160.stdout:0/574: chown d2b/d46/d63/da7/da6 41 1 2024-05-25T06:11:39.739 INFO:tasks.workunit.client.1.smithi203.stdout:8/921: dread d2/d5/d10/d11/de/f1d [0,4194304] 0 2024-05-25T06:11:39.739 INFO:tasks.workunit.client.0.smithi160.stdout:0/575: write d2b/d46/f5a [3172835,122204] 0 2024-05-25T06:11:39.740 INFO:tasks.workunit.client.0.smithi160.stdout:5/563: dread db/dc/d13/d1d/d23/d24/d2e/f5b [0,4194304] 0 2024-05-25T06:11:39.741 INFO:tasks.workunit.client.0.smithi160.stdout:1/411: write f5 [3281330,78979] 0 2024-05-25T06:11:39.743 INFO:tasks.workunit.client.0.smithi160.stdout:0/576: write d2b/d31/d35/d36/f89 [880394,40814] 0 2024-05-25T06:11:39.743 INFO:tasks.workunit.client.1.smithi203.stdout:7/816: rename d1/l7e to d1/dbd/dc1/df3/l116 0 2024-05-25T06:11:39.744 INFO:tasks.workunit.client.1.smithi203.stdout:7/817: chown d1/d9/d2c/d3a/d4e/l67 9070 1 2024-05-25T06:11:39.747 INFO:tasks.workunit.client.0.smithi160.stdout:5/564: read db/dc/d13/d1d/d31/d3b/f2b [739044,125805] 0 2024-05-25T06:11:39.748 INFO:tasks.workunit.client.0.smithi160.stdout:1/412: creat de/d21/d49/d1c/d59/d67/f83 x:0 0 0 2024-05-25T06:11:39.750 INFO:tasks.workunit.client.0.smithi160.stdout:0/577: dread f1f [0,4194304] 0 2024-05-25T06:11:39.750 INFO:tasks.workunit.client.1.smithi203.stdout:7/818: rmdir d1/d9/d2c/d36/d54/dd0 0 2024-05-25T06:11:39.753 INFO:tasks.workunit.client.0.smithi160.stdout:1/413: creat de/d43/f84 x:0 0 0 2024-05-25T06:11:39.753 INFO:tasks.workunit.client.0.smithi160.stdout:5/565: truncate db/dc/d13/f1f 1339320 0 2024-05-25T06:11:39.754 INFO:tasks.workunit.client.1.smithi203.stdout:7/819: dread d1/d9/d2c/d3a/d4e/f5b [0,4194304] 0 2024-05-25T06:11:39.754 INFO:tasks.workunit.client.1.smithi203.stdout:8/922: write d2/d5/d10/d4a/db6/f126 [1009318,73213] 0 2024-05-25T06:11:39.754 INFO:tasks.workunit.client.0.smithi160.stdout:1/414: chown de/l1e 54250275 1 2024-05-25T06:11:39.755 INFO:tasks.workunit.client.0.smithi160.stdout:5/566: read f1 [1534462,33264] 0 2024-05-25T06:11:39.757 INFO:tasks.workunit.client.1.smithi203.stdout:8/923: symlink d2/d5/d10/d45/d5d/d68/d118/l13c 0 2024-05-25T06:11:39.758 INFO:tasks.workunit.client.0.smithi160.stdout:5/567: creat db/dc/d13/d1d/d3f/d72/d98/da5/fac x:0 0 0 2024-05-25T06:11:39.759 INFO:tasks.workunit.client.1.smithi203.stdout:8/924: fsync d2/d5/d18/d31/dac/dc2/d8a/d8f/ff4 0 2024-05-25T06:11:39.760 INFO:tasks.workunit.client.1.smithi203.stdout:7/820: dwrite d1/d9/d2c/d3a/dd1/f115 [0,4194304] 0 2024-05-25T06:11:39.761 INFO:tasks.workunit.client.1.smithi203.stdout:8/925: fdatasync d2/d5/d10/d45/d90/d100/f111 0 2024-05-25T06:11:39.763 INFO:tasks.workunit.client.1.smithi203.stdout:7/821: write d1/d11/d3d/d4c/f93 [4668365,37426] 0 2024-05-25T06:11:39.764 INFO:tasks.workunit.client.0.smithi160.stdout:5/568: dwrite db/dc/d13/d54/f95 [0,4194304] 0 2024-05-25T06:11:39.770 INFO:tasks.workunit.client.0.smithi160.stdout:4/496: write d4/d1a/f50 [1261297,85772] 0 2024-05-25T06:11:39.771 INFO:tasks.workunit.client.0.smithi160.stdout:4/497: chown d4/d63/d64/c95 11121 1 2024-05-25T06:11:39.776 INFO:tasks.workunit.client.0.smithi160.stdout:5/569: dwrite db/fa9 [0,4194304] 0 2024-05-25T06:11:39.778 INFO:tasks.workunit.client.0.smithi160.stdout:4/498: getdents d4/d6/d35/d3d 0 2024-05-25T06:11:39.779 INFO:tasks.workunit.client.1.smithi203.stdout:8/926: dwrite d2/d5/d18/fb1 [0,4194304] 0 2024-05-25T06:11:39.781 INFO:tasks.workunit.client.1.smithi203.stdout:7/822: dwrite d1/d9/d2c/fe7 [0,4194304] 0 2024-05-25T06:11:39.782 INFO:tasks.workunit.client.0.smithi160.stdout:5/570: mknod db/dc/d13/d1d/cad 0 2024-05-25T06:11:39.783 INFO:tasks.workunit.client.0.smithi160.stdout:4/499: symlink d4/l99 0 2024-05-25T06:11:39.784 INFO:tasks.workunit.client.1.smithi203.stdout:8/927: write d2/d5/d18/dbf/f11b [2530547,8238] 0 2024-05-25T06:11:39.784 INFO:tasks.workunit.client.0.smithi160.stdout:4/500: chown d4/d6/d5b/d66/c6f 43051 1 2024-05-25T06:11:39.788 INFO:tasks.workunit.client.0.smithi160.stdout:0/578: write f1b [1845796,4162] 0 2024-05-25T06:11:39.790 INFO:tasks.workunit.client.1.smithi203.stdout:8/928: mkdir d2/d5/d10/dfa/d13d 0 2024-05-25T06:11:39.790 INFO:tasks.workunit.client.0.smithi160.stdout:5/571: rename db/dc/d13/d1d/d31/d3b/d69/d94 to db/dc/d13/d1d/d3f/d72/d98/da5/dae 0 2024-05-25T06:11:39.793 INFO:tasks.workunit.client.0.smithi160.stdout:4/501: symlink d4/l9a 0 2024-05-25T06:11:39.796 INFO:tasks.workunit.client.0.smithi160.stdout:5/572: dread db/dc/d13/d54/f95 [0,4194304] 0 2024-05-25T06:11:39.797 INFO:tasks.workunit.client.0.smithi160.stdout:4/502: mkdir d4/d1c/d54/d9b 0 2024-05-25T06:11:39.798 INFO:tasks.workunit.client.0.smithi160.stdout:5/573: truncate db/dc/d13/d54/f90 784260 0 2024-05-25T06:11:39.799 INFO:tasks.workunit.client.1.smithi203.stdout:8/929: link d2/d5/d28/d38/d4e/d97/c7c d2/d5/d18/d31/dac/dc2/d8a/d8f/c13e 0 2024-05-25T06:11:39.800 INFO:tasks.workunit.client.0.smithi160.stdout:0/579: creat d2b/d31/d35/d36/fb6 x:0 0 0 2024-05-25T06:11:39.808 INFO:tasks.workunit.client.0.smithi160.stdout:0/580: rename d2b/d31/d35/d36/d86/f9c to d2b/d46/d63/fb7 0 2024-05-25T06:11:39.808 INFO:tasks.workunit.client.1.smithi203.stdout:8/930: dwrite d2/d5/d18/d31/dac/dc2/f129 [0,4194304] 0 2024-05-25T06:11:39.809 INFO:tasks.workunit.client.0.smithi160.stdout:5/574: getdents db/dc/d13/d1d/d31/d3b/d69 0 2024-05-25T06:11:39.812 INFO:tasks.workunit.client.0.smithi160.stdout:5/575: unlink f4 0 2024-05-25T06:11:39.817 INFO:tasks.workunit.client.0.smithi160.stdout:5/576: rename db/dc/d13/d54 to db/dc/d13/d1d/d23/d24/daf 0 2024-05-25T06:11:39.818 INFO:tasks.workunit.client.0.smithi160.stdout:5/577: chown db/dc/d13/l18 241170 1 2024-05-25T06:11:39.820 INFO:tasks.workunit.client.0.smithi160.stdout:5/578: mknod db/dc/d13/d1d/d23/cb0 0 2024-05-25T06:11:39.828 INFO:tasks.workunit.client.0.smithi160.stdout:0/581: dread f2 [0,4194304] 0 2024-05-25T06:11:39.849 INFO:tasks.workunit.client.0.smithi160.stdout:5/579: fsync db/dc/d13/d1d/d31/d3b/d69/f7e 0 2024-05-25T06:11:39.851 INFO:tasks.workunit.client.0.smithi160.stdout:5/580: creat db/dc/d13/d9a/fb1 x:0 0 0 2024-05-25T06:11:39.854 INFO:tasks.workunit.client.0.smithi160.stdout:5/581: creat db/dc/d13/d1d/d23/d24/d2e/fb2 x:0 0 0 2024-05-25T06:11:39.855 INFO:tasks.workunit.client.0.smithi160.stdout:0/582: truncate d2b/f42 2571963 0 2024-05-25T06:11:39.857 INFO:tasks.workunit.client.0.smithi160.stdout:0/583: mkdir d2b/d46/d63/da7/d67/db8 0 2024-05-25T06:11:39.858 INFO:tasks.workunit.client.0.smithi160.stdout:5/582: dread db/dc/d13/d1d/d31/d3b/d69/f79 [0,4194304] 0 2024-05-25T06:11:39.860 INFO:tasks.workunit.client.0.smithi160.stdout:5/583: write db/dc/d13/d1d/d23/d24/f3e [2905511,11532] 0 2024-05-25T06:11:39.863 INFO:tasks.workunit.client.0.smithi160.stdout:5/584: fdatasync db/dc/d13/d1d/d31/d3b/d69/f79 0 2024-05-25T06:11:39.864 INFO:tasks.workunit.client.0.smithi160.stdout:5/585: write db/d47/d5e/f99 [165067,117988] 0 2024-05-25T06:11:39.867 INFO:tasks.workunit.client.0.smithi160.stdout:5/586: mkdir db/dc/d13/d1d/d23/d24/daf/db3 0 2024-05-25T06:11:39.869 INFO:tasks.workunit.client.0.smithi160.stdout:0/584: dwrite d2b/d46/d63/f7e [0,4194304] 0 2024-05-25T06:11:39.870 INFO:tasks.workunit.client.0.smithi160.stdout:5/587: write db/dc/d13/d1d/d31/d56/f82 [106369,122220] 0 2024-05-25T06:11:39.877 INFO:tasks.workunit.client.0.smithi160.stdout:5/588: dwrite db/dc/d13/d1d/d23/d45/f5f [4194304,4194304] 0 2024-05-25T06:11:39.882 INFO:tasks.workunit.client.0.smithi160.stdout:5/589: creat db/dc/fb4 x:0 0 0 2024-05-25T06:11:39.883 INFO:tasks.workunit.client.0.smithi160.stdout:0/585: dread f1b [0,4194304] 0 2024-05-25T06:11:39.884 INFO:tasks.workunit.client.0.smithi160.stdout:3/433: sync 2024-05-25T06:11:39.885 INFO:tasks.workunit.client.0.smithi160.stdout:0/586: write d2b/d46/d63/f6e [772301,4701] 0 2024-05-25T06:11:39.887 INFO:tasks.workunit.client.0.smithi160.stdout:3/434: write d19/d5d/d6a/f97 [103035,124105] 0 2024-05-25T06:11:39.889 INFO:tasks.workunit.client.0.smithi160.stdout:5/590: dwrite db/fa7 [0,4194304] 0 2024-05-25T06:11:39.891 INFO:tasks.workunit.client.0.smithi160.stdout:3/435: mknod d19/c9a 0 2024-05-25T06:11:39.892 INFO:tasks.workunit.client.0.smithi160.stdout:3/436: truncate d19/d2f/d34/f40 2357913 0 2024-05-25T06:11:39.893 INFO:tasks.workunit.client.0.smithi160.stdout:0/587: link d2b/d31/d35/d36/d86/fae d2b/da8/fb9 0 2024-05-25T06:11:39.894 INFO:tasks.workunit.client.0.smithi160.stdout:5/591: mkdir db/dc/d13/d1d/d3f/d72/d98/da5/db5 0 2024-05-25T06:11:39.895 INFO:tasks.workunit.client.0.smithi160.stdout:0/588: mknod d2b/d31/d35/d36/d37/cba 0 2024-05-25T06:11:39.897 INFO:tasks.workunit.client.0.smithi160.stdout:3/437: dread d19/f45 [0,4194304] 0 2024-05-25T06:11:39.899 INFO:tasks.workunit.client.0.smithi160.stdout:3/438: write d19/d22/f2c [21327,123435] 0 2024-05-25T06:11:39.900 INFO:tasks.workunit.client.0.smithi160.stdout:0/589: read d2b/d46/d63/da7/f85 [660594,59371] 0 2024-05-25T06:11:39.902 INFO:tasks.workunit.client.0.smithi160.stdout:3/439: rename d19/d2f/l8c to d19/d2f/d38/d60/l9b 0 2024-05-25T06:11:39.903 INFO:tasks.workunit.client.0.smithi160.stdout:3/440: readlink d19/l91 0 2024-05-25T06:11:39.904 INFO:tasks.workunit.client.0.smithi160.stdout:5/592: dwrite db/dc/d13/d1d/d23/f2d [0,4194304] 0 2024-05-25T06:11:39.904 INFO:tasks.workunit.client.0.smithi160.stdout:5/593: chown db/d47 15 1 2024-05-25T06:11:39.943 INFO:tasks.workunit.client.0.smithi160.stdout:0/590: rename d2b/d31/d35/d36/d37/l83 to d2b/d46/lbb 0 2024-05-25T06:11:39.945 INFO:tasks.workunit.client.0.smithi160.stdout:0/591: chown d2b/d31/d35/d36/d37 0 1 2024-05-25T06:11:39.947 INFO:tasks.workunit.client.0.smithi160.stdout:0/592: dread - d2b/d46/d63/da7/d67/fad zero size 2024-05-25T06:11:39.950 INFO:tasks.workunit.client.0.smithi160.stdout:3/441: dwrite d19/d5d/f61 [0,4194304] 0 2024-05-25T06:11:39.951 INFO:tasks.workunit.client.0.smithi160.stdout:0/593: write f2 [1894180,117592] 0 2024-05-25T06:11:39.958 INFO:tasks.workunit.client.0.smithi160.stdout:5/594: dwrite db/dc/d13/d1d/d31/d56/f7d [0,4194304] 0 2024-05-25T06:11:39.958 INFO:tasks.workunit.client.0.smithi160.stdout:5/595: fsync db/f85 0 2024-05-25T06:11:39.958 INFO:tasks.workunit.client.0.smithi160.stdout:3/442: truncate d19/d2f/d38/f78 815512 0 2024-05-25T06:11:39.958 INFO:tasks.workunit.client.0.smithi160.stdout:3/443: dread d19/f45 [0,4194304] 0 2024-05-25T06:11:39.959 INFO:tasks.workunit.client.0.smithi160.stdout:5/596: truncate db/d47/d5e/f99 484318 0 2024-05-25T06:11:39.961 INFO:tasks.workunit.client.0.smithi160.stdout:0/594: rename d2b/d31/d35/d36/d37/c69 to d2b/d46/d63/da7/d67/cbc 0 2024-05-25T06:11:39.963 INFO:tasks.workunit.client.0.smithi160.stdout:0/595: mkdir d2b/d46/dbd 0 2024-05-25T06:11:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.964+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb04000bc0 con 0x7fbb1006b860 2024-05-25T06:11:39.965 INFO:tasks.workunit.client.0.smithi160.stdout:5/597: rmdir db/dc/d13/d1d/d23/d24/d2e/da6 0 2024-05-25T06:11:39.965 INFO:tasks.workunit.client.0.smithi160.stdout:9/487: sync 2024-05-25T06:11:39.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.966+0000 7fbb19ffb700 1 -- 172.21.15.160:0/3845372836 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fbb04000bc0 con 0x7fbb1006b860 2024-05-25T06:11:39.966 INFO:tasks.workunit.client.0.smithi160.stdout:0/596: link d2b/d31/f7d d2b/d46/d63/da7/da6/fbe 0 2024-05-25T06:11:39.968 INFO:tasks.workunit.client.0.smithi160.stdout:5/598: rename db/dc/d13/d1d/d31/d3b/f38 to db/dc/d13/d1d/d3f/d72/d98/da5/dae/fb6 0 2024-05-25T06:11:39.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fbb1006b860 msgr2=0x7fbb1006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:39.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fbb1006b860 0x7fbb1006dd10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009bf0 tx=0x7fbb140091a0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 msgr2=0x7fbb24114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:39.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 0x7fbb24114000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000b3b0 tx=0x7fbb20009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:39.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 shutdown_connections 2024-05-25T06:11:39.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb240fb1e0 0x7fbb24114000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fbb1006b860 0x7fbb1006dd10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 --2- 172.21.15.160:0/3845372836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb240fbb30 0x7fbb24114540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:39.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 >> 172.21.15.160:0/3845372836 conn(0x7fbb240f6b80 msgr2=0x7fbb24106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:39.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 shutdown_connections 2024-05-25T06:11:39.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:39.969+0000 7fbb0f7fe700 1 -- 172.21.15.160:0/3845372836 wait complete. 2024-05-25T06:11:39.970 INFO:tasks.workunit.client.0.smithi160.stdout:5/599: stat db/d47/d5e/ca3 0 2024-05-25T06:11:39.971 INFO:tasks.workunit.client.0.smithi160.stdout:0/597: creat d2b/d31/d6d/fbf x:0 0 0 2024-05-25T06:11:39.971 INFO:tasks.workunit.client.0.smithi160.stdout:9/488: mkdir d4/d11/d1e/d3e/d25/d27/d95 0 2024-05-25T06:11:39.973 INFO:tasks.workunit.client.0.smithi160.stdout:9/489: creat d4/d11/d1e/d3e/f96 x:0 0 0 2024-05-25T06:11:39.978 INFO:tasks.workunit.client.0.smithi160.stdout:5/600: dread db/dc/d13/d1d/d23/d24/daf/f95 [0,4194304] 0 2024-05-25T06:11:39.978 INFO:tasks.workunit.client.0.smithi160.stdout:0/598: dwrite d2b/d46/d63/fa2 [0,4194304] 0 2024-05-25T06:11:39.980 INFO:tasks.workunit.client.0.smithi160.stdout:0/599: dread - d2b/da8/fb9 zero size 2024-05-25T06:11:39.981 INFO:tasks.workunit.client.0.smithi160.stdout:3/444: dwrite f13 [0,4194304] 0 2024-05-25T06:11:39.983 INFO:tasks.workunit.client.0.smithi160.stdout:5/601: truncate db/dc/d13/d9a/fb1 601149 0 2024-05-25T06:11:39.985 INFO:tasks.workunit.client.1.smithi203.stdout:5/990: sync 2024-05-25T06:11:39.985 INFO:tasks.workunit.client.1.smithi203.stdout:1/717: sync 2024-05-25T06:11:39.985 INFO:tasks.workunit.client.1.smithi203.stdout:9/706: sync 2024-05-25T06:11:39.985 INFO:tasks.workunit.client.1.smithi203.stdout:3/947: sync 2024-05-25T06:11:39.985 INFO:tasks.workunit.client.1.smithi203.stdout:0/728: sync 2024-05-25T06:11:39.987 INFO:tasks.workunit.client.1.smithi203.stdout:9/707: write d0/df/fd1 [573015,103723] 0 2024-05-25T06:11:39.988 INFO:tasks.workunit.client.1.smithi203.stdout:9/708: dread - d0/df/d2d/d92/fdf zero size 2024-05-25T06:11:39.988 INFO:tasks.workunit.client.0.smithi160.stdout:3/445: creat d19/d5d/f9c x:0 0 0 2024-05-25T06:11:39.988 INFO:tasks.workunit.client.0.smithi160.stdout:3/446: dread - d19/d2f/d38/f6b zero size 2024-05-25T06:11:39.989 INFO:tasks.workunit.client.0.smithi160.stdout:3/447: fsync d19/d2f/d38/f3e 0 2024-05-25T06:11:39.989 INFO:tasks.workunit.client.1.smithi203.stdout:0/729: rmdir dd/d32/d37 39 2024-05-25T06:11:39.991 INFO:tasks.workunit.client.0.smithi160.stdout:0/600: creat d2b/d31/d35/d36/d37/fc0 x:0 0 0 2024-05-25T06:11:39.992 INFO:tasks.workunit.client.1.smithi203.stdout:1/718: symlink d14/d57/d29/d94/db8/lfa 0 2024-05-25T06:11:39.993 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:11:39.995 INFO:tasks.workunit.client.1.smithi203.stdout:9/709: rename d0/df/dc5 to d0/d17/dec 0 2024-05-25T06:11:39.995 INFO:tasks.workunit.client.0.smithi160.stdout:0/601: write f1b [719184,30592] 0 2024-05-25T06:11:39.997 INFO:tasks.workunit.client.0.smithi160.stdout:3/448: dwrite d19/d2f/d34/f55 [0,4194304] 0 2024-05-25T06:11:39.997 INFO:tasks.workunit.client.1.smithi203.stdout:9/710: chown d0/df/d2d/l89 0 1 2024-05-25T06:11:39.998 INFO:tasks.workunit.client.1.smithi203.stdout:0/730: dwrite dd/d3d/da0/db9/df9/d9d/fef [0,4194304] 0 2024-05-25T06:11:39.999 INFO:tasks.workunit.client.0.smithi160.stdout:3/449: truncate d19/d2f/d38/f6c 187962 0 2024-05-25T06:11:40.001 INFO:tasks.workunit.client.1.smithi203.stdout:0/731: rmdir dd/d3d/df3 39 2024-05-25T06:11:40.002 INFO:tasks.workunit.client.0.smithi160.stdout:0/602: rename d2b/d31/d35/d7b/f9d to d2b/d31/fc1 0 2024-05-25T06:11:40.003 INFO:tasks.workunit.client.1.smithi203.stdout:1/719: dwrite d14/d57/d1d/f2a [0,4194304] 0 2024-05-25T06:11:40.004 INFO:tasks.workunit.client.1.smithi203.stdout:5/991: write dc/d11/d17/d1a/d22/f39 [1037401,86383] 0 2024-05-25T06:11:40.006 INFO:tasks.workunit.client.1.smithi203.stdout:1/720: chown d14/d57/d97/dec 16217512 1 2024-05-25T06:11:40.010 INFO:tasks.workunit.client.1.smithi203.stdout:1/721: creat d14/d7c/ffb x:0 0 0 2024-05-25T06:11:40.011 INFO:tasks.workunit.client.0.smithi160.stdout:0/603: dwrite d2b/d31/d35/d36/f8b [0,4194304] 0 2024-05-25T06:11:40.017 INFO:tasks.workunit.client.1.smithi203.stdout:5/992: dwrite dc/d11/d17/d1a/d2d/d4d/ded/d13f/f142 [0,4194304] 0 2024-05-25T06:11:40.021 INFO:tasks.workunit.client.0.smithi160.stdout:0/604: symlink d2b/d46/d63/db3/lc2 0 2024-05-25T06:11:40.021 INFO:tasks.workunit.client.0.smithi160.stdout:0/605: chown c9 15888458 1 2024-05-25T06:11:40.021 INFO:tasks.workunit.client.0.smithi160.stdout:0/606: stat d2b/d46/d63/db3/lb4 0 2024-05-25T06:11:40.023 INFO:tasks.workunit.client.1.smithi203.stdout:3/948: dread d10/fb5 [0,4194304] 0 2024-05-25T06:11:40.026 INFO:tasks.workunit.client.1.smithi203.stdout:1/722: dwrite d14/d57/d97/ff2 [0,4194304] 0 2024-05-25T06:11:40.027 INFO:tasks.workunit.client.0.smithi160.stdout:0/607: creat d2b/da8/fc3 x:0 0 0 2024-05-25T06:11:40.027 INFO:tasks.workunit.client.0.smithi160.stdout:0/608: chown d2b/d31/d35/d36/l5c 12305685 1 2024-05-25T06:11:40.028 INFO:tasks.workunit.client.1.smithi203.stdout:5/993: write dc/d11/d17/d1a/d2d/d4d/ded/d13f/dd4/f147 [412003,36621] 0 2024-05-25T06:11:40.028 INFO:tasks.workunit.client.1.smithi203.stdout:1/723: readlink d14/le3 0 2024-05-25T06:11:40.029 INFO:tasks.workunit.client.1.smithi203.stdout:3/949: stat d10/d16/d23/d4a/d88/dca/ff4 0 2024-05-25T06:11:40.034 INFO:tasks.workunit.client.1.smithi203.stdout:5/994: symlink dc/daa/l158 0 2024-05-25T06:11:40.034 INFO:tasks.workunit.client.1.smithi203.stdout:9/711: dwrite d0/d17/d23/f90 [0,4194304] 0 2024-05-25T06:11:40.035 INFO:tasks.workunit.client.0.smithi160.stdout:5/602: dwrite db/dc/d13/f1f [0,4194304] 0 2024-05-25T06:11:40.038 INFO:tasks.workunit.client.0.smithi160.stdout:3/450: write d19/d22/f24 [1270930,48932] 0 2024-05-25T06:11:40.039 INFO:tasks.workunit.client.1.smithi203.stdout:3/950: fdatasync d10/d16/d23/fdd 0 2024-05-25T06:11:40.042 INFO:tasks.workunit.client.0.smithi160.stdout:3/451: chown d19/l91 210580728 1 2024-05-25T06:11:40.042 INFO:tasks.workunit.client.0.smithi160.stdout:5/603: creat db/dc/d13/d1d/d23/d24/d2e/fb7 x:0 0 0 2024-05-25T06:11:40.044 INFO:tasks.workunit.client.1.smithi203.stdout:5/995: unlink dc/d11/d17/d1a/d2d/d53/da2/fd6 0 2024-05-25T06:11:40.044 INFO:tasks.workunit.client.0.smithi160.stdout:3/452: write d19/d22/f8f [863903,96273] 0 2024-05-25T06:11:40.046 INFO:tasks.workunit.client.1.smithi203.stdout:0/732: dread dd/d10/d34/d4c/d99/fbe [0,4194304] 0 2024-05-25T06:11:40.049 INFO:tasks.workunit.client.1.smithi203.stdout:5/996: creat dc/d11/d17/d1a/dda/f159 x:0 0 0 2024-05-25T06:11:40.050 INFO:tasks.workunit.client.0.smithi160.stdout:3/453: read d19/f52 [390467,112899] 0 2024-05-25T06:11:40.050 INFO:tasks.workunit.client.0.smithi160.stdout:3/454: chown d19/d22/l31 133949380 1 2024-05-25T06:11:40.054 INFO:tasks.workunit.client.0.smithi160.stdout:3/455: creat d19/d2f/d34/f9d x:0 0 0 2024-05-25T06:11:40.056 INFO:tasks.workunit.client.1.smithi203.stdout:0/733: rename dd/d3d to dd/d5c/d63/d106 0 2024-05-25T06:11:40.058 INFO:tasks.workunit.client.1.smithi203.stdout:5/997: mknod dc/d11/d17/d1a/d72/c15a 0 2024-05-25T06:11:40.058 INFO:tasks.workunit.client.1.smithi203.stdout:9/712: write d0/d57/f76 [1955164,45211] 0 2024-05-25T06:11:40.060 INFO:tasks.workunit.client.1.smithi203.stdout:0/734: mknod dd/d10/d34/d4c/d99/c107 0 2024-05-25T06:11:40.061 INFO:tasks.workunit.client.1.smithi203.stdout:9/713: creat d0/d17/d20/d3c/fed x:0 0 0 2024-05-25T06:11:40.062 INFO:tasks.workunit.client.1.smithi203.stdout:9/714: write d0/df/d2d/f7c [1434780,98252] 0 2024-05-25T06:11:40.063 INFO:tasks.workunit.client.1.smithi203.stdout:5/998: rename dc/daa/d10c/l12e to dc/d11/d17/d1a/d2d/d53/l15b 0 2024-05-25T06:11:40.063 INFO:tasks.workunit.client.1.smithi203.stdout:5/999: readlink dc/d11/d18/l71 0 2024-05-25T06:11:40.064 INFO:tasks.workunit.client.1.smithi203.stdout:1/724: truncate d14/d57/d1d/f67 8154738 0 2024-05-25T06:11:40.064 INFO:tasks.workunit.client.1.smithi203.stdout:1/725: chown d14/d57/d7e 0 1 2024-05-25T06:11:40.065 INFO:tasks.workunit.client.1.smithi203.stdout:0/735: rmdir dd/d5c/d63/d106/d48 39 2024-05-25T06:11:40.065 INFO:tasks.workunit.client.1.smithi203.stdout:0/736: readlink dd/d10/l22 0 2024-05-25T06:11:40.070 INFO:tasks.workunit.client.1.smithi203.stdout:9/715: mknod d0/df/d37/cee 0 2024-05-25T06:11:40.073 INFO:tasks.workunit.client.1.smithi203.stdout:9/716: mkdir d0/d17/d23/def 0 2024-05-25T06:11:40.074 INFO:tasks.workunit.client.1.smithi203.stdout:9/717: chown d0/l7f 7573004 1 2024-05-25T06:11:40.075 INFO:tasks.workunit.client.1.smithi203.stdout:3/951: dread d10/d16/d23/d45/f4e [0,4194304] 0 2024-05-25T06:11:40.077 INFO:tasks.workunit.client.0.smithi160.stdout:8/578: sync 2024-05-25T06:11:40.077 INFO:tasks.workunit.client.0.smithi160.stdout:6/514: sync 2024-05-25T06:11:40.077 INFO:tasks.workunit.client.0.smithi160.stdout:2/473: sync 2024-05-25T06:11:40.077 INFO:tasks.workunit.client.0.smithi160.stdout:3/456: dread d19/d2f/d38/d47/f4b [0,4194304] 0 2024-05-25T06:11:40.079 INFO:tasks.workunit.client.0.smithi160.stdout:0/609: write f5 [823017,112326] 0 2024-05-25T06:11:40.080 INFO:tasks.workunit.client.1.smithi203.stdout:9/718: fdatasync d0/d17/d23/f8e 0 2024-05-25T06:11:40.080 INFO:tasks.workunit.client.0.smithi160.stdout:5/604: truncate db/dc/d13/f87 486534 0 2024-05-25T06:11:40.080 INFO:tasks.workunit.client.0.smithi160.stdout:8/579: write d5/d44/d1f/d58/d65/da2/fb7 [226518,12725] 0 2024-05-25T06:11:40.081 INFO:tasks.workunit.client.0.smithi160.stdout:3/457: fdatasync d19/d2f/d34/f4d 0 2024-05-25T06:11:40.081 INFO:tasks.workunit.client.0.smithi160.stdout:0/610: write d2b/d31/f7a [1822484,88405] 0 2024-05-25T06:11:40.082 INFO:tasks.workunit.client.1.smithi203.stdout:1/726: dread d14/d57/f58 [0,4194304] 0 2024-05-25T06:11:40.082 INFO:tasks.workunit.client.0.smithi160.stdout:3/458: chown f11 10226458 1 2024-05-25T06:11:40.084 INFO:tasks.workunit.client.0.smithi160.stdout:8/580: write d5/d7/d9a/db5/fbd [134485,73620] 0 2024-05-25T06:11:40.085 INFO:tasks.workunit.client.0.smithi160.stdout:2/474: mknod d12/d1c/d2a/d7e/d90/ca5 0 2024-05-25T06:11:40.087 INFO:tasks.workunit.client.0.smithi160.stdout:2/475: dread - d12/d1c/d59/f6d zero size 2024-05-25T06:11:40.088 INFO:tasks.workunit.client.0.smithi160.stdout:6/515: creat d5/d30/d36/d73/d87/da4/fb2 x:0 0 0 2024-05-25T06:11:40.088 INFO:tasks.workunit.client.1.smithi203.stdout:9/719: stat d0/df/d2d/l40 0 2024-05-25T06:11:40.089 INFO:tasks.workunit.client.0.smithi160.stdout:5/605: symlink db/dc/d13/d1d/d31/d3b/lb8 0 2024-05-25T06:11:40.090 INFO:tasks.workunit.client.0.smithi160.stdout:3/459: chown d19/d2f/d38/d43/d5b/d65 24915 1 2024-05-25T06:11:40.091 INFO:tasks.workunit.client.1.smithi203.stdout:1/727: rename d14/d57/d6b/da1/de2/fd7 to d14/d57/db7/dd2/ddc/ffc 0 2024-05-25T06:11:40.091 INFO:tasks.workunit.client.0.smithi160.stdout:8/581: mkdir d5/d9/d72/daf/dbf 0 2024-05-25T06:11:40.093 INFO:tasks.workunit.client.0.smithi160.stdout:6/516: symlink d5/d45/lb3 0 2024-05-25T06:11:40.095 INFO:tasks.workunit.client.1.smithi203.stdout:9/720: dwrite d0/df/d2d/f7c [0,4194304] 0 2024-05-25T06:11:40.096 INFO:tasks.workunit.client.0.smithi160.stdout:0/611: creat d2b/d31/d35/d43/fc4 x:0 0 0 2024-05-25T06:11:40.097 INFO:tasks.workunit.client.1.smithi203.stdout:3/952: link d10/d16/d71/dfa/c12b d10/c135 0 2024-05-25T06:11:40.097 INFO:tasks.workunit.client.0.smithi160.stdout:8/582: dread d5/d44/d1f/da3/d28/f5e [0,4194304] 0 2024-05-25T06:11:40.099 INFO:tasks.workunit.client.0.smithi160.stdout:2/476: getdents d12/d5c/d85/d9b 0 2024-05-25T06:11:40.099 INFO:tasks.workunit.client.0.smithi160.stdout:0/612: chown d2b/da8/fb0 3140897 1 2024-05-25T06:11:40.099 INFO:tasks.workunit.client.0.smithi160.stdout:3/460: chown d19/d2f/d34/d67/l92 1 1 2024-05-25T06:11:40.102 INFO:tasks.workunit.client.1.smithi203.stdout:0/737: dwrite dd/d5c/d63/d106/d4f/ff4 [0,4194304] 0 2024-05-25T06:11:40.102 INFO:tasks.workunit.client.1.smithi203.stdout:1/728: fsync d14/d57/d1d/f67 0 2024-05-25T06:11:40.103 INFO:tasks.workunit.client.0.smithi160.stdout:3/461: fsync d19/d2f/d34/f40 0 2024-05-25T06:11:40.104 INFO:tasks.workunit.client.0.smithi160.stdout:5/606: dwrite f6 [4194304,4194304] 0 2024-05-25T06:11:40.105 INFO:tasks.workunit.client.1.smithi203.stdout:9/721: mkdir d0/d17/d20/d6e/d75/df0 0 2024-05-25T06:11:40.105 INFO:tasks.workunit.client.1.smithi203.stdout:0/738: dread - dd/d5c/f73 zero size 2024-05-25T06:11:40.108 INFO:tasks.workunit.client.0.smithi160.stdout:3/462: write d19/d2f/d38/d47/f64 [303160,53055] 0 2024-05-25T06:11:40.109 INFO:tasks.workunit.client.0.smithi160.stdout:8/583: rename d5/d7/l10 to d5/d44/lc0 0 2024-05-25T06:11:40.109 INFO:tasks.workunit.client.0.smithi160.stdout:5/607: dread - db/dc/d13/d1d/d3f/d72/d98/f9b zero size 2024-05-25T06:11:40.112 INFO:tasks.workunit.client.1.smithi203.stdout:1/729: symlink d14/d81/lfd 0 2024-05-25T06:11:40.112 INFO:tasks.workunit.client.0.smithi160.stdout:1/415: sync 2024-05-25T06:11:40.112 INFO:tasks.workunit.client.0.smithi160.stdout:7/563: sync 2024-05-25T06:11:40.113 INFO:tasks.workunit.client.0.smithi160.stdout:4/503: sync 2024-05-25T06:11:40.113 INFO:tasks.workunit.client.1.smithi203.stdout:0/739: fsync dd/f44 0 2024-05-25T06:11:40.116 INFO:tasks.workunit.client.1.smithi203.stdout:1/730: mknod d14/d57/db7/dd2/cfe 0 2024-05-25T06:11:40.117 INFO:tasks.workunit.client.0.smithi160.stdout:3/463: rmdir d19/d2f/d38 39 2024-05-25T06:11:40.117 INFO:tasks.workunit.client.0.smithi160.stdout:7/564: read d6/dc/d64/f78 [238982,92853] 0 2024-05-25T06:11:40.118 INFO:tasks.workunit.client.0.smithi160.stdout:8/584: symlink d5/d44/d1f/da3/d28/d4a/lc1 0 2024-05-25T06:11:40.119 INFO:tasks.workunit.client.0.smithi160.stdout:5/608: symlink db/dc/d13/d1d/d23/d24/d2e/lb9 0 2024-05-25T06:11:40.119 INFO:tasks.workunit.client.1.smithi203.stdout:1/731: write d14/d57/db7/dd2/ddc/ffc [140995,73489] 0 2024-05-25T06:11:40.124 INFO:tasks.workunit.client.1.smithi203.stdout:4/972: sync 2024-05-25T06:11:40.124 INFO:tasks.workunit.client.1.smithi203.stdout:2/878: sync 2024-05-25T06:11:40.124 INFO:tasks.workunit.client.1.smithi203.stdout:7/823: sync 2024-05-25T06:11:40.124 INFO:tasks.workunit.client.1.smithi203.stdout:8/931: sync 2024-05-25T06:11:40.125 INFO:tasks.workunit.client.0.smithi160.stdout:0/613: dwrite d2b/d31/d35/d36/d37/f75 [0,4194304] 0 2024-05-25T06:11:40.126 INFO:tasks.workunit.client.1.smithi203.stdout:4/973: write d0/f131 [278440,9316] 0 2024-05-25T06:11:40.128 INFO:tasks.workunit.client.1.smithi203.stdout:2/879: creat d2/d1a/dd3/f128 x:0 0 0 2024-05-25T06:11:40.130 INFO:tasks.workunit.client.0.smithi160.stdout:7/565: chown d6/d3e/f9d 28156 1 2024-05-25T06:11:40.131 INFO:tasks.workunit.client.1.smithi203.stdout:7/824: unlink d1/d11/d3d/d9a/fab 0 2024-05-25T06:11:40.132 INFO:tasks.workunit.client.0.smithi160.stdout:0/614: creat d2b/d46/d63/da7/da6/fc5 x:0 0 0 2024-05-25T06:11:40.132 INFO:tasks.workunit.client.0.smithi160.stdout:1/416: rename de/d21/d49/d1c/f3f to de/d39/f85 0 2024-05-25T06:11:40.132 INFO:tasks.workunit.client.0.smithi160.stdout:8/585: mkdir d5/d44/d1f/da3/d28/d4a/d73/dc2 0 2024-05-25T06:11:40.134 INFO:tasks.workunit.client.1.smithi203.stdout:3/953: dread d10/d16/d23/fb9 [0,4194304] 0 2024-05-25T06:11:40.136 INFO:tasks.workunit.client.1.smithi203.stdout:8/932: rename d2/d5/d18/d31/l79 to d2/d5/dae/l13f 0 2024-05-25T06:11:40.137 INFO:tasks.workunit.client.1.smithi203.stdout:8/933: chown d2/d5/d10/d11/de/f16 31 1 2024-05-25T06:11:40.138 INFO:tasks.workunit.client.0.smithi160.stdout:4/504: dwrite d4/d6/d21/f27 [0,4194304] 0 2024-05-25T06:11:40.139 INFO:tasks.workunit.client.0.smithi160.stdout:3/464: readlink d19/d2f/d34/d67/d96/l98 0 2024-05-25T06:11:40.141 INFO:tasks.workunit.client.1.smithi203.stdout:7/825: write d1/d9/d2c/d62/db3/ff6 [1080056,92723] 0 2024-05-25T06:11:40.143 INFO:tasks.workunit.client.0.smithi160.stdout:2/477: write d12/d75/f88 [1237805,43006] 0 2024-05-25T06:11:40.146 INFO:tasks.workunit.client.0.smithi160.stdout:0/615: stat c28 0 2024-05-25T06:11:40.149 INFO:tasks.workunit.client.1.smithi203.stdout:2/880: dwrite d2/d1a/dd3/d100/d10/d5f/f7b [0,4194304] 0 2024-05-25T06:11:40.149 INFO:tasks.workunit.client.1.smithi203.stdout:7/826: rmdir d1/d9/d2c/d3a/d4e 39 2024-05-25T06:11:40.150 INFO:tasks.workunit.client.0.smithi160.stdout:6/517: dwrite d5/de/f12 [0,4194304] 0 2024-05-25T06:11:40.150 INFO:tasks.workunit.client.0.smithi160.stdout:7/566: write d6/d3e/d6f/f76 [1076077,32127] 0 2024-05-25T06:11:40.151 INFO:tasks.workunit.client.0.smithi160.stdout:0/616: stat d2b/d31/d35/c64 0 2024-05-25T06:11:40.151 INFO:tasks.workunit.client.1.smithi203.stdout:0/740: dwrite dd/d45/d96/f9f [0,4194304] 0 2024-05-25T06:11:40.152 INFO:tasks.workunit.client.1.smithi203.stdout:3/954: write d10/d16/d23/d6c/f63 [1212041,84908] 0 2024-05-25T06:11:40.154 INFO:tasks.workunit.client.0.smithi160.stdout:4/505: unlink d4/d1c/f62 0 2024-05-25T06:11:40.156 INFO:tasks.workunit.client.1.smithi203.stdout:9/722: dread d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:40.158 INFO:tasks.workunit.client.0.smithi160.stdout:5/609: dwrite db/dc/d13/d1d/d31/d56/f92 [4194304,4194304] 0 2024-05-25T06:11:40.160 INFO:tasks.workunit.client.1.smithi203.stdout:2/881: truncate d2/d1a/d89/daa/def/d107/d113/f122 330196 0 2024-05-25T06:11:40.163 INFO:tasks.workunit.client.0.smithi160.stdout:6/518: dread d5/d39/f6c [0,4194304] 0 2024-05-25T06:11:40.163 INFO:tasks.workunit.client.0.smithi160.stdout:8/586: dread d5/d7/d9a/db5/fbd [0,4194304] 0 2024-05-25T06:11:40.164 INFO:tasks.workunit.client.0.smithi160.stdout:0/617: unlink d2b/d31/d35/d43/f4c 0 2024-05-25T06:11:40.167 INFO:tasks.workunit.client.1.smithi203.stdout:0/741: dread - dd/d32/d37/f101 zero size 2024-05-25T06:11:40.168 INFO:tasks.workunit.client.1.smithi203.stdout:9/723: creat d0/df/d73/ff1 x:0 0 0 2024-05-25T06:11:40.168 INFO:tasks.workunit.client.1.smithi203.stdout:2/882: unlink d2/d1a/db0/le1 0 2024-05-25T06:11:40.169 INFO:tasks.workunit.client.0.smithi160.stdout:6/519: write d5/de/d58/f80 [683340,15096] 0 2024-05-25T06:11:40.171 INFO:tasks.workunit.client.1.smithi203.stdout:3/955: link d10/d16/d23/d45/f4e d10/d16/d23/f136 0 2024-05-25T06:11:40.171 INFO:tasks.workunit.client.0.smithi160.stdout:1/417: truncate de/d43/f46 2546074 0 2024-05-25T06:11:40.177 INFO:tasks.workunit.client.0.smithi160.stdout:2/478: dwrite d12/d1c/f2c [4194304,4194304] 0 2024-05-25T06:11:40.178 INFO:tasks.workunit.client.1.smithi203.stdout:9/724: fdatasync d0/df/d2d/d92/fc2 0 2024-05-25T06:11:40.178 INFO:tasks.workunit.client.1.smithi203.stdout:4/974: dwrite d0/d8/d1f/d26/f4a [0,4194304] 0 2024-05-25T06:11:40.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.179+0000 7f3811851700 1 -- 172.21.15.160:0/3776431344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fca60 msgr2=0x7f380c0fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:40.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.179+0000 7f3811851700 1 --2- 172.21.15.160:0/3776431344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fca60 0x7f380c0fae70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3800004940 tx=0x7f3800009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:40.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.179+0000 7f3811851700 1 -- 172.21.15.160:0/3776431344 shutdown_connections 2024-05-25T06:11:40.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.179+0000 7f3811851700 1 --2- 172.21.15.160:0/3776431344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fca60 0x7f380c0fae70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.179+0000 7f3811851700 1 --2- 172.21.15.160:0/3776431344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f380c0fc110 0x7f380c0fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.179+0000 7f3811851700 1 -- 172.21.15.160:0/3776431344 >> 172.21.15.160:0/3776431344 conn(0x7f380c0f6b80 msgr2=0x7f380c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:40.181 INFO:tasks.workunit.client.1.smithi203.stdout:1/732: dread d14/d57/d1d/f2f [0,4194304] 0 2024-05-25T06:11:40.181 INFO:tasks.workunit.client.0.smithi160.stdout:5/610: creat db/dc/d13/d1d/d23/d24/daf/db3/fba x:0 0 0 2024-05-25T06:11:40.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.180+0000 7f3811851700 1 -- 172.21.15.160:0/3776431344 shutdown_connections 2024-05-25T06:11:40.181 INFO:tasks.workunit.client.1.smithi203.stdout:2/883: symlink d2/d38/d43/d60/d6b/l129 0 2024-05-25T06:11:40.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.181+0000 7f3811851700 1 -- 172.21.15.160:0/3776431344 wait complete. 2024-05-25T06:11:40.182 INFO:tasks.workunit.client.0.smithi160.stdout:0/618: write d2b/f57 [3658120,16200] 0 2024-05-25T06:11:40.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.182+0000 7f3811851700 1 Processor -- start 2024-05-25T06:11:40.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.182+0000 7f3811851700 1 -- start start 2024-05-25T06:11:40.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.183+0000 7f3811851700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 0x7f380c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:40.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.183+0000 7f3811851700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f380c0fca60 0x7f380c114470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:40.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.183+0000 7f3811851700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c114a70 con 0x7f380c0fca60 2024-05-25T06:11:40.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.183+0000 7f3811851700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c1152c0 con 0x7f380c0fc110 2024-05-25T06:11:40.184 INFO:tasks.workunit.client.1.smithi203.stdout:8/934: dwrite d2/d5/d10/d11/d59/f9f [0,4194304] 0 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.184+0000 7f380affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 0x7f380c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.184+0000 7f380affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 0x7f380c113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34730/0 (socket says 172.21.15.160:34730) 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.184+0000 7f380affd700 1 -- 172.21.15.160:0/731985280 learned_addr learned my addr 172.21.15.160:0/731985280 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.184+0000 7f380a7fc700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f380c0fca60 0x7f380c114470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.185+0000 7f380affd700 1 -- 172.21.15.160:0/731985280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f380c0fca60 msgr2=0x7f380c114470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.185+0000 7f380affd700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f380c0fca60 0x7f380c114470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.185+0000 7f380affd700 1 -- 172.21.15.160:0/731985280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800005040 con 0x7f380c0fc110 2024-05-25T06:11:40.186 INFO:tasks.workunit.client.1.smithi203.stdout:9/725: mkdir d0/d17/d20/d6e/d75/df2 0 2024-05-25T06:11:40.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.186+0000 7f380affd700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 0x7f380c113f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00eb40 tx=0x7f37fc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:40.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.186+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37fc007cc0 con 0x7f380c0fc110 2024-05-25T06:11:40.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.187+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37fc00f040 con 0x7f380c0fc110 2024-05-25T06:11:40.188 INFO:tasks.workunit.client.1.smithi203.stdout:8/935: creat d2/d5/d18/d31/f140 x:0 0 0 2024-05-25T06:11:40.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.187+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c1154c0 con 0x7f380c0fc110 2024-05-25T06:11:40.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.187+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37fc009560 con 0x7f380c0fc110 2024-05-25T06:11:40.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.187+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c115a80 con 0x7f380c0fc110 2024-05-25T06:11:40.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.189+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380c05f5c0 con 0x7f380c0fc110 2024-05-25T06:11:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.191+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f37fc0097d0 con 0x7f380c0fc110 2024-05-25T06:11:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.191+0000 7f37f37fe700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f37f406b7e0 0x7f37f406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.191+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f37fc08a310 con 0x7f380c0fc110 2024-05-25T06:11:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.192+0000 7f380a7fc700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f37f406b7e0 0x7f37f406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:40.192 INFO:tasks.workunit.client.1.smithi203.stdout:9/726: chown d0/d17/d23/f6c 3739313 1 2024-05-25T06:11:40.192 INFO:tasks.workunit.client.1.smithi203.stdout:4/975: creat d0/d8/d1f/d26/d3b/d9c/f148 x:0 0 0 2024-05-25T06:11:40.193 INFO:tasks.workunit.client.1.smithi203.stdout:2/884: mkdir d2/d1a/d33/d112/d12a 0 2024-05-25T06:11:40.193 INFO:tasks.workunit.client.0.smithi160.stdout:6/520: symlink d5/de/d9e/lb4 0 2024-05-25T06:11:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.193+0000 7f380a7fc700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f37f406b7e0 0x7f37f406dc90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f380000b0d0 tx=0x7f3800009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:40.194 INFO:tasks.workunit.client.0.smithi160.stdout:2/479: mknod d12/d47/ca6 0 2024-05-25T06:11:40.197 INFO:tasks.workunit.client.0.smithi160.stdout:3/465: dread d19/f8e [0,4194304] 0 2024-05-25T06:11:40.198 INFO:tasks.workunit.client.1.smithi203.stdout:2/885: creat d2/d1a/d89/daa/f12b x:0 0 0 2024-05-25T06:11:40.198 INFO:tasks.workunit.client.0.smithi160.stdout:1/418: rename de/c2d to de/d21/d49/d1c/d41/d45/d7d/c86 0 2024-05-25T06:11:40.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.199+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f37fc057a30 con 0x7f380c0fc110 2024-05-25T06:11:40.200 INFO:tasks.workunit.client.0.smithi160.stdout:0/619: mkdir d2b/d31/d35/d43/dab/dc6 0 2024-05-25T06:11:40.201 INFO:tasks.workunit.client.1.smithi203.stdout:1/733: rename d14/d57/d29/d94 to d14/dff 0 2024-05-25T06:11:40.202 INFO:tasks.workunit.client.1.smithi203.stdout:2/886: chown d2/d38/d2f/f31 31 1 2024-05-25T06:11:40.202 INFO:tasks.workunit.client.0.smithi160.stdout:4/506: dwrite d4/d6/d21/f48 [0,4194304] 0 2024-05-25T06:11:40.202 INFO:tasks.workunit.client.0.smithi160.stdout:0/620: chown d2b/d46/d63/db3/lc2 3541 1 2024-05-25T06:11:40.203 INFO:tasks.workunit.client.1.smithi203.stdout:0/742: dread dd/d5c/d63/d106/d4f/f87 [0,4194304] 0 2024-05-25T06:11:40.203 INFO:tasks.workunit.client.0.smithi160.stdout:5/611: creat db/dc/d13/d1d/fbb x:0 0 0 2024-05-25T06:11:40.204 INFO:tasks.workunit.client.0.smithi160.stdout:4/507: chown d4/f4f 66 1 2024-05-25T06:11:40.205 INFO:tasks.workunit.client.1.smithi203.stdout:4/976: dwrite d0/d9/d50/d6b/fc1 [4194304,4194304] 0 2024-05-25T06:11:40.205 INFO:tasks.workunit.client.0.smithi160.stdout:3/466: mkdir d19/d5d/d6a/d9e 0 2024-05-25T06:11:40.206 INFO:tasks.workunit.client.0.smithi160.stdout:3/467: stat d19/d22 0 2024-05-25T06:11:40.208 INFO:tasks.workunit.client.0.smithi160.stdout:8/587: rename d5/d44/d1f/da3/d28/f4d to d5/d44/d1f/da3/d28/d49/fc3 0 2024-05-25T06:11:40.209 INFO:tasks.workunit.client.1.smithi203.stdout:0/743: dread - dd/d5c/d63/d106/df3/ffe zero size 2024-05-25T06:11:40.209 INFO:tasks.workunit.client.1.smithi203.stdout:2/887: write d2/d38/d43/d60/d6b/dae/f67 [2156382,66172] 0 2024-05-25T06:11:40.212 INFO:tasks.workunit.client.1.smithi203.stdout:4/977: stat d0/d9/d9d/fbf 0 2024-05-25T06:11:40.217 INFO:tasks.workunit.client.1.smithi203.stdout:1/734: dwrite d14/d57/d6b/da1/de2/f55 [0,4194304] 0 2024-05-25T06:11:40.218 INFO:tasks.workunit.client.0.smithi160.stdout:4/508: dread d4/d63/d64/f80 [0,4194304] 0 2024-05-25T06:11:40.219 INFO:tasks.workunit.client.0.smithi160.stdout:2/480: rmdir d12/d3e/d84 0 2024-05-25T06:11:40.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:40 smithi160 ceph-mon[29041]: pgmap v20: 65 pgs: 65 active+clean; 2.5 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 90 MiB/s rd, 255 MiB/s wr, 471 op/s 2024-05-25T06:11:40.221 INFO:tasks.workunit.client.0.smithi160.stdout:3/468: dwrite d19/d5d/f61 [4194304,4194304] 0 2024-05-25T06:11:40.224 INFO:tasks.workunit.client.0.smithi160.stdout:5/612: dwrite db/dc/d13/d1d/d3f/d72/d98/da5/fac [0,4194304] 0 2024-05-25T06:11:40.224 INFO:tasks.workunit.client.0.smithi160.stdout:6/521: link d5/de/d9e/lb4 d5/d39/d8f/lb5 0 2024-05-25T06:11:40.224 INFO:tasks.workunit.client.1.smithi203.stdout:0/744: write dd/d5c/d63/d106/d48/f7a [484979,50067] 0 2024-05-25T06:11:40.225 INFO:tasks.workunit.client.1.smithi203.stdout:2/888: rename d2/d1a/d33/d112/d12a to d2/d1a/d33/d12c 0 2024-05-25T06:11:40.227 INFO:tasks.workunit.client.1.smithi203.stdout:2/889: fdatasync d2/d1a/d33/d80/fed 0 2024-05-25T06:11:40.227 INFO:tasks.workunit.client.0.smithi160.stdout:8/588: link d5/d9/fbb d5/d44/d1f/da3/d28/d49/fc4 0 2024-05-25T06:11:40.228 INFO:tasks.workunit.client.1.smithi203.stdout:0/745: write dd/d10/d34/fa8 [3613246,52264] 0 2024-05-25T06:11:40.231 INFO:tasks.workunit.client.0.smithi160.stdout:0/621: getdents d2b/d46/d63/da7/d67/d6a 0 2024-05-25T06:11:40.238 INFO:tasks.workunit.client.1.smithi203.stdout:3/956: truncate d10/d16/d23/d45/f76 648813 0 2024-05-25T06:11:40.239 INFO:tasks.workunit.client.1.smithi203.stdout:2/890: readlink d2/d1a/dd3/d100/d10/d5f/l4c 0 2024-05-25T06:11:40.241 INFO:tasks.workunit.client.1.smithi203.stdout:0/746: mkdir dd/d32/d108 0 2024-05-25T06:11:40.241 INFO:tasks.workunit.client.1.smithi203.stdout:3/957: write d10/d16/d23/d67/ffd [1311182,130138] 0 2024-05-25T06:11:40.242 INFO:tasks.workunit.client.1.smithi203.stdout:8/936: dread d2/d5/d2d/fbe [0,4194304] 0 2024-05-25T06:11:40.244 INFO:tasks.workunit.client.0.smithi160.stdout:4/509: mknod d4/d63/d70/d94/c9c 0 2024-05-25T06:11:40.245 INFO:tasks.workunit.client.0.smithi160.stdout:0/622: chown d2b/d46/d63/da7/f8c 209781845 1 2024-05-25T06:11:40.248 INFO:tasks.workunit.client.1.smithi203.stdout:2/891: link d2/d42/de5/l10f d2/d1a/dd3/d100/d10/d32/l12d 0 2024-05-25T06:11:40.249 INFO:tasks.workunit.client.1.smithi203.stdout:9/727: dread d0/d17/d20/f3a [0,4194304] 0 2024-05-25T06:11:40.250 INFO:tasks.workunit.client.1.smithi203.stdout:9/728: chown d0/d17/d20/f2c 1440457649 1 2024-05-25T06:11:40.251 INFO:tasks.workunit.client.0.smithi160.stdout:8/589: creat d5/fc5 x:0 0 0 2024-05-25T06:11:40.252 INFO:tasks.workunit.client.0.smithi160.stdout:3/469: link d19/d2f/c77 d19/d2f/d34/d67/d96/c9f 0 2024-05-25T06:11:40.254 INFO:tasks.workunit.client.1.smithi203.stdout:9/729: creat d0/df/d37/ff3 x:0 0 0 2024-05-25T06:11:40.254 INFO:tasks.workunit.client.0.smithi160.stdout:6/522: getdents d5/de/d9e 0 2024-05-25T06:11:40.255 INFO:tasks.workunit.client.1.smithi203.stdout:2/892: dread d2/d6/f46 [0,4194304] 0 2024-05-25T06:11:40.256 INFO:tasks.workunit.client.1.smithi203.stdout:9/730: readlink d0/d17/d20/l2a 0 2024-05-25T06:11:40.257 INFO:tasks.workunit.client.0.smithi160.stdout:1/419: creat de/f87 x:0 0 0 2024-05-25T06:11:40.259 INFO:tasks.workunit.client.1.smithi203.stdout:2/893: symlink d2/d1a/dd3/d100/d10/l12e 0 2024-05-25T06:11:40.260 INFO:tasks.workunit.client.1.smithi203.stdout:9/731: write d0/df/d37/da8/dd6/f9e [1398179,99638] 0 2024-05-25T06:11:40.261 INFO:tasks.workunit.client.0.smithi160.stdout:8/590: write d5/d39/f7d [182760,9347] 0 2024-05-25T06:11:40.262 INFO:tasks.workunit.client.0.smithi160.stdout:4/510: getdents d4 0 2024-05-25T06:11:40.263 INFO:tasks.workunit.client.1.smithi203.stdout:9/732: write d0/d17/d20/d6e/fc3 [218586,96219] 0 2024-05-25T06:11:40.263 INFO:tasks.workunit.client.1.smithi203.stdout:3/958: dread d10/d22/d2a/f4d [0,4194304] 0 2024-05-25T06:11:40.265 INFO:tasks.workunit.client.0.smithi160.stdout:2/481: dread f9 [0,4194304] 0 2024-05-25T06:11:40.266 INFO:tasks.workunit.client.0.smithi160.stdout:1/420: rmdir de/d21/d49/d28 39 2024-05-25T06:11:40.267 INFO:tasks.workunit.client.0.smithi160.stdout:6/523: dwrite d5/de/d32/d35/f82 [0,4194304] 0 2024-05-25T06:11:40.271 INFO:tasks.workunit.client.0.smithi160.stdout:3/470: rename d19/d2f/d38/c3b to d19/d2f/d34/d67/ca0 0 2024-05-25T06:11:40.273 INFO:tasks.workunit.client.0.smithi160.stdout:4/511: creat d4/d6/d28/f9d x:0 0 0 2024-05-25T06:11:40.274 INFO:tasks.workunit.client.1.smithi203.stdout:9/733: creat d0/d17/d20/d29/ff4 x:0 0 0 2024-05-25T06:11:40.276 INFO:tasks.workunit.client.1.smithi203.stdout:9/734: stat d0/d17/d20/d3c/db2/cc7 0 2024-05-25T06:11:40.279 INFO:tasks.workunit.client.0.smithi160.stdout:1/421: dwrite de/d21/d49/d1c/d52/f56 [0,4194304] 0 2024-05-25T06:11:40.280 INFO:tasks.workunit.client.0.smithi160.stdout:2/482: read d12/d1c/d2a/f3a [402417,109785] 0 2024-05-25T06:11:40.281 INFO:tasks.workunit.client.1.smithi203.stdout:0/747: write dd/d5c/d63/d106/d75/f79 [4230949,47962] 0 2024-05-25T06:11:40.281 INFO:tasks.workunit.client.1.smithi203.stdout:8/937: write d2/d5/d18/d31/f34 [532885,7928] 0 2024-05-25T06:11:40.284 INFO:tasks.workunit.client.1.smithi203.stdout:1/735: dwrite d14/d57/d6b/f50 [0,4194304] 0 2024-05-25T06:11:40.284 INFO:tasks.workunit.client.0.smithi160.stdout:3/471: creat d19/d2f/d34/fa1 x:0 0 0 2024-05-25T06:11:40.284 INFO:tasks.workunit.client.0.smithi160.stdout:1/422: dread de/d21/d49/d1c/d59/f5d [0,4194304] 0 2024-05-25T06:11:40.286 INFO:tasks.workunit.client.0.smithi160.stdout:5/613: dread db/dc/d13/d1d/d23/d45/f5f [0,4194304] 0 2024-05-25T06:11:40.288 INFO:tasks.workunit.client.1.smithi203.stdout:0/748: rmdir dd/d5c/d63/d106/d59 39 2024-05-25T06:11:40.288 INFO:tasks.workunit.client.0.smithi160.stdout:5/614: stat f6 0 2024-05-25T06:11:40.289 INFO:tasks.workunit.client.1.smithi203.stdout:0/749: chown dd/f1f 17465 1 2024-05-25T06:11:40.289 INFO:tasks.workunit.client.1.smithi203.stdout:3/959: write d10/d16/d23/d4a/d88/fc0 [212025,27313] 0 2024-05-25T06:11:40.290 INFO:tasks.workunit.client.0.smithi160.stdout:6/524: symlink d5/d30/d36/d73/d87/da4/lb6 0 2024-05-25T06:11:40.290 INFO:tasks.workunit.client.0.smithi160.stdout:4/512: mkdir d4/d6/d35/d3d/d9e 0 2024-05-25T06:11:40.293 INFO:tasks.workunit.client.0.smithi160.stdout:9/490: sync 2024-05-25T06:11:40.294 INFO:tasks.workunit.client.1.smithi203.stdout:9/735: link d0/l22 d0/d17/db6/lf5 0 2024-05-25T06:11:40.294 INFO:tasks.workunit.client.1.smithi203.stdout:1/736: mkdir d14/d81/d100 0 2024-05-25T06:11:40.295 INFO:tasks.workunit.client.1.smithi203.stdout:2/894: dread d2/d38/d43/d60/fa4 [0,4194304] 0 2024-05-25T06:11:40.295 INFO:tasks.workunit.client.0.smithi160.stdout:8/591: rmdir d5/d44/d1f/da3/d28/d49/db2 0 2024-05-25T06:11:40.296 INFO:tasks.workunit.client.0.smithi160.stdout:0/623: write d2b/d46/d63/fb7 [817083,25514] 0 2024-05-25T06:11:40.297 INFO:tasks.workunit.client.1.smithi203.stdout:9/736: write d0/df/d2d/f7c [3510017,118219] 0 2024-05-25T06:11:40.301 INFO:tasks.workunit.client.0.smithi160.stdout:5/615: creat db/dc/d13/d1d/d31/d56/fbc x:0 0 0 2024-05-25T06:11:40.301 INFO:tasks.workunit.client.1.smithi203.stdout:1/737: mkdir d14/d57/db7/dd2/d101 0 2024-05-25T06:11:40.302 INFO:tasks.workunit.client.0.smithi160.stdout:9/491: mkdir d4/d11/d12/d58/d97 0 2024-05-25T06:11:40.304 INFO:tasks.workunit.client.1.smithi203.stdout:9/737: truncate d0/df/d37/fb3 4788040 0 2024-05-25T06:11:40.305 INFO:tasks.workunit.client.0.smithi160.stdout:0/624: creat d2b/d46/d63/fc7 x:0 0 0 2024-05-25T06:11:40.306 INFO:tasks.workunit.client.0.smithi160.stdout:1/423: mkdir de/d21/d49/d88 0 2024-05-25T06:11:40.307 INFO:tasks.workunit.client.0.smithi160.stdout:0/625: chown d2b/d31/d35/d36/d37/f75 288 1 2024-05-25T06:11:40.307 INFO:tasks.workunit.client.1.smithi203.stdout:2/895: symlink d2/d1a/l12f 0 2024-05-25T06:11:40.309 INFO:tasks.workunit.client.1.smithi203.stdout:1/738: unlink d14/d57/db7/dd2/db3/c87 0 2024-05-25T06:11:40.309 INFO:tasks.workunit.client.0.smithi160.stdout:8/592: rename d5/d44/d1f/d6b to d5/d44/d1f/da3/d28/dc6 0 2024-05-25T06:11:40.310 INFO:tasks.workunit.client.0.smithi160.stdout:1/424: read de/d21/d49/d1c/d52/f6d [1951975,30069] 0 2024-05-25T06:11:40.310 INFO:tasks.workunit.client.1.smithi203.stdout:3/960: getdents d10/d16/d23/d4a/d88/deb 0 2024-05-25T06:11:40.311 INFO:tasks.workunit.client.0.smithi160.stdout:6/525: creat d5/de/d32/d9f/fb7 x:0 0 0 2024-05-25T06:11:40.312 INFO:tasks.workunit.client.1.smithi203.stdout:2/896: symlink d2/d1a/dd3/d100/d10/dc8/l130 0 2024-05-25T06:11:40.315 INFO:tasks.workunit.client.0.smithi160.stdout:6/526: stat d5/de/d58/c81 0 2024-05-25T06:11:40.315 INFO:tasks.workunit.client.0.smithi160.stdout:8/593: chown d5/d44/d1f/da3/fa8 11261891 1 2024-05-25T06:11:40.316 INFO:tasks.workunit.client.0.smithi160.stdout:4/513: getdents d4/d6/d5b/d66/d78 0 2024-05-25T06:11:40.317 INFO:tasks.workunit.client.1.smithi203.stdout:8/938: truncate d2/d5/d10/d45/d5d/f117 1303012 0 2024-05-25T06:11:40.318 INFO:tasks.workunit.client.0.smithi160.stdout:4/514: stat d4/d6/d28/l90 0 2024-05-25T06:11:40.320 INFO:tasks.workunit.client.1.smithi203.stdout:1/739: mknod d14/d7c/d9d/c102 0 2024-05-25T06:11:40.320 INFO:tasks.workunit.client.0.smithi160.stdout:0/626: dwrite f24 [4194304,4194304] 0 2024-05-25T06:11:40.321 INFO:tasks.workunit.client.1.smithi203.stdout:1/740: read d14/d57/d6b/f50 [3130918,6045] 0 2024-05-25T06:11:40.321 INFO:tasks.workunit.client.0.smithi160.stdout:1/425: rename de/d21/d49/f27 to de/d21/d49/d1c/d52/d77/f89 0 2024-05-25T06:11:40.322 INFO:tasks.workunit.client.0.smithi160.stdout:4/515: dread - d4/d6/d28/f9d zero size 2024-05-25T06:11:40.323 INFO:tasks.workunit.client.0.smithi160.stdout:8/594: mknod d5/d44/d1f/da3/d28/d4a/d93/cc7 0 2024-05-25T06:11:40.323 INFO:tasks.workunit.client.1.smithi203.stdout:3/961: mknod d10/d131/c137 0 2024-05-25T06:11:40.327 INFO:tasks.workunit.client.0.smithi160.stdout:3/472: dread d19/d22/f2c [0,4194304] 0 2024-05-25T06:11:40.327 INFO:tasks.workunit.client.0.smithi160.stdout:0/627: creat d2b/d46/d63/da7/fc8 x:0 0 0 2024-05-25T06:11:40.327 INFO:tasks.workunit.client.0.smithi160.stdout:8/595: chown d5/d44/d1f/da3/d28/d4a/d93/cc7 61417296 1 2024-05-25T06:11:40.328 INFO:tasks.workunit.client.1.smithi203.stdout:9/738: truncate d0/df/d37/f58 67878 0 2024-05-25T06:11:40.329 INFO:tasks.workunit.client.0.smithi160.stdout:1/426: mkdir de/d21/d8a 0 2024-05-25T06:11:40.331 INFO:tasks.workunit.client.1.smithi203.stdout:0/750: dwrite dd/d5c/d63/d106/d59/fee [0,4194304] 0 2024-05-25T06:11:40.331 INFO:tasks.workunit.client.1.smithi203.stdout:1/741: stat d14/dff/lc1 0 2024-05-25T06:11:40.331 INFO:tasks.workunit.client.0.smithi160.stdout:6/527: dwrite d5/f1e [0,4194304] 0 2024-05-25T06:11:40.337 INFO:tasks.workunit.client.1.smithi203.stdout:9/739: write d0/d17/d20/d3c/d5c/fbd [299623,47365] 0 2024-05-25T06:11:40.337 INFO:tasks.workunit.client.1.smithi203.stdout:2/897: link d2/d1a/dd3/d100/d10/f9b d2/d1a/d89/daa/def/d107/d113/f131 0 2024-05-25T06:11:40.340 INFO:tasks.workunit.client.1.smithi203.stdout:3/962: dwrite d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:40.341 INFO:tasks.workunit.client.1.smithi203.stdout:0/751: creat dd/d10/d34/d4c/d5d/f109 x:0 0 0 2024-05-25T06:11:40.342 INFO:tasks.workunit.client.1.smithi203.stdout:9/740: dread - d0/d17/d20/d29/fe8 zero size 2024-05-25T06:11:40.343 INFO:tasks.workunit.client.1.smithi203.stdout:8/939: dread d2/d5/d10/dfa/f11d [0,4194304] 0 2024-05-25T06:11:40.345 INFO:tasks.workunit.client.1.smithi203.stdout:2/898: mknod d2/d1a/dd3/d100/d10/dc8/c132 0 2024-05-25T06:11:40.351 INFO:tasks.workunit.client.1.smithi203.stdout:8/940: symlink d2/d5/d10/d4a/db6/l141 0 2024-05-25T06:11:40.354 INFO:tasks.workunit.client.1.smithi203.stdout:0/752: mknod dd/d5c/d63/d106/d75/d93/c10a 0 2024-05-25T06:11:40.355 INFO:tasks.workunit.client.1.smithi203.stdout:2/899: chown d2/d1a/d33/ce3 3499 1 2024-05-25T06:11:40.356 INFO:tasks.workunit.client.1.smithi203.stdout:9/741: dwrite d0/df/d37/fd8 [0,4194304] 0 2024-05-25T06:11:40.359 INFO:tasks.workunit.client.1.smithi203.stdout:2/900: mkdir d2/d1a/d33/d80/d133 0 2024-05-25T06:11:40.363 INFO:tasks.workunit.client.1.smithi203.stdout:1/742: dwrite fc [0,4194304] 0 2024-05-25T06:11:40.368 INFO:tasks.workunit.client.1.smithi203.stdout:1/743: chown d14/f24 765 1 2024-05-25T06:11:40.374 INFO:tasks.workunit.client.1.smithi203.stdout:3/963: dwrite d10/d16/d23/d3a/dc6/ff7 [0,4194304] 0 2024-05-25T06:11:40.375 INFO:tasks.workunit.client.1.smithi203.stdout:2/901: dwrite d2/d1a/d87/d9e/ff7 [0,4194304] 0 2024-05-25T06:11:40.378 INFO:tasks.workunit.client.1.smithi203.stdout:9/742: write d0/d17/d20/f3a [1369520,40985] 0 2024-05-25T06:11:40.383 INFO:tasks.workunit.client.1.smithi203.stdout:3/964: mkdir d10/d16/d71/dfa/d107/d138 0 2024-05-25T06:11:40.389 INFO:tasks.workunit.client.1.smithi203.stdout:2/902: write d2/d1a/dd3/d100/d10/d32/dca/de8/f11d [947754,93143] 0 2024-05-25T06:11:40.394 INFO:tasks.workunit.client.1.smithi203.stdout:9/743: dwrite d0/d17/d20/d3c/db2/fe7 [0,4194304] 0 2024-05-25T06:11:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:40 smithi203 ceph-mon[36030]: pgmap v20: 65 pgs: 65 active+clean; 2.5 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 90 MiB/s rd, 255 MiB/s wr, 471 op/s 2024-05-25T06:11:40.399 INFO:tasks.workunit.client.1.smithi203.stdout:8/941: dwrite d2/d5/d18/d31/f5e [0,4194304] 0 2024-05-25T06:11:40.401 INFO:tasks.workunit.client.1.smithi203.stdout:1/744: dwrite d14/dff/db8/dc2/f61 [0,4194304] 0 2024-05-25T06:11:40.406 INFO:tasks.workunit.client.1.smithi203.stdout:1/745: read d14/d57/db7/dd2/f8d [789876,58830] 0 2024-05-25T06:11:40.408 INFO:tasks.workunit.client.1.smithi203.stdout:0/753: dread dd/f1f [4194304,4194304] 0 2024-05-25T06:11:40.409 INFO:tasks.workunit.client.1.smithi203.stdout:3/965: truncate d10/d16/d23/d4a/d88/dca/f101 161027 0 2024-05-25T06:11:40.413 INFO:tasks.workunit.client.1.smithi203.stdout:8/942: rename d2/d5/d18/d31/dac/dc2/d88/d93/deb to d2/d5/d10/dfa/d142 0 2024-05-25T06:11:40.415 INFO:tasks.workunit.client.1.smithi203.stdout:9/744: write d0/d17/fd7 [234055,127673] 0 2024-05-25T06:11:40.416 INFO:tasks.workunit.client.1.smithi203.stdout:0/754: symlink dd/d5c/d63/d106/d4f/d62/d78/l10b 0 2024-05-25T06:11:40.419 INFO:tasks.workunit.client.1.smithi203.stdout:1/746: rename d14/dff/db8/dc2/c45 to d14/d81/c103 0 2024-05-25T06:11:40.420 INFO:tasks.workunit.client.1.smithi203.stdout:8/943: read - d2/d5/d28/d38/d4e/d97/f102 zero size 2024-05-25T06:11:40.421 INFO:tasks.workunit.client.1.smithi203.stdout:9/745: unlink d0/f21 0 2024-05-25T06:11:40.422 INFO:tasks.workunit.client.1.smithi203.stdout:0/755: mkdir dd/d5c/d63/d106/df3/d10c 0 2024-05-25T06:11:40.422 INFO:tasks.workunit.client.1.smithi203.stdout:0/756: stat dd/d32/de5 0 2024-05-25T06:11:40.424 INFO:tasks.workunit.client.1.smithi203.stdout:8/944: symlink d2/d5/d10/dfa/d13d/l143 0 2024-05-25T06:11:40.424 INFO:tasks.workunit.client.1.smithi203.stdout:9/746: mkdir d0/d57/de0/df6 0 2024-05-25T06:11:40.425 INFO:tasks.workunit.client.1.smithi203.stdout:0/757: readlink dd/d5c/d63/d106/d48/d91/ld6 0 2024-05-25T06:11:40.428 INFO:tasks.workunit.client.1.smithi203.stdout:8/945: symlink d2/d5/d10/d11/de/l144 0 2024-05-25T06:11:40.430 INFO:tasks.workunit.client.0.smithi160.stdout:2/483: truncate d12/d3e/f68 2331074 0 2024-05-25T06:11:40.431 INFO:tasks.workunit.client.0.smithi160.stdout:6/528: write d5/d30/f5a [392236,105227] 0 2024-05-25T06:11:40.432 INFO:tasks.workunit.client.1.smithi203.stdout:9/747: symlink d0/d57/dae/lf7 0 2024-05-25T06:11:40.433 INFO:tasks.workunit.client.0.smithi160.stdout:7/567: sync 2024-05-25T06:11:40.434 INFO:tasks.workunit.client.0.smithi160.stdout:0/628: write f1e [3871286,13075] 0 2024-05-25T06:11:40.435 INFO:tasks.workunit.client.0.smithi160.stdout:3/473: write d19/f45 [125902,1114] 0 2024-05-25T06:11:40.437 INFO:tasks.workunit.client.0.smithi160.stdout:8/596: dread d5/d44/d1f/da3/d28/d4a/d73/f91 [0,4194304] 0 2024-05-25T06:11:40.437 INFO:tasks.workunit.client.0.smithi160.stdout:2/484: creat d12/d1c/d2a/d7e/fa7 x:0 0 0 2024-05-25T06:11:40.439 INFO:tasks.workunit.client.0.smithi160.stdout:0/629: dread - d2b/d31/f7d zero size 2024-05-25T06:11:40.440 INFO:tasks.workunit.client.0.smithi160.stdout:1/427: unlink de/d39/f69 0 2024-05-25T06:11:40.441 INFO:tasks.workunit.client.1.smithi203.stdout:1/747: write d14/dff/db8/dc2/f72 [494510,108538] 0 2024-05-25T06:11:40.442 INFO:tasks.workunit.client.0.smithi160.stdout:5/616: dwrite db/dc/d13/f65 [0,4194304] 0 2024-05-25T06:11:40.443 INFO:tasks.workunit.client.1.smithi203.stdout:8/946: dwrite d2/d5/d10/d11/d1f/d75/f119 [0,4194304] 0 2024-05-25T06:11:40.444 INFO:tasks.workunit.client.0.smithi160.stdout:2/485: fdatasync d12/d1c/d2a/f80 0 2024-05-25T06:11:40.446 INFO:tasks.workunit.client.0.smithi160.stdout:7/568: rename d6/d2b/d5d/l7b to d6/d32/d5b/lb8 0 2024-05-25T06:11:40.449 INFO:tasks.workunit.client.1.smithi203.stdout:9/748: dwrite d0/d17/fd7 [0,4194304] 0 2024-05-25T06:11:40.450 INFO:tasks.workunit.client.0.smithi160.stdout:3/474: creat d19/d2f/d38/d47/fa2 x:0 0 0 2024-05-25T06:11:40.450 INFO:tasks.workunit.client.0.smithi160.stdout:6/529: dwrite d5/d45/f9a [0,4194304] 0 2024-05-25T06:11:40.453 INFO:tasks.workunit.client.0.smithi160.stdout:0/630: fdatasync d2b/d46/d63/da7/f85 0 2024-05-25T06:11:40.453 INFO:tasks.workunit.client.1.smithi203.stdout:3/966: dread d10/d16/d50/d7d/f9d [0,4194304] 0 2024-05-25T06:11:40.455 INFO:tasks.workunit.client.0.smithi160.stdout:5/617: dread - db/dc/d13/f50 zero size 2024-05-25T06:11:40.458 INFO:tasks.workunit.client.0.smithi160.stdout:7/569: dwrite d6/dc/d21/f63 [0,4194304] 0 2024-05-25T06:11:40.466 INFO:tasks.workunit.client.0.smithi160.stdout:3/475: truncate d19/d22/f2a 1103901 0 2024-05-25T06:11:40.466 INFO:tasks.workunit.client.0.smithi160.stdout:6/530: mknod d5/d30/d36/d73/d87/da4/cb8 0 2024-05-25T06:11:40.466 INFO:tasks.workunit.client.0.smithi160.stdout:1/428: dread fb [0,4194304] 0 2024-05-25T06:11:40.468 INFO:tasks.workunit.client.0.smithi160.stdout:6/531: fsync d5/d39/d8f/f9d 0 2024-05-25T06:11:40.469 INFO:tasks.workunit.client.0.smithi160.stdout:8/597: symlink d5/d44/d1f/da3/d28/d4a/db8/lc8 0 2024-05-25T06:11:40.469 INFO:tasks.workunit.client.0.smithi160.stdout:2/486: chown d12/d3e/d5f/d74/d7a/c96 52 1 2024-05-25T06:11:40.470 INFO:tasks.workunit.client.0.smithi160.stdout:5/618: mkdir db/dc/d13/d1d/d23/dbd 0 2024-05-25T06:11:40.471 INFO:tasks.workunit.client.0.smithi160.stdout:7/570: symlink d6/d3e/da1/lb9 0 2024-05-25T06:11:40.472 INFO:tasks.workunit.client.0.smithi160.stdout:3/476: read d19/d2f/d38/d47/f6e [3809740,90955] 0 2024-05-25T06:11:40.472 INFO:tasks.workunit.client.0.smithi160.stdout:8/598: mkdir d5/d39/d7a/dc9 0 2024-05-25T06:11:40.475 INFO:tasks.workunit.client.0.smithi160.stdout:1/429: symlink de/d21/d49/d1c/d41/l8b 0 2024-05-25T06:11:40.475 INFO:tasks.workunit.client.0.smithi160.stdout:3/477: write d19/f45 [603743,63300] 0 2024-05-25T06:11:40.476 INFO:tasks.workunit.client.0.smithi160.stdout:7/571: mkdir d6/dba 0 2024-05-25T06:11:40.477 INFO:tasks.workunit.client.0.smithi160.stdout:3/478: dread - d19/d2f/d34/f5c zero size 2024-05-25T06:11:40.478 INFO:tasks.workunit.client.0.smithi160.stdout:8/599: creat d5/d7/d9a/fca x:0 0 0 2024-05-25T06:11:40.479 INFO:tasks.workunit.client.0.smithi160.stdout:1/430: rmdir de/d21/d49/d1c/d41/d45 39 2024-05-25T06:11:40.480 INFO:tasks.workunit.client.0.smithi160.stdout:1/431: chown de/d21/d49/f6f 100 1 2024-05-25T06:11:40.484 INFO:tasks.workunit.client.0.smithi160.stdout:1/432: chown de/d21/d49/d35/c4b 292 1 2024-05-25T06:11:40.487 INFO:tasks.workunit.client.0.smithi160.stdout:7/572: dwrite d6/dc/d64/d5a/f8f [0,4194304] 0 2024-05-25T06:11:40.488 INFO:tasks.workunit.client.0.smithi160.stdout:1/433: read de/d21/d49/d1c/d59/d71/f5c [3878193,9238] 0 2024-05-25T06:11:40.494 INFO:tasks.workunit.client.0.smithi160.stdout:7/573: read d6/dc/f25 [827916,26818] 0 2024-05-25T06:11:40.495 INFO:tasks.workunit.client.0.smithi160.stdout:6/532: truncate d5/de/d32/d35/f78 82742 0 2024-05-25T06:11:40.495 INFO:tasks.workunit.client.0.smithi160.stdout:7/574: chown d6/d32/lad 12736437 1 2024-05-25T06:11:40.500 INFO:tasks.workunit.client.0.smithi160.stdout:6/533: read - d5/d45/d4a/fa8 zero size 2024-05-25T06:11:40.504 INFO:tasks.workunit.client.0.smithi160.stdout:1/434: dwrite de/d21/d49/d1c/f53 [0,4194304] 0 2024-05-25T06:11:40.507 INFO:tasks.workunit.client.0.smithi160.stdout:8/600: dwrite d5/d7/f5f [0,4194304] 0 2024-05-25T06:11:40.509 INFO:tasks.workunit.client.0.smithi160.stdout:6/534: chown d5/de/d32/d74/d77/d89/f8a 214328 1 2024-05-25T06:11:40.511 INFO:tasks.workunit.client.0.smithi160.stdout:2/487: dread d12/f25 [0,4194304] 0 2024-05-25T06:11:40.513 INFO:tasks.workunit.client.0.smithi160.stdout:6/535: fdatasync d5/de/d32/d9f/fb7 0 2024-05-25T06:11:40.513 INFO:tasks.workunit.client.0.smithi160.stdout:3/479: dread d19/d2f/d38/d43/d5b/f6d [0,4194304] 0 2024-05-25T06:11:40.514 INFO:tasks.workunit.client.0.smithi160.stdout:5/619: dread db/dc/d13/d1d/f20 [0,4194304] 0 2024-05-25T06:11:40.514 INFO:tasks.workunit.client.0.smithi160.stdout:1/435: unlink de/d21/d49/d1c/d59/f5d 0 2024-05-25T06:11:40.516 INFO:tasks.workunit.client.0.smithi160.stdout:3/480: rename d19/d2f/d38/d43/d5b/d65 to d19/d2f/d38/d43/d5b/d65/d83/da3 22 2024-05-25T06:11:40.519 INFO:tasks.workunit.client.0.smithi160.stdout:6/536: mknod d5/de/d32/d74/d77/d89/cb9 0 2024-05-25T06:11:40.523 INFO:tasks.workunit.client.0.smithi160.stdout:6/537: read d5/de/d58/f80 [902537,1446] 0 2024-05-25T06:11:40.523 INFO:tasks.workunit.client.0.smithi160.stdout:5/620: mkdir db/dc/d13/d1d/d3f/d4a/dbe 0 2024-05-25T06:11:40.525 INFO:tasks.workunit.client.0.smithi160.stdout:1/436: write de/f3d [1359968,95938] 0 2024-05-25T06:11:40.526 INFO:tasks.workunit.client.0.smithi160.stdout:3/481: dwrite d19/d2f/d38/d43/d5b/d65/f89 [0,4194304] 0 2024-05-25T06:11:40.531 INFO:tasks.workunit.client.0.smithi160.stdout:7/575: dread d6/dc/d21/f2a [0,4194304] 0 2024-05-25T06:11:40.532 INFO:tasks.workunit.client.0.smithi160.stdout:6/538: dread d5/f1e [0,4194304] 0 2024-05-25T06:11:40.534 INFO:tasks.workunit.client.0.smithi160.stdout:1/437: rename de/d21/d49/d1c/d52/f6d to de/d21/d49/d1c/f8c 0 2024-05-25T06:11:40.536 INFO:tasks.workunit.client.0.smithi160.stdout:6/539: write d5/de/d32/d35/f5f [4857073,54607] 0 2024-05-25T06:11:40.539 INFO:tasks.workunit.client.0.smithi160.stdout:6/540: dread - d5/de/d32/d9f/fb7 zero size 2024-05-25T06:11:40.542 INFO:tasks.workunit.client.0.smithi160.stdout:2/488: write f8 [2359075,123539] 0 2024-05-25T06:11:40.543 INFO:tasks.workunit.client.0.smithi160.stdout:1/438: dwrite de/d21/d49/d1c/d59/d71/f74 [0,4194304] 0 2024-05-25T06:11:40.549 INFO:tasks.workunit.client.0.smithi160.stdout:1/439: truncate de/f13 3311726 0 2024-05-25T06:11:40.551 INFO:tasks.workunit.client.0.smithi160.stdout:3/482: dread d19/f1e [0,4194304] 0 2024-05-25T06:11:40.552 INFO:tasks.workunit.client.0.smithi160.stdout:2/489: dread d12/d1c/d2a/f3a [0,4194304] 0 2024-05-25T06:11:40.554 INFO:tasks.workunit.client.0.smithi160.stdout:1/440: mknod de/d21/d49/d1c/d59/d7f/c8d 0 2024-05-25T06:11:40.556 INFO:tasks.workunit.client.0.smithi160.stdout:5/621: truncate db/dc/d13/f1f 2343464 0 2024-05-25T06:11:40.558 INFO:tasks.workunit.client.0.smithi160.stdout:1/441: unlink de/d21/d49/c38 0 2024-05-25T06:11:40.561 INFO:tasks.workunit.client.0.smithi160.stdout:8/601: dwrite d5/d9/fbb [0,4194304] 0 2024-05-25T06:11:40.566 INFO:tasks.workunit.client.0.smithi160.stdout:5/622: link db/d47/l49 db/dc/d13/d1d/d3f/d4a/lbf 0 2024-05-25T06:11:40.567 INFO:tasks.workunit.client.0.smithi160.stdout:2/490: dread d12/d13/f15 [0,4194304] 0 2024-05-25T06:11:40.570 INFO:tasks.workunit.client.0.smithi160.stdout:2/491: truncate fe 505102 0 2024-05-25T06:11:40.572 INFO:tasks.workunit.client.0.smithi160.stdout:2/492: chown d12/d13/f60 7 1 2024-05-25T06:11:40.574 INFO:tasks.workunit.client.0.smithi160.stdout:8/602: getdents d5/d44/d1f/da3/d28/d4a/d4e 0 2024-05-25T06:11:40.577 INFO:tasks.workunit.client.0.smithi160.stdout:5/623: dwrite db/dc/d13/d1d/d3f/d72/fa2 [0,4194304] 0 2024-05-25T06:11:40.578 INFO:tasks.workunit.client.0.smithi160.stdout:3/483: dread d19/d2f/d34/f40 [0,4194304] 0 2024-05-25T06:11:40.580 INFO:tasks.workunit.client.0.smithi160.stdout:7/576: dread d6/d32/d5b/f62 [0,4194304] 0 2024-05-25T06:11:40.580 INFO:tasks.workunit.client.0.smithi160.stdout:3/484: chown d19/f44 0 1 2024-05-25T06:11:40.581 INFO:tasks.workunit.client.0.smithi160.stdout:5/624: fdatasync db/dc/d13/d1d/fbb 0 2024-05-25T06:11:40.581 INFO:tasks.workunit.client.0.smithi160.stdout:8/603: fdatasync d5/d44/d1f/da3/d28/d49/d61/f9f 0 2024-05-25T06:11:40.581 INFO:tasks.workunit.client.0.smithi160.stdout:1/442: truncate de/f3d 3386247 0 2024-05-25T06:11:40.582 INFO:tasks.workunit.client.1.smithi203.stdout:0/758: creat dd/d32/f10d x:0 0 0 2024-05-25T06:11:40.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/443: chown de/d21/d49/d1c/d63/f7b 311 1 2024-05-25T06:11:40.584 INFO:tasks.workunit.client.1.smithi203.stdout:8/947: rename d2/d5/d10 to d2/d5/d10/d11/d145 22 2024-05-25T06:11:40.585 INFO:tasks.workunit.client.0.smithi160.stdout:5/625: chown db/dc/d13/d1d/d23/d24/d2e/fb2 104480 1 2024-05-25T06:11:40.587 INFO:tasks.workunit.client.0.smithi160.stdout:2/493: truncate f8 3941733 0 2024-05-25T06:11:40.587 INFO:tasks.workunit.client.1.smithi203.stdout:3/967: mknod d10/d16/da1/c139 0 2024-05-25T06:11:40.588 INFO:tasks.workunit.client.0.smithi160.stdout:2/494: chown d12/d97 4641 1 2024-05-25T06:11:40.588 INFO:tasks.workunit.client.0.smithi160.stdout:5/626: chown db/dc/d13/f65 1 1 2024-05-25T06:11:40.589 INFO:tasks.workunit.client.1.smithi203.stdout:0/759: creat dd/d5c/d63/d106/d75/d93/f10e x:0 0 0 2024-05-25T06:11:40.589 INFO:tasks.workunit.client.0.smithi160.stdout:1/444: creat de/d21/d49/d1c/d59/d7f/f8e x:0 0 0 2024-05-25T06:11:40.590 INFO:tasks.workunit.client.1.smithi203.stdout:1/748: symlink d14/d57/d97/l104 0 2024-05-25T06:11:40.591 INFO:tasks.workunit.client.0.smithi160.stdout:8/604: rename d5/d39 to d5/d7/dcb 0 2024-05-25T06:11:40.592 INFO:tasks.workunit.client.0.smithi160.stdout:3/485: dwrite d19/d22/f2d [0,4194304] 0 2024-05-25T06:11:40.593 INFO:tasks.workunit.client.1.smithi203.stdout:0/760: stat dd/d10/d34/d4c/d99/c97 0 2024-05-25T06:11:40.594 INFO:tasks.workunit.client.0.smithi160.stdout:1/445: mkdir de/d39/d8f 0 2024-05-25T06:11:40.595 INFO:tasks.workunit.client.1.smithi203.stdout:1/749: truncate d14/d23/f42 2773241 0 2024-05-25T06:11:40.597 INFO:tasks.workunit.client.0.smithi160.stdout:2/495: rename d12/d1c/f6f to d12/d3e/d5a/d8d/fa8 0 2024-05-25T06:11:40.597 INFO:tasks.workunit.client.0.smithi160.stdout:1/446: chown de/d21/d8a 0 1 2024-05-25T06:11:40.597 INFO:tasks.workunit.client.1.smithi203.stdout:1/750: mkdir d14/d7c/d9d/dc8/d105 0 2024-05-25T06:11:40.599 INFO:tasks.workunit.client.0.smithi160.stdout:8/605: symlink d5/d9c/lcc 0 2024-05-25T06:11:40.603 INFO:tasks.workunit.client.1.smithi203.stdout:0/761: link dd/d10/d34/d4c/cce dd/d10/d34/d4c/d99/c10f 0 2024-05-25T06:11:40.605 INFO:tasks.workunit.client.0.smithi160.stdout:8/606: creat d5/d7/dcb/d7a/fcd x:0 0 0 2024-05-25T06:11:40.605 INFO:tasks.workunit.client.1.smithi203.stdout:1/751: dwrite d14/dff/db8/dc2/f90 [0,4194304] 0 2024-05-25T06:11:40.606 INFO:tasks.workunit.client.0.smithi160.stdout:8/607: chown d5/d44/d1f/da3/d28/d4a/db8/lc8 104975720 1 2024-05-25T06:11:40.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.609+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f380c115e20 con 0x7f37f406b7e0 2024-05-25T06:11:40.610 INFO:tasks.workunit.client.1.smithi203.stdout:0/762: creat dd/d32/d108/f110 x:0 0 0 2024-05-25T06:11:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.610+0000 7f37f37fe700 1 -- 172.21.15.160:0/731985280 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f380c115e20 con 0x7f37f406b7e0 2024-05-25T06:11:40.611 INFO:tasks.workunit.client.1.smithi203.stdout:1/752: symlink d14/d57/db7/dd2/db3/l106 0 2024-05-25T06:11:40.611 INFO:tasks.workunit.client.1.smithi203.stdout:0/763: stat dd/d5c/d63/d106/d75/fcf 0 2024-05-25T06:11:40.614 INFO:tasks.workunit.client.0.smithi160.stdout:8/608: dread d5/d7/dcb/d7a/f85 [0,4194304] 0 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f37f406b7e0 msgr2=0x7f37f406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f37f406b7e0 0x7f37f406dc90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f380000b0d0 tx=0x7f3800009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 msgr2=0x7f380c113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 0x7f380c113f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00eb40 tx=0x7f37fc00b040 comp rx=0 tx=0).stop 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 shutdown_connections 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f380c0fc110 0x7f380c113f30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f37f406b7e0 0x7f37f406dc90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.615+0000 7f3811851700 1 --2- 172.21.15.160:0/731985280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f380c0fca60 0x7f380c114470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.616+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 >> 172.21.15.160:0/731985280 conn(0x7f380c0f6b80 msgr2=0x7f380c0f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:40.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.616+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 shutdown_connections 2024-05-25T06:11:40.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.616+0000 7f3811851700 1 -- 172.21.15.160:0/731985280 wait complete. 2024-05-25T06:11:40.617 INFO:tasks.workunit.client.0.smithi160.stdout:8/609: creat d5/d44/d1f/da3/d28/d4a/d73/fce x:0 0 0 2024-05-25T06:11:40.634 INFO:tasks.workunit.client.0.smithi160.stdout:9/492: sync 2024-05-25T06:11:40.639 INFO:tasks.workunit.client.1.smithi203.stdout:1/753: write d14/d57/d1d/f40 [1577218,68377] 0 2024-05-25T06:11:40.640 INFO:tasks.workunit.client.0.smithi160.stdout:1/447: dwrite de/d21/d49/f2b [0,4194304] 0 2024-05-25T06:11:40.640 INFO:tasks.workunit.client.1.smithi203.stdout:1/754: write d14/d57/db7/dd2/fcd [1599779,10333] 0 2024-05-25T06:11:40.643 INFO:tasks.workunit.client.1.smithi203.stdout:0/764: mknod dd/d5c/d63/d106/d4f/d62/d78/d100/c111 0 2024-05-25T06:11:40.646 INFO:tasks.workunit.client.0.smithi160.stdout:8/610: creat d5/d7/dcb/d6a/fcf x:0 0 0 2024-05-25T06:11:40.647 INFO:tasks.workunit.client.0.smithi160.stdout:9/493: creat d4/d11/d1e/d3e/d2f/f98 x:0 0 0 2024-05-25T06:11:40.648 INFO:tasks.workunit.client.0.smithi160.stdout:1/448: mkdir de/d21/d49/d90 0 2024-05-25T06:11:40.650 INFO:tasks.workunit.client.0.smithi160.stdout:8/611: symlink d5/d44/d7e/ld0 0 2024-05-25T06:11:40.650 INFO:tasks.workunit.client.0.smithi160.stdout:1/449: chown de/d21/d49/d1c/d59/d67/c79 199850343 1 2024-05-25T06:11:40.652 INFO:tasks.workunit.client.0.smithi160.stdout:8/612: dread - d5/d7/dcb/d75/fa5 zero size 2024-05-25T06:11:40.652 INFO:tasks.workunit.client.0.smithi160.stdout:2/496: dread ff [0,4194304] 0 2024-05-25T06:11:40.653 INFO:tasks.workunit.client.0.smithi160.stdout:5/627: dread db/f22 [0,4194304] 0 2024-05-25T06:11:40.654 INFO:tasks.workunit.client.0.smithi160.stdout:9/494: truncate d4/d11/d1e/d3e/d25/f8f 791974 0 2024-05-25T06:11:40.654 INFO:tasks.workunit.client.0.smithi160.stdout:1/450: creat de/f91 x:0 0 0 2024-05-25T06:11:40.655 INFO:tasks.workunit.client.0.smithi160.stdout:5/628: read - db/dc/d13/d1d/faa zero size 2024-05-25T06:11:40.656 INFO:tasks.workunit.client.0.smithi160.stdout:8/613: mkdir d5/d44/d1f/d58/d65/da2/dd1 0 2024-05-25T06:11:40.657 INFO:tasks.workunit.client.0.smithi160.stdout:8/614: dread - d5/d44/d1f/da3/d28/d49/fb4 zero size 2024-05-25T06:11:40.659 INFO:tasks.workunit.client.0.smithi160.stdout:1/451: creat de/f92 x:0 0 0 2024-05-25T06:11:40.663 INFO:tasks.workunit.client.1.smithi203.stdout:9/749: truncate d0/d17/d20/d6e/fc3 2861798 0 2024-05-25T06:11:40.665 INFO:tasks.workunit.client.0.smithi160.stdout:5/629: mkdir db/dc/d13/d1d/d3f/dc0 0 2024-05-25T06:11:40.665 INFO:tasks.workunit.client.0.smithi160.stdout:9/495: mknod d4/d11/d1e/d3e/d25/d27/c99 0 2024-05-25T06:11:40.668 INFO:tasks.workunit.client.1.smithi203.stdout:3/968: dwrite d10/d16/d23/f136 [4194304,4194304] 0 2024-05-25T06:11:40.671 INFO:tasks.workunit.client.1.smithi203.stdout:9/750: link d0/d17/f4b d0/df/d2d/d92/ff8 0 2024-05-25T06:11:40.671 INFO:tasks.workunit.client.1.smithi203.stdout:0/765: write dd/d5c/d63/d106/d48/d91/dad/fd4 [302252,44266] 0 2024-05-25T06:11:40.675 INFO:tasks.workunit.client.1.smithi203.stdout:9/751: mknod d0/d57/dae/cf9 0 2024-05-25T06:11:40.676 INFO:tasks.workunit.client.1.smithi203.stdout:1/755: dwrite d14/d57/db7/dd2/f62 [0,4194304] 0 2024-05-25T06:11:40.678 INFO:tasks.workunit.client.1.smithi203.stdout:0/766: dread dd/d5c/d63/d106/d4f/f87 [0,4194304] 0 2024-05-25T06:11:40.678 INFO:tasks.workunit.client.1.smithi203.stdout:1/756: readlink d14/dff/db8/dc2/lc6 0 2024-05-25T06:11:40.678 INFO:tasks.workunit.client.0.smithi160.stdout:1/452: rename de/d43 to de/d21/d49/d1c/d59/d7c/d93 0 2024-05-25T06:11:40.679 INFO:tasks.workunit.client.0.smithi160.stdout:8/615: getdents d5/d44/d1f/d58/d65/da2 0 2024-05-25T06:11:40.682 INFO:tasks.workunit.client.0.smithi160.stdout:1/453: mknod de/d21/d49/d1c/d59/d7c/d93/c94 0 2024-05-25T06:11:40.682 INFO:tasks.workunit.client.1.smithi203.stdout:0/767: rmdir dd/d5c/d63/d106/d4f/d62/d78 39 2024-05-25T06:11:40.684 INFO:tasks.workunit.client.0.smithi160.stdout:8/616: symlink d5/d9/d72/daf/ld2 0 2024-05-25T06:11:40.686 INFO:tasks.workunit.client.0.smithi160.stdout:1/454: creat de/d21/d49/d1c/d59/f95 x:0 0 0 2024-05-25T06:11:40.687 INFO:tasks.workunit.client.0.smithi160.stdout:1/455: dread - de/f92 zero size 2024-05-25T06:11:40.687 INFO:tasks.workunit.client.1.smithi203.stdout:0/768: rename dd/d5c/d63/d106/d59/fac to dd/d32/de5/f112 0 2024-05-25T06:11:40.689 INFO:tasks.workunit.client.1.smithi203.stdout:1/757: dwrite d14/d57/d6b/da1/faa [0,4194304] 0 2024-05-25T06:11:40.689 INFO:tasks.workunit.client.0.smithi160.stdout:1/456: fdatasync fa 0 2024-05-25T06:11:40.693 INFO:tasks.workunit.client.1.smithi203.stdout:1/758: symlink d14/d57/d6b/da1/l107 0 2024-05-25T06:11:40.693 INFO:tasks.workunit.client.1.smithi203.stdout:9/752: dwrite d0/fa6 [0,4194304] 0 2024-05-25T06:11:40.694 INFO:tasks.workunit.client.0.smithi160.stdout:9/496: write d4/d11/d1e/d3e/d25/d27/f7d [810278,54593] 0 2024-05-25T06:11:40.695 INFO:tasks.workunit.client.1.smithi203.stdout:9/753: chown d0/df/d37 3746683 1 2024-05-25T06:11:40.696 INFO:tasks.workunit.client.1.smithi203.stdout:0/769: symlink dd/d5c/d63/d106/da0/db9/df9/d9d/l113 0 2024-05-25T06:11:40.697 INFO:tasks.workunit.client.0.smithi160.stdout:5/630: dwrite db/f53 [0,4194304] 0 2024-05-25T06:11:40.698 INFO:tasks.workunit.client.0.smithi160.stdout:9/497: chown d4/d11/d1e/d3e/d25/d27/c2d 190 1 2024-05-25T06:11:40.700 INFO:tasks.workunit.client.1.smithi203.stdout:0/770: read - dd/d5c/f68 zero size 2024-05-25T06:11:40.706 INFO:tasks.workunit.client.1.smithi203.stdout:1/759: getdents d14/d57/db7/dd2/dca 0 2024-05-25T06:11:40.709 INFO:tasks.workunit.client.1.smithi203.stdout:0/771: unlink dd/d10/d34/d4c/l9e 0 2024-05-25T06:11:40.710 INFO:tasks.workunit.client.1.smithi203.stdout:0/772: chown dd 430 1 2024-05-25T06:11:40.711 INFO:tasks.workunit.client.0.smithi160.stdout:9/498: symlink d4/d11/d12/d2e/d45/d90/l9a 0 2024-05-25T06:11:40.711 INFO:tasks.workunit.client.1.smithi203.stdout:9/754: dwrite d0/d17/d20/d3c/fea [0,4194304] 0 2024-05-25T06:11:40.713 INFO:tasks.workunit.client.1.smithi203.stdout:3/969: dread d10/dea/f127 [0,4194304] 0 2024-05-25T06:11:40.713 INFO:tasks.workunit.client.0.smithi160.stdout:5/631: unlink db/dc/d13/d1d/d3f/d72/d98/da5/dae/fb6 0 2024-05-25T06:11:40.716 INFO:tasks.workunit.client.1.smithi203.stdout:0/773: creat dd/d10/d34/d4c/d99/d70/f114 x:0 0 0 2024-05-25T06:11:40.719 INFO:tasks.workunit.client.1.smithi203.stdout:1/760: dwrite d14/f24 [0,4194304] 0 2024-05-25T06:11:40.720 INFO:tasks.workunit.client.1.smithi203.stdout:0/774: mkdir dd/d10/d34/d4c/d5d/d115 0 2024-05-25T06:11:40.721 INFO:tasks.workunit.client.0.smithi160.stdout:5/632: dwrite db/dc/d13/d1d/d31/d56/f61 [0,4194304] 0 2024-05-25T06:11:40.721 INFO:tasks.workunit.client.1.smithi203.stdout:1/761: chown d14/c15 207 1 2024-05-25T06:11:40.722 INFO:tasks.workunit.client.1.smithi203.stdout:0/775: readlink dd/d32/la6 0 2024-05-25T06:11:40.722 INFO:tasks.workunit.client.0.smithi160.stdout:5/633: chown db/dc/d13/d9a 17 1 2024-05-25T06:11:40.723 INFO:tasks.workunit.client.1.smithi203.stdout:0/776: readlink dd/d5c/d63/d106/l6b 0 2024-05-25T06:11:40.724 INFO:tasks.workunit.client.1.smithi203.stdout:3/970: dwrite d10/d16/d23/d6c/f63 [0,4194304] 0 2024-05-25T06:11:40.725 INFO:tasks.workunit.client.0.smithi160.stdout:5/634: stat db/dc/c88 0 2024-05-25T06:11:40.728 INFO:tasks.workunit.client.1.smithi203.stdout:0/777: write dd/d5c/d63/d106/df3/f3b [234342,72330] 0 2024-05-25T06:11:40.730 INFO:tasks.workunit.client.0.smithi160.stdout:5/635: mkdir db/d47/d81/dc1 0 2024-05-25T06:11:40.730 INFO:tasks.workunit.client.1.smithi203.stdout:0/778: readlink dd/d10/d34/lc3 0 2024-05-25T06:11:40.732 INFO:tasks.workunit.client.1.smithi203.stdout:1/762: dread d14/d57/d29/f80 [0,4194304] 0 2024-05-25T06:11:40.733 INFO:tasks.workunit.client.0.smithi160.stdout:5/636: getdents db/d47/d81/dc1 0 2024-05-25T06:11:40.735 INFO:tasks.workunit.client.1.smithi203.stdout:4/978: sync 2024-05-25T06:11:40.735 INFO:tasks.workunit.client.1.smithi203.stdout:7/827: sync 2024-05-25T06:11:40.740 INFO:tasks.workunit.client.1.smithi203.stdout:4/979: mknod d0/d8/d3a/d81/d93/d98/c149 0 2024-05-25T06:11:40.740 INFO:tasks.workunit.client.0.smithi160.stdout:5/637: rename db/dc/c3d to db/dc/d13/d1d/d31/cc2 0 2024-05-25T06:11:40.742 INFO:tasks.workunit.client.1.smithi203.stdout:1/763: dwrite d14/d57/db7/dd2/fce [0,4194304] 0 2024-05-25T06:11:40.745 INFO:tasks.workunit.client.1.smithi203.stdout:4/980: rename d0/d8/d52/cdf to d0/d9/d50/d79/c14a 0 2024-05-25T06:11:40.746 INFO:tasks.workunit.client.1.smithi203.stdout:1/764: creat d14/d7c/d9d/dc8/f108 x:0 0 0 2024-05-25T06:11:40.750 INFO:tasks.workunit.client.1.smithi203.stdout:4/981: dread d0/d9/d50/dce/f75 [0,4194304] 0 2024-05-25T06:11:40.752 INFO:tasks.workunit.client.1.smithi203.stdout:1/765: getdents d14/d57/db7/dd2 0 2024-05-25T06:11:40.756 INFO:tasks.workunit.client.1.smithi203.stdout:9/755: write d0/f39 [808852,59403] 0 2024-05-25T06:11:40.757 INFO:tasks.workunit.client.1.smithi203.stdout:4/982: dread d0/d8/d1f/d26/f4a [0,4194304] 0 2024-05-25T06:11:40.760 INFO:tasks.workunit.client.1.smithi203.stdout:4/983: fsync d0/d8/d3a/d81/d93/d98/d99/ded/d10b/f11d 0 2024-05-25T06:11:40.761 INFO:tasks.workunit.client.1.smithi203.stdout:1/766: dwrite d14/f32 [0,4194304] 0 2024-05-25T06:11:40.763 INFO:tasks.workunit.client.1.smithi203.stdout:1/767: chown d14/d81/c9c 7 1 2024-05-25T06:11:40.764 INFO:tasks.workunit.client.1.smithi203.stdout:0/779: dread dd/f72 [0,4194304] 0 2024-05-25T06:11:40.765 INFO:tasks.workunit.client.1.smithi203.stdout:4/984: write d0/d8/d3a/f119 [2220639,113576] 0 2024-05-25T06:11:40.766 INFO:tasks.workunit.client.0.smithi160.stdout:5/638: write db/dc/d13/f2c [1666374,98168] 0 2024-05-25T06:11:40.769 INFO:tasks.workunit.client.1.smithi203.stdout:1/768: creat d14/dff/f109 x:0 0 0 2024-05-25T06:11:40.769 INFO:tasks.workunit.client.0.smithi160.stdout:5/639: creat db/d47/d5e/fc3 x:0 0 0 2024-05-25T06:11:40.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 -- 172.21.15.160:0/4221904715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fe3e0 msgr2=0x7fcea00fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:40.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 --2- 172.21.15.160:0/4221904715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fe3e0 0x7fcea00fe7f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce90004940 tx=0x7fce90009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:40.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 -- 172.21.15.160:0/4221904715 shutdown_connections 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 --2- 172.21.15.160:0/4221904715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00fed30 0x7fcea00fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 --2- 172.21.15.160:0/4221904715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fe3e0 0x7fcea00fe7f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 -- 172.21.15.160:0/4221904715 >> 172.21.15.160:0/4221904715 conn(0x7fcea00f6b80 msgr2=0x7fcea00f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 -- 172.21.15.160:0/4221904715 shutdown_connections 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.769+0000 7fcea6469700 1 -- 172.21.15.160:0/4221904715 wait complete. 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.770+0000 7fcea6469700 1 Processor -- start 2024-05-25T06:11:40.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.770+0000 7fcea6469700 1 -- start start 2024-05-25T06:11:40.772 INFO:tasks.workunit.client.1.smithi203.stdout:7/828: dread d1/d9/d2c/d3a/d4e/f6c [0,4194304] 0 2024-05-25T06:11:40.772 INFO:tasks.workunit.client.1.smithi203.stdout:0/780: creat dd/d5c/d63/d106/d48/f116 x:0 0 0 2024-05-25T06:11:40.772 INFO:tasks.workunit.client.1.smithi203.stdout:4/985: symlink d0/d8/d1f/d26/d3b/d9c/l14b 0 2024-05-25T06:11:40.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fcea6469700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00fe3e0 0x7fcea010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:40.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fcea6469700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 0x7fcea010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:40.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fcea6469700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea010e5b0 con 0x7fcea00fed30 2024-05-25T06:11:40.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fcea6469700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea010eda0 con 0x7fcea00fe3e0 2024-05-25T06:11:40.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 0x7fcea010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 0x7fcea010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48496/0 (socket says 172.21.15.160:48496) 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.771+0000 7fce9f7fe700 1 -- 172.21.15.160:0/2930498661 learned_addr learned my addr 172.21.15.160:0/2930498661 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.772+0000 7fce9ffff700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00fe3e0 0x7fcea010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.772+0000 7fce9f7fe700 1 -- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00fe3e0 msgr2=0x7fcea010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.772+0000 7fce9f7fe700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00fe3e0 0x7fcea010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.772+0000 7fce9f7fe700 1 -- 172.21.15.160:0/2930498661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90005040 con 0x7fcea00fed30 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.772+0000 7fce9f7fe700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 0x7fcea010df20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b790 tx=0x7fce94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:40.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.773+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9400f6f0 con 0x7fcea00fed30 2024-05-25T06:11:40.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.773+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce9400fd20 con 0x7fcea00fed30 2024-05-25T06:11:40.775 INFO:tasks.workunit.client.0.smithi160.stdout:5/640: mknod db/dc/d13/d1d/d23/d24/daf/db3/cc4 0 2024-05-25T06:11:40.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.773+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce940103a0 con 0x7fcea00fed30 2024-05-25T06:11:40.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.773+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea010f080 con 0x7fcea00fed30 2024-05-25T06:11:40.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.773+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea010f5f0 con 0x7fcea00fed30 2024-05-25T06:11:40.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.774+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea005f5c0 con 0x7fcea00fed30 2024-05-25T06:11:40.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.776+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fce94011400 con 0x7fcea00fed30 2024-05-25T06:11:40.777 INFO:tasks.workunit.client.1.smithi203.stdout:1/769: symlink d14/d7c/l10a 0 2024-05-25T06:11:40.777 INFO:tasks.workunit.client.1.smithi203.stdout:4/986: dread - d0/d8/d10/f7f zero size 2024-05-25T06:11:40.777 INFO:tasks.workunit.client.1.smithi203.stdout:1/770: creat d14/d7c/f10b x:0 0 0 2024-05-25T06:11:40.778 INFO:tasks.workunit.client.1.smithi203.stdout:4/987: chown d0/d9/d50/d79/d9a/dbe/de0 462212 1 2024-05-25T06:11:40.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.776+0000 7fce9cff9700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fce8806b790 0x7fce8806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:40.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.777+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fce9408abd0 con 0x7fcea00fed30 2024-05-25T06:11:40.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.777+0000 7fce9ffff700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fce8806b790 0x7fce8806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:40.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.777+0000 7fce9ffff700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fce8806b790 0x7fce8806dc40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b970 tx=0x7fce90009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:40.778 INFO:tasks.workunit.client.1.smithi203.stdout:1/771: write d14/d57/db7/fd1 [188445,10126] 0 2024-05-25T06:11:40.779 INFO:tasks.workunit.client.1.smithi203.stdout:4/988: chown d0/d8/d3a/l67 21 1 2024-05-25T06:11:40.779 INFO:tasks.workunit.client.1.smithi203.stdout:0/781: getdents dd/d10/d34/d4c 0 2024-05-25T06:11:40.779 INFO:tasks.workunit.client.0.smithi160.stdout:5/641: dwrite db/dc/d13/d1d/d31/d56/f61 [0,4194304] 0 2024-05-25T06:11:40.780 INFO:tasks.workunit.client.1.smithi203.stdout:0/782: chown dd/d32/f10d 31886 1 2024-05-25T06:11:40.781 INFO:tasks.workunit.client.1.smithi203.stdout:0/783: readlink dd/d5c/d63/d106/l61 0 2024-05-25T06:11:40.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:40.783+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fce94058390 con 0x7fcea00fed30 2024-05-25T06:11:40.786 INFO:tasks.workunit.client.1.smithi203.stdout:4/989: rename d0/d9/d50/d6b/lda to d0/d9/d9d/d13a/l14c 0 2024-05-25T06:11:40.787 INFO:tasks.workunit.client.1.smithi203.stdout:0/784: unlink dd/d5c/f68 0 2024-05-25T06:11:40.788 INFO:tasks.workunit.client.0.smithi160.stdout:5/642: dwrite db/dc/d13/d1d/d23/d24/d2e/fb7 [0,4194304] 0 2024-05-25T06:11:40.790 INFO:tasks.workunit.client.0.smithi160.stdout:5/643: readlink db/l39 0 2024-05-25T06:11:40.791 INFO:tasks.workunit.client.0.smithi160.stdout:5/644: chown db/dc/d13/d1d/d23/cb0 9246 1 2024-05-25T06:11:40.793 INFO:tasks.workunit.client.1.smithi203.stdout:7/829: dwrite d1/d9/d2c/d36/d54/fc3 [0,4194304] 0 2024-05-25T06:11:40.794 INFO:tasks.workunit.client.0.smithi160.stdout:5/645: mkdir db/dc/d13/d1d/d3f/d72/dc5 0 2024-05-25T06:11:40.806 INFO:tasks.workunit.client.0.smithi160.stdout:5/646: dread db/dc/d13/d1d/d23/d24/d2e/f32 [4194304,4194304] 0 2024-05-25T06:11:40.827 INFO:tasks.workunit.client.0.smithi160.stdout:5/647: truncate db/dc/d13/d1d/d3f/d72/d98/da5/fac 1881108 0 2024-05-25T06:11:40.829 INFO:tasks.workunit.client.1.smithi203.stdout:7/830: dread d1/d11/d3d/d4c/f93 [0,4194304] 0 2024-05-25T06:11:40.829 INFO:tasks.workunit.client.0.smithi160.stdout:5/648: mkdir db/dc/d13/d1d/d31/dc6 0 2024-05-25T06:11:40.832 INFO:tasks.workunit.client.0.smithi160.stdout:5/649: dread db/dc/d13/d1d/d23/d24/d2e/fb7 [0,4194304] 0 2024-05-25T06:11:40.835 INFO:tasks.workunit.client.0.smithi160.stdout:5/650: mknod db/cc7 0 2024-05-25T06:11:40.836 INFO:tasks.workunit.client.0.smithi160.stdout:5/651: dread - db/dc/d13/d1d/d3f/d72/d98/fa0 zero size 2024-05-25T06:11:40.837 INFO:tasks.workunit.client.0.smithi160.stdout:5/652: dread - db/dc/d13/d1d/d23/d24/f77 zero size 2024-05-25T06:11:40.839 INFO:tasks.workunit.client.0.smithi160.stdout:5/653: mknod db/dc/d13/d1d/d3f/d72/cc8 0 2024-05-25T06:11:40.842 INFO:tasks.workunit.client.0.smithi160.stdout:5/654: link db/dc/d13/d1d/d23/d24/d2e/f4e db/dc/d13/d1d/fc9 0 2024-05-25T06:11:40.856 INFO:tasks.workunit.client.0.smithi160.stdout:5/655: dread db/dc/d13/d1d/fc9 [0,4194304] 0 2024-05-25T06:11:40.859 INFO:tasks.workunit.client.0.smithi160.stdout:5/656: rmdir db/dc/d13/d1d/d3f/d72/d9c 0 2024-05-25T06:11:40.861 INFO:tasks.workunit.client.1.smithi203.stdout:0/785: mkdir dd/d5c/d63/d106/d75/d117 0 2024-05-25T06:11:40.862 INFO:tasks.workunit.client.1.smithi203.stdout:1/772: link d14/d57/d6b/da1/de2/l86 d14/d57/d6b/l10c 0 2024-05-25T06:11:40.863 INFO:tasks.workunit.client.1.smithi203.stdout:1/773: chown d14/d57/d1d/l8b 2742757 1 2024-05-25T06:11:40.865 INFO:tasks.workunit.client.1.smithi203.stdout:0/786: symlink dd/d5c/d63/d106/d75/d93/l118 0 2024-05-25T06:11:40.867 INFO:tasks.workunit.client.0.smithi160.stdout:5/657: write db/f5d [4024147,105757] 0 2024-05-25T06:11:40.867 INFO:tasks.workunit.client.0.smithi160.stdout:4/516: sync 2024-05-25T06:11:40.867 INFO:tasks.workunit.client.0.smithi160.stdout:0/631: sync 2024-05-25T06:11:40.868 INFO:tasks.workunit.client.1.smithi203.stdout:1/774: creat d14/d57/d7e/f10d x:0 0 0 2024-05-25T06:11:40.870 INFO:tasks.workunit.client.0.smithi160.stdout:4/517: creat d4/d1c/f9f x:0 0 0 2024-05-25T06:11:40.871 INFO:tasks.workunit.client.1.smithi203.stdout:7/831: link d1/d9/d2c/d3a/f3e d1/d11/d20/de4/f117 0 2024-05-25T06:11:40.872 INFO:tasks.workunit.client.1.smithi203.stdout:4/990: write d0/d8/d3a/f77 [17313,51522] 0 2024-05-25T06:11:40.872 INFO:tasks.workunit.client.0.smithi160.stdout:5/658: symlink db/dc/d13/d1d/d31/d56/lca 0 2024-05-25T06:11:40.873 INFO:tasks.workunit.client.1.smithi203.stdout:0/787: fsync dd/d5c/d63/d106/d4f/f8a 0 2024-05-25T06:11:40.874 INFO:tasks.workunit.client.1.smithi203.stdout:1/775: stat d14/d57/d97/da4/cc3 0 2024-05-25T06:11:40.875 INFO:tasks.workunit.client.0.smithi160.stdout:0/632: write d2b/d46/d63/da7/f8c [1070748,54572] 0 2024-05-25T06:11:40.876 INFO:tasks.workunit.client.0.smithi160.stdout:4/518: link d4/d6/l16 d4/d6/d35/d3d/d55/la0 0 2024-05-25T06:11:40.877 INFO:tasks.workunit.client.1.smithi203.stdout:4/991: creat d0/d139/f14d x:0 0 0 2024-05-25T06:11:40.878 INFO:tasks.workunit.client.0.smithi160.stdout:0/633: rmdir d2b/d46/d63/db3 39 2024-05-25T06:11:40.878 INFO:tasks.workunit.client.0.smithi160.stdout:0/634: stat c9 0 2024-05-25T06:11:40.880 INFO:tasks.workunit.client.1.smithi203.stdout:1/776: mknod d14/d57/c10e 0 2024-05-25T06:11:40.880 INFO:tasks.workunit.client.0.smithi160.stdout:5/659: rename db/f85 to db/d47/fcb 0 2024-05-25T06:11:40.881 INFO:tasks.workunit.client.0.smithi160.stdout:4/519: mkdir d4/d6/d5b/d66/d78/da1 0 2024-05-25T06:11:40.882 INFO:tasks.workunit.client.1.smithi203.stdout:1/777: unlink d14/d7c/d9d/c102 0 2024-05-25T06:11:40.885 INFO:tasks.workunit.client.1.smithi203.stdout:2/903: sync 2024-05-25T06:11:40.885 INFO:tasks.workunit.client.0.smithi160.stdout:5/660: creat db/dc/d13/d1d/d23/dbd/fcc x:0 0 0 2024-05-25T06:11:40.887 INFO:tasks.workunit.client.0.smithi160.stdout:0/635: rename d2b/d46/d63/fb7 to d2b/d31/d35/fc9 0 2024-05-25T06:11:40.888 INFO:tasks.workunit.client.0.smithi160.stdout:4/520: dwrite d4/d1a/f50 [0,4194304] 0 2024-05-25T06:11:40.889 INFO:tasks.workunit.client.1.smithi203.stdout:7/832: dwrite d1/d11/d3d/d4c/f93 [4194304,4194304] 0 2024-05-25T06:11:40.890 INFO:tasks.workunit.client.1.smithi203.stdout:0/788: dread f5 [0,4194304] 0 2024-05-25T06:11:40.891 INFO:tasks.workunit.client.0.smithi160.stdout:0/636: creat d2b/d46/d63/da7/d67/fca x:0 0 0 2024-05-25T06:11:40.894 INFO:tasks.workunit.client.0.smithi160.stdout:0/637: chown d2b/d31/d6d 1 1 2024-05-25T06:11:40.894 INFO:tasks.workunit.client.1.smithi203.stdout:1/778: dwrite d14/d57/d7e/fd0 [0,4194304] 0 2024-05-25T06:11:40.899 INFO:tasks.workunit.client.0.smithi160.stdout:0/638: mkdir d2b/d46/dcb 0 2024-05-25T06:11:40.901 INFO:tasks.workunit.client.0.smithi160.stdout:4/521: dwrite d4/f76 [0,4194304] 0 2024-05-25T06:11:40.901 INFO:tasks.workunit.client.0.smithi160.stdout:5/661: dwrite db/dc/d13/d1d/d31/d56/f82 [0,4194304] 0 2024-05-25T06:11:40.902 INFO:tasks.workunit.client.1.smithi203.stdout:2/904: dwrite d2/d1a/d89/daa/def/d107/d113/f124 [0,4194304] 0 2024-05-25T06:11:40.905 INFO:tasks.workunit.client.0.smithi160.stdout:5/662: chown db/dc/d13/d1d/d31/d56/lca 666761 1 2024-05-25T06:11:40.905 INFO:tasks.workunit.client.0.smithi160.stdout:4/522: chown d4/d1a/c1b 15509 1 2024-05-25T06:11:40.906 INFO:tasks.workunit.client.1.smithi203.stdout:4/992: dwrite d0/d9/d11/f66 [0,4194304] 0 2024-05-25T06:11:40.908 INFO:tasks.workunit.client.0.smithi160.stdout:5/663: write db/dc/d13/d1d/d3f/d72/f7f [881395,106562] 0 2024-05-25T06:11:40.909 INFO:tasks.workunit.client.0.smithi160.stdout:4/523: fsync d4/d1c/d54/f65 0 2024-05-25T06:11:40.911 INFO:tasks.workunit.client.1.smithi203.stdout:0/789: dread dd/d5c/f98 [0,4194304] 0 2024-05-25T06:11:40.912 INFO:tasks.workunit.client.0.smithi160.stdout:0/639: mknod d2b/d46/d63/db3/ccc 0 2024-05-25T06:11:40.914 INFO:tasks.workunit.client.0.smithi160.stdout:4/524: creat d4/d1a/fa2 x:0 0 0 2024-05-25T06:11:40.915 INFO:tasks.workunit.client.1.smithi203.stdout:2/905: creat d2/d1a/d33/d12c/f134 x:0 0 0 2024-05-25T06:11:40.916 INFO:tasks.workunit.client.1.smithi203.stdout:4/993: mknod d0/d8/d1f/d26/d3b/c14e 0 2024-05-25T06:11:40.916 INFO:tasks.workunit.client.1.smithi203.stdout:7/833: dread d1/d11/d20/f27 [0,4194304] 0 2024-05-25T06:11:40.918 INFO:tasks.workunit.client.1.smithi203.stdout:4/994: dread - d0/d8/d3a/db6/f117 zero size 2024-05-25T06:11:40.918 INFO:tasks.workunit.client.1.smithi203.stdout:0/790: write dd/d5c/d63/d106/d4f/f87 [1458695,98805] 0 2024-05-25T06:11:40.919 INFO:tasks.workunit.client.0.smithi160.stdout:0/640: mkdir d2b/d46/d63/da7/da6/dcd 0 2024-05-25T06:11:40.923 INFO:tasks.workunit.client.1.smithi203.stdout:2/906: symlink d2/d1a/d33/l135 0 2024-05-25T06:11:40.925 INFO:tasks.workunit.client.0.smithi160.stdout:0/641: creat d2b/d46/d63/da7/da6/fce x:0 0 0 2024-05-25T06:11:40.926 INFO:tasks.workunit.client.0.smithi160.stdout:5/664: dread db/dc/d13/d1d/d31/d3b/d69/f79 [0,4194304] 0 2024-05-25T06:11:40.926 INFO:tasks.workunit.client.0.smithi160.stdout:4/525: link d4/d6/l16 d4/d6/d5b/d66/d78/la3 0 2024-05-25T06:11:40.927 INFO:tasks.workunit.client.1.smithi203.stdout:0/791: mknod dd/d5c/d63/d106/d4f/db7/c119 0 2024-05-25T06:11:40.928 INFO:tasks.workunit.client.0.smithi160.stdout:4/526: stat d4/d6/d21/d42 0 2024-05-25T06:11:40.929 INFO:tasks.workunit.client.1.smithi203.stdout:2/907: creat d2/d38/d43/d60/f136 x:0 0 0 2024-05-25T06:11:40.929 INFO:tasks.workunit.client.1.smithi203.stdout:7/834: mkdir d1/d9/d2c/d118 0 2024-05-25T06:11:40.930 INFO:tasks.workunit.client.1.smithi203.stdout:1/779: dwrite d14/d57/d6b/f4f [0,4194304] 0 2024-05-25T06:11:40.931 INFO:tasks.workunit.client.1.smithi203.stdout:4/995: link d0/d9/f17 d0/d9/d50/f14f 0 2024-05-25T06:11:40.932 INFO:tasks.workunit.client.0.smithi160.stdout:4/527: creat d4/d6/d21/d42/fa4 x:0 0 0 2024-05-25T06:11:40.935 INFO:tasks.workunit.client.1.smithi203.stdout:7/835: dread - d1/dbd/dc1/fd8 zero size 2024-05-25T06:11:40.935 INFO:tasks.workunit.client.1.smithi203.stdout:1/780: mkdir d14/d57/d1d/d10f 0 2024-05-25T06:11:40.937 INFO:tasks.workunit.client.0.smithi160.stdout:4/528: unlink d4/d63/d64/d88/c8e 0 2024-05-25T06:11:40.937 INFO:tasks.workunit.client.1.smithi203.stdout:0/792: getdents dd/d10/d34/d4c/d99/d70 0 2024-05-25T06:11:40.938 INFO:tasks.workunit.client.1.smithi203.stdout:7/836: creat d1/d11/d20/f119 x:0 0 0 2024-05-25T06:11:40.938 INFO:tasks.workunit.client.0.smithi160.stdout:5/665: dread db/f57 [0,4194304] 0 2024-05-25T06:11:40.939 INFO:tasks.workunit.client.1.smithi203.stdout:1/781: symlink d14/d57/dbb/df8/l110 0 2024-05-25T06:11:40.940 INFO:tasks.workunit.client.0.smithi160.stdout:4/529: creat d4/d63/d64/d88/fa5 x:0 0 0 2024-05-25T06:11:40.940 INFO:tasks.workunit.client.1.smithi203.stdout:4/996: truncate d0/d8/d3a/d81/d93/d98/d99/fd7 195369 0 2024-05-25T06:11:40.942 INFO:tasks.workunit.client.0.smithi160.stdout:4/530: truncate d4/d63/d64/d88/fa5 171161 0 2024-05-25T06:11:40.942 INFO:tasks.workunit.client.1.smithi203.stdout:7/837: mknod d1/d9/d2c/d62/db3/c11a 0 2024-05-25T06:11:40.944 INFO:tasks.workunit.client.1.smithi203.stdout:4/997: read d0/d8/d52/d59/f120 [2759231,43554] 0 2024-05-25T06:11:40.945 INFO:tasks.workunit.client.0.smithi160.stdout:5/666: read db/dc/d13/d1d/d23/d45/f8b [2598850,12072] 0 2024-05-25T06:11:40.945 INFO:tasks.workunit.client.0.smithi160.stdout:0/642: dread d2b/d31/d35/d36/d37/f4d [0,4194304] 0 2024-05-25T06:11:40.946 INFO:tasks.workunit.client.1.smithi203.stdout:7/838: symlink d1/d9/d2c/d62/db3/de5/l11b 0 2024-05-25T06:11:40.947 INFO:tasks.workunit.client.0.smithi160.stdout:4/531: rename d4/d6/l68 to d4/d6/d28/la6 0 2024-05-25T06:11:40.947 INFO:tasks.workunit.client.1.smithi203.stdout:1/782: dwrite d14/d57/db7/dd2/ddc/ffc [0,4194304] 0 2024-05-25T06:11:40.948 INFO:tasks.workunit.client.0.smithi160.stdout:4/532: chown d4/d6/d35/d3d/d55/f79 27 1 2024-05-25T06:11:40.949 INFO:tasks.workunit.client.0.smithi160.stdout:0/643: creat d2b/d46/d63/da7/d67/d6a/fcf x:0 0 0 2024-05-25T06:11:40.950 INFO:tasks.workunit.client.1.smithi203.stdout:4/998: symlink d0/d9/d50/l150 0 2024-05-25T06:11:40.951 INFO:tasks.workunit.client.0.smithi160.stdout:4/533: creat d4/d1a/fa7 x:0 0 0 2024-05-25T06:11:40.952 INFO:tasks.workunit.client.1.smithi203.stdout:1/783: creat d14/d23/f111 x:0 0 0 2024-05-25T06:11:40.953 INFO:tasks.workunit.client.0.smithi160.stdout:0/644: getdents d2b/d46/d63/da7/db5 0 2024-05-25T06:11:40.954 INFO:tasks.workunit.client.0.smithi160.stdout:5/667: dread db/d47/d5e/f99 [0,4194304] 0 2024-05-25T06:11:40.956 INFO:tasks.workunit.client.1.smithi203.stdout:0/793: dread dd/d5c/d63/d106/df3/f3b [0,4194304] 0 2024-05-25T06:11:40.957 INFO:tasks.workunit.client.0.smithi160.stdout:5/668: read db/dc/d13/d1d/d31/d3b/f2b [304158,26838] 0 2024-05-25T06:11:40.958 INFO:tasks.workunit.client.1.smithi203.stdout:0/794: chown dd/d5c/d63/d106/d4f/f87 1 1 2024-05-25T06:11:40.959 INFO:tasks.workunit.client.1.smithi203.stdout:1/784: dwrite d14/d7c/d9d/dc8/f108 [0,4194304] 0 2024-05-25T06:11:40.959 INFO:tasks.workunit.client.0.smithi160.stdout:4/534: link d4/d6/d35/f53 d4/d6/d5b/d66/d82/fa8 0 2024-05-25T06:11:40.960 INFO:tasks.workunit.client.1.smithi203.stdout:0/795: write dd/d32/d108/f110 [494975,56643] 0 2024-05-25T06:11:40.962 INFO:tasks.workunit.client.1.smithi203.stdout:1/785: chown d14/d7c/d9d/dc8/f108 7820 1 2024-05-25T06:11:40.962 INFO:tasks.workunit.client.1.smithi203.stdout:1/786: readlink d14/d7c/lf1 0 2024-05-25T06:11:40.964 INFO:tasks.workunit.client.0.smithi160.stdout:5/669: creat db/dc/d13/d1d/d3f/d72/d98/da5/db5/fcd x:0 0 0 2024-05-25T06:11:40.965 INFO:tasks.workunit.client.0.smithi160.stdout:4/535: unlink d4/c2d 0 2024-05-25T06:11:40.966 INFO:tasks.workunit.client.0.smithi160.stdout:5/670: write db/dc/d13/d1d/d23/f59 [192876,65667] 0 2024-05-25T06:11:40.967 INFO:tasks.workunit.client.1.smithi203.stdout:0/796: symlink dd/d10/d34/l11a 0 2024-05-25T06:11:40.968 INFO:tasks.workunit.client.1.smithi203.stdout:1/787: mknod d14/d81/c112 0 2024-05-25T06:11:40.969 INFO:tasks.workunit.client.1.smithi203.stdout:1/788: fdatasync d14/d57/d6b/f8e 0 2024-05-25T06:11:40.970 INFO:tasks.workunit.client.1.smithi203.stdout:2/908: write d2/d38/d2f/f61 [144752,47685] 0 2024-05-25T06:11:40.974 INFO:tasks.workunit.client.1.smithi203.stdout:0/797: rename dd/d5c/d63/d106/df3/cf6 to dd/d5c/d63/d106/df3/c11b 0 2024-05-25T06:11:40.974 INFO:tasks.workunit.client.1.smithi203.stdout:1/789: dread d14/dff/db8/dc2/f9a [0,4194304] 0 2024-05-25T06:11:40.977 INFO:tasks.workunit.client.0.smithi160.stdout:5/671: dwrite db/dc/d13/d1d/d23/dbd/fcc [0,4194304] 0 2024-05-25T06:11:40.978 INFO:tasks.workunit.client.1.smithi203.stdout:0/798: chown dd/d10/d34/d4c/d5d 23741383 1 2024-05-25T06:11:40.980 INFO:tasks.workunit.client.1.smithi203.stdout:1/790: dread d14/d57/db7/dd2/f62 [0,4194304] 0 2024-05-25T06:11:40.983 INFO:tasks.workunit.client.1.smithi203.stdout:4/999: rmdir d0 39 2024-05-25T06:11:40.984 INFO:tasks.workunit.client.1.smithi203.stdout:0/799: dwrite dd/d5c/d63/d106/da0/db9/df9/d9d/fef [0,4194304] 0 2024-05-25T06:11:40.988 INFO:tasks.workunit.client.1.smithi203.stdout:1/791: getdents d14/d57/d6b/da1/de2 0 2024-05-25T06:11:40.989 INFO:tasks.workunit.client.1.smithi203.stdout:2/909: truncate d2/d38/d2f/ff0 3234454 0 2024-05-25T06:11:40.990 INFO:tasks.workunit.client.1.smithi203.stdout:2/910: chown d2/d38/d2f/f31 10577 1 2024-05-25T06:11:40.990 INFO:tasks.workunit.client.1.smithi203.stdout:0/800: chown dd/d5c/d63/d106/df3/c11b 4148286 1 2024-05-25T06:11:40.993 INFO:tasks.workunit.client.0.smithi160.stdout:4/536: unlink d4/d6/d28/la6 0 2024-05-25T06:11:40.994 INFO:tasks.workunit.client.0.smithi160.stdout:0/645: rmdir d2b/d46/d63 39 2024-05-25T06:11:40.996 INFO:tasks.workunit.client.0.smithi160.stdout:4/537: creat d4/d6/d35/d3d/d55/fa9 x:0 0 0 2024-05-25T06:11:40.996 INFO:tasks.workunit.client.0.smithi160.stdout:0/646: fsync d2b/d31/fc1 0 2024-05-25T06:11:40.997 INFO:tasks.workunit.client.0.smithi160.stdout:4/538: truncate d4/d6/d21/f77 798242 0 2024-05-25T06:11:40.999 INFO:tasks.workunit.client.0.smithi160.stdout:5/672: write db/dc/d13/d1d/d3f/d72/d98/da5/fac [137519,54575] 0 2024-05-25T06:11:41.001 INFO:tasks.workunit.client.0.smithi160.stdout:0/647: creat d2b/d46/d63/da7/d67/db8/fd0 x:0 0 0 2024-05-25T06:11:41.001 INFO:tasks.workunit.client.0.smithi160.stdout:0/648: stat c9 0 2024-05-25T06:11:41.002 INFO:tasks.workunit.client.0.smithi160.stdout:4/539: creat d4/d6/faa x:0 0 0 2024-05-25T06:11:41.002 INFO:tasks.workunit.client.1.smithi203.stdout:2/911: write d2/d1a/f27 [2544151,111289] 0 2024-05-25T06:11:41.002 INFO:tasks.workunit.client.0.smithi160.stdout:0/649: write d2b/f2e [1177910,1638] 0 2024-05-25T06:11:41.007 INFO:tasks.workunit.client.1.smithi203.stdout:0/801: dwrite dd/f5a [4194304,4194304] 0 2024-05-25T06:11:41.007 INFO:tasks.workunit.client.1.smithi203.stdout:2/912: symlink d2/d38/d43/l137 0 2024-05-25T06:11:41.008 INFO:tasks.workunit.client.1.smithi203.stdout:0/802: chown dd/d10/d34/d4c/d99/d70/fb0 469 1 2024-05-25T06:11:41.012 INFO:tasks.workunit.client.0.smithi160.stdout:4/540: dwrite d4/d6/d21/f3f [0,4194304] 0 2024-05-25T06:11:41.013 INFO:tasks.workunit.client.0.smithi160.stdout:0/650: dwrite d2b/d31/fc1 [0,4194304] 0 2024-05-25T06:11:41.018 INFO:tasks.workunit.client.1.smithi203.stdout:2/913: creat d2/d1a/dd3/d100/d10/f138 x:0 0 0 2024-05-25T06:11:41.019 INFO:tasks.workunit.client.1.smithi203.stdout:2/914: read d2/d1a/dd3/d100/d10/d32/f1f [844570,7505] 0 2024-05-25T06:11:41.024 INFO:tasks.workunit.client.1.smithi203.stdout:2/915: creat d2/d1a/dd3/d100/d10/dc8/f139 x:0 0 0 2024-05-25T06:11:41.026 INFO:tasks.workunit.client.0.smithi160.stdout:5/673: dread db/dc/d13/d1d/d23/d24/f3e [0,4194304] 0 2024-05-25T06:11:41.028 INFO:tasks.workunit.client.1.smithi203.stdout:2/916: stat d2/d1a/dd3/d100/d10/d32/l12d 0 2024-05-25T06:11:41.029 INFO:tasks.workunit.client.0.smithi160.stdout:6/541: sync 2024-05-25T06:11:41.030 INFO:tasks.workunit.client.1.smithi203.stdout:2/917: write d2/d38/d2f/f10a [4880383,29266] 0 2024-05-25T06:11:41.031 INFO:tasks.workunit.client.0.smithi160.stdout:5/674: link db/d47/fcb db/fce 0 2024-05-25T06:11:41.033 INFO:tasks.workunit.client.1.smithi203.stdout:0/803: dwrite dd/d5c/d63/d106/d4f/d62/d78/ff8 [4194304,4194304] 0 2024-05-25T06:11:41.034 INFO:tasks.workunit.client.0.smithi160.stdout:5/675: creat db/dc/d13/d1d/d3f/d72/d98/da5/fcf x:0 0 0 2024-05-25T06:11:41.038 INFO:tasks.workunit.client.1.smithi203.stdout:0/804: mknod dd/d5c/d63/d106/df3/d10c/c11c 0 2024-05-25T06:11:41.040 INFO:tasks.workunit.client.1.smithi203.stdout:0/805: chown dd/d5c/d63/d106/da0/db9/df9/d86/fbf 67 1 2024-05-25T06:11:41.043 INFO:tasks.workunit.client.1.smithi203.stdout:0/806: creat dd/d10/d34/d4c/f11d x:0 0 0 2024-05-25T06:11:41.043 INFO:tasks.workunit.client.1.smithi203.stdout:0/807: dread - dd/d10/d34/d4c/fd0 zero size 2024-05-25T06:11:41.044 INFO:tasks.workunit.client.0.smithi160.stdout:4/541: dread d4/d1c/d54/f65 [0,4194304] 0 2024-05-25T06:11:41.046 INFO:tasks.workunit.client.1.smithi203.stdout:0/808: truncate dd/d10/d34/d4c/d99/d70/fb0 2959357 0 2024-05-25T06:11:41.046 INFO:tasks.workunit.client.0.smithi160.stdout:4/542: mkdir d4/d6/d5b/d66/d78/da1/dab 0 2024-05-25T06:11:41.047 INFO:tasks.workunit.client.0.smithi160.stdout:4/543: chown d4/d6/d35/d3d/d55/f56 2 1 2024-05-25T06:11:41.051 INFO:tasks.workunit.client.1.smithi203.stdout:0/809: dwrite dd/d5c/d63/d106/da0/db9/df9/d9d/fef [0,4194304] 0 2024-05-25T06:11:41.052 INFO:tasks.workunit.client.0.smithi160.stdout:4/544: dwrite d4/d1a/fa2 [0,4194304] 0 2024-05-25T06:11:41.053 INFO:tasks.workunit.client.1.smithi203.stdout:0/810: rmdir dd/d5c/d63/d106/da0 39 2024-05-25T06:11:41.055 INFO:tasks.workunit.client.0.smithi160.stdout:4/545: rmdir d4/d6/d5b/d66/d78/da1 39 2024-05-25T06:11:41.056 INFO:tasks.workunit.client.1.smithi203.stdout:0/811: fdatasync dd/d5c/d63/d106/d4f/d62/f74 0 2024-05-25T06:11:41.059 INFO:tasks.workunit.client.0.smithi160.stdout:4/546: rename d4/d1c/d54/c60 to d4/d6/d5b/d66/d82/cac 0 2024-05-25T06:11:41.060 INFO:tasks.workunit.client.0.smithi160.stdout:4/547: dread - d4/d1c/f9f zero size 2024-05-25T06:11:41.060 INFO:tasks.workunit.client.0.smithi160.stdout:4/548: chown d4/d1a/f50 254801 1 2024-05-25T06:11:41.061 INFO:tasks.workunit.client.1.smithi203.stdout:0/812: mknod dd/d5c/d63/d106/d75/d117/c11e 0 2024-05-25T06:11:41.063 INFO:tasks.workunit.client.0.smithi160.stdout:4/549: dread d4/d1a/f8b [0,4194304] 0 2024-05-25T06:11:41.066 INFO:tasks.workunit.client.0.smithi160.stdout:4/550: symlink d4/d6/d21/lad 0 2024-05-25T06:11:41.067 INFO:tasks.workunit.client.0.smithi160.stdout:4/551: write d4/f2a [4796239,67174] 0 2024-05-25T06:11:41.069 INFO:tasks.workunit.client.0.smithi160.stdout:4/552: mkdir d4/d6/d21/d42/dae 0 2024-05-25T06:11:41.070 INFO:tasks.workunit.client.1.smithi203.stdout:2/918: dwrite d2/d38/d2f/fa7 [0,4194304] 0 2024-05-25T06:11:41.070 INFO:tasks.workunit.client.0.smithi160.stdout:4/553: write d4/d1c/f9f [618023,86141] 0 2024-05-25T06:11:41.073 INFO:tasks.workunit.client.1.smithi203.stdout:2/919: symlink d2/d1a/d87/dd9/l13a 0 2024-05-25T06:11:41.077 INFO:tasks.workunit.client.1.smithi203.stdout:8/948: sync 2024-05-25T06:11:41.078 INFO:tasks.workunit.client.0.smithi160.stdout:4/554: dwrite d4/d6/d35/d3d/d55/f56 [4194304,4194304] 0 2024-05-25T06:11:41.080 INFO:tasks.workunit.client.0.smithi160.stdout:4/555: write d4/d6/d21/f27 [2083106,122847] 0 2024-05-25T06:11:41.083 INFO:tasks.workunit.client.0.smithi160.stdout:4/556: mknod d4/d1c/d54/d9b/caf 0 2024-05-25T06:11:41.084 INFO:tasks.workunit.client.1.smithi203.stdout:0/813: write dd/f44 [480376,78016] 0 2024-05-25T06:11:41.088 INFO:tasks.workunit.client.1.smithi203.stdout:0/814: symlink dd/d5c/d63/d106/da0/db9/l11f 0 2024-05-25T06:11:41.090 INFO:tasks.workunit.client.1.smithi203.stdout:0/815: mkdir dd/d5c/d120 0 2024-05-25T06:11:41.091 INFO:tasks.workunit.client.1.smithi203.stdout:0/816: chown dd/d5c/d63/d106/d4f/d62/d78/fdb 3 1 2024-05-25T06:11:41.100 INFO:tasks.workunit.client.1.smithi203.stdout:0/817: dwrite dd/d10/d34/d4c/d5d/fb2 [0,4194304] 0 2024-05-25T06:11:41.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.115+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcea010f960 con 0x7fce8806b790 2024-05-25T06:11:41.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.126+0000 7fce9cff9700 1 -- 172.21.15.160:0/2930498661 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fcea010f960 con 0x7fce8806b790 2024-05-25T06:11:41.126 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:11:41.126 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (7m) 23s ago 8m 24.9M - 0.25.0 c8568f914cd2 86108650b602 2024-05-25T06:11:41.126 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (8m) 23s ago 8m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:11:41.126 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (7m) 11s ago 7m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (7m) 23s ago 8m 84.4M - 9.4.7 954c08fa6188 59fd8b61d321 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (3m) 23s ago 3m 57.5M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (3m) 23s ago 3m 99.2M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (3m) 11s ago 3m 66.4M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (3m) 11s ago 3m 102M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:9283 running (10m) 23s ago 10m 385M - 17.2.7-958-g7b01852c baf1547092e7 fbeba8913b6d 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (56s) 11s ago 7m 490M - 19.0.0-2843-gea0684c0 f9a9809cafd6 fbc025680ca5 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (10m) 23s ago 10m 49.6M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (7m) 11s ago 7m 39.3M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (8m) 23s ago 8m 15.6M - 1.5.0 0da6a335fe13 3a8c0c37653a 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (7m) 11s ago 7m 15.5M - 1.5.0 0da6a335fe13 aec5d8f8dcff 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (6m) 23s ago 6m 112M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (6m) 23s ago 6m 125M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:11:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (5m) 23s ago 5m 104M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:11:41.128 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (5m) 11s ago 5m 132M 5312M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:11:41.128 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (5m) 11s ago 5m 121M 5312M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:11:41.128 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (4m) 11s ago 4m 109M 5312M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:11:41.128 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (7m) 23s ago 8m 59.3M - 2.43.0 a07b618ecd1d 6cc9a4b6650c 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fce8806b790 msgr2=0x7fce8806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fce8806b790 0x7fce8806dc40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b970 tx=0x7fce90009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 msgr2=0x7fcea010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 0x7fcea010df20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b790 tx=0x7fce94009250 comp rx=0 tx=0).stop 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 shutdown_connections 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00fe3e0 0x7fcea010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fce8806b790 0x7fce8806dc40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 --2- 172.21.15.160:0/2930498661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea00fed30 0x7fcea010df20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 >> 172.21.15.160:0/2930498661 conn(0x7fcea00f6b80 msgr2=0x7fcea00f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 shutdown_connections 2024-05-25T06:11:41.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.128+0000 7fcea6469700 1 -- 172.21.15.160:0/2930498661 wait complete. 2024-05-25T06:11:41.164 INFO:tasks.workunit.client.1.smithi203.stdout:0/818: rmdir dd/d5c/d63 39 2024-05-25T06:11:41.165 INFO:tasks.workunit.client.1.smithi203.stdout:0/819: stat dd/d5c/dd5/dde 0 2024-05-25T06:11:41.175 INFO:tasks.workunit.client.1.smithi203.stdout:0/820: dwrite dd/d10/f4a [0,4194304] 0 2024-05-25T06:11:41.183 INFO:tasks.workunit.client.1.smithi203.stdout:0/821: dread dd/d10/d34/d4c/d99/fbe [0,4194304] 0 2024-05-25T06:11:41.188 INFO:tasks.workunit.client.1.smithi203.stdout:0/822: unlink dd/d5c/d63/d106/df3/lc5 0 2024-05-25T06:11:41.194 INFO:tasks.workunit.client.1.smithi203.stdout:0/823: write dd/d5c/d63/d106/df3/fc8 [125226,13747] 0 2024-05-25T06:11:41.201 INFO:tasks.workunit.client.1.smithi203.stdout:0/824: dwrite f9 [0,4194304] 0 2024-05-25T06:11:41.205 INFO:tasks.workunit.client.0.smithi160.stdout:3/486: sync 2024-05-25T06:11:41.205 INFO:tasks.workunit.client.0.smithi160.stdout:7/577: sync 2024-05-25T06:11:41.206 INFO:tasks.workunit.client.0.smithi160.stdout:3/487: read - d19/d2f/d34/f9d zero size 2024-05-25T06:11:41.212 INFO:tasks.workunit.client.1.smithi203.stdout:0/825: dwrite f5 [0,4194304] 0 2024-05-25T06:11:41.216 INFO:tasks.workunit.client.0.smithi160.stdout:7/578: dread d6/d3e/d6f/f55 [0,4194304] 0 2024-05-25T06:11:41.216 INFO:tasks.workunit.client.0.smithi160.stdout:3/488: dwrite d19/d2f/d38/d43/d5b/d65/f89 [4194304,4194304] 0 2024-05-25T06:11:41.217 INFO:tasks.workunit.client.1.smithi203.stdout:0/826: mkdir dd/d5c/d63/d106/d75/d121 0 2024-05-25T06:11:41.217 INFO:tasks.workunit.client.0.smithi160.stdout:7/579: readlink d6/dc/d21/l3b 0 2024-05-25T06:11:41.218 INFO:tasks.workunit.client.0.smithi160.stdout:7/580: read - d6/d32/d92/fab zero size 2024-05-25T06:11:41.220 INFO:tasks.workunit.client.0.smithi160.stdout:3/489: mknod d19/d2f/d38/d60/ca4 0 2024-05-25T06:11:41.222 INFO:tasks.workunit.client.0.smithi160.stdout:3/490: readlink d19/d2f/d38/d43/l63 0 2024-05-25T06:11:41.226 INFO:tasks.workunit.client.0.smithi160.stdout:7/581: dwrite d6/d3e/f70 [0,4194304] 0 2024-05-25T06:11:41.226 INFO:tasks.workunit.client.0.smithi160.stdout:3/491: creat d19/d2f/fa5 x:0 0 0 2024-05-25T06:11:41.228 INFO:tasks.workunit.client.1.smithi203.stdout:3/971: sync 2024-05-25T06:11:41.228 INFO:tasks.workunit.client.1.smithi203.stdout:9/756: sync 2024-05-25T06:11:41.228 INFO:tasks.workunit.client.1.smithi203.stdout:7/839: sync 2024-05-25T06:11:41.228 INFO:tasks.workunit.client.0.smithi160.stdout:7/582: write d6/d32/f99 [720360,103878] 0 2024-05-25T06:11:41.229 INFO:tasks.workunit.client.1.smithi203.stdout:7/840: stat d1/d9/d60 0 2024-05-25T06:11:41.229 INFO:tasks.workunit.client.0.smithi160.stdout:7/583: chown d6/d3e/f9d 718 1 2024-05-25T06:11:41.230 INFO:tasks.workunit.client.0.smithi160.stdout:7/584: write d6/d3e/d6f/f6e [2632995,38659] 0 2024-05-25T06:11:41.233 INFO:tasks.workunit.client.1.smithi203.stdout:9/757: creat d0/df/d37/da8/dd6/ffa x:0 0 0 2024-05-25T06:11:41.235 INFO:tasks.workunit.client.1.smithi203.stdout:9/758: dread - d0/d17/d20/d29/fe4 zero size 2024-05-25T06:11:41.236 INFO:tasks.workunit.client.0.smithi160.stdout:3/492: rename d19/d5d to d19/da6 0 2024-05-25T06:11:41.239 INFO:tasks.workunit.client.0.smithi160.stdout:7/585: dwrite d6/dc/d64/f45 [0,4194304] 0 2024-05-25T06:11:41.239 INFO:tasks.workunit.client.0.smithi160.stdout:7/586: chown d6/d3e/d6f 1 1 2024-05-25T06:11:41.240 INFO:tasks.workunit.client.0.smithi160.stdout:7/587: fdatasync d6/dc/d64/f38 0 2024-05-25T06:11:41.243 INFO:tasks.workunit.client.0.smithi160.stdout:3/493: fdatasync d19/d2f/d38/d43/f4e 0 2024-05-25T06:11:41.244 INFO:tasks.workunit.client.0.smithi160.stdout:7/588: creat d6/d32/d5b/fbb x:0 0 0 2024-05-25T06:11:41.245 INFO:tasks.workunit.client.0.smithi160.stdout:3/494: truncate d19/f99 174552 0 2024-05-25T06:11:41.248 INFO:tasks.workunit.client.1.smithi203.stdout:7/841: write d1/d9/d2c/d62/d7a/d89/f8c [503019,75144] 0 2024-05-25T06:11:41.248 INFO:tasks.workunit.client.0.smithi160.stdout:2/497: sync 2024-05-25T06:11:41.249 INFO:tasks.workunit.client.0.smithi160.stdout:8/617: sync 2024-05-25T06:11:41.249 INFO:tasks.workunit.client.1.smithi203.stdout:9/759: write d0/df/f5f [778138,83160] 0 2024-05-25T06:11:41.253 INFO:tasks.workunit.client.0.smithi160.stdout:3/495: dwrite d19/da6/d6a/f97 [0,4194304] 0 2024-05-25T06:11:41.253 INFO:tasks.workunit.client.0.smithi160.stdout:7/589: link d6/dc/d21/l28 d6/d2b/da0/lbc 0 2024-05-25T06:11:41.254 INFO:tasks.workunit.client.0.smithi160.stdout:2/498: symlink d12/d1c/d2a/d7e/d90/la9 0 2024-05-25T06:11:41.255 INFO:tasks.workunit.client.0.smithi160.stdout:8/618: getdents d5/d9/d72/daf/dbf 0 2024-05-25T06:11:41.257 INFO:tasks.workunit.client.0.smithi160.stdout:3/496: mkdir d19/da7 0 2024-05-25T06:11:41.258 INFO:tasks.workunit.client.1.smithi203.stdout:9/760: truncate d0/df/d37/da8/db8/fd4 218466 0 2024-05-25T06:11:41.258 INFO:tasks.workunit.client.1.smithi203.stdout:3/972: dwrite d10/d16/d23/d45/f76 [0,4194304] 0 2024-05-25T06:11:41.258 INFO:tasks.workunit.client.0.smithi160.stdout:2/499: fsync d12/d1c/d37/f71 0 2024-05-25T06:11:41.258 INFO:tasks.workunit.client.1.smithi203.stdout:0/827: dwrite dd/d5c/d63/d106/d4f/d62/f74 [0,4194304] 0 2024-05-25T06:11:41.259 INFO:tasks.workunit.client.0.smithi160.stdout:7/590: dread d6/d2b/f6b [0,4194304] 0 2024-05-25T06:11:41.263 INFO:tasks.workunit.client.1.smithi203.stdout:7/842: rename d1/d9/d2c/d36/ff5 to d1/d9/d2c/d79/dc7/f11c 0 2024-05-25T06:11:41.264 INFO:tasks.workunit.client.1.smithi203.stdout:3/973: creat d10/d16/d50/d7d/d132/f13a x:0 0 0 2024-05-25T06:11:41.266 INFO:tasks.workunit.client.1.smithi203.stdout:0/828: mkdir dd/d5c/d63/d106/d75/d121/d122 0 2024-05-25T06:11:41.266 INFO:tasks.workunit.client.0.smithi160.stdout:8/619: rename d5/fb to d5/d44/d1f/da3/d28/d4a/d73/fd3 0 2024-05-25T06:11:41.266 INFO:tasks.workunit.client.1.smithi203.stdout:0/829: readlink dd/d10/l2c 0 2024-05-25T06:11:41.269 INFO:tasks.workunit.client.1.smithi203.stdout:3/974: creat d10/d16/d23/d4a/f13b x:0 0 0 2024-05-25T06:11:41.270 INFO:tasks.workunit.client.0.smithi160.stdout:3/497: rename d19/da6/d6a/d9e to d19/d2f/d34/da8 0 2024-05-25T06:11:41.271 INFO:tasks.workunit.client.0.smithi160.stdout:8/620: mkdir d5/d44/d1f/da3/d28/d4a/d4e/d86/dd4 0 2024-05-25T06:11:41.273 INFO:tasks.workunit.client.0.smithi160.stdout:2/500: dread d12/d13/f24 [0,4194304] 0 2024-05-25T06:11:41.274 INFO:tasks.workunit.client.0.smithi160.stdout:8/621: fdatasync d5/d7/d9a/db5/fbd 0 2024-05-25T06:11:41.275 INFO:tasks.workunit.client.0.smithi160.stdout:8/622: fdatasync d5/d44/d1f/da3/fa8 0 2024-05-25T06:11:41.275 INFO:tasks.workunit.client.0.smithi160.stdout:3/498: mknod d19/d2f/d34/da8/ca9 0 2024-05-25T06:11:41.277 INFO:tasks.workunit.client.0.smithi160.stdout:7/591: rename d6/dc/d64/f9f to d6/dc/d64/d5a/fbd 0 2024-05-25T06:11:41.277 INFO:tasks.workunit.client.0.smithi160.stdout:2/501: creat d12/d97/faa x:0 0 0 2024-05-25T06:11:41.278 INFO:tasks.workunit.client.0.smithi160.stdout:7/592: chown d6/lb6 144 1 2024-05-25T06:11:41.278 INFO:tasks.workunit.client.0.smithi160.stdout:8/623: mknod d5/d7/dcb/d6a/cd5 0 2024-05-25T06:11:41.279 INFO:tasks.workunit.client.1.smithi203.stdout:3/975: link d10/f51 d10/dea/f13c 0 2024-05-25T06:11:41.280 INFO:tasks.workunit.client.0.smithi160.stdout:7/593: creat d6/d32/d5b/fbe x:0 0 0 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.280+0000 7f03610bc700 1 -- 172.21.15.160:0/295268931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 msgr2=0x7f035c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.280+0000 7f03610bc700 1 --2- 172.21.15.160:0/295268931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c0fc0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f034c004940 tx=0x7f034c009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.280+0000 7f03610bc700 1 -- 172.21.15.160:0/295268931 shutdown_connections 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.280+0000 7f03610bc700 1 --2- 172.21.15.160:0/295268931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.280+0000 7f03610bc700 1 --2- 172.21.15.160:0/295268931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c0fc0c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.280+0000 7f03610bc700 1 -- 172.21.15.160:0/295268931 >> 172.21.15.160:0/295268931 conn(0x7f035c0f7430 msgr2=0x7f035c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.281+0000 7f03610bc700 1 -- 172.21.15.160:0/295268931 shutdown_connections 2024-05-25T06:11:41.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.281+0000 7f03610bc700 1 -- 172.21.15.160:0/295268931 wait complete. 2024-05-25T06:11:41.281 INFO:tasks.workunit.client.0.smithi160.stdout:8/624: write d5/d44/d1f/da3/fa8 [684867,105785] 0 2024-05-25T06:11:41.282 INFO:tasks.workunit.client.1.smithi203.stdout:3/976: chown d10/d16/c3c 237 1 2024-05-25T06:11:41.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.282+0000 7f03610bc700 1 Processor -- start 2024-05-25T06:11:41.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.282+0000 7f03610bc700 1 -- start start 2024-05-25T06:11:41.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.282+0000 7f03610bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c10c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:41.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.282+0000 7f03610bc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c10cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:41.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.282+0000 7f03610bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c10d3c0 con 0x7f035c0fbcb0 2024-05-25T06:11:41.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.282+0000 7f03610bc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c10dba0 con 0x7f035c0fcf60 2024-05-25T06:11:41.283 INFO:tasks.workunit.client.0.smithi160.stdout:2/502: rename d12/d1c/d37/f4f to d12/d1c/d37/d4b/fab 0 2024-05-25T06:11:41.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.283+0000 7f035a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c10cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.283+0000 7f035a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c10cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34780/0 (socket says 172.21.15.160:34780) 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.283+0000 7f035a59c700 1 -- 172.21.15.160:0/2595891350 learned_addr learned my addr 172.21.15.160:0/2595891350 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.283+0000 7f035ad9d700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c10c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.283+0000 7f035ad9d700 1 -- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 msgr2=0x7f035c10cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f035ad9d700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c10cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f035ad9d700 1 -- 172.21.15.160:0/2595891350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c005040 con 0x7f035c0fbcb0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f035a59c700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f035ad9d700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c10c880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f034c000c00 tx=0x7f034c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f034c00e070 con 0x7f035c0fbcb0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f034c01c400 con 0x7f035c0fbcb0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.284+0000 7f03610bc700 1 -- 172.21.15.160:0/2595891350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c10de20 con 0x7f035c0fbcb0 2024-05-25T06:11:41.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.285+0000 7f03610bc700 1 -- 172.21.15.160:0/2595891350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c10e300 con 0x7f035c0fbcb0 2024-05-25T06:11:41.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.285+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f034c0255e0 con 0x7f035c0fbcb0 2024-05-25T06:11:41.286 INFO:tasks.workunit.client.1.smithi203.stdout:9/761: dread d0/d17/d20/f2c [0,4194304] 0 2024-05-25T06:11:41.286 INFO:tasks.workunit.client.1.smithi203.stdout:9/762: stat d0/d17/d20/d3c/db2/d7a 0 2024-05-25T06:11:41.286 INFO:tasks.workunit.client.0.smithi160.stdout:2/503: read - d12/d1c/d37/d4b/f79 zero size 2024-05-25T06:11:41.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.286+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03400052b0 con 0x7f035c0fbcb0 2024-05-25T06:11:41.288 INFO:tasks.workunit.client.0.smithi160.stdout:7/594: dwrite d6/d2b/d5d/f6a [0,4194304] 0 2024-05-25T06:11:41.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.288+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f034c01ca50 con 0x7f035c0fbcb0 2024-05-25T06:11:41.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.288+0000 7f03537fe700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f034806b7e0 0x7f034806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:41.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.289+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f034c08c1d0 con 0x7f035c0fbcb0 2024-05-25T06:11:41.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.289+0000 7f035a59c700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f034806b7e0 0x7f034806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:41.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.289+0000 7f035a59c700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f034806b7e0 0x7f034806dc90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03440041d0 tx=0x7f034400b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:41.292 INFO:tasks.workunit.client.0.smithi160.stdout:7/595: creat d6/d3e/da1/fbf x:0 0 0 2024-05-25T06:11:41.292 INFO:tasks.workunit.client.1.smithi203.stdout:9/763: read d0/d17/d20/d3c/f43 [2250034,49235] 0 2024-05-25T06:11:41.293 INFO:tasks.workunit.client.0.smithi160.stdout:7/596: truncate d6/d2b/da0/fb7 110695 0 2024-05-25T06:11:41.294 INFO:tasks.workunit.client.1.smithi203.stdout:0/830: write dd/d5c/dd5/fd9 [1171897,256] 0 2024-05-25T06:11:41.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.296+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f034c059840 con 0x7f035c0fbcb0 2024-05-25T06:11:41.297 INFO:tasks.workunit.client.0.smithi160.stdout:7/597: creat d6/d32/d7e/fc0 x:0 0 0 2024-05-25T06:11:41.298 INFO:tasks.workunit.client.1.smithi203.stdout:9/764: unlink d0/df/d37/c82 0 2024-05-25T06:11:41.299 INFO:tasks.workunit.client.1.smithi203.stdout:0/831: creat dd/d5c/d63/d106/d75/d121/f123 x:0 0 0 2024-05-25T06:11:41.300 INFO:tasks.workunit.client.1.smithi203.stdout:7/843: dwrite d1/dbd/dc1/df3/ff4 [4194304,4194304] 0 2024-05-25T06:11:41.301 INFO:tasks.workunit.client.1.smithi203.stdout:9/765: readlink d0/d17/d20/d3c/l62 0 2024-05-25T06:11:41.303 INFO:tasks.workunit.client.0.smithi160.stdout:7/598: rename d6/d2b/d5d/f8d to d6/d2b/d5d/fc1 0 2024-05-25T06:11:41.305 INFO:tasks.workunit.client.1.smithi203.stdout:0/832: dread dd/d5c/d63/d106/df3/fc8 [0,4194304] 0 2024-05-25T06:11:41.309 INFO:tasks.workunit.client.0.smithi160.stdout:7/599: creat d6/d2b/d5d/fc2 x:0 0 0 2024-05-25T06:11:41.309 INFO:tasks.workunit.client.1.smithi203.stdout:0/833: unlink dd/d32/la6 0 2024-05-25T06:11:41.310 INFO:tasks.workunit.client.1.smithi203.stdout:7/844: link d1/d9/f76 d1/d11/d3d/f11d 0 2024-05-25T06:11:41.311 INFO:tasks.workunit.client.1.smithi203.stdout:8/949: sync 2024-05-25T06:11:41.311 INFO:tasks.workunit.client.1.smithi203.stdout:1/792: sync 2024-05-25T06:11:41.312 INFO:tasks.workunit.client.1.smithi203.stdout:2/920: sync 2024-05-25T06:11:41.312 INFO:tasks.workunit.client.0.smithi160.stdout:2/504: rmdir d12/d1c/d37 39 2024-05-25T06:11:41.313 INFO:tasks.workunit.client.1.smithi203.stdout:0/834: creat dd/d5c/d63/d106/d48/f124 x:0 0 0 2024-05-25T06:11:41.314 INFO:tasks.workunit.client.1.smithi203.stdout:7/845: rename d1/d11/d20/lbb to d1/d9/d2c/d118/l11e 0 2024-05-25T06:11:41.315 INFO:tasks.workunit.client.0.smithi160.stdout:7/600: getdents d6/dc 0 2024-05-25T06:11:41.315 INFO:tasks.workunit.client.1.smithi203.stdout:3/977: dread d10/d16/d23/d67/fb7 [0,4194304] 0 2024-05-25T06:11:41.316 INFO:tasks.workunit.client.1.smithi203.stdout:7/846: chown d1/l52 2 1 2024-05-25T06:11:41.317 INFO:tasks.workunit.client.1.smithi203.stdout:1/793: creat d14/d81/d100/f113 x:0 0 0 2024-05-25T06:11:41.317 INFO:tasks.workunit.client.0.smithi160.stdout:2/505: creat d12/d3e/d5f/d74/d7a/fac x:0 0 0 2024-05-25T06:11:41.318 INFO:tasks.workunit.client.1.smithi203.stdout:0/835: truncate dd/d5c/d63/d106/d4f/f87 2048247 0 2024-05-25T06:11:41.318 INFO:tasks.workunit.client.1.smithi203.stdout:2/921: read d2/d1a/dd3/d100/d10/d5f/fc0 [1924778,85271] 0 2024-05-25T06:11:41.319 INFO:tasks.workunit.client.0.smithi160.stdout:2/506: write d12/f7b [1015222,54937] 0 2024-05-25T06:11:41.323 INFO:tasks.workunit.client.0.smithi160.stdout:2/507: symlink d12/d3e/d5f/lad 0 2024-05-25T06:11:41.324 INFO:tasks.workunit.client.1.smithi203.stdout:8/950: write d2/d5/d28/d38/f3c [658594,39700] 0 2024-05-25T06:11:41.325 INFO:tasks.workunit.client.0.smithi160.stdout:2/508: dread d12/d3c/f61 [0,4194304] 0 2024-05-25T06:11:41.325 INFO:tasks.workunit.client.0.smithi160.stdout:2/509: readlink d12/d1c/l23 0 2024-05-25T06:11:41.328 INFO:tasks.workunit.client.0.smithi160.stdout:2/510: dread - d12/d1c/d37/f95 zero size 2024-05-25T06:11:41.330 INFO:tasks.workunit.client.1.smithi203.stdout:9/766: dwrite d0/df/d2d/f72 [0,4194304] 0 2024-05-25T06:11:41.330 INFO:tasks.workunit.client.0.smithi160.stdout:7/601: dwrite d6/d2b/d5d/d89/f90 [0,4194304] 0 2024-05-25T06:11:41.330 INFO:tasks.workunit.client.1.smithi203.stdout:0/836: getdents dd/d5c/d63/d106/d75/d121 0 2024-05-25T06:11:41.333 INFO:tasks.workunit.client.0.smithi160.stdout:2/511: write d12/d3c/f61 [329000,104851] 0 2024-05-25T06:11:41.333 INFO:tasks.workunit.client.1.smithi203.stdout:3/978: dwrite d10/d22/f35 [0,4194304] 0 2024-05-25T06:11:41.335 INFO:tasks.workunit.client.0.smithi160.stdout:7/602: rmdir d6/dc/d64 39 2024-05-25T06:11:41.335 INFO:tasks.workunit.client.0.smithi160.stdout:2/512: truncate d12/d1c/d37/f3b 1870097 0 2024-05-25T06:11:41.337 INFO:tasks.workunit.client.1.smithi203.stdout:1/794: dwrite d14/d57/d97/fa3 [4194304,4194304] 0 2024-05-25T06:11:41.338 INFO:tasks.workunit.client.1.smithi203.stdout:8/951: mknod d2/d5/d28/c146 0 2024-05-25T06:11:41.339 INFO:tasks.workunit.client.1.smithi203.stdout:2/922: truncate d2/d1a/dd3/d100/d10/d32/f1f 1684548 0 2024-05-25T06:11:41.341 INFO:tasks.workunit.client.1.smithi203.stdout:7/847: dread d1/d9/d2c/d3a/d4e/f6c [0,4194304] 0 2024-05-25T06:11:41.341 INFO:tasks.workunit.client.0.smithi160.stdout:7/603: stat d6/dc/d64/d5a/l6d 0 2024-05-25T06:11:41.342 INFO:tasks.workunit.client.0.smithi160.stdout:7/604: read - d6/d3e/d6f/d73/f84 zero size 2024-05-25T06:11:41.345 INFO:tasks.workunit.client.1.smithi203.stdout:1/795: write d14/d57/d6b/da1/de2/ff5 [946139,100232] 0 2024-05-25T06:11:41.347 INFO:tasks.workunit.client.0.smithi160.stdout:7/605: link d6/l7 d6/d32/d7e/lc3 0 2024-05-25T06:11:41.347 INFO:tasks.workunit.client.1.smithi203.stdout:0/837: truncate dd/d10/d34/d4c/fd0 981371 0 2024-05-25T06:11:41.349 INFO:tasks.workunit.client.1.smithi203.stdout:9/767: dwrite d0/df/d37/da8/db8/fd4 [0,4194304] 0 2024-05-25T06:11:41.349 INFO:tasks.workunit.client.1.smithi203.stdout:8/952: chown d2/d5/d18/d31/f34 440 1 2024-05-25T06:11:41.350 INFO:tasks.workunit.client.1.smithi203.stdout:7/848: truncate d1/d11/d20/f119 189149 0 2024-05-25T06:11:41.351 INFO:tasks.workunit.client.1.smithi203.stdout:1/796: fdatasync d14/d57/d6b/f5d 0 2024-05-25T06:11:41.352 INFO:tasks.workunit.client.1.smithi203.stdout:7/849: stat d1/d9/d2c/d3a/d4e/l112 0 2024-05-25T06:11:41.355 INFO:tasks.workunit.client.1.smithi203.stdout:8/953: mknod d2/d5/c147 0 2024-05-25T06:11:41.356 INFO:tasks.workunit.client.1.smithi203.stdout:9/768: chown d0/d17/d20/d29/l4c 1 1 2024-05-25T06:11:41.356 INFO:tasks.workunit.client.0.smithi160.stdout:7/606: dwrite d6/dc/d64/d5a/fa6 [0,4194304] 0 2024-05-25T06:11:41.358 INFO:tasks.workunit.client.1.smithi203.stdout:3/979: creat d10/d16/d23/d4a/f13d x:0 0 0 2024-05-25T06:11:41.363 INFO:tasks.workunit.client.1.smithi203.stdout:0/838: creat dd/d10/d34/d4c/d5d/d115/f125 x:0 0 0 2024-05-25T06:11:41.363 INFO:tasks.workunit.client.1.smithi203.stdout:3/980: read d10/fb5 [2799450,109362] 0 2024-05-25T06:11:41.364 INFO:tasks.workunit.client.1.smithi203.stdout:7/850: unlink d1/d11/d20/f28 0 2024-05-25T06:11:41.368 INFO:tasks.workunit.client.1.smithi203.stdout:7/851: dread - d1/d9/d2c/d36/d3b/faf zero size 2024-05-25T06:11:41.372 INFO:tasks.workunit.client.1.smithi203.stdout:7/852: fdatasync d1/d11/d3d/f11d 0 2024-05-25T06:11:41.372 INFO:tasks.workunit.client.1.smithi203.stdout:0/839: dwrite dd/d5c/d63/d106/da0/fc4 [0,4194304] 0 2024-05-25T06:11:41.375 INFO:tasks.workunit.client.0.smithi160.stdout:2/513: write f8 [644841,7230] 0 2024-05-25T06:11:41.375 INFO:tasks.workunit.client.1.smithi203.stdout:8/954: dread d2/d5/d18/dbf/f11b [0,4194304] 0 2024-05-25T06:11:41.377 INFO:tasks.workunit.client.1.smithi203.stdout:8/955: fdatasync d2/d5/d10/d4a/db6/f126 0 2024-05-25T06:11:41.377 INFO:tasks.workunit.client.0.smithi160.stdout:7/607: rmdir d6/d32/d7e 39 2024-05-25T06:11:41.378 INFO:tasks.workunit.client.0.smithi160.stdout:7/608: chown d6/fb 384 1 2024-05-25T06:11:41.378 INFO:tasks.workunit.client.1.smithi203.stdout:0/840: mkdir dd/d5c/d63/d106/df3/d10c/d126 0 2024-05-25T06:11:41.379 INFO:tasks.workunit.client.0.smithi160.stdout:2/514: mkdir d12/d75/dae 0 2024-05-25T06:11:41.380 INFO:tasks.workunit.client.0.smithi160.stdout:2/515: chown d12/d1c/f6c 1082479 1 2024-05-25T06:11:41.382 INFO:tasks.workunit.client.1.smithi203.stdout:0/841: fdatasync dd/d5c/d63/d106/d4f/d62/d78/ff8 0 2024-05-25T06:11:41.382 INFO:tasks.workunit.client.0.smithi160.stdout:7/609: truncate d6/dc/d21/f97 631324 0 2024-05-25T06:11:41.382 INFO:tasks.workunit.client.1.smithi203.stdout:8/956: dread - d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/f124 zero size 2024-05-25T06:11:41.383 INFO:tasks.workunit.client.1.smithi203.stdout:7/853: dwrite d1/d9/d2c/d3a/d4e/f75 [0,4194304] 0 2024-05-25T06:11:41.384 INFO:tasks.workunit.client.0.smithi160.stdout:2/516: mknod d12/d1c/caf 0 2024-05-25T06:11:41.385 INFO:tasks.workunit.client.1.smithi203.stdout:2/923: truncate d2/d1a/dd3/d100/f2e 1266170 0 2024-05-25T06:11:41.386 INFO:tasks.workunit.client.0.smithi160.stdout:7/610: mkdir d6/dc/d64/d5a/dc4 0 2024-05-25T06:11:41.387 INFO:tasks.workunit.client.1.smithi203.stdout:0/842: write dd/d5c/d63/d106/df3/de2/fe9 [356537,4526] 0 2024-05-25T06:11:41.387 INFO:tasks.workunit.client.1.smithi203.stdout:9/769: write d0/d17/d20/f44 [1287324,870] 0 2024-05-25T06:11:41.388 INFO:tasks.workunit.client.0.smithi160.stdout:2/517: symlink d12/d3e/d5f/lb0 0 2024-05-25T06:11:41.389 INFO:tasks.workunit.client.1.smithi203.stdout:2/924: fdatasync d2/d42/fc3 0 2024-05-25T06:11:41.389 INFO:tasks.workunit.client.1.smithi203.stdout:2/925: stat d2/d38/lf 0 2024-05-25T06:11:41.390 INFO:tasks.workunit.client.0.smithi160.stdout:7/611: creat d6/d3e/d6f/d73/fc5 x:0 0 0 2024-05-25T06:11:41.392 INFO:tasks.workunit.client.0.smithi160.stdout:9/499: sync 2024-05-25T06:11:41.392 INFO:tasks.workunit.client.0.smithi160.stdout:1/457: sync 2024-05-25T06:11:41.392 INFO:tasks.workunit.client.0.smithi160.stdout:0/651: sync 2024-05-25T06:11:41.392 INFO:tasks.workunit.client.0.smithi160.stdout:5/676: sync 2024-05-25T06:11:41.392 INFO:tasks.workunit.client.0.smithi160.stdout:6/542: sync 2024-05-25T06:11:41.392 INFO:tasks.workunit.client.0.smithi160.stdout:4/557: sync 2024-05-25T06:11:41.394 INFO:tasks.workunit.client.0.smithi160.stdout:2/518: getdents d12/d3e/d5a 0 2024-05-25T06:11:41.395 INFO:tasks.workunit.client.1.smithi203.stdout:8/957: creat d2/d5/d28/d38/d4e/d97/ddf/df0/f148 x:0 0 0 2024-05-25T06:11:41.396 INFO:tasks.workunit.client.0.smithi160.stdout:7/612: creat d6/d2b/fc6 x:0 0 0 2024-05-25T06:11:41.396 INFO:tasks.workunit.client.1.smithi203.stdout:7/854: rename d1/f3 to d1/d9/d2c/d118/f11f 0 2024-05-25T06:11:41.397 INFO:tasks.workunit.client.1.smithi203.stdout:9/770: rmdir d0/d17 39 2024-05-25T06:11:41.398 INFO:tasks.workunit.client.1.smithi203.stdout:0/843: dwrite dd/d5c/d63/d106/d4f/d62/f74 [0,4194304] 0 2024-05-25T06:11:41.399 INFO:tasks.workunit.client.0.smithi160.stdout:1/458: dread - de/d21/d49/d1c/d59/d71/f5a zero size 2024-05-25T06:11:41.399 INFO:tasks.workunit.client.1.smithi203.stdout:3/981: dread d10/fb5 [0,4194304] 0 2024-05-25T06:11:41.400 INFO:tasks.workunit.client.0.smithi160.stdout:9/500: unlink d4/d11/d1e/d3e/f96 0 2024-05-25T06:11:41.401 INFO:tasks.workunit.client.0.smithi160.stdout:0/652: fsync d2b/d46/d63/da7/f85 0 2024-05-25T06:11:41.401 INFO:tasks.workunit.client.1.smithi203.stdout:3/982: stat d10/d16/d23/def/f118 0 2024-05-25T06:11:41.402 INFO:tasks.workunit.client.1.smithi203.stdout:2/926: dwrite d2/d1a/d89/fcc [0,4194304] 0 2024-05-25T06:11:41.402 INFO:tasks.workunit.client.0.smithi160.stdout:2/519: mknod d12/d3e/d5a/d8d/cb1 0 2024-05-25T06:11:41.405 INFO:tasks.workunit.client.0.smithi160.stdout:6/543: rmdir d5/d45/d63/da7 0 2024-05-25T06:11:41.406 INFO:tasks.workunit.client.0.smithi160.stdout:4/558: dwrite d4/d1a/fa7 [0,4194304] 0 2024-05-25T06:11:41.407 INFO:tasks.workunit.client.1.smithi203.stdout:0/844: mknod dd/d32/de5/c127 0 2024-05-25T06:11:41.407 INFO:tasks.workunit.client.1.smithi203.stdout:3/983: dread - d10/d16/d23/def/f118 zero size 2024-05-25T06:11:41.409 INFO:tasks.workunit.client.1.smithi203.stdout:7/855: creat d1/d11/d20/d10b/f120 x:0 0 0 2024-05-25T06:11:41.409 INFO:tasks.workunit.client.0.smithi160.stdout:4/559: fsync d4/d6/f97 0 2024-05-25T06:11:41.409 INFO:tasks.workunit.client.0.smithi160.stdout:4/560: chown d4/d6/d35/l41 119218 1 2024-05-25T06:11:41.411 INFO:tasks.workunit.client.0.smithi160.stdout:9/501: creat d4/d11/d1e/d3e/d2f/d79/f9b x:0 0 0 2024-05-25T06:11:41.411 INFO:tasks.workunit.client.0.smithi160.stdout:5/677: creat db/dc/d13/fd0 x:0 0 0 2024-05-25T06:11:41.413 INFO:tasks.workunit.client.0.smithi160.stdout:9/502: fsync d4/d11/d14/f61 0 2024-05-25T06:11:41.416 INFO:tasks.workunit.client.0.smithi160.stdout:1/459: mkdir de/d21/d49/d1c/d41/d96 0 2024-05-25T06:11:41.417 INFO:tasks.workunit.client.0.smithi160.stdout:7/613: read d6/d32/d5b/f96 [594186,1408] 0 2024-05-25T06:11:41.418 INFO:tasks.workunit.client.0.smithi160.stdout:6/544: mkdir d5/d45/d4a/dba 0 2024-05-25T06:11:41.419 INFO:tasks.workunit.client.1.smithi203.stdout:3/984: unlink d10/d16/ce9 0 2024-05-25T06:11:41.421 INFO:tasks.workunit.client.1.smithi203.stdout:7/856: mkdir d1/d9/d2c/d3a/dd1/d121 0 2024-05-25T06:11:41.421 INFO:tasks.workunit.client.1.smithi203.stdout:8/958: symlink d2/d5/d18/d31/dac/dc2/d88/d93/d9d/de7/l149 0 2024-05-25T06:11:41.422 INFO:tasks.workunit.client.1.smithi203.stdout:9/771: dwrite d0/d17/fd7 [0,4194304] 0 2024-05-25T06:11:41.422 INFO:tasks.workunit.client.0.smithi160.stdout:0/653: creat d2b/d46/dcb/fd1 x:0 0 0 2024-05-25T06:11:41.425 INFO:tasks.workunit.client.1.smithi203.stdout:9/772: write d0/d17/d20/d29/ff4 [903474,33996] 0 2024-05-25T06:11:41.428 INFO:tasks.workunit.client.0.smithi160.stdout:2/520: creat d12/d3e/d5f/d74/fb2 x:0 0 0 2024-05-25T06:11:41.431 INFO:tasks.workunit.client.1.smithi203.stdout:3/985: creat d10/d16/d23/d38/d11e/f13e x:0 0 0 2024-05-25T06:11:41.431 INFO:tasks.workunit.client.0.smithi160.stdout:9/503: fsync d4/d11/d1e/d3e/f44 0 2024-05-25T06:11:41.432 INFO:tasks.workunit.client.1.smithi203.stdout:7/857: dread d1/d11/d3d/f6a [0,4194304] 0 2024-05-25T06:11:41.433 INFO:tasks.workunit.client.1.smithi203.stdout:8/959: dwrite d2/d5/d10/d4a/fba [0,4194304] 0 2024-05-25T06:11:41.433 INFO:tasks.workunit.client.1.smithi203.stdout:3/986: chown d10/l11d 151541130 1 2024-05-25T06:11:41.435 INFO:tasks.workunit.client.0.smithi160.stdout:1/460: rename de/f87 to de/d21/d49/d1c/d59/d7c/f97 0 2024-05-25T06:11:41.435 INFO:tasks.workunit.client.0.smithi160.stdout:7/614: mkdir d6/d2b/da0/dc7 0 2024-05-25T06:11:41.436 INFO:tasks.workunit.client.1.smithi203.stdout:9/773: dwrite d0/df/fd1 [0,4194304] 0 2024-05-25T06:11:41.440 INFO:tasks.workunit.client.0.smithi160.stdout:0/654: fsync d2b/d31/d35/d36/d37/f4d 0 2024-05-25T06:11:41.441 INFO:tasks.workunit.client.0.smithi160.stdout:5/678: mknod db/dc/d13/d1d/d31/dc6/cd1 0 2024-05-25T06:11:41.442 INFO:tasks.workunit.client.0.smithi160.stdout:5/679: readlink db/dc/d13/d1d/d23/d24/l89 0 2024-05-25T06:11:41.446 INFO:tasks.workunit.client.0.smithi160.stdout:2/521: rename d12/d1c/d37/c4a to d12/d1c/d2a/cb3 0 2024-05-25T06:11:41.447 INFO:tasks.workunit.client.0.smithi160.stdout:2/522: dread - d12/d3e/d5f/d74/fb2 zero size 2024-05-25T06:11:41.447 INFO:tasks.workunit.client.1.smithi203.stdout:3/987: mknod d10/d16/d23/d67/c13f 0 2024-05-25T06:11:41.447 INFO:tasks.workunit.client.0.smithi160.stdout:2/523: fsync d12/d13/f24 0 2024-05-25T06:11:41.450 INFO:tasks.workunit.client.0.smithi160.stdout:9/504: mkdir d4/d11/d14/d8e/d9c 0 2024-05-25T06:11:41.450 INFO:tasks.workunit.client.0.smithi160.stdout:2/524: dread d12/d13/f15 [0,4194304] 0 2024-05-25T06:11:41.451 INFO:tasks.workunit.client.0.smithi160.stdout:5/680: mknod db/d47/d81/cd2 0 2024-05-25T06:11:41.454 INFO:tasks.workunit.client.0.smithi160.stdout:0/655: truncate d2b/f42 2083334 0 2024-05-25T06:11:41.455 INFO:tasks.workunit.client.0.smithi160.stdout:2/525: read d12/d1c/f1e [3052883,101623] 0 2024-05-25T06:11:41.455 INFO:tasks.workunit.client.1.smithi203.stdout:2/927: truncate d2/d1a/dd3/d100/f74 1233569 0 2024-05-25T06:11:41.456 INFO:tasks.workunit.client.0.smithi160.stdout:0/656: chown d2b/d31/d35/d36/d86/l4f 52508 1 2024-05-25T06:11:41.458 INFO:tasks.workunit.client.1.smithi203.stdout:0/845: write dd/d10/d34/d4c/f7e [639585,99124] 0 2024-05-25T06:11:41.459 INFO:tasks.workunit.client.0.smithi160.stdout:2/526: mkdir d12/d3e/d5a/d8d/db4 0 2024-05-25T06:11:41.460 INFO:tasks.workunit.client.0.smithi160.stdout:9/505: dwrite d4/d11/d12/d58/f64 [0,4194304] 0 2024-05-25T06:11:41.461 INFO:tasks.workunit.client.0.smithi160.stdout:9/506: chown d4/d11/d1e/d3e/d25 43 1 2024-05-25T06:11:41.463 INFO:tasks.workunit.client.0.smithi160.stdout:4/561: write d4/d6/d5b/d66/d82/fa8 [4598781,48459] 0 2024-05-25T06:11:41.464 INFO:tasks.workunit.client.0.smithi160.stdout:5/681: dwrite db/dc/d13/d1d/d23/d24/f3e [0,4194304] 0 2024-05-25T06:11:41.465 INFO:tasks.workunit.client.0.smithi160.stdout:4/562: fdatasync d4/d1a/f87 0 2024-05-25T06:11:41.467 INFO:tasks.workunit.client.0.smithi160.stdout:9/507: creat d4/d11/d1e/d3e/d2f/f9d x:0 0 0 2024-05-25T06:11:41.469 INFO:tasks.workunit.client.0.smithi160.stdout:6/545: truncate d5/de/d32/d35/f5f 1760114 0 2024-05-25T06:11:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:41 smithi160 ceph-mon[29041]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:41.472 INFO:tasks.workunit.client.1.smithi203.stdout:7/858: dwrite d1/d11/f30 [0,4194304] 0 2024-05-25T06:11:41.473 INFO:tasks.workunit.client.0.smithi160.stdout:4/563: readlink d4/d6/d5b/d66/d78/la3 0 2024-05-25T06:11:41.480 INFO:tasks.workunit.client.0.smithi160.stdout:7/615: dwrite d6/d3e/f9d [0,4194304] 0 2024-05-25T06:11:41.491 INFO:tasks.workunit.client.0.smithi160.stdout:7/616: dwrite d6/d32/d5b/fbb [0,4194304] 0 2024-05-25T06:11:41.496 INFO:tasks.workunit.client.0.smithi160.stdout:4/564: write d4/f26 [5417431,30328] 0 2024-05-25T06:11:41.505 INFO:tasks.workunit.client.0.smithi160.stdout:9/508: dwrite d4/d11/d1e/f1f [4194304,4194304] 0 2024-05-25T06:11:41.507 INFO:tasks.workunit.client.0.smithi160.stdout:4/565: dwrite d4/d6/d5b/d66/f71 [0,4194304] 0 2024-05-25T06:11:41.512 INFO:tasks.workunit.client.0.smithi160.stdout:9/509: rmdir d4/d11 39 2024-05-25T06:11:41.518 INFO:tasks.workunit.client.0.smithi160.stdout:4/566: dread d4/d1a/f8b [0,4194304] 0 2024-05-25T06:11:41.523 INFO:tasks.workunit.client.0.smithi160.stdout:6/546: dread d5/d45/d63/f83 [0,4194304] 0 2024-05-25T06:11:41.524 INFO:tasks.workunit.client.0.smithi160.stdout:9/510: dwrite d4/f5c [0,4194304] 0 2024-05-25T06:11:41.524 INFO:tasks.workunit.client.0.smithi160.stdout:4/567: mknod d4/d6/d21/d42/cb0 0 2024-05-25T06:11:41.525 INFO:tasks.workunit.client.0.smithi160.stdout:6/547: chown d5/de/d32/d35 230 1 2024-05-25T06:11:41.525 INFO:tasks.workunit.client.0.smithi160.stdout:4/568: stat d4/d63/d64/d88/fa5 0 2024-05-25T06:11:41.528 INFO:tasks.workunit.client.0.smithi160.stdout:4/569: fsync d4/d6/d5b/d66/d82/fa8 0 2024-05-25T06:11:41.528 INFO:tasks.workunit.client.0.smithi160.stdout:4/570: readlink d4/d6/d5b/d66/d78/la3 0 2024-05-25T06:11:41.530 INFO:tasks.workunit.client.0.smithi160.stdout:6/548: dread d5/f1e [0,4194304] 0 2024-05-25T06:11:41.530 INFO:tasks.workunit.client.0.smithi160.stdout:9/511: creat d4/d11/d12/d2e/d45/f9e x:0 0 0 2024-05-25T06:11:41.531 INFO:tasks.workunit.client.0.smithi160.stdout:6/549: fdatasync d5/de/d32/d9f/fb7 0 2024-05-25T06:11:41.533 INFO:tasks.workunit.client.0.smithi160.stdout:6/550: write d5/de/d32/d9f/fb7 [680494,108861] 0 2024-05-25T06:11:41.533 INFO:tasks.workunit.client.0.smithi160.stdout:3/499: sync 2024-05-25T06:11:41.535 INFO:tasks.workunit.client.0.smithi160.stdout:4/571: unlink d4/d1c/f3e 0 2024-05-25T06:11:41.539 INFO:tasks.workunit.client.0.smithi160.stdout:4/572: symlink d4/d6/d5b/d66/d82/lb1 0 2024-05-25T06:11:41.539 INFO:tasks.workunit.client.0.smithi160.stdout:9/512: dread d4/d11/d1e/d3e/d25/d27/f7d [0,4194304] 0 2024-05-25T06:11:41.542 INFO:tasks.workunit.client.0.smithi160.stdout:4/573: creat d4/d6/d35/fb2 x:0 0 0 2024-05-25T06:11:41.543 INFO:tasks.workunit.client.0.smithi160.stdout:4/574: readlink d4/d6/d35/d3d/l72 0 2024-05-25T06:11:41.546 INFO:tasks.workunit.client.0.smithi160.stdout:9/513: rename d4/d11/d1e/d3e/d25/f8f to d4/d11/d12/f9f 0 2024-05-25T06:11:41.547 INFO:tasks.workunit.client.0.smithi160.stdout:4/575: dread d4/d1a/fa2 [0,4194304] 0 2024-05-25T06:11:41.548 INFO:tasks.workunit.client.0.smithi160.stdout:4/576: stat d4/d1c/d54 0 2024-05-25T06:11:41.549 INFO:tasks.workunit.client.0.smithi160.stdout:4/577: chown d4/d6/f22 2114998 1 2024-05-25T06:11:41.549 INFO:tasks.workunit.client.0.smithi160.stdout:6/551: link d5/d45/d4a/fa5 d5/de/d32/d74/fbb 0 2024-05-25T06:11:41.552 INFO:tasks.workunit.client.0.smithi160.stdout:7/617: dread d6/dc/d21/f54 [4194304,4194304] 0 2024-05-25T06:11:41.552 INFO:tasks.workunit.client.0.smithi160.stdout:9/514: creat d4/d11/d1e/d24/d66/fa0 x:0 0 0 2024-05-25T06:11:41.554 INFO:tasks.workunit.client.0.smithi160.stdout:6/552: dread - d5/de/d58/f5b zero size 2024-05-25T06:11:41.555 INFO:tasks.workunit.client.0.smithi160.stdout:7/618: chown d6/dc/d64/d5a/d7c/l9a 519 1 2024-05-25T06:11:41.557 INFO:tasks.workunit.client.0.smithi160.stdout:7/619: chown d6/d3e/f70 1824 1 2024-05-25T06:11:41.558 INFO:tasks.workunit.client.0.smithi160.stdout:9/515: mknod d4/d11/d1e/d3e/d25/d27/ca1 0 2024-05-25T06:11:41.561 INFO:tasks.workunit.client.0.smithi160.stdout:7/620: creat d6/d3e/d59/fc8 x:0 0 0 2024-05-25T06:11:41.561 INFO:tasks.workunit.client.0.smithi160.stdout:4/578: dwrite d4/d1a/fa2 [4194304,4194304] 0 2024-05-25T06:11:41.563 INFO:tasks.workunit.client.0.smithi160.stdout:7/621: fsync d6/dc/d64/f53 0 2024-05-25T06:11:41.565 INFO:tasks.workunit.client.0.smithi160.stdout:7/622: fsync d6/d2b/d5d/d89/f8a 0 2024-05-25T06:11:41.568 INFO:tasks.workunit.client.0.smithi160.stdout:4/579: rmdir d4/d1c/d54/d9b 39 2024-05-25T06:11:41.571 INFO:tasks.workunit.client.0.smithi160.stdout:6/553: dread d5/d39/f51 [0,4194304] 0 2024-05-25T06:11:41.573 INFO:tasks.workunit.client.0.smithi160.stdout:6/554: write d5/d45/d4a/f76 [8628015,93192] 0 2024-05-25T06:11:41.574 INFO:tasks.workunit.client.0.smithi160.stdout:7/623: dwrite d6/dc/d21/f9b [0,4194304] 0 2024-05-25T06:11:41.576 INFO:tasks.workunit.client.0.smithi160.stdout:6/555: mkdir d5/d45/d4a/dbc 0 2024-05-25T06:11:41.576 INFO:tasks.workunit.client.0.smithi160.stdout:7/624: creat d6/d32/d92/fc9 x:0 0 0 2024-05-25T06:11:41.581 INFO:tasks.workunit.client.0.smithi160.stdout:7/625: creat d6/d3e/fca x:0 0 0 2024-05-25T06:11:41.581 INFO:tasks.workunit.client.0.smithi160.stdout:7/626: chown d6/d2b/d5d/d89/f90 6482 1 2024-05-25T06:11:41.581 INFO:tasks.workunit.client.0.smithi160.stdout:7/627: stat d6/dc/d64/d5a/fa6 0 2024-05-25T06:11:41.582 INFO:tasks.workunit.client.0.smithi160.stdout:6/556: rename d5/de/d32/d74/d77/d89/f8a to d5/d45/d4a/fbd 0 2024-05-25T06:11:41.588 INFO:tasks.workunit.client.0.smithi160.stdout:6/557: rename d5/d45/lb3 to d5/d30/d36/d73/d6e/lbe 0 2024-05-25T06:11:41.589 INFO:tasks.workunit.client.0.smithi160.stdout:9/516: truncate d4/d11/f38 3982388 0 2024-05-25T06:11:41.590 INFO:tasks.workunit.client.0.smithi160.stdout:7/628: mknod d6/d3e/d59/d65/ccb 0 2024-05-25T06:11:41.590 INFO:tasks.workunit.client.0.smithi160.stdout:6/558: fdatasync d5/d45/d4a/fbd 0 2024-05-25T06:11:41.590 INFO:tasks.workunit.client.0.smithi160.stdout:9/517: write d4/d11/d12/f48 [2183333,109003] 0 2024-05-25T06:11:41.592 INFO:tasks.workunit.client.0.smithi160.stdout:7/629: getdents d6/dc/d64/d5a/dc4 0 2024-05-25T06:11:41.593 INFO:tasks.workunit.client.1.smithi203.stdout:1/797: sync 2024-05-25T06:11:41.596 INFO:tasks.workunit.client.1.smithi203.stdout:8/960: symlink d2/d5/d10/dfa/dfb/l14a 0 2024-05-25T06:11:41.598 INFO:tasks.workunit.client.1.smithi203.stdout:8/961: write d2/d5/d10/d45/d5d/f117 [2718554,128898] 0 2024-05-25T06:11:41.599 INFO:tasks.workunit.client.1.smithi203.stdout:2/928: mkdir d2/d1a/db0/d13b 0 2024-05-25T06:11:41.600 INFO:tasks.workunit.client.0.smithi160.stdout:7/630: dwrite d6/d3e/f70 [0,4194304] 0 2024-05-25T06:11:41.600 INFO:tasks.workunit.client.1.smithi203.stdout:8/962: fsync d2/d5/d28/d38/d4e/d97/da1/fdc 0 2024-05-25T06:11:41.601 INFO:tasks.workunit.client.0.smithi160.stdout:7/631: chown d6/lb6 1 1 2024-05-25T06:11:41.603 INFO:tasks.workunit.client.1.smithi203.stdout:0/846: mkdir dd/d5c/d63/d128 0 2024-05-25T06:11:41.604 INFO:tasks.workunit.client.0.smithi160.stdout:9/518: write d4/d11/d1e/d24/d66/f94 [416882,13192] 0 2024-05-25T06:11:41.605 INFO:tasks.workunit.client.0.smithi160.stdout:6/559: truncate d5/d39/f51 3314747 0 2024-05-25T06:11:41.608 INFO:tasks.workunit.client.1.smithi203.stdout:2/929: dwrite d2/d38/d2f/f10a [4194304,4194304] 0 2024-05-25T06:11:41.608 INFO:tasks.workunit.client.0.smithi160.stdout:9/519: mkdir d4/d11/d1e/d3e/d25/d5e/da2 0 2024-05-25T06:11:41.609 INFO:tasks.workunit.client.0.smithi160.stdout:9/520: write d4/d11/d1e/d3e/d25/f8b [674059,84266] 0 2024-05-25T06:11:41.610 INFO:tasks.workunit.client.1.smithi203.stdout:1/798: rename d14/l3d to d14/d57/db7/dd2/dca/dcf/l114 0 2024-05-25T06:11:41.613 INFO:tasks.workunit.client.1.smithi203.stdout:1/799: read d14/d57/d97/da4/fe8 [475527,18739] 0 2024-05-25T06:11:41.614 INFO:tasks.workunit.client.0.smithi160.stdout:6/560: mknod d5/d30/d36/d73/cbf 0 2024-05-25T06:11:41.614 INFO:tasks.workunit.client.0.smithi160.stdout:9/521: fsync d4/d11/f26 0 2024-05-25T06:11:41.614 INFO:tasks.workunit.client.0.smithi160.stdout:7/632: getdents d6/d2b/d8b 0 2024-05-25T06:11:41.617 INFO:tasks.workunit.client.1.smithi203.stdout:9/774: getdents d0/d17 0 2024-05-25T06:11:41.622 INFO:tasks.workunit.client.0.smithi160.stdout:6/561: fsync d5/d45/f8e 0 2024-05-25T06:11:41.623 INFO:tasks.workunit.client.0.smithi160.stdout:7/633: mkdir d6/d2b/dcc 0 2024-05-25T06:11:41.624 INFO:tasks.workunit.client.0.smithi160.stdout:9/522: symlink d4/d11/d14/d8e/d9c/la3 0 2024-05-25T06:11:41.625 INFO:tasks.workunit.client.1.smithi203.stdout:1/800: rename d14/d57/d29 to d14/d23/d115 0 2024-05-25T06:11:41.627 INFO:tasks.workunit.client.0.smithi160.stdout:9/523: symlink d4/d11/d7a/la4 0 2024-05-25T06:11:41.628 INFO:tasks.workunit.client.0.smithi160.stdout:7/634: fdatasync d6/d2b/f5c 0 2024-05-25T06:11:41.631 INFO:tasks.workunit.client.1.smithi203.stdout:9/775: rename d0/d17/d20/d3c/db2/d7a/c9c to d0/d17/d20/d3c/db2/cfb 0 2024-05-25T06:11:41.632 INFO:tasks.workunit.client.0.smithi160.stdout:7/635: symlink d6/d3e/d6f/d73/lcd 0 2024-05-25T06:11:41.634 INFO:tasks.workunit.client.1.smithi203.stdout:2/930: dread d2/d38/d2f/f31 [4194304,4194304] 0 2024-05-25T06:11:41.634 INFO:tasks.workunit.client.0.smithi160.stdout:7/636: fdatasync d6/d32/d5b/fbe 0 2024-05-25T06:11:41.635 INFO:tasks.workunit.client.0.smithi160.stdout:9/524: dwrite d4/f69 [4194304,4194304] 0 2024-05-25T06:11:41.636 INFO:tasks.workunit.client.1.smithi203.stdout:9/776: rename d0/d17/d23/l26 to d0/df/d37/da8/lfc 0 2024-05-25T06:11:41.638 INFO:tasks.workunit.client.0.smithi160.stdout:7/637: mkdir d6/d32/dce 0 2024-05-25T06:11:41.639 INFO:tasks.workunit.client.1.smithi203.stdout:2/931: rename d2/d1a/dd3/d100/d10/d32/dca/de8/f11d to d2/d1a/d33/d80/d133/f13c 0 2024-05-25T06:11:41.640 INFO:tasks.workunit.client.0.smithi160.stdout:7/638: mknod d6/d3e/d6f/d73/ccf 0 2024-05-25T06:11:41.640 INFO:tasks.workunit.client.1.smithi203.stdout:2/932: write d2/d38/f29 [2832234,108982] 0 2024-05-25T06:11:41.641 INFO:tasks.workunit.client.1.smithi203.stdout:9/777: symlink d0/df/d37/da8/lfd 0 2024-05-25T06:11:41.643 INFO:tasks.workunit.client.1.smithi203.stdout:9/778: write d0/df/d2d/d92/fdf [722917,51441] 0 2024-05-25T06:11:41.643 INFO:tasks.workunit.client.0.smithi160.stdout:7/639: creat d6/dc/d64/d5a/d7c/fd0 x:0 0 0 2024-05-25T06:11:41.644 INFO:tasks.workunit.client.0.smithi160.stdout:7/640: stat d6/dc/d64/dae/lb4 0 2024-05-25T06:11:41.645 INFO:tasks.workunit.client.0.smithi160.stdout:9/525: link d4/d11/d1e/d3e/d25/d27/f77 d4/d11/d12/d58/d97/fa5 0 2024-05-25T06:11:41.649 INFO:tasks.workunit.client.1.smithi203.stdout:9/779: rename d0/l22 to d0/d17/dec/lfe 0 2024-05-25T06:11:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:41 smithi203 ceph-mon[36030]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:41.650 INFO:tasks.workunit.client.0.smithi160.stdout:7/641: creat d6/dc/fd1 x:0 0 0 2024-05-25T06:11:41.650 INFO:tasks.workunit.client.0.smithi160.stdout:9/526: dread d4/d11/f4c [0,4194304] 0 2024-05-25T06:11:41.651 INFO:tasks.workunit.client.0.smithi160.stdout:7/642: creat d6/dc/d64/d5a/d7c/fd2 x:0 0 0 2024-05-25T06:11:41.653 INFO:tasks.workunit.client.1.smithi203.stdout:9/780: link d0/d17/f81 d0/df/d37/da8/db8/fff 0 2024-05-25T06:11:41.653 INFO:tasks.workunit.client.0.smithi160.stdout:9/527: getdents d4/d11/d12/d2e 0 2024-05-25T06:11:41.654 INFO:tasks.workunit.client.0.smithi160.stdout:7/643: mknod d6/da8/cd3 0 2024-05-25T06:11:41.655 INFO:tasks.workunit.client.1.smithi203.stdout:9/781: truncate d0/d17/d20/d3c/d5c/fbd 550445 0 2024-05-25T06:11:41.657 INFO:tasks.workunit.client.0.smithi160.stdout:9/528: creat d4/d11/d1e/d3e/d25/d5e/fa6 x:0 0 0 2024-05-25T06:11:41.658 INFO:tasks.workunit.client.0.smithi160.stdout:7/644: rmdir d6/da8 39 2024-05-25T06:11:41.658 INFO:tasks.workunit.client.0.smithi160.stdout:7/645: chown d6/d3e/d6f/d73/l74 0 1 2024-05-25T06:11:41.659 INFO:tasks.workunit.client.0.smithi160.stdout:9/529: mkdir d4/d11/d1e/d3e/d2f/da7 0 2024-05-25T06:11:41.661 INFO:tasks.workunit.client.0.smithi160.stdout:7/646: dread d6/d32/f3f [0,4194304] 0 2024-05-25T06:11:41.671 INFO:tasks.workunit.client.0.smithi160.stdout:9/530: dwrite d4/fa [0,4194304] 0 2024-05-25T06:11:41.674 INFO:tasks.workunit.client.0.smithi160.stdout:7/647: dwrite d6/d3e/fca [0,4194304] 0 2024-05-25T06:11:41.676 INFO:tasks.workunit.client.1.smithi203.stdout:3/988: write d10/d16/d50/d7d/d132/f79 [1784345,32682] 0 2024-05-25T06:11:41.677 INFO:tasks.workunit.client.1.smithi203.stdout:3/989: stat d10/d16/d23/def/ff9 0 2024-05-25T06:11:41.678 INFO:tasks.workunit.client.0.smithi160.stdout:9/531: creat d4/d11/d14/d8e/fa8 x:0 0 0 2024-05-25T06:11:41.685 INFO:tasks.workunit.client.1.smithi203.stdout:3/990: symlink d10/dea/l140 0 2024-05-25T06:11:41.685 INFO:tasks.workunit.client.1.smithi203.stdout:7/859: write d1/fa7 [912000,1831] 0 2024-05-25T06:11:41.686 INFO:tasks.workunit.client.1.smithi203.stdout:7/860: dread - d1/d11/fc0 zero size 2024-05-25T06:11:41.690 INFO:tasks.workunit.client.1.smithi203.stdout:3/991: fsync d10/d16/d23/d4a/fe3 0 2024-05-25T06:11:41.691 INFO:tasks.workunit.client.0.smithi160.stdout:9/532: getdents d4/d11/d1e/d3e/d2f 0 2024-05-25T06:11:41.693 INFO:tasks.workunit.client.1.smithi203.stdout:8/963: write d2/d5/d18/d31/dac/dc2/d88/fea [431106,38477] 0 2024-05-25T06:11:41.695 INFO:tasks.workunit.client.1.smithi203.stdout:3/992: truncate d10/d16/d23/d38/fe2 712176 0 2024-05-25T06:11:41.696 INFO:tasks.workunit.client.1.smithi203.stdout:3/993: stat d10/d16/d23/d45/fdc 0 2024-05-25T06:11:41.697 INFO:tasks.workunit.client.1.smithi203.stdout:8/964: mkdir d2/d5/d18/d31/dac/dc2/d88/d93/d14b 0 2024-05-25T06:11:41.698 INFO:tasks.workunit.client.0.smithi160.stdout:7/648: write d6/dc/d64/f45 [4535219,94505] 0 2024-05-25T06:11:41.698 INFO:tasks.workunit.client.1.smithi203.stdout:3/994: mknod d10/d10d/c141 0 2024-05-25T06:11:41.699 INFO:tasks.workunit.client.1.smithi203.stdout:8/965: fsync d2/d5/d10/f116 0 2024-05-25T06:11:41.704 INFO:tasks.workunit.client.0.smithi160.stdout:7/649: dread d6/d2b/fb2 [0,4194304] 0 2024-05-25T06:11:41.705 INFO:tasks.workunit.client.1.smithi203.stdout:3/995: rename d10/d16/d23/d38/l59 to d10/d16/d71/dfa/d107/d138/l142 0 2024-05-25T06:11:41.706 INFO:tasks.workunit.client.0.smithi160.stdout:7/650: dread - d6/dc/d64/fb0 zero size 2024-05-25T06:11:41.706 INFO:tasks.workunit.client.0.smithi160.stdout:9/533: dwrite d4/d11/d1e/d3e/d25/d27/f77 [0,4194304] 0 2024-05-25T06:11:41.710 INFO:tasks.workunit.client.0.smithi160.stdout:9/534: creat d4/d11/d12/d2e/d45/d83/fa9 x:0 0 0 2024-05-25T06:11:41.713 INFO:tasks.workunit.client.1.smithi203.stdout:3/996: symlink d10/d16/d23/d6c/l143 0 2024-05-25T06:11:41.713 INFO:tasks.workunit.client.0.smithi160.stdout:7/651: link d6/d2b/d5d/l75 d6/d32/dce/ld4 0 2024-05-25T06:11:41.715 INFO:tasks.workunit.client.0.smithi160.stdout:9/535: creat d4/d11/d1e/d3e/d25/d5e/da2/faa x:0 0 0 2024-05-25T06:11:41.715 INFO:tasks.workunit.client.0.smithi160.stdout:7/652: creat d6/d32/d92/fd5 x:0 0 0 2024-05-25T06:11:41.716 INFO:tasks.workunit.client.0.smithi160.stdout:9/536: fdatasync d4/d11/f93 0 2024-05-25T06:11:41.717 INFO:tasks.workunit.client.1.smithi203.stdout:3/997: truncate d10/d16/d23/fb9 1619117 0 2024-05-25T06:11:41.718 INFO:tasks.workunit.client.0.smithi160.stdout:7/653: mkdir d6/d2b/d5d/d89/dd6 0 2024-05-25T06:11:41.718 INFO:tasks.workunit.client.0.smithi160.stdout:8/625: sync 2024-05-25T06:11:41.719 INFO:tasks.workunit.client.0.smithi160.stdout:8/626: stat d5/d44/f68 0 2024-05-25T06:11:41.719 INFO:tasks.workunit.client.0.smithi160.stdout:9/537: mknod d4/cab 0 2024-05-25T06:11:41.720 INFO:tasks.workunit.client.0.smithi160.stdout:9/538: write f2 [61499,73903] 0 2024-05-25T06:11:41.720 INFO:tasks.workunit.client.0.smithi160.stdout:7/654: symlink d6/dc/d64/d5a/d7c/ld7 0 2024-05-25T06:11:41.723 INFO:tasks.workunit.client.0.smithi160.stdout:9/539: chown d4/d11/d1e/d24/f1b 4 1 2024-05-25T06:11:41.723 INFO:tasks.workunit.client.1.smithi203.stdout:2/933: mkdir d2/d1a/d87/d13d 0 2024-05-25T06:11:41.723 INFO:tasks.workunit.client.1.smithi203.stdout:3/998: dwrite d10/d16/d23/f10b [0,4194304] 0 2024-05-25T06:11:41.725 INFO:tasks.workunit.client.1.smithi203.stdout:3/999: dread - d10/d16/d23/d38/d11e/f13e zero size 2024-05-25T06:11:41.726 INFO:tasks.workunit.client.0.smithi160.stdout:7/655: rename d6/l1c to d6/d2b/d8b/ld8 0 2024-05-25T06:11:41.727 INFO:tasks.workunit.client.0.smithi160.stdout:8/627: dwrite d5/d7/f5f [0,4194304] 0 2024-05-25T06:11:41.728 INFO:tasks.workunit.client.1.smithi203.stdout:7/861: unlink d1/d9/d2c/f5d 0 2024-05-25T06:11:41.728 INFO:tasks.workunit.client.0.smithi160.stdout:7/656: write d6/d3e/d6f/d73/f84 [270188,17775] 0 2024-05-25T06:11:41.729 INFO:tasks.workunit.client.1.smithi203.stdout:2/934: mkdir d2/d38/d43/d60/d6b/d13e 0 2024-05-25T06:11:41.730 INFO:tasks.workunit.client.1.smithi203.stdout:9/782: dwrite d0/d17/d20/d3c/db2/fc6 [0,4194304] 0 2024-05-25T06:11:41.730 INFO:tasks.workunit.client.1.smithi203.stdout:2/935: chown d2/d38/cbd 211 1 2024-05-25T06:11:41.733 INFO:tasks.workunit.client.1.smithi203.stdout:9/783: write d0/df/d2d/d92/fdf [1008705,49838] 0 2024-05-25T06:11:41.733 INFO:tasks.workunit.client.0.smithi160.stdout:9/540: creat d4/d11/d1e/fac x:0 0 0 2024-05-25T06:11:41.736 INFO:tasks.workunit.client.1.smithi203.stdout:7/862: creat d1/d9/d2c/d36/d3b/f122 x:0 0 0 2024-05-25T06:11:41.737 INFO:tasks.workunit.client.0.smithi160.stdout:7/657: mknod d6/dc/cd9 0 2024-05-25T06:11:41.738 INFO:tasks.workunit.client.1.smithi203.stdout:9/784: mknod d0/d17/d20/d3c/db2/c100 0 2024-05-25T06:11:41.740 INFO:tasks.workunit.client.0.smithi160.stdout:9/541: creat d4/d11/d14/d8e/fad x:0 0 0 2024-05-25T06:11:41.740 INFO:tasks.workunit.client.1.smithi203.stdout:7/863: symlink d1/d9/d60/dea/l123 0 2024-05-25T06:11:41.742 INFO:tasks.workunit.client.0.smithi160.stdout:7/658: creat d6/d2b/da0/dc7/fda x:0 0 0 2024-05-25T06:11:41.744 INFO:tasks.workunit.client.1.smithi203.stdout:2/936: getdents d2/d1a/d33/d80/d133 0 2024-05-25T06:11:41.745 INFO:tasks.workunit.client.0.smithi160.stdout:9/542: creat d4/fae x:0 0 0 2024-05-25T06:11:41.746 INFO:tasks.workunit.client.0.smithi160.stdout:7/659: mkdir d6/d3e/da1/ddb 0 2024-05-25T06:11:41.747 INFO:tasks.workunit.client.0.smithi160.stdout:9/543: rmdir d4/d11/d14/d8e/d9c 39 2024-05-25T06:11:41.747 INFO:tasks.workunit.client.0.smithi160.stdout:7/660: write d6/d3e/f70 [2964729,111147] 0 2024-05-25T06:11:41.748 INFO:tasks.workunit.client.1.smithi203.stdout:9/785: creat d0/d74/f101 x:0 0 0 2024-05-25T06:11:41.749 INFO:tasks.workunit.client.0.smithi160.stdout:9/544: write d4/d11/d1e/d3e/d25/d5e/fa6 [66151,106660] 0 2024-05-25T06:11:41.753 INFO:tasks.workunit.client.0.smithi160.stdout:7/661: truncate d6/dc/ff 5599007 0 2024-05-25T06:11:41.754 INFO:tasks.workunit.client.1.smithi203.stdout:8/966: dwrite d2/d5/d28/d38/d4e/d97/f66 [8388608,4194304] 0 2024-05-25T06:11:41.754 INFO:tasks.workunit.client.0.smithi160.stdout:9/545: stat d4/d11/d1e/d3e/d25/d27/l7c 0 2024-05-25T06:11:41.755 INFO:tasks.workunit.client.0.smithi160.stdout:9/546: chown d4 15127 1 2024-05-25T06:11:41.755 INFO:tasks.workunit.client.1.smithi203.stdout:8/967: write d2/d5/d28/f7a [1717081,101803] 0 2024-05-25T06:11:41.756 INFO:tasks.workunit.client.0.smithi160.stdout:7/662: creat d6/d2b/da0/dc7/fdc x:0 0 0 2024-05-25T06:11:41.757 INFO:tasks.workunit.client.1.smithi203.stdout:9/786: symlink d0/d57/de0/df6/l102 0 2024-05-25T06:11:41.757 INFO:tasks.workunit.client.0.smithi160.stdout:7/663: stat d6/d3e/d6f/d73/c7d 0 2024-05-25T06:11:41.758 INFO:tasks.workunit.client.1.smithi203.stdout:8/968: write d2/d5/d18/d31/dac/dc2/f129 [1542552,68398] 0 2024-05-25T06:11:41.759 INFO:tasks.workunit.client.1.smithi203.stdout:8/969: stat d2/d5/d10/d11 0 2024-05-25T06:11:41.760 INFO:tasks.workunit.client.1.smithi203.stdout:7/864: getdents d1/d9/d2c/d62/db3/de5 0 2024-05-25T06:11:41.760 INFO:tasks.workunit.client.0.smithi160.stdout:7/664: mkdir d6/dc/d21/ddd 0 2024-05-25T06:11:41.763 INFO:tasks.workunit.client.0.smithi160.stdout:9/547: rename d4/d11/d7a/c8c to d4/d11/d14/d8e/caf 0 2024-05-25T06:11:41.763 INFO:tasks.workunit.client.1.smithi203.stdout:9/787: creat d0/d57/de0/f103 x:0 0 0 2024-05-25T06:11:41.765 INFO:tasks.workunit.client.0.smithi160.stdout:7/665: creat d6/d2b/d5d/d89/dd6/fde x:0 0 0 2024-05-25T06:11:41.766 INFO:tasks.workunit.client.1.smithi203.stdout:2/937: write d2/d1a/db0/feb [2936932,18798] 0 2024-05-25T06:11:41.767 INFO:tasks.workunit.client.1.smithi203.stdout:9/788: mknod d0/df/d37/da8/c104 0 2024-05-25T06:11:41.769 INFO:tasks.workunit.client.1.smithi203.stdout:8/970: mkdir d2/d5/d10/dfa/d14c 0 2024-05-25T06:11:41.771 INFO:tasks.workunit.client.0.smithi160.stdout:9/548: dwrite d4/d11/d1e/d24/d66/f94 [0,4194304] 0 2024-05-25T06:11:41.774 INFO:tasks.workunit.client.0.smithi160.stdout:9/549: write d4/d11/d1e/d3e/d25/d27/f77 [4365586,69160] 0 2024-05-25T06:11:41.784 INFO:tasks.workunit.client.1.smithi203.stdout:2/938: rename d2/d38/d2f to d2/d1a/d13f 0 2024-05-25T06:11:41.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.785+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0340005830 con 0x7f035c0fbcb0 2024-05-25T06:11:41.785 INFO:tasks.workunit.client.1.smithi203.stdout:7/865: dwrite d1/d9/d2c/d36/fbc [0,4194304] 0 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.786+0000 7f03537fe700 1 -- 172.21.15.160:0/2595891350 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+844 (secure 0 0 0) 0x7f034c0593d0 con 0x7f035c0fbcb0 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 1, 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:11:41.786 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 13, 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:11:41.787 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:11:41.789 INFO:tasks.workunit.client.1.smithi203.stdout:2/939: unlink d2/d42/d8e/cf4 0 2024-05-25T06:11:41.789 INFO:tasks.workunit.client.1.smithi203.stdout:9/789: write d0/fb [1002767,63786] 0 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f034806b7e0 msgr2=0x7f034806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f034806b7e0 0x7f034806dc90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03440041d0 tx=0x7f034400b040 comp rx=0 tx=0).stop 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 msgr2=0x7f035c10c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c10c880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f034c000c00 tx=0x7f034c009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 shutdown_connections 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f034806b7e0 0x7f034806dc90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f035c0fbcb0 0x7f035c10c880 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 --2- 172.21.15.160:0/2595891350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f035c0fcf60 0x7f035c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.789+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 >> 172.21.15.160:0/2595891350 conn(0x7f035c0f7430 msgr2=0x7f035c0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:41.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.790+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 shutdown_connections 2024-05-25T06:11:41.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.790+0000 7f03517fa700 1 -- 172.21.15.160:0/2595891350 wait complete. 2024-05-25T06:11:41.791 INFO:tasks.workunit.client.1.smithi203.stdout:8/971: truncate d2/d5/d18/d31/dac/dc2/f129 2954221 0 2024-05-25T06:11:41.795 INFO:tasks.workunit.client.0.smithi160.stdout:9/550: dread d4/d11/d12/d2e/f32 [0,4194304] 0 2024-05-25T06:11:41.796 INFO:tasks.workunit.client.0.smithi160.stdout:9/551: dread - d4/d11/d1e/d3e/d2f/f9d zero size 2024-05-25T06:11:41.797 INFO:tasks.workunit.client.0.smithi160.stdout:9/552: write d4/d11/d1e/d3e/d2f/f9d [10591,44410] 0 2024-05-25T06:11:41.798 INFO:tasks.workunit.client.0.smithi160.stdout:7/666: truncate d6/d3e/f9d 3841095 0 2024-05-25T06:11:41.799 INFO:tasks.workunit.client.0.smithi160.stdout:7/667: write d6/d3e/f70 [508292,76491] 0 2024-05-25T06:11:41.800 INFO:tasks.workunit.client.1.smithi203.stdout:7/866: write d1/d11/d3d/f6a [608083,3744] 0 2024-05-25T06:11:41.801 INFO:tasks.workunit.client.1.smithi203.stdout:8/972: truncate d2/d5/d28/d38/d81/fb3 421555 0 2024-05-25T06:11:41.801 INFO:tasks.workunit.client.0.smithi160.stdout:9/553: creat d4/d11/d1e/d3e/d25/d5e/da2/fb0 x:0 0 0 2024-05-25T06:11:41.801 INFO:tasks.workunit.client.0.smithi160.stdout:7/668: mkdir d6/d2b/d8b/ddf 0 2024-05-25T06:11:41.803 INFO:tasks.workunit.client.0.smithi160.stdout:9/554: creat d4/fb1 x:0 0 0 2024-05-25T06:11:41.807 INFO:tasks.workunit.client.1.smithi203.stdout:8/973: truncate d2/d5/d28/d38/d4e/f74 39750 0 2024-05-25T06:11:41.808 INFO:tasks.workunit.client.1.smithi203.stdout:8/974: chown d2/d5/d18/d31/c128 139571 1 2024-05-25T06:11:41.808 INFO:tasks.workunit.client.1.smithi203.stdout:7/867: write d1/d11/d3d/fa6 [984052,57] 0 2024-05-25T06:11:41.811 INFO:tasks.workunit.client.1.smithi203.stdout:8/975: creat d2/d5/d18/d31/dac/dc2/d88/d93/d9d/d9e/de6/f14d x:0 0 0 2024-05-25T06:11:41.811 INFO:tasks.workunit.client.1.smithi203.stdout:7/868: symlink d1/d9/d2c/d3a/dd1/df8/l124 0 2024-05-25T06:11:41.812 INFO:tasks.workunit.client.1.smithi203.stdout:8/976: rmdir d2/d5/d18/d139 39 2024-05-25T06:11:41.813 INFO:tasks.workunit.client.1.smithi203.stdout:8/977: fsync d2/d5/d10/d11/de/f1e 0 2024-05-25T06:11:41.816 INFO:tasks.workunit.client.0.smithi160.stdout:7/669: dwrite d6/d32/d5b/fbe [0,4194304] 0 2024-05-25T06:11:41.816 INFO:tasks.workunit.client.1.smithi203.stdout:2/940: dread d2/d42/d8e/fc4 [0,4194304] 0 2024-05-25T06:11:41.819 INFO:tasks.workunit.client.1.smithi203.stdout:8/978: rename d2/d5/d10/dfa/dfb/lfd to d2/d5/d18/d31/dac/dc2/d8a/d8f/dfc/l14e 0 2024-05-25T06:11:41.821 INFO:tasks.workunit.client.0.smithi160.stdout:7/670: dread d6/d3e/fca [0,4194304] 0 2024-05-25T06:11:41.824 INFO:tasks.workunit.client.1.smithi203.stdout:2/941: rename d2/d1a/dd3/d100/da6/le6 to d2/d1a/d33/d80/dd1/ddf/d10c/l140 0 2024-05-25T06:11:41.827 INFO:tasks.workunit.client.1.smithi203.stdout:8/979: getdents d2/d5/d10/d11 0 2024-05-25T06:11:41.828 INFO:tasks.workunit.client.1.smithi203.stdout:2/942: symlink d2/d1a/db0/l141 0 2024-05-25T06:11:41.829 INFO:tasks.workunit.client.1.smithi203.stdout:7/869: dwrite d1/d9/d2c/d79/dd6/fd7 [0,4194304] 0 2024-05-25T06:11:41.831 INFO:tasks.workunit.client.1.smithi203.stdout:8/980: read - d2/d5/d28/d38/fde zero size 2024-05-25T06:11:41.837 INFO:tasks.workunit.client.1.smithi203.stdout:7/870: rename d1/d9/d2c/d62/l9f to d1/d11/d3d/d9a/l125 0 2024-05-25T06:11:41.837 INFO:tasks.workunit.client.1.smithi203.stdout:8/981: truncate d2/d5/d18/d31/f50 1721112 0 2024-05-25T06:11:41.839 INFO:tasks.workunit.client.1.smithi203.stdout:9/790: dread d0/f16 [0,4194304] 0 2024-05-25T06:11:41.839 INFO:tasks.workunit.client.1.smithi203.stdout:8/982: truncate d2/d5/d28/d11a/f134 174651 0 2024-05-25T06:11:41.840 INFO:tasks.workunit.client.1.smithi203.stdout:9/791: chown d0/f36 183399 1 2024-05-25T06:11:41.840 INFO:tasks.workunit.client.1.smithi203.stdout:7/871: getdents d1/d9/d2c/d3a/d4e 0 2024-05-25T06:11:41.842 INFO:tasks.workunit.client.1.smithi203.stdout:9/792: write d0/df/d37/da8/dd6/ffa [944630,89068] 0 2024-05-25T06:11:41.845 INFO:tasks.workunit.client.1.smithi203.stdout:8/983: rename d2/d5/dae/cf7 to d2/d5/d10/d45/d5d/d68/c14f 0 2024-05-25T06:11:41.849 INFO:tasks.workunit.client.1.smithi203.stdout:8/984: fdatasync d2/d5/d10/d45/d5d/f5c 0 2024-05-25T06:11:41.849 INFO:tasks.workunit.client.1.smithi203.stdout:9/793: link d0/d17/f4b d0/d17/d20/d6e/d75/df0/f105 0 2024-05-25T06:11:41.852 INFO:tasks.workunit.client.1.smithi203.stdout:9/794: mknod d0/d57/de0/c106 0 2024-05-25T06:11:41.852 INFO:tasks.workunit.client.1.smithi203.stdout:9/795: chown d0/d17/d20/d29 3628511 1 2024-05-25T06:11:41.854 INFO:tasks.workunit.client.1.smithi203.stdout:8/985: creat d2/d5/d18/d31/dac/dc2/d88/f150 x:0 0 0 2024-05-25T06:11:41.855 INFO:tasks.workunit.client.1.smithi203.stdout:2/943: dread d2/d1a/d87/d9e/ff7 [0,4194304] 0 2024-05-25T06:11:41.856 INFO:tasks.workunit.client.1.smithi203.stdout:9/796: creat d0/df/d2b/f107 x:0 0 0 2024-05-25T06:11:41.858 INFO:tasks.workunit.client.1.smithi203.stdout:9/797: write d0/d17/f33 [7783317,101043] 0 2024-05-25T06:11:41.859 INFO:tasks.workunit.client.1.smithi203.stdout:2/944: mkdir d2/d1a/db0/d13b/d142 0 2024-05-25T06:11:41.861 INFO:tasks.workunit.client.1.smithi203.stdout:7/872: dwrite d1/d11/d20/de4/f117 [0,4194304] 0 2024-05-25T06:11:41.862 INFO:tasks.workunit.client.1.smithi203.stdout:7/873: chown d1/d9/d2c/d36/d3b/l4a 7 1 2024-05-25T06:11:41.863 INFO:tasks.workunit.client.1.smithi203.stdout:7/874: chown d1/d11/d20/l64 1048086682 1 2024-05-25T06:11:41.867 INFO:tasks.workunit.client.1.smithi203.stdout:2/945: getdents d2/d38 0 2024-05-25T06:11:41.867 INFO:tasks.workunit.client.1.smithi203.stdout:9/798: write d0/df/d73/f53 [111313,73837] 0 2024-05-25T06:11:41.868 INFO:tasks.workunit.client.1.smithi203.stdout:2/946: chown d2/d1a/c98 221668 1 2024-05-25T06:11:41.868 INFO:tasks.workunit.client.1.smithi203.stdout:9/799: stat d0/df/d37/da8/cd5 0 2024-05-25T06:11:41.871 INFO:tasks.workunit.client.1.smithi203.stdout:9/800: chown d0/f4 761647 1 2024-05-25T06:11:41.873 INFO:tasks.workunit.client.1.smithi203.stdout:9/801: stat d0/d17/d20/d6e/f65 0 2024-05-25T06:11:41.873 INFO:tasks.workunit.client.1.smithi203.stdout:7/875: dwrite d1/d9/d2c/d36/d54/f8f [0,4194304] 0 2024-05-25T06:11:41.875 INFO:tasks.workunit.client.1.smithi203.stdout:2/947: fsync d2/d1a/dd3/d100/d10/d58/f6d 0 2024-05-25T06:11:41.877 INFO:tasks.workunit.client.1.smithi203.stdout:9/802: dread - d0/d17/d20/d29/f6b zero size 2024-05-25T06:11:41.877 INFO:tasks.workunit.client.1.smithi203.stdout:7/876: read d1/f24 [2137388,8567] 0 2024-05-25T06:11:41.878 INFO:tasks.workunit.client.1.smithi203.stdout:9/803: chown d0/d17/d20/d3c/f5e 835 1 2024-05-25T06:11:41.880 INFO:tasks.workunit.client.1.smithi203.stdout:9/804: dread d0/f16 [0,4194304] 0 2024-05-25T06:11:41.881 INFO:tasks.workunit.client.1.smithi203.stdout:7/877: mknod d1/d11/d3d/d4c/de1/c126 0 2024-05-25T06:11:41.883 INFO:tasks.workunit.client.1.smithi203.stdout:7/878: write d1/d9/d2c/d3a/f3e [1297329,106247] 0 2024-05-25T06:11:41.884 INFO:tasks.workunit.client.1.smithi203.stdout:7/879: read - d1/d9/d2c/d36/d3b/faf zero size 2024-05-25T06:11:41.885 INFO:tasks.workunit.client.1.smithi203.stdout:9/805: unlink d0/df/d37/da8/lfc 0 2024-05-25T06:11:41.886 INFO:tasks.workunit.client.1.smithi203.stdout:9/806: chown d0/df/d37/da8/db8/fff 854 1 2024-05-25T06:11:41.889 INFO:tasks.workunit.client.1.smithi203.stdout:7/880: rename d1/d11/d20 to d1/d9/d2c/d118/d127 0 2024-05-25T06:11:41.892 INFO:tasks.workunit.client.1.smithi203.stdout:9/807: getdents d0/df/d73 0 2024-05-25T06:11:41.893 INFO:tasks.workunit.client.1.smithi203.stdout:7/881: mknod d1/d9/d2c/d36/dd3/c128 0 2024-05-25T06:11:41.894 INFO:tasks.workunit.client.1.smithi203.stdout:7/882: chown d1/d9/d2c/d79/dc7/f11c 3296356 1 2024-05-25T06:11:41.896 INFO:tasks.workunit.client.1.smithi203.stdout:9/808: link d0/f1e d0/df/d37/d4f/f108 0 2024-05-25T06:11:41.899 INFO:tasks.workunit.client.1.smithi203.stdout:7/883: mkdir d1/d9/d2c/d3a/d4e/d129 0 2024-05-25T06:11:41.900 INFO:tasks.workunit.client.1.smithi203.stdout:7/884: fsync d1/d9/d2c/d62/db3/f111 0 2024-05-25T06:11:41.900 INFO:tasks.workunit.client.1.smithi203.stdout:2/948: dread d2/d42/d4e/d68/faf [0,4194304] 0 2024-05-25T06:11:41.900 INFO:tasks.workunit.client.1.smithi203.stdout:8/986: dread d2/d5/d10/d11/de/f1e [0,4194304] 0 2024-05-25T06:11:41.902 INFO:tasks.workunit.client.1.smithi203.stdout:2/949: symlink d2/d1a/d33/d80/l143 0 2024-05-25T06:11:41.905 INFO:tasks.workunit.client.1.smithi203.stdout:7/885: creat d1/d9/d2c/d79/f12a x:0 0 0 2024-05-25T06:11:41.906 INFO:tasks.workunit.client.1.smithi203.stdout:2/950: fsync d2/d1a/dd3/d100/d10/f21 0 2024-05-25T06:11:41.906 INFO:tasks.workunit.client.1.smithi203.stdout:7/886: write d1/d9/d2c/d36/d66/fba [1401813,10962] 0 2024-05-25T06:11:41.909 INFO:tasks.workunit.client.1.smithi203.stdout:8/987: dread d2/d5/d18/d31/f6f [0,4194304] 0 2024-05-25T06:11:41.909 INFO:tasks.workunit.client.1.smithi203.stdout:7/887: creat d1/d9/d2c/d79/dd6/f12b x:0 0 0 2024-05-25T06:11:41.911 INFO:tasks.workunit.client.1.smithi203.stdout:2/951: rename d2/d1a/dd3/d100/d10/d32/fe7 to d2/d42/d4e/f144 0 2024-05-25T06:11:41.914 INFO:tasks.workunit.client.1.smithi203.stdout:7/888: mkdir d1/d9/dc2/d12c 0 2024-05-25T06:11:41.915 INFO:tasks.workunit.client.1.smithi203.stdout:2/952: creat d2/d1a/d33/d80/dd1/ddf/d10c/f145 x:0 0 0 2024-05-25T06:11:41.916 INFO:tasks.workunit.client.1.smithi203.stdout:8/988: dread d2/d5/d28/d38/d4e/f74 [0,4194304] 0 2024-05-25T06:11:41.916 INFO:tasks.workunit.client.0.smithi160.stdout:9/555: rename d4/d11/d12/d58/d97 to d4/db2 0 2024-05-25T06:11:41.917 INFO:tasks.workunit.client.1.smithi203.stdout:2/953: dread d2/d1a/d89/daa/def/d107/d113/f122 [0,4194304] 0 2024-05-25T06:11:41.917 INFO:tasks.workunit.client.1.smithi203.stdout:7/889: creat d1/d9/d2c/d3a/dd1/dff/f12d x:0 0 0 2024-05-25T06:11:41.919 INFO:tasks.workunit.client.0.smithi160.stdout:7/671: getdents d6/d3e/d59 0 2024-05-25T06:11:41.920 INFO:tasks.workunit.client.0.smithi160.stdout:7/672: read - d6/dc/fd1 zero size 2024-05-25T06:11:41.921 INFO:tasks.workunit.client.1.smithi203.stdout:8/989: mkdir d2/d5/d28/d38/d4e/d97/da1/d151 0 2024-05-25T06:11:41.922 INFO:tasks.workunit.client.0.smithi160.stdout:9/556: write d4/d11/d1e/d3e/f74 [4645914,60001] 0 2024-05-25T06:11:41.923 INFO:tasks.workunit.client.1.smithi203.stdout:7/890: fsync d1/f24 0 2024-05-25T06:11:41.923 INFO:tasks.workunit.client.1.smithi203.stdout:2/954: rmdir d2/d38/d43/d60/d6b/dae 39 2024-05-25T06:11:41.928 INFO:tasks.workunit.client.1.smithi203.stdout:7/891: mkdir d1/d9/d2c/d62/d7a/d89/d12e 0 2024-05-25T06:11:41.928 INFO:tasks.workunit.client.1.smithi203.stdout:7/892: fdatasync d1/d9/d2c/d36/f9d 0 2024-05-25T06:11:41.933 INFO:tasks.workunit.client.1.smithi203.stdout:2/955: rmdir d2/d42/de5/d116 0 2024-05-25T06:11:41.935 INFO:tasks.workunit.client.1.smithi203.stdout:7/893: getdents d1/dbd 0 2024-05-25T06:11:41.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.937+0000 7fecadda0700 1 -- 172.21.15.160:0/842970721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fcda0 msgr2=0x7feca80fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.937+0000 7fecadda0700 1 --2- 172.21.15.160:0/842970721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fcda0 0x7feca80fd210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec94004940 tx=0x7fec94009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:41.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.937+0000 7fecadda0700 1 -- 172.21.15.160:0/842970721 shutdown_connections 2024-05-25T06:11:41.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.937+0000 7fecadda0700 1 --2- 172.21.15.160:0/842970721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fcda0 0x7feca80fd210 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec94004940 tx=0x7fec94009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:41.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.937+0000 7fecadda0700 1 --2- 172.21.15.160:0/842970721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca80fbaf0 0x7feca80fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.937+0000 7fecadda0700 1 -- 172.21.15.160:0/842970721 >> 172.21.15.160:0/842970721 conn(0x7feca80f7250 msgr2=0x7feca80f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:41.938 INFO:tasks.workunit.client.1.smithi203.stdout:8/990: dwrite d2/d5/d28/d38/d4e/fbb [0,4194304] 0 2024-05-25T06:11:41.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.938+0000 7fecadda0700 1 -- 172.21.15.160:0/842970721 shutdown_connections 2024-05-25T06:11:41.939 INFO:tasks.workunit.client.0.smithi160.stdout:9/557: dread d4/f75 [0,4194304] 0 2024-05-25T06:11:41.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.938+0000 7fecadda0700 1 -- 172.21.15.160:0/842970721 wait complete. 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.938+0000 7fecadda0700 1 Processor -- start 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecadda0700 1 -- start start 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecadda0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 0x7feca810b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecadda0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca810bcb0 0x7feca8111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecadda0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca8112340 con 0x7feca80fbaf0 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecadda0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca81124b0 con 0x7feca810bcb0 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecacd9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 0x7feca810b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecacd9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 0x7feca810b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48524/0 (socket says 172.21.15.160:48524) 2024-05-25T06:11:41.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.939+0000 7fecacd9e700 1 -- 172.21.15.160:0/3839213594 learned_addr learned my addr 172.21.15.160:0/3839213594 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:11:41.941 INFO:tasks.workunit.client.1.smithi203.stdout:7/894: fdatasync d1/f24 0 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.940+0000 7feca7fff700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca810bcb0 0x7feca8111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.940+0000 7fecacd9e700 1 -- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca810bcb0 msgr2=0x7feca8111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.940+0000 7fecacd9e700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca810bcb0 0x7feca8111db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.940+0000 7fecacd9e700 1 -- 172.21.15.160:0/3839213594 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94005040 con 0x7feca80fbaf0 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.940+0000 7fecacd9e700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 0x7feca810b770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00eb70 tx=0x7fec9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.941+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9c007cc0 con 0x7feca80fbaf0 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.941+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca810c1e0 con 0x7feca80fbaf0 2024-05-25T06:11:41.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.941+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca810cd60 con 0x7feca80fbaf0 2024-05-25T06:11:41.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.941+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec9c00f040 con 0x7feca80fbaf0 2024-05-25T06:11:41.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.941+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9c009640 con 0x7feca80fbaf0 2024-05-25T06:11:41.943 INFO:tasks.workunit.client.0.smithi160.stdout:9/558: rename d4/f75 to d4/d11/d1e/d3e/d25/fb3 0 2024-05-25T06:11:41.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.942+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca805f5c0 con 0x7feca80fbaf0 2024-05-25T06:11:41.943 INFO:tasks.workunit.client.0.smithi160.stdout:9/559: write d4/d11/d12/d2e/d45/d83/fa9 [688369,57060] 0 2024-05-25T06:11:41.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.944+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fec9c0097a0 con 0x7feca80fbaf0 2024-05-25T06:11:41.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.944+0000 7feca57fa700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fec9806b860 0x7fec9806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:11:41.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.945+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fec9c08a020 con 0x7feca80fbaf0 2024-05-25T06:11:41.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.945+0000 7feca7fff700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fec9806b860 0x7fec9806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:11:41.946 INFO:tasks.workunit.client.0.smithi160.stdout:9/560: write d4/d11/d1e/d24/f4f [1285558,119113] 0 2024-05-25T06:11:41.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.947+0000 7feca7fff700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fec9806b860 0x7fec9806dd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec94000c00 tx=0x7fec94009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:11:41.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:41.951+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fec9c053ae0 con 0x7feca80fbaf0 2024-05-25T06:11:41.951 INFO:tasks.workunit.client.1.smithi203.stdout:8/991: creat d2/d5/d18/d31/dac/dc2/d88/f152 x:0 0 0 2024-05-25T06:11:41.952 INFO:tasks.workunit.client.1.smithi203.stdout:7/895: write d1/dbd/dc1/fd8 [585106,93118] 0 2024-05-25T06:11:41.955 INFO:tasks.workunit.client.1.smithi203.stdout:2/956: dwrite d2/d1a/d33/d80/fc9 [0,4194304] 0 2024-05-25T06:11:41.963 INFO:tasks.workunit.client.1.smithi203.stdout:2/957: dwrite d2/d1a/dd3/d100/f11c [0,4194304] 0 2024-05-25T06:11:41.966 INFO:tasks.workunit.client.0.smithi160.stdout:1/461: sync 2024-05-25T06:11:41.966 INFO:tasks.workunit.client.1.smithi203.stdout:2/958: write d2/d1a/dd3/d100/d10/dc8/f139 [527973,119509] 0 2024-05-25T06:11:41.967 INFO:tasks.workunit.client.1.smithi203.stdout:7/896: dread d1/d9/d2c/d118/d127/f23 [0,4194304] 0 2024-05-25T06:11:41.967 INFO:tasks.workunit.client.0.smithi160.stdout:1/462: dread - de/d21/d49/d1c/d59/d67/f83 zero size 2024-05-25T06:11:41.968 INFO:tasks.workunit.client.1.smithi203.stdout:8/992: dwrite d2/d5/d10/d45/d5d/fc5 [0,4194304] 0 2024-05-25T06:11:41.974 INFO:tasks.workunit.client.1.smithi203.stdout:2/959: truncate d2/d1a/dd3/d100/d10/f9b 4182013 0 2024-05-25T06:11:41.976 INFO:tasks.workunit.client.1.smithi203.stdout:7/897: rename d1/f1c to d1/f12f 0 2024-05-25T06:11:41.978 INFO:tasks.workunit.client.1.smithi203.stdout:2/960: symlink d2/d1a/dd3/d100/d10/d32/dca/l146 0 2024-05-25T06:11:41.979 INFO:tasks.workunit.client.1.smithi203.stdout:7/898: truncate d1/d9/f76 523777 0 2024-05-25T06:11:41.981 INFO:tasks.workunit.client.1.smithi203.stdout:2/961: chown d2/d42/c6a 4343536 1 2024-05-25T06:11:41.983 INFO:tasks.workunit.client.1.smithi203.stdout:8/993: dwrite d2/d5/d10/d45/d5d/fe9 [0,4194304] 0 2024-05-25T06:11:41.988 INFO:tasks.workunit.client.1.smithi203.stdout:8/994: rename d2/d5/d10/l86 to d2/d5/d28/d38/d4e/d97/l153 0 2024-05-25T06:11:41.988 INFO:tasks.workunit.client.1.smithi203.stdout:2/962: link d2/d1a/d89/fd2 d2/d42/d8e/f147 0 2024-05-25T06:11:41.988 INFO:tasks.workunit.client.1.smithi203.stdout:7/899: link d1/d9/d2c/d36/d54/cdf d1/d11/d3d/d9a/da4/c130 0 2024-05-25T06:11:41.989 INFO:tasks.workunit.client.1.smithi203.stdout:8/995: fsync d2/d5/d28/ff9 0 2024-05-25T06:11:41.989 INFO:tasks.workunit.client.1.smithi203.stdout:7/900: readlink d1/l4 0 2024-05-25T06:11:41.990 INFO:tasks.workunit.client.1.smithi203.stdout:2/963: unlink d2/d6/f46 0 2024-05-25T06:11:41.991 INFO:tasks.workunit.client.1.smithi203.stdout:8/996: read - d2/d5/d18/d31/dac/dc2/d88/d93/f12d zero size 2024-05-25T06:11:41.993 INFO:tasks.workunit.client.1.smithi203.stdout:7/901: mknod d1/d9/d2c/d62/db3/de5/c131 0 2024-05-25T06:11:41.993 INFO:tasks.workunit.client.1.smithi203.stdout:2/964: rmdir d2/d1a/d89 39 2024-05-25T06:11:41.994 INFO:tasks.workunit.client.1.smithi203.stdout:8/997: stat d2/d5/d28/d38/d4e/d97/c7c 0 2024-05-25T06:11:41.998 INFO:tasks.workunit.client.1.smithi203.stdout:2/965: dread d2/d1a/dd3/d100/d10/f11a [0,4194304] 0 2024-05-25T06:11:41.998 INFO:tasks.workunit.client.1.smithi203.stdout:7/902: dread - d1/fd9 zero size 2024-05-25T06:11:41.998 INFO:tasks.workunit.client.1.smithi203.stdout:8/998: stat d2/d5/d10/d45/d5d/d76/cec 0 2024-05-25T06:11:42.000 INFO:tasks.workunit.client.1.smithi203.stdout:8/999: mkdir d2/d5/d10/dfa/d154 0 2024-05-25T06:11:42.000 INFO:tasks.workunit.client.1.smithi203.stdout:2/966: dread d2/d1a/d33/d80/d133/f13c [0,4194304] 0 2024-05-25T06:11:42.001 INFO:tasks.workunit.client.1.smithi203.stdout:2/967: write d2/d38/f29 [173409,14035] 0 2024-05-25T06:11:42.002 INFO:tasks.workunit.client.1.smithi203.stdout:2/968: readlink d2/d1a/dd3/d100/d10/d58/da0/lee 0 2024-05-25T06:11:42.002 INFO:tasks.workunit.client.0.smithi160.stdout:9/561: mknod d4/d11/d12/d2e/d45/d90/cb4 0 2024-05-25T06:11:42.004 INFO:tasks.workunit.client.0.smithi160.stdout:1/463: symlink de/d21/d49/d1c/d59/l98 0 2024-05-25T06:11:42.004 INFO:tasks.workunit.client.1.smithi203.stdout:2/969: creat d2/d1a/d33/d80/dd1/ddf/d10c/f148 x:0 0 0 2024-05-25T06:11:42.005 INFO:tasks.workunit.client.0.smithi160.stdout:1/464: readlink de/d21/d49/d1c/d41/l8b 0 2024-05-25T06:11:42.007 INFO:tasks.workunit.client.1.smithi203.stdout:2/970: truncate d2/d1a/d89/daa/fdd 608945 0 2024-05-25T06:11:42.079 INFO:tasks.workunit.client.0.smithi160.stdout:9/562: dwrite d4/d11/d14/f61 [0,4194304] 0 2024-05-25T06:11:42.084 INFO:tasks.workunit.client.0.smithi160.stdout:9/563: unlink d4/d11/d1e/d3e/d25/d27/c99 0 2024-05-25T06:11:42.089 INFO:tasks.workunit.client.0.smithi160.stdout:9/564: creat d4/d11/d14/d8e/d9c/fb5 x:0 0 0 2024-05-25T06:11:42.089 INFO:tasks.workunit.client.1.smithi203.stdout:7/903: write d1/d11/f41 [4710479,29986] 0 2024-05-25T06:11:42.090 INFO:tasks.workunit.client.1.smithi203.stdout:0/847: sync 2024-05-25T06:11:42.092 INFO:tasks.workunit.client.1.smithi203.stdout:2/971: write d2/f6c [480719,1948] 0 2024-05-25T06:11:42.092 INFO:tasks.workunit.client.0.smithi160.stdout:9/565: creat d4/d11/d14/fb6 x:0 0 0 2024-05-25T06:11:42.094 INFO:tasks.workunit.client.0.smithi160.stdout:9/566: mkdir d4/d11/d1e/d3e/d2f/d79/db7 0 2024-05-25T06:11:42.095 INFO:tasks.workunit.client.0.smithi160.stdout:9/567: fdatasync d4/d11/d1e/fac 0 2024-05-25T06:11:42.095 INFO:tasks.workunit.client.1.smithi203.stdout:0/848: creat dd/d5c/d63/d106/d4f/f129 x:0 0 0 2024-05-25T06:11:42.096 INFO:tasks.workunit.client.0.smithi160.stdout:9/568: fsync d4/d11/d14/d8e/fa8 0 2024-05-25T06:11:42.097 INFO:tasks.workunit.client.1.smithi203.stdout:0/849: creat dd/d32/d108/f12a x:0 0 0 2024-05-25T06:11:42.098 INFO:tasks.workunit.client.1.smithi203.stdout:0/850: dread - dd/d5c/f73 zero size 2024-05-25T06:11:42.098 INFO:tasks.workunit.client.1.smithi203.stdout:2/972: getdents d2/d1a/d33/d12c 0 2024-05-25T06:11:42.099 INFO:tasks.workunit.client.1.smithi203.stdout:0/851: read dd/d5c/f98 [367027,101763] 0 2024-05-25T06:11:42.100 INFO:tasks.workunit.client.1.smithi203.stdout:2/973: fsync d2/d1a/d33/d80/d133/f13c 0 2024-05-25T06:11:42.100 INFO:tasks.workunit.client.1.smithi203.stdout:0/852: fdatasync dd/d5c/d63/d106/da0/db9/f105 0 2024-05-25T06:11:42.103 INFO:tasks.workunit.client.0.smithi160.stdout:1/465: dread de/f19 [0,4194304] 0 2024-05-25T06:11:42.103 INFO:tasks.workunit.client.1.smithi203.stdout:2/974: mkdir d2/d42/d8e/d149 0 2024-05-25T06:11:42.104 INFO:tasks.workunit.client.1.smithi203.stdout:0/853: creat dd/d5c/d63/d106/df3/d10c/f12b x:0 0 0 2024-05-25T06:11:42.104 INFO:tasks.workunit.client.1.smithi203.stdout:2/975: symlink d2/d1a/dd3/d100/l14a 0 2024-05-25T06:11:42.105 INFO:tasks.workunit.client.0.smithi160.stdout:1/466: truncate de/f13 4221254 0 2024-05-25T06:11:42.105 INFO:tasks.workunit.client.1.smithi203.stdout:0/854: symlink dd/d5c/d63/d106/d48/de6/l12c 0 2024-05-25T06:11:42.106 INFO:tasks.workunit.client.0.smithi160.stdout:1/467: stat de/d21/d49/d1c/d59/d7f/c8d 0 2024-05-25T06:11:42.106 INFO:tasks.workunit.client.1.smithi203.stdout:2/976: read d2/d1a/dd3/d100/d10/f5e [1660003,13744] 0 2024-05-25T06:11:42.108 INFO:tasks.workunit.client.1.smithi203.stdout:2/977: rmdir d2/d42/d4e 39 2024-05-25T06:11:42.108 INFO:tasks.workunit.client.1.smithi203.stdout:2/978: truncate d2/f6c 1165936 0 2024-05-25T06:11:42.109 INFO:tasks.workunit.client.1.smithi203.stdout:2/979: read d2/d38/f7d [55655,130410] 0 2024-05-25T06:11:42.110 INFO:tasks.workunit.client.1.smithi203.stdout:0/855: write dd/d32/f54 [5574832,37856] 0 2024-05-25T06:11:42.111 INFO:tasks.workunit.client.1.smithi203.stdout:0/856: stat dd/d5c/d63/d106/d59/c5e 0 2024-05-25T06:11:42.112 INFO:tasks.workunit.client.1.smithi203.stdout:2/980: unlink d2/d1a/dd3/d100/d10/l40 0 2024-05-25T06:11:42.113 INFO:tasks.workunit.client.1.smithi203.stdout:2/981: chown d2/d1a/d95 3 1 2024-05-25T06:11:42.113 INFO:tasks.workunit.client.0.smithi160.stdout:1/468: dwrite de/f92 [0,4194304] 0 2024-05-25T06:11:42.115 INFO:tasks.workunit.client.1.smithi203.stdout:2/982: rmdir d2/d1a/d95 39 2024-05-25T06:11:42.117 INFO:tasks.workunit.client.0.smithi160.stdout:1/469: mknod de/d21/d49/d90/c99 0 2024-05-25T06:11:42.119 INFO:tasks.workunit.client.0.smithi160.stdout:1/470: write de/f15 [2315379,102923] 0 2024-05-25T06:11:42.122 INFO:tasks.workunit.client.0.smithi160.stdout:1/471: creat de/d21/f9a x:0 0 0 2024-05-25T06:11:42.124 INFO:tasks.workunit.client.1.smithi203.stdout:2/983: dread d2/d38/d43/d60/f108 [0,4194304] 0 2024-05-25T06:11:42.125 INFO:tasks.workunit.client.1.smithi203.stdout:2/984: write d2/d42/f118 [251188,53183] 0 2024-05-25T06:11:42.132 INFO:tasks.workunit.client.1.smithi203.stdout:2/985: dwrite d2/d1a/d87/d9e/ff7 [0,4194304] 0 2024-05-25T06:11:42.132 INFO:tasks.workunit.client.1.smithi203.stdout:0/857: dread dd/d5c/d63/d106/d59/f83 [0,4194304] 0 2024-05-25T06:11:42.134 INFO:tasks.workunit.client.0.smithi160.stdout:1/472: truncate de/d21/d49/f1d 5879159 0 2024-05-25T06:11:42.136 INFO:tasks.workunit.client.1.smithi203.stdout:0/858: mknod dd/d5c/d63/d106/c12d 0 2024-05-25T06:11:42.137 INFO:tasks.workunit.client.1.smithi203.stdout:2/986: truncate d2/d1a/dd3/d100/d10/d5f/f85 1207000 0 2024-05-25T06:11:42.139 INFO:tasks.workunit.client.1.smithi203.stdout:2/987: creat d2/d1a/d89/f14b x:0 0 0 2024-05-25T06:11:42.139 INFO:tasks.workunit.client.0.smithi160.stdout:2/527: sync 2024-05-25T06:11:42.140 INFO:tasks.workunit.client.0.smithi160.stdout:0/657: sync 2024-05-25T06:11:42.140 INFO:tasks.workunit.client.0.smithi160.stdout:2/528: write ff [1005634,118264] 0 2024-05-25T06:11:42.142 INFO:tasks.workunit.client.1.smithi203.stdout:0/859: symlink dd/d5c/d63/d106/da0/db9/df9/d86/l12e 0 2024-05-25T06:11:42.142 INFO:tasks.workunit.client.0.smithi160.stdout:2/529: dread - d12/d1c/d37/d4b/f79 zero size 2024-05-25T06:11:42.145 INFO:tasks.workunit.client.0.smithi160.stdout:2/530: getdents d12/d13 0 2024-05-25T06:11:42.145 INFO:tasks.workunit.client.1.smithi203.stdout:2/988: read d2/d1a/dd3/d100/f74 [427934,99626] 0 2024-05-25T06:11:42.146 INFO:tasks.workunit.client.1.smithi203.stdout:0/860: rename dd/d10/l3f to dd/d5c/d63/d106/d75/l12f 0 2024-05-25T06:11:42.147 INFO:tasks.workunit.client.1.smithi203.stdout:2/989: chown d2/d1a/d33/c3f 1033132 1 2024-05-25T06:11:42.147 INFO:tasks.workunit.client.0.smithi160.stdout:2/531: mkdir d12/d3e/db5 0 2024-05-25T06:11:42.147 INFO:tasks.workunit.client.1.smithi203.stdout:0/861: rename dd to dd/d5c/d63/d106/d75/d117/d130 22 2024-05-25T06:11:42.149 INFO:tasks.workunit.client.0.smithi160.stdout:2/532: creat d12/d5c/d85/fb6 x:0 0 0 2024-05-25T06:11:42.149 INFO:tasks.workunit.client.1.smithi203.stdout:2/990: mknod d2/d1a/d33/d80/dd1/c14c 0 2024-05-25T06:11:42.149 INFO:tasks.workunit.client.0.smithi160.stdout:2/533: chown d12/d1c/f1e 3 1 2024-05-25T06:11:42.152 INFO:tasks.workunit.client.0.smithi160.stdout:2/534: mknod d12/cb7 0 2024-05-25T06:11:42.153 INFO:tasks.workunit.client.1.smithi203.stdout:2/991: mknod d2/d1a/d33/d80/dd1/ddf/c14d 0 2024-05-25T06:11:42.153 INFO:tasks.workunit.client.1.smithi203.stdout:0/862: dread dd/d5c/f98 [0,4194304] 0 2024-05-25T06:11:42.153 INFO:tasks.workunit.client.0.smithi160.stdout:2/535: dread - d12/f8f zero size 2024-05-25T06:11:42.153 INFO:tasks.workunit.client.0.smithi160.stdout:1/473: dread de/d21/d49/d1c/d52/f56 [0,4194304] 0 2024-05-25T06:11:42.154 INFO:tasks.workunit.client.1.smithi203.stdout:2/992: mknod d2/d1a/dd3/c14e 0 2024-05-25T06:11:42.155 INFO:tasks.workunit.client.1.smithi203.stdout:2/993: chown d2/d1a/l59 1 1 2024-05-25T06:11:42.155 INFO:tasks.workunit.client.1.smithi203.stdout:0/863: creat dd/d5c/d63/d106/d4f/db7/df0/f131 x:0 0 0 2024-05-25T06:11:42.157 INFO:tasks.workunit.client.1.smithi203.stdout:0/864: chown dd/d5c/d63/d106/df3/d10c/f12b 644527014 1 2024-05-25T06:11:42.157 INFO:tasks.workunit.client.1.smithi203.stdout:2/994: truncate d2/d38/fe2 793383 0 2024-05-25T06:11:42.158 INFO:tasks.workunit.client.1.smithi203.stdout:0/865: write dd/d45/ffa [233934,101359] 0 2024-05-25T06:11:42.161 INFO:tasks.workunit.client.0.smithi160.stdout:1/474: rmdir de/d21/d49/d88 0 2024-05-25T06:11:42.162 INFO:tasks.workunit.client.0.smithi160.stdout:0/658: dread d2b/d31/d35/f3e [0,4194304] 0 2024-05-25T06:11:42.163 INFO:tasks.workunit.client.0.smithi160.stdout:1/475: mknod de/d21/c9b 0 2024-05-25T06:11:42.163 INFO:tasks.workunit.client.1.smithi203.stdout:2/995: symlink d2/d38/d43/d60/d6b/d13e/l14f 0 2024-05-25T06:11:42.165 INFO:tasks.workunit.client.1.smithi203.stdout:2/996: creat d2/d1a/d89/f150 x:0 0 0 2024-05-25T06:11:42.168 INFO:tasks.workunit.client.1.smithi203.stdout:2/997: readlink d2/d1a/d13f/l47 0 2024-05-25T06:11:42.169 INFO:tasks.workunit.client.0.smithi160.stdout:2/536: dwrite d12/d13/f60 [0,4194304] 0 2024-05-25T06:11:42.169 INFO:tasks.workunit.client.0.smithi160.stdout:1/476: mknod de/d21/d49/d1c/d41/d45/d7d/c9c 0 2024-05-25T06:11:42.172 INFO:tasks.workunit.client.1.smithi203.stdout:2/998: fdatasync d2/d1a/dd3/d100/d10/f21 0 2024-05-25T06:11:42.172 INFO:tasks.workunit.client.0.smithi160.stdout:2/537: fsync d12/d3e/d5a/d8d/fa8 0 2024-05-25T06:11:42.176 INFO:tasks.workunit.client.0.smithi160.stdout:2/538: unlink d12/d1c/l5d 0 2024-05-25T06:11:42.179 INFO:tasks.workunit.client.0.smithi160.stdout:2/539: chown d12/d1c/d2a/d7e/fa2 44 1 2024-05-25T06:11:42.179 INFO:tasks.workunit.client.0.smithi160.stdout:1/477: dwrite de/f91 [0,4194304] 0 2024-05-25T06:11:42.184 INFO:tasks.workunit.client.0.smithi160.stdout:2/540: creat d12/d3e/d5a/d8d/fb8 x:0 0 0 2024-05-25T06:11:42.185 INFO:tasks.workunit.client.0.smithi160.stdout:1/478: truncate de/d21/d49/d28/f65 30999 0 2024-05-25T06:11:42.188 INFO:tasks.workunit.client.0.smithi160.stdout:1/479: write de/d21/d49/d1c/d59/d67/f83 [704953,48601] 0 2024-05-25T06:11:42.189 INFO:tasks.workunit.client.1.smithi203.stdout:2/999: rmdir d2/d38/d43/d60/d6b 39 2024-05-25T06:11:42.192 INFO:tasks.workunit.client.0.smithi160.stdout:0/659: dread d2b/d46/f5a [0,4194304] 0 2024-05-25T06:11:42.192 INFO:tasks.workunit.client.0.smithi160.stdout:1/480: fdatasync de/d21/d49/d1c/d59/f62 0 2024-05-25T06:11:42.193 INFO:tasks.workunit.client.0.smithi160.stdout:2/541: dwrite d12/d3c/f61 [0,4194304] 0 2024-05-25T06:11:42.196 INFO:tasks.workunit.client.0.smithi160.stdout:1/481: truncate de/d21/d49/d1c/d59/d7f/f8e 865330 0 2024-05-25T06:11:42.197 INFO:tasks.workunit.client.0.smithi160.stdout:0/660: rename d2b/d31/d35/d36/d37/f4d to d2b/d46/d63/da7/da6/fd2 0 2024-05-25T06:11:42.201 INFO:tasks.workunit.client.0.smithi160.stdout:1/482: chown de/l1e 778974 1 2024-05-25T06:11:42.201 INFO:tasks.workunit.client.1.smithi203.stdout:0/866: write dd/d10/f29 [7552929,65665] 0 2024-05-25T06:11:42.202 INFO:tasks.workunit.client.0.smithi160.stdout:2/542: dwrite d12/d75/f88 [0,4194304] 0 2024-05-25T06:11:42.205 INFO:tasks.workunit.client.1.smithi203.stdout:0/867: truncate dd/d5c/d63/d106/df3/f7f 1860958 0 2024-05-25T06:11:42.206 INFO:tasks.workunit.client.0.smithi160.stdout:2/543: chown d12/d3e/d5f/d74/d7a/fac 36145 1 2024-05-25T06:11:42.207 INFO:tasks.workunit.client.1.smithi203.stdout:0/868: write dd/d32/d37/f101 [668988,24665] 0 2024-05-25T06:11:42.207 INFO:tasks.workunit.client.0.smithi160.stdout:0/661: getdents d2b/d31/d35 0 2024-05-25T06:11:42.207 INFO:tasks.workunit.client.0.smithi160.stdout:1/483: creat de/d21/d49/d1c/f9d x:0 0 0 2024-05-25T06:11:42.208 INFO:tasks.workunit.client.0.smithi160.stdout:2/544: creat d12/d3e/d5a/d8d/fb9 x:0 0 0 2024-05-25T06:11:42.210 INFO:tasks.workunit.client.0.smithi160.stdout:0/662: readlink d2b/d46/lbb 0 2024-05-25T06:11:42.210 INFO:tasks.workunit.client.1.smithi203.stdout:0/869: mkdir dd/d5c/d63/d106/d4f/db7/d132 0 2024-05-25T06:11:42.212 INFO:tasks.workunit.client.0.smithi160.stdout:1/484: symlink de/d21/d49/d1c/l9e 0 2024-05-25T06:11:42.214 INFO:tasks.workunit.client.1.smithi203.stdout:0/870: fdatasync dd/d5c/d63/d106/df3/f7f 0 2024-05-25T06:11:42.215 INFO:tasks.workunit.client.0.smithi160.stdout:0/663: creat d2b/d31/d35/d36/fd3 x:0 0 0 2024-05-25T06:11:42.219 INFO:tasks.workunit.client.0.smithi160.stdout:0/664: creat d2b/d46/d63/da7/d67/db8/fd4 x:0 0 0 2024-05-25T06:11:42.220 INFO:tasks.workunit.client.0.smithi160.stdout:1/485: dwrite de/d21/d49/d1c/d59/f95 [0,4194304] 0 2024-05-25T06:11:42.221 INFO:tasks.workunit.client.1.smithi203.stdout:9/809: sync 2024-05-25T06:11:42.221 INFO:tasks.workunit.client.1.smithi203.stdout:1/801: sync 2024-05-25T06:11:42.223 INFO:tasks.workunit.client.0.smithi160.stdout:1/486: write de/d21/d49/d1c/d59/d7c/d93/f84 [631005,79594] 0 2024-05-25T06:11:42.223 INFO:tasks.workunit.client.0.smithi160.stdout:0/665: mkdir d2b/d31/d35/d36/d86/dd5 0 2024-05-25T06:11:42.229 INFO:tasks.workunit.client.1.smithi203.stdout:1/802: mknod d14/d57/db7/dd2/dca/dcf/df7/c116 0 2024-05-25T06:11:42.229 INFO:tasks.workunit.client.0.smithi160.stdout:1/487: symlink de/d21/d49/d1c/d59/d67/l9f 0 2024-05-25T06:11:42.231 INFO:tasks.workunit.client.0.smithi160.stdout:2/545: truncate d12/d5c/f6b 378416 0 2024-05-25T06:11:42.232 INFO:tasks.workunit.client.0.smithi160.stdout:1/488: stat de/d21/c61 0 2024-05-25T06:11:42.233 INFO:tasks.workunit.client.1.smithi203.stdout:9/810: write d0/d17/d23/f8e [898880,28171] 0 2024-05-25T06:11:42.233 INFO:tasks.workunit.client.0.smithi160.stdout:0/666: dwrite d2b/d46/d63/da7/f8c [0,4194304] 0 2024-05-25T06:11:42.233 INFO:tasks.workunit.client.0.smithi160.stdout:1/489: fsync fa 0 2024-05-25T06:11:42.234 INFO:tasks.workunit.client.1.smithi203.stdout:9/811: chown d0/d57/de0 89399 1 2024-05-25T06:11:42.236 INFO:tasks.workunit.client.0.smithi160.stdout:2/546: dread d12/d13/f60 [0,4194304] 0 2024-05-25T06:11:42.238 INFO:tasks.workunit.client.0.smithi160.stdout:0/667: mkdir d2b/d46/d63/db3/dd6 0 2024-05-25T06:11:42.240 INFO:tasks.workunit.client.0.smithi160.stdout:0/668: write d2b/d31/d35/d36/d86/f9b [290203,55378] 0 2024-05-25T06:11:42.240 INFO:tasks.workunit.client.0.smithi160.stdout:1/490: creat de/d21/d49/d1c/d59/d67/fa0 x:0 0 0 2024-05-25T06:11:42.242 INFO:tasks.workunit.client.1.smithi203.stdout:9/812: creat d0/df/d37/d4f/f109 x:0 0 0 2024-05-25T06:11:42.242 INFO:tasks.workunit.client.1.smithi203.stdout:0/871: dwrite dd/d5c/d63/d106/d4f/d62/d78/fdb [0,4194304] 0 2024-05-25T06:11:42.244 INFO:tasks.workunit.client.0.smithi160.stdout:1/491: unlink de/d21/d49/d35/c57 0 2024-05-25T06:11:42.245 INFO:tasks.workunit.client.1.smithi203.stdout:0/872: chown dd/f23 107 1 2024-05-25T06:11:42.249 INFO:tasks.workunit.client.1.smithi203.stdout:0/873: symlink dd/d5c/d63/d106/df3/d10c/l133 0 2024-05-25T06:11:42.250 INFO:tasks.workunit.client.1.smithi203.stdout:0/874: rename dd/d32 to dd/d32/d134 22 2024-05-25T06:11:42.253 INFO:tasks.workunit.client.1.smithi203.stdout:9/813: link d0/l7f d0/d17/d20/d6e/d75/df0/l10a 0 2024-05-25T06:11:42.253 INFO:tasks.workunit.client.0.smithi160.stdout:1/492: getdents de 0 2024-05-25T06:11:42.254 INFO:tasks.workunit.client.1.smithi203.stdout:9/814: write d0/d57/de0/f103 [199861,98516] 0 2024-05-25T06:11:42.255 INFO:tasks.workunit.client.1.smithi203.stdout:9/815: stat d0/d17/d20/d3c/l62 0 2024-05-25T06:11:42.257 INFO:tasks.workunit.client.1.smithi203.stdout:0/875: creat dd/d5c/d63/d106/da0/db9/df9/f135 x:0 0 0 2024-05-25T06:11:42.258 INFO:tasks.workunit.client.0.smithi160.stdout:0/669: dread d2b/f42 [0,4194304] 0 2024-05-25T06:11:42.258 INFO:tasks.workunit.client.0.smithi160.stdout:1/493: mkdir de/d21/d49/d1c/d59/d7f/da1 0 2024-05-25T06:11:42.258 INFO:tasks.workunit.client.1.smithi203.stdout:9/816: creat d0/d17/d20/d6e/f10b x:0 0 0 2024-05-25T06:11:42.259 INFO:tasks.workunit.client.0.smithi160.stdout:2/547: dread f9 [0,4194304] 0 2024-05-25T06:11:42.260 INFO:tasks.workunit.client.1.smithi203.stdout:0/876: rmdir dd/d5c/d63/d106/d75/d121 39 2024-05-25T06:11:42.260 INFO:tasks.workunit.client.0.smithi160.stdout:0/670: dread - d2b/d46/d63/da7/fc8 zero size 2024-05-25T06:11:42.262 INFO:tasks.workunit.client.1.smithi203.stdout:9/817: creat d0/df/d37/da8/f10c x:0 0 0 2024-05-25T06:11:42.265 INFO:tasks.workunit.client.1.smithi203.stdout:0/877: symlink dd/d5c/d63/l136 0 2024-05-25T06:11:42.268 INFO:tasks.workunit.client.1.smithi203.stdout:9/818: unlink d0/df/d73/ff1 0 2024-05-25T06:11:42.268 INFO:tasks.workunit.client.0.smithi160.stdout:1/494: getdents de/d21/d49 0 2024-05-25T06:11:42.268 INFO:tasks.workunit.client.1.smithi203.stdout:9/819: dread - d0/dd0/fdb zero size 2024-05-25T06:11:42.270 INFO:tasks.workunit.client.0.smithi160.stdout:1/495: write de/d21/d49/d1c/d59/d7c/d93/f84 [352471,126108] 0 2024-05-25T06:11:42.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.271+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feca810d810 con 0x7fec9806b860 2024-05-25T06:11:42.271 INFO:tasks.workunit.client.0.smithi160.stdout:2/548: write d12/d5c/f72 [1311916,118553] 0 2024-05-25T06:11:42.272 INFO:tasks.workunit.client.0.smithi160.stdout:0/671: link d2b/d31/d35/d36/f84 d2b/d46/fd7 0 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.273+0000 7feca57fa700 1 -- 172.21.15.160:0/3839213594 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7feca810d810 con 0x7fec9806b860 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:11:42.273 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "1/2 daemons upgraded", 2024-05-25T06:11:42.274 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "", 2024-05-25T06:11:42.274 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:11:42.274 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:11:42.274 INFO:tasks.workunit.client.1.smithi203.stdout:0/878: getdents dd/d5c/d63/d106/d4f/d62/d78/d100 0 2024-05-25T06:11:42.274 INFO:tasks.workunit.client.0.smithi160.stdout:5/682: sync 2024-05-25T06:11:42.275 INFO:tasks.workunit.client.1.smithi203.stdout:9/820: dwrite d0/df/d37/f70 [0,4194304] 0 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.275+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fec9806b860 msgr2=0x7fec9806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.275+0000 7fecadda0700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fec9806b860 0x7fec9806dd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec94000c00 tx=0x7fec94009e30 comp rx=0 tx=0).stop 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 msgr2=0x7feca810b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 0x7feca810b770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00eb70 tx=0x7fec9c00b040 comp rx=0 tx=0).stop 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 shutdown_connections 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fec9806b860 0x7fec9806dd10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca80fbaf0 0x7feca810b770 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 --2- 172.21.15.160:0/3839213594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca810bcb0 0x7feca8111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:11:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.276+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 >> 172.21.15.160:0/3839213594 conn(0x7feca80f7250 msgr2=0x7feca8065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:11:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.277+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 shutdown_connections 2024-05-25T06:11:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:11:42.277+0000 7fecadda0700 1 -- 172.21.15.160:0/3839213594 wait complete. 2024-05-25T06:11:42.279 INFO:tasks.workunit.client.1.smithi203.stdout:9/821: stat d0/f1e 0 2024-05-25T06:11:42.281 INFO:tasks.workunit.client.1.smithi203.stdout:0/879: symlink dd/d5c/d63/d106/df3/l137 0 2024-05-25T06:11:42.281 INFO:tasks.workunit.client.0.smithi160.stdout:5/683: mknod db/dc/d13/d1d/d31/cd3 0 2024-05-25T06:11:42.283 INFO:tasks.workunit.client.0.smithi160.stdout:2/549: truncate d12/d3e/f41 165950 0 2024-05-25T06:11:42.284 INFO:tasks.workunit.client.0.smithi160.stdout:0/672: getdents d2b/d46/d63/da7/db5 0 2024-05-25T06:11:42.284 INFO:tasks.workunit.client.1.smithi203.stdout:9/822: mknod d0/df/d2d/c10d 0 2024-05-25T06:11:42.286 INFO:tasks.workunit.client.0.smithi160.stdout:5/684: symlink db/dc/d13/d1d/d23/d24/daf/ld4 0 2024-05-25T06:11:42.288 INFO:tasks.workunit.client.0.smithi160.stdout:2/550: symlink d12/d3e/d5a/lba 0 2024-05-25T06:11:42.289 INFO:tasks.workunit.client.0.smithi160.stdout:2/551: read d12/d3c/f61 [2982603,80] 0 2024-05-25T06:11:42.289 INFO:tasks.workunit.client.0.smithi160.stdout:2/552: chown d12/d5c 3 1 2024-05-25T06:11:42.290 INFO:tasks.workunit.client.0.smithi160.stdout:2/553: chown d12/d1c/d59/c83 7159 1 2024-05-25T06:11:42.290 INFO:tasks.workunit.client.0.smithi160.stdout:0/673: unlink d2b/d31/d35/d36/d37/f44 0 2024-05-25T06:11:42.292 INFO:tasks.workunit.client.1.smithi203.stdout:9/823: dwrite d0/d17/d20/d29/ff4 [0,4194304] 0 2024-05-25T06:11:42.293 INFO:tasks.workunit.client.1.smithi203.stdout:9/824: write d0/f39 [669128,30702] 0 2024-05-25T06:11:42.295 INFO:tasks.workunit.client.0.smithi160.stdout:2/554: dread f9 [4194304,4194304] 0 2024-05-25T06:11:42.296 INFO:tasks.workunit.client.0.smithi160.stdout:1/496: dread de/d21/d49/d35/f40 [0,4194304] 0 2024-05-25T06:11:42.297 INFO:tasks.workunit.client.0.smithi160.stdout:5/685: getdents db/dc/d13/d1d/d23/d45 0 2024-05-25T06:11:42.297 INFO:tasks.workunit.client.0.smithi160.stdout:1/497: readlink de/d21/d49/d28/l48 0 2024-05-25T06:11:42.300 INFO:tasks.workunit.client.0.smithi160.stdout:1/498: write de/d21/d49/d28/f65 [87802,26828] 0 2024-05-25T06:11:42.301 INFO:tasks.workunit.client.0.smithi160.stdout:2/555: mknod d12/d1c/d2a/d7e/d90/cbb 0 2024-05-25T06:11:42.305 INFO:tasks.workunit.client.1.smithi203.stdout:9/825: link d0/df/d37/fb7 d0/df/d37/f10e 0 2024-05-25T06:11:42.308 INFO:tasks.workunit.client.0.smithi160.stdout:1/499: chown de/d21/d49/d1c/d59/d7c/f97 231771 1 2024-05-25T06:11:42.312 INFO:tasks.workunit.client.0.smithi160.stdout:1/500: write de/d21/d49/d1c/d59/d67/fa0 [222584,130082] 0 2024-05-25T06:11:42.312 INFO:tasks.workunit.client.0.smithi160.stdout:2/556: getdents d12/d5c 0 2024-05-25T06:11:42.315 INFO:tasks.workunit.client.1.smithi203.stdout:0/880: dwrite dd/d10/d34/d4c/fd0 [0,4194304] 0 2024-05-25T06:11:42.316 INFO:tasks.workunit.client.0.smithi160.stdout:5/686: dwrite db/dc/d13/f50 [0,4194304] 0 2024-05-25T06:11:42.317 INFO:tasks.workunit.client.1.smithi203.stdout:0/881: dread - dd/d5c/d63/d106/d75/fcf zero size 2024-05-25T06:11:42.317 INFO:tasks.workunit.client.0.smithi160.stdout:2/557: unlink d12/d3e/d5f/d74/d7a/l91 0 2024-05-25T06:11:42.318 INFO:tasks.workunit.client.1.smithi203.stdout:9/826: write d0/d17/d20/d6e/d75/f98 [1797500,119139] 0 2024-05-25T06:11:42.318 INFO:tasks.workunit.client.1.smithi203.stdout:0/882: write dd/d45/f5b [4959223,109132] 0 2024-05-25T06:11:42.320 INFO:tasks.workunit.client.0.smithi160.stdout:1/501: creat de/d21/d49/fa2 x:0 0 0 2024-05-25T06:11:42.322 INFO:tasks.workunit.client.0.smithi160.stdout:2/558: rmdir d12/d13 39 2024-05-25T06:11:42.323 INFO:tasks.workunit.client.0.smithi160.stdout:2/559: chown d12/d1c/d2a/d7e/fa7 15 1 2024-05-25T06:11:42.325 INFO:tasks.workunit.client.1.smithi203.stdout:0/883: mknod dd/d5c/d120/c138 0 2024-05-25T06:11:42.325 INFO:tasks.workunit.client.0.smithi160.stdout:5/687: link db/dc/d13/d1d/c6b db/dc/d13/d1d/d3f/dc0/cd5 0 2024-05-25T06:11:42.328 INFO:tasks.workunit.client.0.smithi160.stdout:2/560: symlink d12/lbc 0 2024-05-25T06:11:42.329 INFO:tasks.workunit.client.1.smithi203.stdout:0/884: mknod dd/d5c/d63/d106/d4f/d62/d78/c139 0 2024-05-25T06:11:42.329 INFO:tasks.workunit.client.0.smithi160.stdout:2/561: dread - d12/d3e/d5f/d74/fb2 zero size 2024-05-25T06:11:42.329 INFO:tasks.workunit.client.0.smithi160.stdout:2/562: dread - d12/d3e/d5a/f93 zero size 2024-05-25T06:11:42.330 INFO:tasks.workunit.client.1.smithi203.stdout:7/904: sync 2024-05-25T06:11:42.332 INFO:tasks.workunit.client.1.smithi203.stdout:7/905: dread - d1/d9/d2c/d118/d127/d10b/f120 zero size 2024-05-25T06:11:42.332 INFO:tasks.workunit.client.0.smithi160.stdout:2/563: creat d12/d5c/fbd x:0 0 0 2024-05-25T06:11:42.332 INFO:tasks.workunit.client.1.smithi203.stdout:9/827: write d0/f1e [1028048,125822] 0 2024-05-25T06:11:42.333 INFO:tasks.workunit.client.0.smithi160.stdout:2/564: readlink d12/d3c/l9a 0 2024-05-25T06:11:42.333 INFO:tasks.workunit.client.0.smithi160.stdout:2/565: dread - d12/f8f zero size 2024-05-25T06:11:42.334 INFO:tasks.workunit.client.1.smithi203.stdout:0/885: getdents dd/d5c/d63/d106/d4f/db7 0 2024-05-25T06:11:42.335 INFO:tasks.workunit.client.0.smithi160.stdout:2/566: symlink d12/d97/lbe 0 2024-05-25T06:11:42.337 INFO:tasks.workunit.client.0.smithi160.stdout:2/567: chown d12/d1c/d2a/d7e/d90/ca5 4377253 1 2024-05-25T06:11:42.337 INFO:tasks.workunit.client.1.smithi203.stdout:0/886: creat dd/d5c/d63/d106/df3/d10c/f13a x:0 0 0 2024-05-25T06:11:42.337 INFO:tasks.workunit.client.1.smithi203.stdout:9/828: creat d0/df/d37/da8/dd6/f10f x:0 0 0 2024-05-25T06:11:42.339 INFO:tasks.workunit.client.1.smithi203.stdout:7/906: unlink d1/d9/d2c/d3a/d4e/l112 0 2024-05-25T06:11:42.339 INFO:tasks.workunit.client.0.smithi160.stdout:2/568: symlink d12/d75/lbf 0 2024-05-25T06:11:42.340 INFO:tasks.workunit.client.1.smithi203.stdout:0/887: write dd/d5c/d63/d106/d59/f83 [4296428,119278] 0 2024-05-25T06:11:42.340 INFO:tasks.workunit.client.1.smithi203.stdout:9/829: creat d0/dd0/f110 x:0 0 0 2024-05-25T06:11:42.341 INFO:tasks.workunit.client.0.smithi160.stdout:5/688: dread f9 [0,4194304] 0 2024-05-25T06:11:42.342 INFO:tasks.workunit.client.0.smithi160.stdout:5/689: write db/dc/d13/f2c [2068313,94107] 0 2024-05-25T06:11:42.343 INFO:tasks.workunit.client.1.smithi203.stdout:0/888: symlink dd/d45/d96/l13b 0 2024-05-25T06:11:42.343 INFO:tasks.workunit.client.0.smithi160.stdout:1/502: write de/f3d [717093,47529] 0 2024-05-25T06:11:42.345 INFO:tasks.workunit.client.0.smithi160.stdout:2/569: rename d12/d3e/c64 to d12/d1c/d37/cc0 0 2024-05-25T06:11:42.348 INFO:tasks.workunit.client.0.smithi160.stdout:2/570: write d12/d3e/d5a/d8d/fb8 [790390,28190] 0 2024-05-25T06:11:42.349 INFO:tasks.workunit.client.1.smithi203.stdout:0/889: mknod dd/d32/d37/c13c 0 2024-05-25T06:11:42.349 INFO:tasks.workunit.client.0.smithi160.stdout:5/690: getdents db/dc/d13/d1d/d31/d3b 0 2024-05-25T06:11:42.350 INFO:tasks.workunit.client.1.smithi203.stdout:9/830: link d0/d17/d20/d3c/f7d d0/d17/d20/d6e/d75/df2/f111 0 2024-05-25T06:11:42.351 INFO:tasks.workunit.client.1.smithi203.stdout:0/890: chown dd/d5c/d63/d106/d4f/db7 0 1 2024-05-25T06:11:42.351 INFO:tasks.workunit.client.0.smithi160.stdout:1/503: dwrite de/d21/f31 [0,4194304] 0 2024-05-25T06:11:42.351 INFO:tasks.workunit.client.0.smithi160.stdout:2/571: unlink d12/d3c/l9a 0 2024-05-25T06:11:42.351 INFO:tasks.workunit.client.0.smithi160.stdout:5/691: chown db/dc/d13/d1d/d23/d45/f5f 15694999 1 2024-05-25T06:11:42.352 INFO:tasks.workunit.client.1.smithi203.stdout:9/831: chown d0/d17/d20/d6e/d75/fc0 37 1 2024-05-25T06:11:42.352 INFO:tasks.workunit.client.0.smithi160.stdout:5/692: chown db/f53 13168223 1 2024-05-25T06:11:42.353 INFO:tasks.workunit.client.1.smithi203.stdout:0/891: rmdir dd/d32/d108 39 2024-05-25T06:11:42.355 INFO:tasks.workunit.client.1.smithi203.stdout:9/832: readlink d0/df/d37/lc4 0 2024-05-25T06:11:42.357 INFO:tasks.workunit.client.1.smithi203.stdout:7/907: dread d1/d9/f29 [4194304,4194304] 0 2024-05-25T06:11:42.357 INFO:tasks.workunit.client.0.smithi160.stdout:5/693: rmdir db/dc/d13/d1d/d23/d24/daf/db3 39 2024-05-25T06:11:42.358 INFO:tasks.workunit.client.0.smithi160.stdout:5/694: chown db/dc/d13/d1d/d3f/d72 246360267 1 2024-05-25T06:11:42.358 INFO:tasks.workunit.client.1.smithi203.stdout:0/892: creat dd/d5c/d63/f13d x:0 0 0 2024-05-25T06:11:42.359 INFO:tasks.workunit.client.1.smithi203.stdout:7/908: read - d1/d9/d2c/d62/db3/f111 zero size 2024-05-25T06:11:42.360 INFO:tasks.workunit.client.1.smithi203.stdout:9/833: creat d0/d17/f112 x:0 0 0 2024-05-25T06:11:42.361 INFO:tasks.workunit.client.0.smithi160.stdout:1/504: mknod de/d21/d49/d1c/d59/d71/ca3 0 2024-05-25T06:11:42.362 INFO:tasks.workunit.client.1.smithi203.stdout:7/909: truncate d1/d9/fe 654047 0 2024-05-25T06:11:42.362 INFO:tasks.workunit.client.1.smithi203.stdout:9/834: write d0/df/d37/ff3 [997536,109034] 0 2024-05-25T06:11:42.362 INFO:tasks.workunit.client.1.smithi203.stdout:0/893: symlink dd/d5c/d63/d106/da0/db9/df9/l13e 0 2024-05-25T06:11:42.363 INFO:tasks.workunit.client.0.smithi160.stdout:5/695: rmdir db/dc/d13/d1d/d3f/d4a/dbe 0 2024-05-25T06:11:42.364 INFO:tasks.workunit.client.0.smithi160.stdout:1/505: symlink de/d21/d49/d1c/d59/d7c/la4 0 2024-05-25T06:11:42.365 INFO:tasks.workunit.client.1.smithi203.stdout:7/910: symlink d1/d9/dc2/d12c/l132 0 2024-05-25T06:11:42.365 INFO:tasks.workunit.client.0.smithi160.stdout:5/696: readlink db/dc/d13/d1d/d3f/d4a/lbf 0 2024-05-25T06:11:42.367 INFO:tasks.workunit.client.0.smithi160.stdout:1/506: rmdir de/d21/d49/d1c/d41 39 2024-05-25T06:11:42.368 INFO:tasks.workunit.client.1.smithi203.stdout:9/835: dwrite d0/f1e [0,4194304] 0 2024-05-25T06:11:42.369 INFO:tasks.workunit.client.1.smithi203.stdout:9/836: chown d0/c10 91451590 1 2024-05-25T06:11:42.374 INFO:tasks.workunit.client.0.smithi160.stdout:5/697: creat db/dc/d13/d1d/d23/d24/daf/db3/fd6 x:0 0 0 2024-05-25T06:11:42.375 INFO:tasks.workunit.client.0.smithi160.stdout:5/698: chown db/dc/d13/d1d/d23/d24/l89 58674247 1 2024-05-25T06:11:42.375 INFO:tasks.workunit.client.0.smithi160.stdout:5/699: fsync db/fa9 0 2024-05-25T06:11:42.377 INFO:tasks.workunit.client.0.smithi160.stdout:1/507: creat de/d21/d49/d1c/fa5 x:0 0 0 2024-05-25T06:11:42.379 INFO:tasks.workunit.client.0.smithi160.stdout:5/700: creat db/dc/fd7 x:0 0 0 2024-05-25T06:11:42.380 INFO:tasks.workunit.client.0.smithi160.stdout:2/572: dwrite d12/d13/f24 [0,4194304] 0 2024-05-25T06:11:42.382 INFO:tasks.workunit.client.0.smithi160.stdout:5/701: creat db/dc/d13/d1d/d23/dbd/fd8 x:0 0 0 2024-05-25T06:11:42.383 INFO:tasks.workunit.client.0.smithi160.stdout:2/573: mkdir d12/d5c/d85/dc1 0 2024-05-25T06:11:42.386 INFO:tasks.workunit.client.0.smithi160.stdout:2/574: creat d12/d3c/fc2 x:0 0 0 2024-05-25T06:11:42.387 INFO:tasks.workunit.client.0.smithi160.stdout:1/508: dwrite de/d21/d49/d1c/d59/d71/f68 [0,4194304] 0 2024-05-25T06:11:42.388 INFO:tasks.workunit.client.0.smithi160.stdout:2/575: stat d12/d13/f15 0 2024-05-25T06:11:42.388 INFO:tasks.workunit.client.1.smithi203.stdout:9/837: rename d0/d17/d20/d6e/d75/df0 to d0/df/ddd/d113 0 2024-05-25T06:11:42.390 INFO:tasks.workunit.client.0.smithi160.stdout:5/702: dwrite db/f22 [0,4194304] 0 2024-05-25T06:11:42.392 INFO:tasks.workunit.client.0.smithi160.stdout:2/576: creat d12/d3e/db5/fc3 x:0 0 0 2024-05-25T06:11:42.393 INFO:tasks.workunit.client.1.smithi203.stdout:7/911: dwrite d1/d9/d2c/d62/d9b/fd5 [0,4194304] 0 2024-05-25T06:11:42.395 INFO:tasks.workunit.client.1.smithi203.stdout:9/838: fsync d0/df/d37/da8/db8/fb9 0 2024-05-25T06:11:42.396 INFO:tasks.workunit.client.0.smithi160.stdout:1/509: dwrite f5 [0,4194304] 0 2024-05-25T06:11:42.397 INFO:tasks.workunit.client.0.smithi160.stdout:1/510: chown cc 121 1 2024-05-25T06:11:42.397 INFO:tasks.workunit.client.1.smithi203.stdout:7/912: rmdir d1/d9/d2c/d62/db3/de5 39 2024-05-25T06:11:42.398 INFO:tasks.workunit.client.0.smithi160.stdout:1/511: write de/f3a [675450,103113] 0 2024-05-25T06:11:42.400 INFO:tasks.workunit.client.0.smithi160.stdout:5/703: mknod db/dc/d13/d1d/d3f/d72/d98/cd9 0 2024-05-25T06:11:42.406 INFO:tasks.workunit.client.0.smithi160.stdout:5/704: read db/f53 [3834835,55433] 0 2024-05-25T06:11:42.406 INFO:tasks.workunit.client.0.smithi160.stdout:2/577: fdatasync d12/d1c/f31 0 2024-05-25T06:11:42.410 INFO:tasks.workunit.client.0.smithi160.stdout:2/578: truncate d12/d3e/d5f/d74/fb2 464575 0 2024-05-25T06:11:42.410 INFO:tasks.workunit.client.0.smithi160.stdout:1/512: dwrite de/d21/d49/d28/f5b [0,4194304] 0 2024-05-25T06:11:42.412 INFO:tasks.workunit.client.0.smithi160.stdout:2/579: write d12/d13/f15 [1637359,40241] 0 2024-05-25T06:11:42.413 INFO:tasks.workunit.client.0.smithi160.stdout:2/580: write d12/d1c/f56 [119667,3898] 0 2024-05-25T06:11:42.420 INFO:tasks.workunit.client.0.smithi160.stdout:2/581: dwrite d12/d1c/d2a/f80 [0,4194304] 0 2024-05-25T06:11:42.423 INFO:tasks.workunit.client.0.smithi160.stdout:1/513: read de/d21/d49/f2b [1576936,49433] 0 2024-05-25T06:11:42.427 INFO:tasks.workunit.client.0.smithi160.stdout:1/514: readlink de/d21/d49/d1c/d41/d45/l58 0 2024-05-25T06:11:42.430 INFO:tasks.workunit.client.0.smithi160.stdout:1/515: write de/d21/d49/d1c/fa5 [940589,31544] 0 2024-05-25T06:11:42.433 INFO:tasks.workunit.client.0.smithi160.stdout:2/582: dwrite d12/d3e/d5f/f63 [0,4194304] 0 2024-05-25T06:11:42.437 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:42 smithi203 ceph-mon[36030]: pgmap v21: 65 pgs: 65 active+clean; 2.5 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 90 MiB/s rd, 255 MiB/s wr, 471 op/s 2024-05-25T06:11:42.437 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:42 smithi203 ceph-mon[36030]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:42.437 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:42 smithi203 ceph-mon[36030]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:42.437 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:42 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2595891350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:11:42.438 INFO:tasks.workunit.client.0.smithi160.stdout:2/583: write d12/d13/f60 [1121960,43025] 0 2024-05-25T06:11:42.438 INFO:tasks.workunit.client.0.smithi160.stdout:3/500: sync 2024-05-25T06:11:42.438 INFO:tasks.workunit.client.0.smithi160.stdout:6/562: sync 2024-05-25T06:11:42.438 INFO:tasks.workunit.client.0.smithi160.stdout:4/580: sync 2024-05-25T06:11:42.440 INFO:tasks.workunit.client.0.smithi160.stdout:1/516: dwrite de/d21/d49/d1c/d63/f7b [0,4194304] 0 2024-05-25T06:11:42.441 INFO:tasks.workunit.client.0.smithi160.stdout:2/584: dread - d12/d5c/d85/fb6 zero size 2024-05-25T06:11:42.446 INFO:tasks.workunit.client.0.smithi160.stdout:1/517: write de/f54 [65735,81577] 0 2024-05-25T06:11:42.446 INFO:tasks.workunit.client.0.smithi160.stdout:4/581: mknod d4/d6/d21/d42/cb3 0 2024-05-25T06:11:42.448 INFO:tasks.workunit.client.0.smithi160.stdout:5/705: dread db/fa9 [0,4194304] 0 2024-05-25T06:11:42.451 INFO:tasks.workunit.client.0.smithi160.stdout:6/563: link d5/de/d32/d35/f82 d5/de/d32/d9f/fc0 0 2024-05-25T06:11:42.452 INFO:tasks.workunit.client.0.smithi160.stdout:1/518: rename de/d21/d49/d1c/d59/d7c/d93/f46 to de/d21/fa6 0 2024-05-25T06:11:42.453 INFO:tasks.workunit.client.0.smithi160.stdout:4/582: symlink d4/d6/d5b/d66/d78/da1/lb4 0 2024-05-25T06:11:42.455 INFO:tasks.workunit.client.0.smithi160.stdout:5/706: write db/dc/d13/d1d/d23/d24/daf/f95 [1698001,82659] 0 2024-05-25T06:11:42.455 INFO:tasks.workunit.client.0.smithi160.stdout:2/585: dwrite d12/d1c/d2a/d7e/fa2 [0,4194304] 0 2024-05-25T06:11:42.455 INFO:tasks.workunit.client.1.smithi203.stdout:1/803: sync 2024-05-25T06:11:42.456 INFO:tasks.workunit.client.0.smithi160.stdout:6/564: fdatasync d5/d45/d63/f83 0 2024-05-25T06:11:42.456 INFO:tasks.workunit.client.0.smithi160.stdout:4/583: read d4/d6/d5b/d66/f71 [1820263,47811] 0 2024-05-25T06:11:42.457 INFO:tasks.workunit.client.0.smithi160.stdout:6/565: readlink d5/d30/d36/l33 0 2024-05-25T06:11:42.459 INFO:tasks.workunit.client.0.smithi160.stdout:1/519: mknod de/d21/d49/d1c/d59/d7f/da1/ca7 0 2024-05-25T06:11:42.460 INFO:tasks.workunit.client.0.smithi160.stdout:6/566: mkdir d5/d39/dc1 0 2024-05-25T06:11:42.462 INFO:tasks.workunit.client.0.smithi160.stdout:4/584: read d4/d6/f7 [619347,113925] 0 2024-05-25T06:11:42.464 INFO:tasks.workunit.client.0.smithi160.stdout:5/707: rename db/dc/d13/d1d/d23/dbd/fd8 to db/d47/d81/dc1/fda 0 2024-05-25T06:11:42.466 INFO:tasks.workunit.client.0.smithi160.stdout:6/567: symlink d5/d39/d8f/lc2 0 2024-05-25T06:11:42.467 INFO:tasks.workunit.client.0.smithi160.stdout:2/586: dwrite d12/d3c/fc2 [0,4194304] 0 2024-05-25T06:11:42.467 INFO:tasks.workunit.client.0.smithi160.stdout:5/708: truncate db/f63 9555922 0 2024-05-25T06:11:42.468 INFO:tasks.workunit.client.0.smithi160.stdout:5/709: chown db/dc/d13 124065 1 2024-05-25T06:11:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:42 smithi160 ceph-mon[29041]: pgmap v21: 65 pgs: 65 active+clean; 2.5 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 90 MiB/s rd, 255 MiB/s wr, 471 op/s 2024-05-25T06:11:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:42 smithi160 ceph-mon[29041]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:42 smithi160 ceph-mon[29041]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:42 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2595891350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:11:42.471 INFO:tasks.workunit.client.0.smithi160.stdout:1/520: link de/d21/d49/d1c/d59/d7c/d93/f84 de/d21/d49/d1c/d41/d96/fa8 0 2024-05-25T06:11:42.472 INFO:tasks.workunit.client.0.smithi160.stdout:6/568: mkdir d5/de/d32/d35/dc3 0 2024-05-25T06:11:42.475 INFO:tasks.workunit.client.0.smithi160.stdout:2/587: dwrite d12/d3e/d5a/d8d/fb8 [0,4194304] 0 2024-05-25T06:11:42.476 INFO:tasks.workunit.client.0.smithi160.stdout:3/501: dread d19/d2f/d38/f3e [0,4194304] 0 2024-05-25T06:11:42.477 INFO:tasks.workunit.client.0.smithi160.stdout:2/588: chown d12/d1c/d37/d4b/f79 22553 1 2024-05-25T06:11:42.477 INFO:tasks.workunit.client.0.smithi160.stdout:5/710: unlink db/dc/d13/d1d/d23/f59 0 2024-05-25T06:11:42.478 INFO:tasks.workunit.client.0.smithi160.stdout:3/502: rename d19 to d19/d2f/d38/d43/d5b/d65/daa 22 2024-05-25T06:11:42.480 INFO:tasks.workunit.client.0.smithi160.stdout:2/589: truncate d12/d5c/d85/fb6 996329 0 2024-05-25T06:11:42.480 INFO:tasks.workunit.client.0.smithi160.stdout:2/590: write d12/f9f [1027507,11548] 0 2024-05-25T06:11:42.482 INFO:tasks.workunit.client.0.smithi160.stdout:2/591: dread - d12/f8f zero size 2024-05-25T06:11:42.483 INFO:tasks.workunit.client.0.smithi160.stdout:2/592: readlink d12/d3e/l98 0 2024-05-25T06:11:42.485 INFO:tasks.workunit.client.0.smithi160.stdout:5/711: read db/dc/d13/d1d/d23/f46 [4155429,1719] 0 2024-05-25T06:11:42.485 INFO:tasks.workunit.client.0.smithi160.stdout:3/503: dwrite d19/d2f/d38/f6b [0,4194304] 0 2024-05-25T06:11:42.487 INFO:tasks.workunit.client.0.smithi160.stdout:1/521: link de/d21/d49/d1c/d41/l8b de/d21/d49/la9 0 2024-05-25T06:11:42.490 INFO:tasks.workunit.client.0.smithi160.stdout:6/569: getdents d5/de 0 2024-05-25T06:11:42.490 INFO:tasks.workunit.client.0.smithi160.stdout:1/522: write f5 [2434793,41546] 0 2024-05-25T06:11:42.491 INFO:tasks.workunit.client.0.smithi160.stdout:5/712: unlink db/dc/d13/c25 0 2024-05-25T06:11:42.492 INFO:tasks.workunit.client.0.smithi160.stdout:5/713: write db/dc/d13/d1d/d23/f46 [175331,53140] 0 2024-05-25T06:11:42.494 INFO:tasks.workunit.client.0.smithi160.stdout:2/593: unlink d12/d3e/f68 0 2024-05-25T06:11:42.497 INFO:tasks.workunit.client.0.smithi160.stdout:3/504: dread d19/d2f/d38/d47/f4c [0,4194304] 0 2024-05-25T06:11:42.502 INFO:tasks.workunit.client.0.smithi160.stdout:5/714: rmdir db/dc/d13/d1d/d23/d24/daf/db3 39 2024-05-25T06:11:42.503 INFO:tasks.workunit.client.1.smithi203.stdout:9/839: write d0/f36 [714961,41203] 0 2024-05-25T06:11:42.505 INFO:tasks.workunit.client.0.smithi160.stdout:2/594: unlink d12/f25 0 2024-05-25T06:11:42.506 INFO:tasks.workunit.client.0.smithi160.stdout:2/595: chown d12/d3e/d5a/d8d/fb8 2 1 2024-05-25T06:11:42.507 INFO:tasks.workunit.client.0.smithi160.stdout:1/523: fsync de/d21/d49/d28/f3c 0 2024-05-25T06:11:42.510 INFO:tasks.workunit.client.0.smithi160.stdout:6/570: mkdir d5/d45/d4a/dbc/dc4 0 2024-05-25T06:11:42.518 INFO:tasks.workunit.client.0.smithi160.stdout:5/715: dread db/d47/f68 [0,4194304] 0 2024-05-25T06:11:42.518 INFO:tasks.workunit.client.1.smithi203.stdout:7/913: chown d1/d9/d2c/d62/d7a/l90 153124 1 2024-05-25T06:11:42.519 INFO:tasks.workunit.client.0.smithi160.stdout:2/596: rename d12/d3e/d5a/lba to d12/d3e/d5f/d74/d7a/lc4 0 2024-05-25T06:11:42.519 INFO:tasks.workunit.client.1.smithi203.stdout:7/914: chown d1/f31 37095 1 2024-05-25T06:11:42.520 INFO:tasks.workunit.client.0.smithi160.stdout:5/716: write db/dc/d13/d1d/d3f/d72/d98/da5/fac [1467385,21259] 0 2024-05-25T06:11:42.522 INFO:tasks.workunit.client.1.smithi203.stdout:9/840: rename d0/d17/d20/d3c to d0/d17/d20/d6e/d75/df2/d114 0 2024-05-25T06:11:42.523 INFO:tasks.workunit.client.0.smithi160.stdout:1/524: write de/d21/d49/f2b [3776105,6612] 0 2024-05-25T06:11:42.523 INFO:tasks.workunit.client.1.smithi203.stdout:1/804: creat d14/d57/db7/dd2/d101/f117 x:0 0 0 2024-05-25T06:11:42.524 INFO:tasks.workunit.client.0.smithi160.stdout:3/505: dread fe [0,4194304] 0 2024-05-25T06:11:42.529 INFO:tasks.workunit.client.1.smithi203.stdout:1/805: chown d14/d81/f8a 599223 1 2024-05-25T06:11:42.529 INFO:tasks.workunit.client.0.smithi160.stdout:5/717: dread db/d47/fcb [0,4194304] 0 2024-05-25T06:11:42.531 INFO:tasks.workunit.client.1.smithi203.stdout:9/841: dwrite d0/d57/de0/f103 [0,4194304] 0 2024-05-25T06:11:42.532 INFO:tasks.workunit.client.0.smithi160.stdout:6/571: symlink d5/de/d32/d74/lc5 0 2024-05-25T06:11:42.533 INFO:tasks.workunit.client.0.smithi160.stdout:5/718: read db/dc/d13/d1d/d31/d3b/d69/f79 [193389,98691] 0 2024-05-25T06:11:42.534 INFO:tasks.workunit.client.1.smithi203.stdout:1/806: rename d14/d57/db7/dd2/db3/l106 to d14/d57/dbb/l118 0 2024-05-25T06:11:42.537 INFO:tasks.workunit.client.1.smithi203.stdout:7/915: dread d1/d9/d2c/d36/d54/fc3 [0,4194304] 0 2024-05-25T06:11:42.539 INFO:tasks.workunit.client.1.smithi203.stdout:1/807: creat d14/d57/db7/dd2/dca/dcf/f119 x:0 0 0 2024-05-25T06:11:42.541 INFO:tasks.workunit.client.1.smithi203.stdout:7/916: creat d1/d9/d2c/d79/df0/f133 x:0 0 0 2024-05-25T06:11:42.545 INFO:tasks.workunit.client.0.smithi160.stdout:2/597: getdents d12/d3e/d5a 0 2024-05-25T06:11:42.547 INFO:tasks.workunit.client.1.smithi203.stdout:7/917: rename d1/f38 to d1/d11/d3d/d4c/de1/f134 0 2024-05-25T06:11:42.547 INFO:tasks.workunit.client.0.smithi160.stdout:1/525: dwrite de/d21/d49/d35/f73 [0,4194304] 0 2024-05-25T06:11:42.548 INFO:tasks.workunit.client.1.smithi203.stdout:7/918: write d1/d9/d2c/d62/d9b/fc9 [99601,79484] 0 2024-05-25T06:11:42.548 INFO:tasks.workunit.client.0.smithi160.stdout:2/598: creat d12/d3e/d5a/fc5 x:0 0 0 2024-05-25T06:11:42.553 INFO:tasks.workunit.client.1.smithi203.stdout:9/842: write d0/df/d37/f10e [2176255,1863] 0 2024-05-25T06:11:42.554 INFO:tasks.workunit.client.0.smithi160.stdout:5/719: write db/dc/f15 [1311479,35615] 0 2024-05-25T06:11:42.557 INFO:tasks.workunit.client.0.smithi160.stdout:3/506: dread d19/d2f/d38/d43/d5b/d65/d7f/f81 [0,4194304] 0 2024-05-25T06:11:42.558 INFO:tasks.workunit.client.0.smithi160.stdout:2/599: mknod d12/d3e/d5f/d74/cc6 0 2024-05-25T06:11:42.558 INFO:tasks.workunit.client.1.smithi203.stdout:9/843: creat d0/df/ddd/d113/f115 x:0 0 0 2024-05-25T06:11:42.559 INFO:tasks.workunit.client.1.smithi203.stdout:7/919: link d1/d9/d60/ca8 d1/d9/d2c/d62/d7a/d89/c135 0 2024-05-25T06:11:42.561 INFO:tasks.workunit.client.0.smithi160.stdout:2/600: read d12/d13/f15 [163692,10740] 0 2024-05-25T06:11:42.562 INFO:tasks.workunit.client.0.smithi160.stdout:6/572: dwrite d5/de/d32/d74/f92 [0,4194304] 0 2024-05-25T06:11:42.563 INFO:tasks.workunit.client.1.smithi203.stdout:7/920: mkdir d1/d9/d136 0 2024-05-25T06:11:42.563 INFO:tasks.workunit.client.0.smithi160.stdout:2/601: dread - d12/d97/f99 zero size 2024-05-25T06:11:42.564 INFO:tasks.workunit.client.1.smithi203.stdout:1/808: dwrite d14/d57/d97/da4/fe8 [0,4194304] 0 2024-05-25T06:11:42.564 INFO:tasks.workunit.client.0.smithi160.stdout:3/507: creat d19/d2f/d38/fab x:0 0 0 2024-05-25T06:11:42.565 INFO:tasks.workunit.client.1.smithi203.stdout:9/844: rename d0/d17/d20/d6e/d75/df2/d114/c66 to d0/d17/db6/c116 0 2024-05-25T06:11:42.566 INFO:tasks.workunit.client.0.smithi160.stdout:6/573: creat d5/d30/fc6 x:0 0 0 2024-05-25T06:11:42.567 INFO:tasks.workunit.client.0.smithi160.stdout:1/526: truncate de/d21/d49/d1c/d59/d71/f5c 1381789 0 2024-05-25T06:11:42.568 INFO:tasks.workunit.client.0.smithi160.stdout:2/602: mknod d12/d47/cc7 0 2024-05-25T06:11:42.570 INFO:tasks.workunit.client.0.smithi160.stdout:2/603: truncate d12/d13/f24 5193617 0 2024-05-25T06:11:42.570 INFO:tasks.workunit.client.1.smithi203.stdout:7/921: rmdir d1/dbd 39 2024-05-25T06:11:42.570 INFO:tasks.workunit.client.0.smithi160.stdout:3/508: mknod d19/d2f/d34/d67/d96/cac 0 2024-05-25T06:11:42.573 INFO:tasks.workunit.client.0.smithi160.stdout:1/527: creat de/d21/d49/d1c/d59/d7f/faa x:0 0 0 2024-05-25T06:11:42.574 INFO:tasks.workunit.client.0.smithi160.stdout:5/720: dread db/dc/d13/d1d/d23/d24/d2e/f4e [0,4194304] 0 2024-05-25T06:11:42.575 INFO:tasks.workunit.client.1.smithi203.stdout:7/922: fsync d1/d9/f48 0 2024-05-25T06:11:42.575 INFO:tasks.workunit.client.0.smithi160.stdout:5/721: chown db/d47/d5e/ca3 92 1 2024-05-25T06:11:42.576 INFO:tasks.workunit.client.0.smithi160.stdout:2/604: rename d12/d1c/d59/f6d to d12/d75/fc8 0 2024-05-25T06:11:42.577 INFO:tasks.workunit.client.0.smithi160.stdout:3/509: mkdir d19/d2f/dad 0 2024-05-25T06:11:42.578 INFO:tasks.workunit.client.0.smithi160.stdout:1/528: creat de/d21/d49/d1c/d59/d7c/d93/fab x:0 0 0 2024-05-25T06:11:42.578 INFO:tasks.workunit.client.0.smithi160.stdout:3/510: read - d19/da6/f76 zero size 2024-05-25T06:11:42.578 INFO:tasks.workunit.client.0.smithi160.stdout:2/605: mkdir d12/d3e/d5a/d8d/dc9 0 2024-05-25T06:11:42.579 INFO:tasks.workunit.client.1.smithi203.stdout:9/845: creat d0/df/d37/f117 x:0 0 0 2024-05-25T06:11:42.579 INFO:tasks.workunit.client.1.smithi203.stdout:9/846: dread - d0/df/d37/f117 zero size 2024-05-25T06:11:42.580 INFO:tasks.workunit.client.0.smithi160.stdout:2/606: fsync d12/d3e/d5f/d74/d7a/fac 0 2024-05-25T06:11:42.581 INFO:tasks.workunit.client.1.smithi203.stdout:1/809: getdents d14/d57/d97/da4 0 2024-05-25T06:11:42.581 INFO:tasks.workunit.client.0.smithi160.stdout:5/722: rename db/dc/d13/d1d/d31/la8 to db/dc/d13/d1d/d3f/dc0/ldb 0 2024-05-25T06:11:42.581 INFO:tasks.workunit.client.0.smithi160.stdout:2/607: write d12/d97/f99 [996578,282] 0 2024-05-25T06:11:42.582 INFO:tasks.workunit.client.0.smithi160.stdout:1/529: mkdir de/d21/d49/d1c/d41/d45/dac 0 2024-05-25T06:11:42.583 INFO:tasks.workunit.client.1.smithi203.stdout:7/923: mkdir d1/d9/d2c/d36/dd3/d137 0 2024-05-25T06:11:42.585 INFO:tasks.workunit.client.0.smithi160.stdout:3/511: rename d19/d22/c26 to d19/d22/cae 0 2024-05-25T06:11:42.586 INFO:tasks.workunit.client.1.smithi203.stdout:9/847: mkdir d0/d17/d20/d29/d118 0 2024-05-25T06:11:42.586 INFO:tasks.workunit.client.1.smithi203.stdout:1/810: dwrite d14/dff/db8/dc2/f61 [0,4194304] 0 2024-05-25T06:11:42.587 INFO:tasks.workunit.client.0.smithi160.stdout:3/512: write d19/f8e [3647641,85284] 0 2024-05-25T06:11:42.588 INFO:tasks.workunit.client.1.smithi203.stdout:9/848: dread - d0/df/d37/da8/dd6/f10f zero size 2024-05-25T06:11:42.589 INFO:tasks.workunit.client.0.smithi160.stdout:5/723: dwrite db/f22 [0,4194304] 0 2024-05-25T06:11:42.590 INFO:tasks.workunit.client.0.smithi160.stdout:3/513: write d19/d2f/d34/f55 [1174411,81416] 0 2024-05-25T06:11:42.597 INFO:tasks.workunit.client.0.smithi160.stdout:7/673: sync 2024-05-25T06:11:42.597 INFO:tasks.workunit.client.0.smithi160.stdout:9/569: sync 2024-05-25T06:11:42.597 INFO:tasks.workunit.client.0.smithi160.stdout:8/628: sync 2024-05-25T06:11:42.598 INFO:tasks.workunit.client.0.smithi160.stdout:1/530: dread de/d21/d49/d1c/fa5 [0,4194304] 0 2024-05-25T06:11:42.602 INFO:tasks.workunit.client.1.smithi203.stdout:0/894: sync 2024-05-25T06:11:42.606 INFO:tasks.workunit.client.0.smithi160.stdout:6/574: dread d5/d39/f51 [0,4194304] 0 2024-05-25T06:11:42.606 INFO:tasks.workunit.client.0.smithi160.stdout:9/570: rmdir d4/d11/d1e/d3e/d25/d5e/da2 39 2024-05-25T06:11:42.607 INFO:tasks.workunit.client.0.smithi160.stdout:5/724: dwrite db/dc/d13/d1d/d31/d3b/f70 [0,4194304] 0 2024-05-25T06:11:42.609 INFO:tasks.workunit.client.0.smithi160.stdout:6/575: stat d5/de/d18 0 2024-05-25T06:11:42.611 INFO:tasks.workunit.client.1.smithi203.stdout:1/811: dwrite d14/d81/fe5 [0,4194304] 0 2024-05-25T06:11:42.611 INFO:tasks.workunit.client.1.smithi203.stdout:9/849: dwrite d0/d17/d20/fc8 [0,4194304] 0 2024-05-25T06:11:42.612 INFO:tasks.workunit.client.0.smithi160.stdout:1/531: creat de/d21/d49/d28/fad x:0 0 0 2024-05-25T06:11:42.614 INFO:tasks.workunit.client.0.smithi160.stdout:5/725: chown db/dc/d13/d1d/d23/d24/d2e/lb9 69 1 2024-05-25T06:11:42.617 INFO:tasks.workunit.client.0.smithi160.stdout:2/608: dwrite d12/d5c/f6b [0,4194304] 0 2024-05-25T06:11:42.617 INFO:tasks.workunit.client.0.smithi160.stdout:9/571: unlink d4/d11/d1e/d24/f1b 0 2024-05-25T06:11:42.618 INFO:tasks.workunit.client.0.smithi160.stdout:3/514: getdents d19/d2f/d38/d60 0 2024-05-25T06:11:42.619 INFO:tasks.workunit.client.1.smithi203.stdout:1/812: getdents d14/d57/db7/dd2/d101 0 2024-05-25T06:11:42.620 INFO:tasks.workunit.client.0.smithi160.stdout:7/674: mkdir d6/d32/d7e/de0 0 2024-05-25T06:11:42.620 INFO:tasks.workunit.client.0.smithi160.stdout:8/629: dwrite d5/d44/d1f/d58/d65/da2/fb7 [0,4194304] 0 2024-05-25T06:11:42.620 INFO:tasks.workunit.client.0.smithi160.stdout:0/674: sync 2024-05-25T06:11:42.620 INFO:tasks.workunit.client.0.smithi160.stdout:4/585: sync 2024-05-25T06:11:42.621 INFO:tasks.workunit.client.0.smithi160.stdout:9/572: stat d4/d11/d14/f3a 0 2024-05-25T06:11:42.623 INFO:tasks.workunit.client.0.smithi160.stdout:6/576: creat d5/de/d32/d35/d40/fc7 x:0 0 0 2024-05-25T06:11:42.623 INFO:tasks.workunit.client.1.smithi203.stdout:0/895: write dd/f64 [1163762,109284] 0 2024-05-25T06:11:42.623 INFO:tasks.workunit.client.1.smithi203.stdout:1/813: dread - d14/dff/db8/fb0 zero size 2024-05-25T06:11:42.624 INFO:tasks.workunit.client.0.smithi160.stdout:3/515: write d19/d2f/d34/f40 [1239061,24063] 0 2024-05-25T06:11:42.626 INFO:tasks.workunit.client.0.smithi160.stdout:6/577: truncate d5/d30/f5a 855972 0 2024-05-25T06:11:42.628 INFO:tasks.workunit.client.0.smithi160.stdout:2/609: rename d12/d13 to d12/d75/dca 0 2024-05-25T06:11:42.629 INFO:tasks.workunit.client.0.smithi160.stdout:2/610: write d12/f7b [1052948,102033] 0 2024-05-25T06:11:42.631 INFO:tasks.workunit.client.0.smithi160.stdout:7/675: chown d6/d3e/f4b 407894 1 2024-05-25T06:11:42.632 INFO:tasks.workunit.client.0.smithi160.stdout:4/586: mknod d4/d63/d70/d94/cb5 0 2024-05-25T06:11:42.633 INFO:tasks.workunit.client.0.smithi160.stdout:4/587: read d4/d63/d64/f80 [88393,61184] 0 2024-05-25T06:11:42.635 INFO:tasks.workunit.client.1.smithi203.stdout:1/814: read d14/d57/d1d/f2a [5684631,104021] 0 2024-05-25T06:11:42.636 INFO:tasks.workunit.client.0.smithi160.stdout:0/675: write d2b/d31/d35/f49 [2603580,99856] 0 2024-05-25T06:11:42.636 INFO:tasks.workunit.client.0.smithi160.stdout:6/578: creat d5/d45/d63/fc8 x:0 0 0 2024-05-25T06:11:42.638 INFO:tasks.workunit.client.1.smithi203.stdout:1/815: mkdir d14/d57/db7/dd2/dca/d11a 0 2024-05-25T06:11:42.639 INFO:tasks.workunit.client.1.smithi203.stdout:1/816: dread - d14/d57/db7/fc9 zero size 2024-05-25T06:11:42.640 INFO:tasks.workunit.client.1.smithi203.stdout:1/817: write d14/d57/db7/dd2/fcd [1191078,43481] 0 2024-05-25T06:11:42.641 INFO:tasks.workunit.client.0.smithi160.stdout:8/630: mkdir d5/d44/d1f/da3/d28/d4a/d73/dc2/dd6 0 2024-05-25T06:11:42.644 INFO:tasks.workunit.client.1.smithi203.stdout:1/818: rename d14/dff/db8/dc2/f90 to d14/d57/db7/dd2/dca/dcf/df7/f11b 0 2024-05-25T06:11:42.645 INFO:tasks.workunit.client.1.smithi203.stdout:1/819: chown d14/d23/d115/f6e 53575712 1 2024-05-25T06:11:42.645 INFO:tasks.workunit.client.0.smithi160.stdout:7/676: dwrite d6/dc/d21/f2a [0,4194304] 0 2024-05-25T06:11:42.645 INFO:tasks.workunit.client.0.smithi160.stdout:1/532: rename de/d21/d49/d1c/d41/d45/l64 to de/d21/d49/d1c/d41/d96/lae 0 2024-05-25T06:11:42.647 INFO:tasks.workunit.client.0.smithi160.stdout:3/516: creat d19/faf x:0 0 0 2024-05-25T06:11:42.648 INFO:tasks.workunit.client.0.smithi160.stdout:6/579: mknod d5/d30/d36/d73/d87/da4/cc9 0 2024-05-25T06:11:42.649 INFO:tasks.workunit.client.0.smithi160.stdout:6/580: fdatasync d5/d45/d63/fc8 0 2024-05-25T06:11:42.649 INFO:tasks.workunit.client.0.smithi160.stdout:9/573: creat d4/d11/d14/d8e/fb8 x:0 0 0 2024-05-25T06:11:42.649 INFO:tasks.workunit.client.0.smithi160.stdout:4/588: symlink d4/d1c/d54/d9b/lb6 0 2024-05-25T06:11:42.650 INFO:tasks.workunit.client.1.smithi203.stdout:9/850: dread d0/d17/d20/d6e/fc3 [0,4194304] 0 2024-05-25T06:11:42.653 INFO:tasks.workunit.client.1.smithi203.stdout:1/820: dread d14/d57/d6b/da1/de2/f76 [0,4194304] 0 2024-05-25T06:11:42.654 INFO:tasks.workunit.client.0.smithi160.stdout:5/726: rename db/dc/d13/d1d/d23/f2d to db/dc/d13/d1d/d3f/d72/dc5/fdc 0 2024-05-25T06:11:42.655 INFO:tasks.workunit.client.0.smithi160.stdout:0/676: mkdir d2b/d46/d63/da7/da6/dcd/dd8 0 2024-05-25T06:11:42.657 INFO:tasks.workunit.client.1.smithi203.stdout:1/821: write d14/d57/d6b/da1/fee [2112897,30618] 0 2024-05-25T06:11:42.658 INFO:tasks.workunit.client.0.smithi160.stdout:6/581: unlink d5/de/d32/d35/f5f 0 2024-05-25T06:11:42.659 INFO:tasks.workunit.client.0.smithi160.stdout:5/727: dread db/dc/d13/d1d/d23/d24/d2e/f4e [0,4194304] 0 2024-05-25T06:11:42.660 INFO:tasks.workunit.client.0.smithi160.stdout:8/631: mknod d5/d7/dcb/d7a/dc9/cd7 0 2024-05-25T06:11:42.660 INFO:tasks.workunit.client.1.smithi203.stdout:9/851: dwrite d0/d17/d20/d6e/d75/df2/d114/db2/f84 [0,4194304] 0 2024-05-25T06:11:42.660 INFO:tasks.workunit.client.0.smithi160.stdout:6/582: write d5/d39/d8f/f55 [4251598,98716] 0 2024-05-25T06:11:42.661 INFO:tasks.workunit.client.1.smithi203.stdout:9/852: fdatasync d0/dd0/fb1 0 2024-05-25T06:11:42.661 INFO:tasks.workunit.client.0.smithi160.stdout:6/583: chown d5/d45 19 1 2024-05-25T06:11:42.662 INFO:tasks.workunit.client.1.smithi203.stdout:1/822: mknod d14/d57/d97/dec/c11c 0 2024-05-25T06:11:42.664 INFO:tasks.workunit.client.0.smithi160.stdout:6/584: write d5/d30/d36/d73/d6e/f84 [4064982,70215] 0 2024-05-25T06:11:42.666 INFO:tasks.workunit.client.0.smithi160.stdout:9/574: mknod d4/cb9 0 2024-05-25T06:11:42.666 INFO:tasks.workunit.client.0.smithi160.stdout:6/585: truncate d5/d45/d63/fc8 466081 0 2024-05-25T06:11:42.667 INFO:tasks.workunit.client.0.smithi160.stdout:9/575: dread - d4/d11/d14/d8e/d9c/fb5 zero size 2024-05-25T06:11:42.667 INFO:tasks.workunit.client.1.smithi203.stdout:1/823: mkdir d14/d57/d97/d11d 0 2024-05-25T06:11:42.667 INFO:tasks.workunit.client.0.smithi160.stdout:9/576: chown d4/fb1 1411070 1 2024-05-25T06:11:42.668 INFO:tasks.workunit.client.0.smithi160.stdout:6/586: truncate d5/d45/d63/f83 4229148 0 2024-05-25T06:11:42.669 INFO:tasks.workunit.client.0.smithi160.stdout:4/589: link d4/d6/d28/f17 d4/d63/fb7 0 2024-05-25T06:11:42.669 INFO:tasks.workunit.client.1.smithi203.stdout:9/853: rename d0/df/d2d/d92/ff8 to d0/d17/dec/f119 0 2024-05-25T06:11:42.670 INFO:tasks.workunit.client.1.smithi203.stdout:0/896: truncate dd/d5c/d63/d106/d4f/f56 58741 0 2024-05-25T06:11:42.670 INFO:tasks.workunit.client.1.smithi203.stdout:9/854: chown d0/f6f 13 1 2024-05-25T06:11:42.673 INFO:tasks.workunit.client.1.smithi203.stdout:9/855: rename d0/df to d0/df/d37/d11a 22 2024-05-25T06:11:42.675 INFO:tasks.workunit.client.0.smithi160.stdout:5/728: creat db/dc/d13/d1d/d3f/d72/d98/da5/db5/fdd x:0 0 0 2024-05-25T06:11:42.676 INFO:tasks.workunit.client.1.smithi203.stdout:0/897: creat dd/d5c/d63/d106/d59/f13f x:0 0 0 2024-05-25T06:11:42.677 INFO:tasks.workunit.client.0.smithi160.stdout:3/517: creat d19/d2f/d38/fb0 x:0 0 0 2024-05-25T06:11:42.677 INFO:tasks.workunit.client.0.smithi160.stdout:0/677: link d2b/d31/fc1 d2b/da8/fd9 0 2024-05-25T06:11:42.677 INFO:tasks.workunit.client.0.smithi160.stdout:9/577: rmdir d4/d11/d12/d2e/d45 39 2024-05-25T06:11:42.678 INFO:tasks.workunit.client.1.smithi203.stdout:0/898: write dd/d5c/d63/d106/df3/f7f [2363245,20080] 0 2024-05-25T06:11:42.679 INFO:tasks.workunit.client.1.smithi203.stdout:9/856: unlink d0/d57/de0/f103 0 2024-05-25T06:11:42.679 INFO:tasks.workunit.client.0.smithi160.stdout:0/678: fdatasync d2b/d46/d63/da7/f8c 0 2024-05-25T06:11:42.680 INFO:tasks.workunit.client.1.smithi203.stdout:1/824: getdents d14/d57/dbb/df8 0 2024-05-25T06:11:42.680 INFO:tasks.workunit.client.1.smithi203.stdout:9/857: write d0/f1e [3118508,12607] 0 2024-05-25T06:11:42.680 INFO:tasks.workunit.client.0.smithi160.stdout:3/518: fsync d19/da6/d6a/f97 0 2024-05-25T06:11:42.680 INFO:tasks.workunit.client.0.smithi160.stdout:4/590: chown d4/d6/d35/d3d/d55/fa9 1016622409 1 2024-05-25T06:11:42.681 INFO:tasks.workunit.client.0.smithi160.stdout:5/729: chown db/dc/d13/d1d/d23/d24/d2e/lb9 14831240 1 2024-05-25T06:11:42.682 INFO:tasks.workunit.client.0.smithi160.stdout:3/519: chown d19/d2f/d38/f6c 2995688 1 2024-05-25T06:11:42.684 INFO:tasks.workunit.client.1.smithi203.stdout:1/825: rmdir d14/d57/db7/dd2/dca 39 2024-05-25T06:11:42.686 INFO:tasks.workunit.client.1.smithi203.stdout:0/899: mknod dd/d10/d34/d4c/d99/c140 0 2024-05-25T06:11:42.687 INFO:tasks.workunit.client.0.smithi160.stdout:4/591: dwrite d4/f3b [4194304,4194304] 0 2024-05-25T06:11:42.688 INFO:tasks.workunit.client.1.smithi203.stdout:9/858: dwrite d0/df/d37/d4f/f108 [0,4194304] 0 2024-05-25T06:11:42.690 INFO:tasks.workunit.client.0.smithi160.stdout:4/592: write d4/d63/d64/d88/fa5 [637330,130793] 0 2024-05-25T06:11:42.690 INFO:tasks.workunit.client.1.smithi203.stdout:1/826: symlink d14/d57/d97/d11d/l11e 0 2024-05-25T06:11:42.691 INFO:tasks.workunit.client.0.smithi160.stdout:1/533: dread de/d21/d49/d35/f24 [0,4194304] 0 2024-05-25T06:11:42.691 INFO:tasks.workunit.client.1.smithi203.stdout:1/827: fdatasync d14/d57/d97/ff2 0 2024-05-25T06:11:42.692 INFO:tasks.workunit.client.0.smithi160.stdout:4/593: read d4/f26 [8109424,104361] 0 2024-05-25T06:11:42.694 INFO:tasks.workunit.client.1.smithi203.stdout:9/859: mknod d0/d57/dae/c11b 0 2024-05-25T06:11:42.696 INFO:tasks.workunit.client.0.smithi160.stdout:2/611: write d12/d3e/f62 [2871935,118949] 0 2024-05-25T06:11:42.696 INFO:tasks.workunit.client.1.smithi203.stdout:0/900: rename dd/d5c/d63/d106/d48/d91/ff7 to dd/d5c/d63/d106/d4f/d62/d78/d100/f141 0 2024-05-25T06:11:42.699 INFO:tasks.workunit.client.1.smithi203.stdout:9/860: mkdir d0/df/ddd/d11c 0 2024-05-25T06:11:42.701 INFO:tasks.workunit.client.1.smithi203.stdout:0/901: mkdir dd/d5c/d63/d106/da0/d142 0 2024-05-25T06:11:42.702 INFO:tasks.workunit.client.1.smithi203.stdout:9/861: symlink d0/d57/de0/df6/l11d 0 2024-05-25T06:11:42.706 INFO:tasks.workunit.client.1.smithi203.stdout:9/862: unlink d0/df/d37/da8/cd5 0 2024-05-25T06:11:42.707 INFO:tasks.workunit.client.1.smithi203.stdout:0/902: mknod dd/d5c/d63/d106/d4f/db7/d132/c143 0 2024-05-25T06:11:42.709 INFO:tasks.workunit.client.1.smithi203.stdout:9/863: creat d0/f11e x:0 0 0 2024-05-25T06:11:42.710 INFO:tasks.workunit.client.0.smithi160.stdout:7/677: getdents d6/d2b/da0/dc7 0 2024-05-25T06:11:42.714 INFO:tasks.workunit.client.1.smithi203.stdout:9/864: link d0/d17/d20/f31 d0/d17/d20/d6e/d75/f11f 0 2024-05-25T06:11:42.716 INFO:tasks.workunit.client.0.smithi160.stdout:0/679: chown d2b/d31/d35/d7b 7 1 2024-05-25T06:11:42.719 INFO:tasks.workunit.client.1.smithi203.stdout:0/903: truncate dd/d10/f4a 1240308 0 2024-05-25T06:11:42.721 INFO:tasks.workunit.client.0.smithi160.stdout:5/730: creat db/d47/d81/dc1/fde x:0 0 0 2024-05-25T06:11:42.723 INFO:tasks.workunit.client.1.smithi203.stdout:9/865: dwrite d0/d17/d20/d6e/d75/df2/d114/fed [0,4194304] 0 2024-05-25T06:11:42.723 INFO:tasks.workunit.client.1.smithi203.stdout:0/904: creat dd/d5c/d120/f144 x:0 0 0 2024-05-25T06:11:42.723 INFO:tasks.workunit.client.0.smithi160.stdout:5/731: read db/dc/d13/d1d/f20 [1188379,108510] 0 2024-05-25T06:11:42.724 INFO:tasks.workunit.client.1.smithi203.stdout:1/828: dread d14/d57/d6b/f8e [0,4194304] 0 2024-05-25T06:11:42.725 INFO:tasks.workunit.client.0.smithi160.stdout:3/520: rename d19/l79 to d19/d2f/d34/da8/lb1 0 2024-05-25T06:11:42.725 INFO:tasks.workunit.client.0.smithi160.stdout:5/732: chown db/dc/d13/d1d/d23/dbd 1011810 1 2024-05-25T06:11:42.725 INFO:tasks.workunit.client.0.smithi160.stdout:3/521: write d19/f99 [1197498,101941] 0 2024-05-25T06:11:42.725 INFO:tasks.workunit.client.1.smithi203.stdout:9/866: fdatasync d0/df/d37/da8/f10c 0 2024-05-25T06:11:42.728 INFO:tasks.workunit.client.0.smithi160.stdout:1/534: mknod de/d21/d49/d1c/d59/d71/caf 0 2024-05-25T06:11:42.730 INFO:tasks.workunit.client.0.smithi160.stdout:4/594: creat d4/d6/d35/d3d/fb8 x:0 0 0 2024-05-25T06:11:42.732 INFO:tasks.workunit.client.0.smithi160.stdout:6/587: truncate d5/de/d32/d35/f78 886479 0 2024-05-25T06:11:42.732 INFO:tasks.workunit.client.1.smithi203.stdout:1/829: dwrite d14/d57/d1d/fda [4194304,4194304] 0 2024-05-25T06:11:42.733 INFO:tasks.workunit.client.0.smithi160.stdout:3/522: dwrite d19/d2f/d34/f55 [0,4194304] 0 2024-05-25T06:11:42.734 INFO:tasks.workunit.client.0.smithi160.stdout:3/523: dread - d19/d2f/d38/d60/f82 zero size 2024-05-25T06:11:42.735 INFO:tasks.workunit.client.1.smithi203.stdout:9/867: symlink d0/d17/d20/d29/d118/l120 0 2024-05-25T06:11:42.735 INFO:tasks.workunit.client.1.smithi203.stdout:0/905: write dd/d5c/d63/d106/df3/fae [782436,108707] 0 2024-05-25T06:11:42.738 INFO:tasks.workunit.client.1.smithi203.stdout:0/906: chown dd/d5c/d63/d106/d75/d93/la5 36 1 2024-05-25T06:11:42.740 INFO:tasks.workunit.client.1.smithi203.stdout:0/907: write dd/d5c/d63/d106/d4f/db7/df0/f131 [394279,75806] 0 2024-05-25T06:11:42.740 INFO:tasks.workunit.client.1.smithi203.stdout:9/868: mknod d0/d17/d20/d29/d118/c121 0 2024-05-25T06:11:42.741 INFO:tasks.workunit.client.0.smithi160.stdout:0/680: rmdir d2b/d31/d35/d7b 39 2024-05-25T06:11:42.742 INFO:tasks.workunit.client.0.smithi160.stdout:0/681: chown d2b/f2e 22300558 1 2024-05-25T06:11:42.743 INFO:tasks.workunit.client.0.smithi160.stdout:9/578: fsync d4/d11/d1e/d3e/d25/d27/f7d 0 2024-05-25T06:11:42.743 INFO:tasks.workunit.client.1.smithi203.stdout:9/869: creat d0/d17/dec/f122 x:0 0 0 2024-05-25T06:11:42.746 INFO:tasks.workunit.client.1.smithi203.stdout:0/908: mkdir dd/d5c/d63/d106/d75/d145 0 2024-05-25T06:11:42.749 INFO:tasks.workunit.client.1.smithi203.stdout:0/909: creat dd/d5c/d63/d106/da0/f146 x:0 0 0 2024-05-25T06:11:42.749 INFO:tasks.workunit.client.0.smithi160.stdout:1/535: creat de/d21/d49/d1c/d52/d77/fb0 x:0 0 0 2024-05-25T06:11:42.751 INFO:tasks.workunit.client.0.smithi160.stdout:2/612: mkdir d12/d3e/d5a/d8d/db4/dcb 0 2024-05-25T06:11:42.753 INFO:tasks.workunit.client.0.smithi160.stdout:6/588: creat d5/d39/d8f/d2b/fca x:0 0 0 2024-05-25T06:11:42.755 INFO:tasks.workunit.client.0.smithi160.stdout:7/678: symlink d6/d2b/le1 0 2024-05-25T06:11:42.756 INFO:tasks.workunit.client.0.smithi160.stdout:6/589: write d5/d45/d4a/fa8 [50694,66099] 0 2024-05-25T06:11:42.758 INFO:tasks.workunit.client.0.smithi160.stdout:9/579: truncate d4/d11/d12/f2b 949417 0 2024-05-25T06:11:42.758 INFO:tasks.workunit.client.0.smithi160.stdout:8/632: truncate d5/d44/d1f/da3/d28/d49/f70 2286719 0 2024-05-25T06:11:42.759 INFO:tasks.workunit.client.0.smithi160.stdout:1/536: dwrite de/f3a [0,4194304] 0 2024-05-25T06:11:42.761 INFO:tasks.workunit.client.0.smithi160.stdout:0/682: dread d2b/d31/f8f [0,4194304] 0 2024-05-25T06:11:42.762 INFO:tasks.workunit.client.0.smithi160.stdout:2/613: creat d12/d1c/d37/fcc x:0 0 0 2024-05-25T06:11:42.763 INFO:tasks.workunit.client.0.smithi160.stdout:3/524: mkdir d19/db2 0 2024-05-25T06:11:42.763 INFO:tasks.workunit.client.0.smithi160.stdout:4/595: dwrite d4/f26 [4194304,4194304] 0 2024-05-25T06:11:42.764 INFO:tasks.workunit.client.0.smithi160.stdout:7/679: mkdir d6/d2b/d5d/de2 0 2024-05-25T06:11:42.766 INFO:tasks.workunit.client.0.smithi160.stdout:5/733: creat db/dc/d13/d1d/d23/fdf x:0 0 0 2024-05-25T06:11:42.766 INFO:tasks.workunit.client.0.smithi160.stdout:4/596: stat d4/d1a/c1b 0 2024-05-25T06:11:42.767 INFO:tasks.workunit.client.0.smithi160.stdout:9/580: chown d4/d11/d12/d2e/d45 93 1 2024-05-25T06:11:42.774 INFO:tasks.workunit.client.1.smithi203.stdout:1/830: truncate d14/d57/db7/dd2/db3/f7a 6215671 0 2024-05-25T06:11:42.775 INFO:tasks.workunit.client.1.smithi203.stdout:9/870: dread d0/d17/d23/f5a [0,4194304] 0 2024-05-25T06:11:42.775 INFO:tasks.workunit.client.1.smithi203.stdout:1/831: dread - d14/d57/db7/fc9 zero size 2024-05-25T06:11:42.776 INFO:tasks.workunit.client.1.smithi203.stdout:9/871: dread - d0/f11e zero size 2024-05-25T06:11:42.776 INFO:tasks.workunit.client.1.smithi203.stdout:0/910: truncate dd/d5c/d63/d106/df3/de2/fe9 192110 0 2024-05-25T06:11:42.777 INFO:tasks.workunit.client.0.smithi160.stdout:6/590: dwrite d5/f11 [0,4194304] 0 2024-05-25T06:11:42.780 INFO:tasks.workunit.client.0.smithi160.stdout:0/683: chown d2b/d31/d35/d36/fb6 23454658 1 2024-05-25T06:11:42.780 INFO:tasks.workunit.client.0.smithi160.stdout:1/537: mkdir de/d21/d49/db1 0 2024-05-25T06:11:42.781 INFO:tasks.workunit.client.1.smithi203.stdout:0/911: chown dd/d5c/d63/d106/da0/lb6 924328041 1 2024-05-25T06:11:42.783 INFO:tasks.workunit.client.1.smithi203.stdout:9/872: dwrite d0/d17/d20/fc8 [4194304,4194304] 0 2024-05-25T06:11:42.784 INFO:tasks.workunit.client.0.smithi160.stdout:5/734: mknod db/dc/d13/d1d/d3f/d4a/ce0 0 2024-05-25T06:11:42.784 INFO:tasks.workunit.client.0.smithi160.stdout:7/680: dread d6/d32/f80 [0,4194304] 0 2024-05-25T06:11:42.785 INFO:tasks.workunit.client.1.smithi203.stdout:0/912: unlink la 0 2024-05-25T06:11:42.789 INFO:tasks.workunit.client.0.smithi160.stdout:5/735: write db/f5d [3178473,115443] 0 2024-05-25T06:11:42.789 INFO:tasks.workunit.client.0.smithi160.stdout:1/538: read - de/d21/d49/d1c/d52/d77/fb0 zero size 2024-05-25T06:11:42.791 INFO:tasks.workunit.client.1.smithi203.stdout:0/913: truncate dd/d5c/d63/d106/d75/fe1 22233 0 2024-05-25T06:11:42.794 INFO:tasks.workunit.client.0.smithi160.stdout:4/597: dwrite d4/d6/d21/f3f [0,4194304] 0 2024-05-25T06:11:42.795 INFO:tasks.workunit.client.0.smithi160.stdout:2/614: dwrite f9 [0,4194304] 0 2024-05-25T06:11:42.795 INFO:tasks.workunit.client.0.smithi160.stdout:3/525: dwrite d19/d2f/d34/f95 [0,4194304] 0 2024-05-25T06:11:42.796 INFO:tasks.workunit.client.1.smithi203.stdout:0/914: creat dd/d10/d34/d4c/f147 x:0 0 0 2024-05-25T06:11:42.798 INFO:tasks.workunit.client.1.smithi203.stdout:1/832: dwrite d14/d57/d6b/da1/de2/f82 [0,4194304] 0 2024-05-25T06:11:42.802 INFO:tasks.workunit.client.1.smithi203.stdout:0/915: chown dd/d5c/d63/d106/d75/d121/d122 1837 1 2024-05-25T06:11:42.802 INFO:tasks.workunit.client.1.smithi203.stdout:9/873: write d0/f52 [397529,110055] 0 2024-05-25T06:11:42.804 INFO:tasks.workunit.client.0.smithi160.stdout:1/539: readlink de/d21/d49/la9 0 2024-05-25T06:11:42.804 INFO:tasks.workunit.client.0.smithi160.stdout:4/598: write d4/d1a/f59 [3250528,103391] 0 2024-05-25T06:11:42.806 INFO:tasks.workunit.client.0.smithi160.stdout:0/684: dwrite f23 [0,4194304] 0 2024-05-25T06:11:42.809 INFO:tasks.workunit.client.0.smithi160.stdout:0/685: fsync f1e 0 2024-05-25T06:11:42.809 INFO:tasks.workunit.client.0.smithi160.stdout:9/581: truncate d4/f6e 2492921 0 2024-05-25T06:11:42.811 INFO:tasks.workunit.client.0.smithi160.stdout:2/615: rename d12/d3e/l78 to d12/d3e/db5/lcd 0 2024-05-25T06:11:42.812 INFO:tasks.workunit.client.1.smithi203.stdout:0/916: creat dd/d5c/f148 x:0 0 0 2024-05-25T06:11:42.813 INFO:tasks.workunit.client.0.smithi160.stdout:5/736: creat db/dc/d13/d1d/d23/d24/d2e/fe1 x:0 0 0 2024-05-25T06:11:42.813 INFO:tasks.workunit.client.1.smithi203.stdout:9/874: dwrite d0/d74/f101 [0,4194304] 0 2024-05-25T06:11:42.815 INFO:tasks.workunit.client.1.smithi203.stdout:9/875: chown d0/df/d2d/ld3 127 1 2024-05-25T06:11:42.816 INFO:tasks.workunit.client.1.smithi203.stdout:9/876: fsync d0/df/d73/f49 0 2024-05-25T06:11:42.821 INFO:tasks.workunit.client.1.smithi203.stdout:9/877: truncate d0/d17/d20/d29/f6b 534600 0 2024-05-25T06:11:42.822 INFO:tasks.workunit.client.1.smithi203.stdout:0/917: dread dd/d32/d37/f101 [0,4194304] 0 2024-05-25T06:11:42.823 INFO:tasks.workunit.client.1.smithi203.stdout:9/878: readlink d0/d17/d20/d6e/d75/la4 0 2024-05-25T06:11:42.823 INFO:tasks.workunit.client.1.smithi203.stdout:9/879: chown d0/fa6 6129172 1 2024-05-25T06:11:42.825 INFO:tasks.workunit.client.1.smithi203.stdout:0/918: write dd/f64 [1570759,35288] 0 2024-05-25T06:11:42.829 INFO:tasks.workunit.client.1.smithi203.stdout:9/880: dread d0/d17/d20/fc8 [0,4194304] 0 2024-05-25T06:11:42.830 INFO:tasks.workunit.client.1.smithi203.stdout:0/919: getdents dd/d5c/d63/d106/da0/db9 0 2024-05-25T06:11:42.835 INFO:tasks.workunit.client.1.smithi203.stdout:1/833: write d14/f25 [1109342,127143] 0 2024-05-25T06:11:42.836 INFO:tasks.workunit.client.1.smithi203.stdout:9/881: getdents d0/d17/d20/d6e 0 2024-05-25T06:11:42.837 INFO:tasks.workunit.client.1.smithi203.stdout:9/882: write d0/fe5 [406152,45800] 0 2024-05-25T06:11:42.839 INFO:tasks.workunit.client.1.smithi203.stdout:1/834: getdents d14/d57/d97/dec 0 2024-05-25T06:11:42.841 INFO:tasks.workunit.client.1.smithi203.stdout:0/920: dread dd/d10/fdd [0,4194304] 0 2024-05-25T06:11:42.843 INFO:tasks.workunit.client.1.smithi203.stdout:1/835: creat d14/d57/f11f x:0 0 0 2024-05-25T06:11:42.843 INFO:tasks.workunit.client.1.smithi203.stdout:0/921: chown dd/d5c/d63/d106/d75/cd7 3 1 2024-05-25T06:11:42.844 INFO:tasks.workunit.client.1.smithi203.stdout:9/883: dwrite d0/fb [0,4194304] 0 2024-05-25T06:11:42.845 INFO:tasks.workunit.client.1.smithi203.stdout:1/836: rmdir d14/d57/db7/dd2/d101 39 2024-05-25T06:11:42.846 INFO:tasks.workunit.client.1.smithi203.stdout:9/884: creat d0/d17/dec/f123 x:0 0 0 2024-05-25T06:11:42.849 INFO:tasks.workunit.client.1.smithi203.stdout:0/922: link dd/d32/d37/f101 dd/d5c/d63/d106/d4f/db7/df0/f149 0 2024-05-25T06:11:42.851 INFO:tasks.workunit.client.1.smithi203.stdout:1/837: read d14/d57/db7/dd2/ddc/ffc [813612,63195] 0 2024-05-25T06:11:42.851 INFO:tasks.workunit.client.1.smithi203.stdout:9/885: chown d0/d57/cac 8259972 1 2024-05-25T06:11:42.851 INFO:tasks.workunit.client.1.smithi203.stdout:9/886: chown d0 3 1 2024-05-25T06:11:42.854 INFO:tasks.workunit.client.1.smithi203.stdout:9/887: creat d0/d57/de0/f124 x:0 0 0 2024-05-25T06:11:42.856 INFO:tasks.workunit.client.1.smithi203.stdout:0/923: rename dd/d10/l2c to dd/d5c/d63/d106/d75/d121/d122/l14a 0 2024-05-25T06:11:42.858 INFO:tasks.workunit.client.1.smithi203.stdout:9/888: rename d0/df/d73/f87 to d0/d17/d20/d6e/d75/f125 0 2024-05-25T06:11:42.859 INFO:tasks.workunit.client.1.smithi203.stdout:9/889: chown d0/df/d2d/d92/lde 23772991 1 2024-05-25T06:11:42.863 INFO:tasks.workunit.client.0.smithi160.stdout:3/526: creat d19/d2f/dad/fb3 x:0 0 0 2024-05-25T06:11:42.864 INFO:tasks.workunit.client.0.smithi160.stdout:0/686: mknod d2b/d31/d35/d36/daf/cda 0 2024-05-25T06:11:42.865 INFO:tasks.workunit.client.1.smithi203.stdout:9/890: creat d0/d17/d20/f126 x:0 0 0 2024-05-25T06:11:42.866 INFO:tasks.workunit.client.0.smithi160.stdout:2/616: fsync d12/d1c/f2f 0 2024-05-25T06:11:42.869 INFO:tasks.workunit.client.0.smithi160.stdout:3/527: unlink d19/f29 0 2024-05-25T06:11:42.869 INFO:tasks.workunit.client.0.smithi160.stdout:0/687: dread - d2b/da8/fc3 zero size 2024-05-25T06:11:42.870 INFO:tasks.workunit.client.1.smithi203.stdout:0/924: write dd/d32/de5/f112 [53688,125325] 0 2024-05-25T06:11:42.871 INFO:tasks.workunit.client.1.smithi203.stdout:1/838: dwrite d14/d57/d7e/fa7 [0,4194304] 0 2024-05-25T06:11:42.872 INFO:tasks.workunit.client.0.smithi160.stdout:0/688: stat l3 0 2024-05-25T06:11:42.872 INFO:tasks.workunit.client.0.smithi160.stdout:8/633: dwrite d5/d44/d1f/da3/d28/d4a/fb3 [0,4194304] 0 2024-05-25T06:11:42.873 INFO:tasks.workunit.client.0.smithi160.stdout:2/617: dread - d12/d3e/d5a/fc5 zero size 2024-05-25T06:11:42.874 INFO:tasks.workunit.client.0.smithi160.stdout:6/591: dwrite d5/de/f17 [8388608,4194304] 0 2024-05-25T06:11:42.881 INFO:tasks.workunit.client.0.smithi160.stdout:6/592: rmdir d5/de 39 2024-05-25T06:11:42.881 INFO:tasks.workunit.client.1.smithi203.stdout:0/925: getdents dd/d5c/dd5 0 2024-05-25T06:11:42.882 INFO:tasks.workunit.client.0.smithi160.stdout:1/540: dread de/d21/d49/d35/f24 [4194304,4194304] 0 2024-05-25T06:11:42.883 INFO:tasks.workunit.client.0.smithi160.stdout:9/582: dwrite d4/d11/d12/f48 [4194304,4194304] 0 2024-05-25T06:11:42.883 INFO:tasks.workunit.client.0.smithi160.stdout:8/634: readlink d5/d44/d1f/da3/d28/d4a/lc1 0 2024-05-25T06:11:42.884 INFO:tasks.workunit.client.0.smithi160.stdout:9/583: write d4/fb1 [652120,31390] 0 2024-05-25T06:11:42.885 INFO:tasks.workunit.client.0.smithi160.stdout:0/689: mknod d2b/d31/d35/d36/d86/dd5/cdb 0 2024-05-25T06:11:42.888 INFO:tasks.workunit.client.0.smithi160.stdout:1/541: mknod de/d21/d49/d1c/d41/d45/d7d/cb2 0 2024-05-25T06:11:42.889 INFO:tasks.workunit.client.1.smithi203.stdout:1/839: write d14/d57/db7/dd2/f59 [1746761,128662] 0 2024-05-25T06:11:42.890 INFO:tasks.workunit.client.0.smithi160.stdout:0/690: rmdir d2b/d31/d6d 39 2024-05-25T06:11:42.890 INFO:tasks.workunit.client.1.smithi203.stdout:0/926: truncate dd/d5c/d63/d106/d4f/d62/d78/ff8 7264912 0 2024-05-25T06:11:42.890 INFO:tasks.workunit.client.1.smithi203.stdout:1/840: stat d14/d57/d97/da4/dad/ce6 0 2024-05-25T06:11:42.893 INFO:tasks.workunit.client.0.smithi160.stdout:1/542: read de/d21/d49/d1c/d63/f7b [2070497,28147] 0 2024-05-25T06:11:42.895 INFO:tasks.workunit.client.1.smithi203.stdout:9/891: dread d0/df/f4a [0,4194304] 0 2024-05-25T06:11:42.895 INFO:tasks.workunit.client.0.smithi160.stdout:8/635: creat d5/d44/d1f/fd8 x:0 0 0 2024-05-25T06:11:42.897 INFO:tasks.workunit.client.1.smithi203.stdout:0/927: rename dd/d32/d37/f42 to dd/d5c/d63/d106/da0/db9/df9/d86/f14b 0 2024-05-25T06:11:42.897 INFO:tasks.workunit.client.0.smithi160.stdout:2/618: dwrite d12/d1c/d37/f43 [4194304,4194304] 0 2024-05-25T06:11:42.899 INFO:tasks.workunit.client.0.smithi160.stdout:1/543: unlink de/f91 0 2024-05-25T06:11:42.899 INFO:tasks.workunit.client.0.smithi160.stdout:6/593: dwrite d5/d45/d4a/fbd [0,4194304] 0 2024-05-25T06:11:42.901 INFO:tasks.workunit.client.0.smithi160.stdout:6/594: read - d5/d39/d8f/d2b/fca zero size 2024-05-25T06:11:42.902 INFO:tasks.workunit.client.0.smithi160.stdout:0/691: creat d2b/d46/fdc x:0 0 0 2024-05-25T06:11:42.903 INFO:tasks.workunit.client.1.smithi203.stdout:0/928: creat dd/d32/f14c x:0 0 0 2024-05-25T06:11:42.903 INFO:tasks.workunit.client.0.smithi160.stdout:8/636: dread - d5/d44/d1f/da3/d28/dc6/f95 zero size 2024-05-25T06:11:42.905 INFO:tasks.workunit.client.1.smithi203.stdout:1/841: dread d14/d23/d115/f7b [0,4194304] 0 2024-05-25T06:11:42.906 INFO:tasks.workunit.client.0.smithi160.stdout:0/692: symlink d2b/d31/d35/d36/d86/dd5/ldd 0 2024-05-25T06:11:42.908 INFO:tasks.workunit.client.0.smithi160.stdout:0/693: write d2b/d46/d63/f7e [1468905,51315] 0 2024-05-25T06:11:42.909 INFO:tasks.workunit.client.0.smithi160.stdout:8/637: rename d5/d44/d7e to d5/d9/d72/dd9 0 2024-05-25T06:11:42.910 INFO:tasks.workunit.client.1.smithi203.stdout:1/842: fdatasync d14/d23/d115/f6e 0 2024-05-25T06:11:42.910 INFO:tasks.workunit.client.0.smithi160.stdout:7/681: dread d6/d32/f42 [0,4194304] 0 2024-05-25T06:11:42.910 INFO:tasks.workunit.client.1.smithi203.stdout:0/929: getdents dd/d5c/d63/d106/da0/db9 0 2024-05-25T06:11:42.914 INFO:tasks.workunit.client.0.smithi160.stdout:6/595: rename d5/d39/d8f/d2b/fca to d5/de/d7d/fcb 0 2024-05-25T06:11:42.916 INFO:tasks.workunit.client.1.smithi203.stdout:0/930: creat dd/d5c/dd5/dde/f14d x:0 0 0 2024-05-25T06:11:42.918 INFO:tasks.workunit.client.0.smithi160.stdout:0/694: symlink d2b/d31/d35/d43/dab/lde 0 2024-05-25T06:11:42.919 INFO:tasks.workunit.client.1.smithi203.stdout:1/843: dwrite d14/f32 [0,4194304] 0 2024-05-25T06:11:42.920 INFO:tasks.workunit.client.0.smithi160.stdout:7/682: creat d6/d32/dce/fe3 x:0 0 0 2024-05-25T06:11:42.922 INFO:tasks.workunit.client.0.smithi160.stdout:8/638: mknod d5/d44/d1f/d58/d65/da2/dd1/cda 0 2024-05-25T06:11:42.923 INFO:tasks.workunit.client.1.smithi203.stdout:9/892: truncate d0/df/d2d/f72 3555776 0 2024-05-25T06:11:42.924 INFO:tasks.workunit.client.0.smithi160.stdout:0/695: mknod d2b/d46/d63/db3/cdf 0 2024-05-25T06:11:42.926 INFO:tasks.workunit.client.0.smithi160.stdout:7/683: mknod d6/d2b/d8b/ce4 0 2024-05-25T06:11:42.926 INFO:tasks.workunit.client.1.smithi203.stdout:0/931: rename dd/d5c/d63/d106/da0/db9/df9/d9d/cbb to dd/d5c/d63/d106/d4f/d62/d78/c14e 0 2024-05-25T06:11:42.927 INFO:tasks.workunit.client.0.smithi160.stdout:6/596: mknod d5/d45/d63/d98/ccc 0 2024-05-25T06:11:42.928 INFO:tasks.workunit.client.1.smithi203.stdout:1/844: write d14/d57/f58 [296604,36300] 0 2024-05-25T06:11:42.929 INFO:tasks.workunit.client.1.smithi203.stdout:9/893: unlink d0/df/d73/c88 0 2024-05-25T06:11:42.930 INFO:tasks.workunit.client.0.smithi160.stdout:3/528: dwrite d19/d2f/d38/d43/f53 [4194304,4194304] 0 2024-05-25T06:11:42.932 INFO:tasks.workunit.client.0.smithi160.stdout:0/696: write d2b/da8/fb0 [1712400,129591] 0 2024-05-25T06:11:42.934 INFO:tasks.workunit.client.1.smithi203.stdout:0/932: mkdir dd/d10/d34/d14f 0 2024-05-25T06:11:42.934 INFO:tasks.workunit.client.1.smithi203.stdout:0/933: chown dd/d5c/d120 588042741 1 2024-05-25T06:11:42.934 INFO:tasks.workunit.client.1.smithi203.stdout:1/845: creat d14/d57/db7/dd2/dca/dcf/f120 x:0 0 0 2024-05-25T06:11:42.936 INFO:tasks.workunit.client.0.smithi160.stdout:7/684: unlink d6/c8e 0 2024-05-25T06:11:42.938 INFO:tasks.workunit.client.0.smithi160.stdout:8/639: mkdir d5/d44/dba/ddb 0 2024-05-25T06:11:42.938 INFO:tasks.workunit.client.1.smithi203.stdout:0/934: mknod dd/d5c/d63/d106/d75/d117/c150 0 2024-05-25T06:11:42.940 INFO:tasks.workunit.client.1.smithi203.stdout:1/846: rename d14/d23/d115/f7b to d14/d57/d97/da4/f121 0 2024-05-25T06:11:42.941 INFO:tasks.workunit.client.1.smithi203.stdout:1/847: read d14/dff/db8/fd8 [732195,597] 0 2024-05-25T06:11:42.942 INFO:tasks.workunit.client.0.smithi160.stdout:6/597: dwrite d5/d39/d8f/f9d [0,4194304] 0 2024-05-25T06:11:42.942 INFO:tasks.workunit.client.0.smithi160.stdout:9/584: dread d4/d11/d1e/d6c/f81 [0,4194304] 0 2024-05-25T06:11:42.943 INFO:tasks.workunit.client.1.smithi203.stdout:1/848: fsync d14/f25 0 2024-05-25T06:11:42.947 INFO:tasks.workunit.client.0.smithi160.stdout:3/529: dwrite d19/d2f/d38/f6b [0,4194304] 0 2024-05-25T06:11:42.950 INFO:tasks.workunit.client.0.smithi160.stdout:8/640: write d5/d44/d1f/f55 [1275342,20808] 0 2024-05-25T06:11:42.952 INFO:tasks.workunit.client.0.smithi160.stdout:6/598: truncate d5/de/f85 914646 0 2024-05-25T06:11:42.953 INFO:tasks.workunit.client.1.smithi203.stdout:0/935: rename dd/d10/d34/d4c/ld2 to dd/d5c/d63/d106/d75/l151 0 2024-05-25T06:11:42.953 INFO:tasks.workunit.client.0.smithi160.stdout:7/685: creat d6/d32/d7e/de0/fe5 x:0 0 0 2024-05-25T06:11:42.954 INFO:tasks.workunit.client.0.smithi160.stdout:1/544: mkdir de/d21/d49/d1c/d41/d45/d7d/db3 0 2024-05-25T06:11:42.957 INFO:tasks.workunit.client.0.smithi160.stdout:0/697: dread d2b/d31/d35/f49 [0,4194304] 0 2024-05-25T06:11:42.959 INFO:tasks.workunit.client.1.smithi203.stdout:0/936: symlink dd/d5c/l152 0 2024-05-25T06:11:42.959 INFO:tasks.workunit.client.0.smithi160.stdout:6/599: mkdir d5/de/d9e/dcd 0 2024-05-25T06:11:42.959 INFO:tasks.workunit.client.0.smithi160.stdout:2/619: dwrite d12/d75/dca/f15 [0,4194304] 0 2024-05-25T06:11:42.960 INFO:tasks.workunit.client.0.smithi160.stdout:7/686: chown d6/l98 99 1 2024-05-25T06:11:42.962 INFO:tasks.workunit.client.0.smithi160.stdout:2/620: write d12/d75/dca/f24 [2102317,78707] 0 2024-05-25T06:11:42.963 INFO:tasks.workunit.client.1.smithi203.stdout:0/937: rename dd/f1f to dd/d5c/d63/f153 0 2024-05-25T06:11:42.964 INFO:tasks.workunit.client.0.smithi160.stdout:3/530: rename d19/c20 to d19/d2f/cb4 0 2024-05-25T06:11:42.965 INFO:tasks.workunit.client.1.smithi203.stdout:0/938: dread - dd/d5c/d63/d106/d75/fcf zero size 2024-05-25T06:11:42.966 INFO:tasks.workunit.client.0.smithi160.stdout:1/545: creat de/d21/fb4 x:0 0 0 2024-05-25T06:11:42.969 INFO:tasks.workunit.client.0.smithi160.stdout:8/641: creat d5/d7/fdc x:0 0 0 2024-05-25T06:11:42.970 INFO:tasks.workunit.client.1.smithi203.stdout:9/894: write d0/df/d37/f58 [160402,41166] 0 2024-05-25T06:11:42.970 INFO:tasks.workunit.client.0.smithi160.stdout:9/585: getdents d4/d11/d12/d2e/d45/d83 0 2024-05-25T06:11:42.970 INFO:tasks.workunit.client.1.smithi203.stdout:1/849: rmdir d14/d57 39 2024-05-25T06:11:42.971 INFO:tasks.workunit.client.0.smithi160.stdout:8/642: dread - d5/d44/d1f/da3/d28/d4a/d4e/d86/f66 zero size 2024-05-25T06:11:42.972 INFO:tasks.workunit.client.1.smithi203.stdout:9/895: fdatasync d0/f4 0 2024-05-25T06:11:42.975 INFO:tasks.workunit.client.1.smithi203.stdout:1/850: symlink d14/d7c/d9d/dc8/l122 0 2024-05-25T06:11:42.977 INFO:tasks.workunit.client.1.smithi203.stdout:9/896: creat d0/d17/d20/d6e/d75/df2/d114/d5c/f127 x:0 0 0 2024-05-25T06:11:42.977 INFO:tasks.workunit.client.0.smithi160.stdout:1/546: mkdir de/d21/d49/d1c/d52/d77/db5 0 2024-05-25T06:11:42.978 INFO:tasks.workunit.client.1.smithi203.stdout:1/851: read - d14/d57/db7/dd2/dca/dcf/f119 zero size 2024-05-25T06:11:42.980 INFO:tasks.workunit.client.1.smithi203.stdout:1/852: fdatasync d14/dff/db8/fd8 0 2024-05-25T06:11:42.982 INFO:tasks.workunit.client.1.smithi203.stdout:9/897: getdents d0/d74 0 2024-05-25T06:11:42.983 INFO:tasks.workunit.client.1.smithi203.stdout:1/853: write d14/d57/db7/dd2/ddc/ffc [807823,44907] 0 2024-05-25T06:11:42.983 INFO:tasks.workunit.client.0.smithi160.stdout:8/643: dread d5/d44/d1f/f55 [0,4194304] 0 2024-05-25T06:11:42.984 INFO:tasks.workunit.client.1.smithi203.stdout:9/898: mkdir d0/df/ddd/d128 0 2024-05-25T06:11:42.984 INFO:tasks.workunit.client.0.smithi160.stdout:3/531: rename d19/d2f/d34/l3d to d19/d2f/d34/da8/lb5 0 2024-05-25T06:11:42.985 INFO:tasks.workunit.client.1.smithi203.stdout:9/899: fsync d0/dd0/fdb 0 2024-05-25T06:11:42.986 INFO:tasks.workunit.client.0.smithi160.stdout:1/547: write de/f92 [2428417,124269] 0 2024-05-25T06:11:42.986 INFO:tasks.workunit.client.0.smithi160.stdout:1/548: stat de/d21/d49/d1c/c4c 0 2024-05-25T06:11:42.988 INFO:tasks.workunit.client.0.smithi160.stdout:1/549: chown de/d21/d49/d35/c4b 15921789 1 2024-05-25T06:11:42.989 INFO:tasks.workunit.client.1.smithi203.stdout:0/939: chown dd/d5c/d63/d106/d75/l151 118097202 1 2024-05-25T06:11:42.990 INFO:tasks.workunit.client.0.smithi160.stdout:3/532: dread d19/f52 [0,4194304] 0 2024-05-25T06:11:42.991 INFO:tasks.workunit.client.0.smithi160.stdout:9/586: rename d4/db2/fa5 to d4/d11/d12/d2e/d45/d90/fba 0 2024-05-25T06:11:42.991 INFO:tasks.workunit.client.0.smithi160.stdout:3/533: dread - d19/d2f/d38/fb0 zero size 2024-05-25T06:11:42.992 INFO:tasks.workunit.client.1.smithi203.stdout:9/900: dwrite d0/df/d2d/d92/fdf [0,4194304] 0 2024-05-25T06:11:42.993 INFO:tasks.workunit.client.1.smithi203.stdout:0/940: stat dd/d5c/d63/d106/d75/fe1 0 2024-05-25T06:11:42.994 INFO:tasks.workunit.client.0.smithi160.stdout:6/600: getdents d5/d39/da6 0 2024-05-25T06:11:42.994 INFO:tasks.workunit.client.0.smithi160.stdout:0/698: write d2b/d31/d35/d43/d98/f99 [536119,86243] 0 2024-05-25T06:11:42.996 INFO:tasks.workunit.client.0.smithi160.stdout:0/699: write d2b/d46/faa [727740,56292] 0 2024-05-25T06:11:42.996 INFO:tasks.workunit.client.1.smithi203.stdout:9/901: creat d0/d57/f129 x:0 0 0 2024-05-25T06:11:42.996 INFO:tasks.workunit.client.1.smithi203.stdout:0/941: mkdir dd/d5c/d63/d106/d4f/db7/df0/d154 0 2024-05-25T06:11:42.998 INFO:tasks.workunit.client.0.smithi160.stdout:7/687: write d6/dc/d64/d5a/d7c/faa [1299560,36806] 0 2024-05-25T06:11:42.998 INFO:tasks.workunit.client.0.smithi160.stdout:1/550: write f7 [3573095,67341] 0 2024-05-25T06:11:42.999 INFO:tasks.workunit.client.1.smithi203.stdout:9/902: unlink d0/f91 0 2024-05-25T06:11:42.999 INFO:tasks.workunit.client.0.smithi160.stdout:2/621: truncate d12/d1c/f1e 1195338 0 2024-05-25T06:11:42.999 INFO:tasks.workunit.client.0.smithi160.stdout:0/700: dread - d2b/d31/d35/d43/fc4 zero size 2024-05-25T06:11:43.000 INFO:tasks.workunit.client.1.smithi203.stdout:0/942: dread - dd/d32/d108/f12a zero size 2024-05-25T06:11:43.002 INFO:tasks.workunit.client.0.smithi160.stdout:9/587: read d4/d11/d14/f37 [1065556,32819] 0 2024-05-25T06:11:43.002 INFO:tasks.workunit.client.0.smithi160.stdout:8/644: write d5/d9/d72/fbe [478929,13964] 0 2024-05-25T06:11:43.004 INFO:tasks.workunit.client.0.smithi160.stdout:3/534: fsync d19/d22/f2a 0 2024-05-25T06:11:43.007 INFO:tasks.workunit.client.0.smithi160.stdout:2/622: creat d12/d3c/fce x:0 0 0 2024-05-25T06:11:43.007 INFO:tasks.workunit.client.0.smithi160.stdout:8/645: creat d5/d7/d9a/fdd x:0 0 0 2024-05-25T06:11:43.007 INFO:tasks.workunit.client.0.smithi160.stdout:0/701: truncate d2b/f78 2711117 0 2024-05-25T06:11:43.007 INFO:tasks.workunit.client.1.smithi203.stdout:9/903: dwrite d0/df/d37/fd8 [0,4194304] 0 2024-05-25T06:11:43.009 INFO:tasks.workunit.client.0.smithi160.stdout:7/688: rename d6/dc/d64/dae/lb4 to d6/d2b/d5d/de2/le6 0 2024-05-25T06:11:43.009 INFO:tasks.workunit.client.1.smithi203.stdout:9/904: write d0/df/d2d/d92/fdf [2436076,71838] 0 2024-05-25T06:11:43.010 INFO:tasks.workunit.client.0.smithi160.stdout:7/689: chown d6/d3e/da1/fbf 163801 1 2024-05-25T06:11:43.011 INFO:tasks.workunit.client.1.smithi203.stdout:1/854: dread d14/d57/db7/dd2/fcd [0,4194304] 0 2024-05-25T06:11:43.012 INFO:tasks.workunit.client.0.smithi160.stdout:9/588: rmdir d4/d11/d14 39 2024-05-25T06:11:43.012 INFO:tasks.workunit.client.0.smithi160.stdout:7/690: readlink d6/dc/d21/l39 0 2024-05-25T06:11:43.013 INFO:tasks.workunit.client.1.smithi203.stdout:9/905: mkdir d0/d17/d20/d12a 0 2024-05-25T06:11:43.014 INFO:tasks.workunit.client.0.smithi160.stdout:3/535: rename d19/da6/f8a to d19/da6/fb6 0 2024-05-25T06:11:43.016 INFO:tasks.workunit.client.0.smithi160.stdout:0/702: fdatasync f20 0 2024-05-25T06:11:43.016 INFO:tasks.workunit.client.1.smithi203.stdout:1/855: creat d14/d7c/d9d/dc8/df3/f123 x:0 0 0 2024-05-25T06:11:43.018 INFO:tasks.workunit.client.0.smithi160.stdout:8/646: symlink d5/d44/lde 0 2024-05-25T06:11:43.020 INFO:tasks.workunit.client.0.smithi160.stdout:1/551: rename de/d21/d49/d1c/d41/d45/d7d/cb2 to de/d21/d49/d1c/d41/d96/cb6 0 2024-05-25T06:11:43.020 INFO:tasks.workunit.client.1.smithi203.stdout:9/906: mkdir d0/d17/d20/d6e/d75/df2/d114/d12b 0 2024-05-25T06:11:43.021 INFO:tasks.workunit.client.0.smithi160.stdout:1/552: readlink de/d21/d49/d28/l48 0 2024-05-25T06:11:43.022 INFO:tasks.workunit.client.0.smithi160.stdout:7/691: mkdir d6/d3e/da1/de7 0 2024-05-25T06:11:43.026 INFO:tasks.workunit.client.1.smithi203.stdout:7/924: sync 2024-05-25T06:11:43.027 INFO:tasks.workunit.client.1.smithi203.stdout:1/856: dwrite d14/d57/d7e/fb5 [0,4194304] 0 2024-05-25T06:11:43.028 INFO:tasks.workunit.client.0.smithi160.stdout:3/536: mknod d19/d2f/d38/d43/d5b/d65/d7f/cb7 0 2024-05-25T06:11:43.029 INFO:tasks.workunit.client.1.smithi203.stdout:7/925: write d1/d9/d2c/d62/d7a/d89/f8c [1097615,80557] 0 2024-05-25T06:11:43.029 INFO:tasks.workunit.client.1.smithi203.stdout:0/943: truncate dd/d10/d34/fa8 1174700 0 2024-05-25T06:11:43.032 INFO:tasks.workunit.client.1.smithi203.stdout:1/857: truncate d14/d57/db7/dd2/dca/ff9 516975 0 2024-05-25T06:11:43.032 INFO:tasks.workunit.client.1.smithi203.stdout:9/907: write d0/d17/d20/d6e/d75/df2/f111 [916717,51187] 0 2024-05-25T06:11:43.034 INFO:tasks.workunit.client.0.smithi160.stdout:7/692: symlink d6/dc/d64/d5a/d7c/le8 0 2024-05-25T06:11:43.035 INFO:tasks.workunit.client.1.smithi203.stdout:0/944: unlink dd/d10/d34/d4c/f147 0 2024-05-25T06:11:43.038 INFO:tasks.workunit.client.0.smithi160.stdout:2/623: dread d12/d75/dca/f4c [0,4194304] 0 2024-05-25T06:11:43.039 INFO:tasks.workunit.client.1.smithi203.stdout:1/858: creat d14/d23/d115/de4/f124 x:0 0 0 2024-05-25T06:11:43.040 INFO:tasks.workunit.client.0.smithi160.stdout:6/601: dwrite d5/de/d32/d35/f78 [0,4194304] 0 2024-05-25T06:11:43.041 INFO:tasks.workunit.client.1.smithi203.stdout:9/908: dwrite d0/d17/d20/d29/fe4 [0,4194304] 0 2024-05-25T06:11:43.042 INFO:tasks.workunit.client.1.smithi203.stdout:1/859: write d14/d57/d6b/da1/de2/ff5 [1360867,121886] 0 2024-05-25T06:11:43.043 INFO:tasks.workunit.client.0.smithi160.stdout:9/589: dread d4/d11/d12/f48 [0,4194304] 0 2024-05-25T06:11:43.044 INFO:tasks.workunit.client.1.smithi203.stdout:7/926: rename d1/d9/d2c/d36/d3b/f122 to d1/d9/d2c/f138 0 2024-05-25T06:11:43.044 INFO:tasks.workunit.client.1.smithi203.stdout:0/945: dread dd/d5c/d63/d106/df3/fc8 [0,4194304] 0 2024-05-25T06:11:43.045 INFO:tasks.workunit.client.0.smithi160.stdout:3/537: dwrite d19/d2f/d34/f40 [0,4194304] 0 2024-05-25T06:11:43.046 INFO:tasks.workunit.client.0.smithi160.stdout:0/703: rename f1e to d2b/d31/fe0 0 2024-05-25T06:11:43.049 INFO:tasks.workunit.client.1.smithi203.stdout:9/909: symlink d0/df/d37/d4f/d93/l12c 0 2024-05-25T06:11:43.050 INFO:tasks.workunit.client.0.smithi160.stdout:2/624: mknod d12/d1c/d59/ccf 0 2024-05-25T06:11:43.051 INFO:tasks.workunit.client.0.smithi160.stdout:8/647: link d5/d7/dcb/l80 d5/d44/d1f/da3/d28/d4a/d4e/d86/dd4/ldf 0 2024-05-25T06:11:43.052 INFO:tasks.workunit.client.0.smithi160.stdout:8/648: stat d5/d7/dcb/d7a 0 2024-05-25T06:11:43.053 INFO:tasks.workunit.client.0.smithi160.stdout:9/590: creat d4/d11/d12/d58/fbb x:0 0 0 2024-05-25T06:11:43.053 INFO:tasks.workunit.client.0.smithi160.stdout:8/649: stat d5/d7/dcb/d7a/c7c 0 2024-05-25T06:11:43.054 INFO:tasks.workunit.client.1.smithi203.stdout:7/927: mknod d1/d9/d2c/d36/d3b/c139 0 2024-05-25T06:11:43.054 INFO:tasks.workunit.client.1.smithi203.stdout:1/860: creat d14/d81/d100/f125 x:0 0 0 2024-05-25T06:11:43.054 INFO:tasks.workunit.client.0.smithi160.stdout:9/591: chown d4/d11/d1e/d3e/d2f 61233 1 2024-05-25T06:11:43.055 INFO:tasks.workunit.client.1.smithi203.stdout:0/946: creat dd/d5c/d63/d106/df3/d10c/f155 x:0 0 0 2024-05-25T06:11:43.056 INFO:tasks.workunit.client.0.smithi160.stdout:7/693: mknod d6/d3e/da1/ddb/ce9 0 2024-05-25T06:11:43.056 INFO:tasks.workunit.client.1.smithi203.stdout:9/910: readlink d0/d17/d20/d29/l4e 0 2024-05-25T06:11:43.058 INFO:tasks.workunit.client.1.smithi203.stdout:9/911: chown d0/df/d37/da8/dd6 766294 1 2024-05-25T06:11:43.059 INFO:tasks.workunit.client.0.smithi160.stdout:6/602: rename d5/de/d58/f80 to d5/d45/d4a/dbc/dc4/fce 0 2024-05-25T06:11:43.060 INFO:tasks.workunit.client.1.smithi203.stdout:0/947: symlink dd/d5c/d63/d106/da0/l156 0 2024-05-25T06:11:43.061 INFO:tasks.workunit.client.0.smithi160.stdout:0/704: readlink d2b/d31/d35/d36/l3d 0 2024-05-25T06:11:43.063 INFO:tasks.workunit.client.0.smithi160.stdout:8/650: creat d5/d44/d1f/da3/d28/d4a/d73/fe0 x:0 0 0 2024-05-25T06:11:43.065 INFO:tasks.workunit.client.1.smithi203.stdout:0/948: creat dd/d32/f157 x:0 0 0 2024-05-25T06:11:43.065 INFO:tasks.workunit.client.0.smithi160.stdout:8/651: truncate d5/d44/d1f/fd8 789387 0 2024-05-25T06:11:43.066 INFO:tasks.workunit.client.1.smithi203.stdout:7/928: dread d1/d9/d2c/d79/dd6/f108 [0,4194304] 0 2024-05-25T06:11:43.066 INFO:tasks.workunit.client.1.smithi203.stdout:0/949: chown dd/d5c/f73 57068 1 2024-05-25T06:11:43.067 INFO:tasks.workunit.client.0.smithi160.stdout:1/553: truncate de/d21/d49/d1c/d52/f56 742026 0 2024-05-25T06:11:43.067 INFO:tasks.workunit.client.0.smithi160.stdout:8/652: dread - d5/d44/d1f/fb6 zero size 2024-05-25T06:11:43.067 INFO:tasks.workunit.client.1.smithi203.stdout:0/950: chown dd/d5c/d63/d106/df3/l137 15438508 1 2024-05-25T06:11:43.067 INFO:tasks.workunit.client.0.smithi160.stdout:7/694: symlink d6/d2b/d5d/d89/dd6/lea 0 2024-05-25T06:11:43.068 INFO:tasks.workunit.client.0.smithi160.stdout:2/625: mknod d12/d3e/d5a/d8d/db4/dcb/cd0 0 2024-05-25T06:11:43.068 INFO:tasks.workunit.client.0.smithi160.stdout:8/653: write d5/d44/d1f/da3/d28/d4a/fb3 [943740,32529] 0 2024-05-25T06:11:43.069 INFO:tasks.workunit.client.0.smithi160.stdout:9/592: getdents d4/d11/d1e/d3e/d2f/da7 0 2024-05-25T06:11:43.070 INFO:tasks.workunit.client.0.smithi160.stdout:1/554: rmdir de 39 2024-05-25T06:11:43.070 INFO:tasks.workunit.client.1.smithi203.stdout:0/951: creat dd/d5c/dd5/f158 x:0 0 0 2024-05-25T06:11:43.070 INFO:tasks.workunit.client.0.smithi160.stdout:0/705: dread d2b/f2e [0,4194304] 0 2024-05-25T06:11:43.071 INFO:tasks.workunit.client.1.smithi203.stdout:0/952: fdatasync dd/d10/d34/d4c/d99/d70/f114 0 2024-05-25T06:11:43.072 INFO:tasks.workunit.client.1.smithi203.stdout:7/929: dwrite d1/d11/d3d/f7f [0,4194304] 0 2024-05-25T06:11:43.073 INFO:tasks.workunit.client.0.smithi160.stdout:7/695: symlink d6/dc/d64/leb 0 2024-05-25T06:11:43.073 INFO:tasks.workunit.client.0.smithi160.stdout:8/654: dread d5/d44/d1f/da3/d28/d4a/fb3 [0,4194304] 0 2024-05-25T06:11:43.074 INFO:tasks.workunit.client.1.smithi203.stdout:0/953: truncate dd/d32/d108/f12a 702450 0 2024-05-25T06:11:43.074 INFO:tasks.workunit.client.0.smithi160.stdout:8/655: readlink d5/d9c/lcc 0 2024-05-25T06:11:43.076 INFO:tasks.workunit.client.1.smithi203.stdout:7/930: unlink d1/d9/d2c/d36/dd3/l101 0 2024-05-25T06:11:43.077 INFO:tasks.workunit.client.1.smithi203.stdout:0/954: symlink dd/d5c/d63/d106/df3/d10c/l159 0 2024-05-25T06:11:43.079 INFO:tasks.workunit.client.1.smithi203.stdout:7/931: mknod d1/d9/d2c/d3a/dde/c13a 0 2024-05-25T06:11:43.081 INFO:tasks.workunit.client.1.smithi203.stdout:7/932: dread - d1/d9/d2c/d3a/dde/f10c zero size 2024-05-25T06:11:43.082 INFO:tasks.workunit.client.1.smithi203.stdout:7/933: chown d1/d9/d2c/d79 8827 1 2024-05-25T06:11:43.083 INFO:tasks.workunit.client.1.smithi203.stdout:7/934: stat d1/d11/fc0 0 2024-05-25T06:11:43.083 INFO:tasks.workunit.client.0.smithi160.stdout:6/603: link d5/de/d58/l6a d5/d39/da6/lcf 0 2024-05-25T06:11:43.083 INFO:tasks.workunit.client.0.smithi160.stdout:1/555: chown de/d21/d49/d1c/d59/d7f/da1 62821 1 2024-05-25T06:11:43.083 INFO:tasks.workunit.client.0.smithi160.stdout:8/656: dwrite d5/d44/d1f/da3/d28/d49/d61/f9f [0,4194304] 0 2024-05-25T06:11:43.084 INFO:tasks.workunit.client.1.smithi203.stdout:7/935: read d1/f31 [696071,14075] 0 2024-05-25T06:11:43.085 INFO:tasks.workunit.client.0.smithi160.stdout:9/593: mkdir d4/d11/d1e/dbc 0 2024-05-25T06:11:43.086 INFO:tasks.workunit.client.0.smithi160.stdout:3/538: dread d19/d2f/d38/d47/f54 [0,4194304] 0 2024-05-25T06:11:43.086 INFO:tasks.workunit.client.1.smithi203.stdout:7/936: write d1/d9/d60/dea/f113 [228000,20844] 0 2024-05-25T06:11:43.087 INFO:tasks.workunit.client.0.smithi160.stdout:1/556: write de/d21/d49/d1c/d59/d7c/f97 [609312,19810] 0 2024-05-25T06:11:43.089 INFO:tasks.workunit.client.0.smithi160.stdout:6/604: creat d5/de/d32/d35/dc3/fd0 x:0 0 0 2024-05-25T06:11:43.089 INFO:tasks.workunit.client.0.smithi160.stdout:1/557: write de/f15 [4922111,31969] 0 2024-05-25T06:11:43.090 INFO:tasks.workunit.client.0.smithi160.stdout:3/539: symlink d19/d2f/d38/d43/d5b/d65/d7e/lb8 0 2024-05-25T06:11:43.090 INFO:tasks.workunit.client.0.smithi160.stdout:8/657: read d5/d44/d1f/da3/d28/f38 [1709708,28947] 0 2024-05-25T06:11:43.091 INFO:tasks.workunit.client.0.smithi160.stdout:3/540: fdatasync d19/d2f/d38/d47/f54 0 2024-05-25T06:11:43.093 INFO:tasks.workunit.client.1.smithi203.stdout:7/937: mkdir d1/d9/d2c/d3a/d4e/d50/d13b 0 2024-05-25T06:11:43.093 INFO:tasks.workunit.client.0.smithi160.stdout:1/558: fsync de/d21/d49/d1c/d59/d71/f5a 0 2024-05-25T06:11:43.095 INFO:tasks.workunit.client.1.smithi203.stdout:0/955: rmdir dd/d5c/d63/d106/da0 39 2024-05-25T06:11:43.097 INFO:tasks.workunit.client.0.smithi160.stdout:3/541: write d19/d2f/d38/d60/f82 [702944,1262] 0 2024-05-25T06:11:43.097 INFO:tasks.workunit.client.0.smithi160.stdout:8/658: dread d5/d44/d1f/fd8 [0,4194304] 0 2024-05-25T06:11:43.101 INFO:tasks.workunit.client.0.smithi160.stdout:2/626: rmdir d12 39 2024-05-25T06:11:43.101 INFO:tasks.workunit.client.0.smithi160.stdout:6/605: dwrite d5/f26 [12582912,4194304] 0 2024-05-25T06:11:43.104 INFO:tasks.workunit.client.1.smithi203.stdout:7/938: dwrite d1/d11/d3d/d4c/f93 [4194304,4194304] 0 2024-05-25T06:11:43.106 INFO:tasks.workunit.client.0.smithi160.stdout:3/542: rename d19/f52 to d19/d2f/d38/d43/d5b/d65/fb9 0 2024-05-25T06:11:43.108 INFO:tasks.workunit.client.0.smithi160.stdout:3/543: readlink d19/d2f/d38/d60/l9b 0 2024-05-25T06:11:43.108 INFO:tasks.workunit.client.0.smithi160.stdout:2/627: creat d12/d1c/d37/d4b/fd1 x:0 0 0 2024-05-25T06:11:43.109 INFO:tasks.workunit.client.1.smithi203.stdout:7/939: chown d1/d9/d2c/d36/d54/c55 26340 1 2024-05-25T06:11:43.109 INFO:tasks.workunit.client.1.smithi203.stdout:0/956: dwrite dd/f24 [0,4194304] 0 2024-05-25T06:11:43.110 INFO:tasks.workunit.client.0.smithi160.stdout:6/606: dwrite d5/de/d32/d35/f4e [0,4194304] 0 2024-05-25T06:11:43.110 INFO:tasks.workunit.client.0.smithi160.stdout:1/559: rename de/d21/d49/d1c/d59/d7c/la4 to de/d21/d49/d1c/d41/d45/d7d/lb7 0 2024-05-25T06:11:43.111 INFO:tasks.workunit.client.1.smithi203.stdout:1/861: dwrite d14/d57/db7/dd2/db3/f85 [4194304,4194304] 0 2024-05-25T06:11:43.112 INFO:tasks.workunit.client.1.smithi203.stdout:1/862: chown d14/d81/c112 40744 1 2024-05-25T06:11:43.120 INFO:tasks.workunit.client.1.smithi203.stdout:1/863: truncate d14/d57/d6b/f50 2298413 0 2024-05-25T06:11:43.120 INFO:tasks.workunit.client.0.smithi160.stdout:2/628: symlink d12/d3c/ld2 0 2024-05-25T06:11:43.121 INFO:tasks.workunit.client.1.smithi203.stdout:1/864: write d14/d57/d1d/da8/fe7 [1662844,41406] 0 2024-05-25T06:11:43.126 INFO:tasks.workunit.client.1.smithi203.stdout:7/940: getdents d1/d9/d2c/d3a/dd1/df8 0 2024-05-25T06:11:43.127 INFO:tasks.workunit.client.1.smithi203.stdout:1/865: mkdir d14/d57/d97/dec/d126 0 2024-05-25T06:11:43.127 INFO:tasks.workunit.client.0.smithi160.stdout:6/607: mknod d5/de/d32/d35/cd1 0 2024-05-25T06:11:43.129 INFO:tasks.workunit.client.0.smithi160.stdout:0/706: dwrite d2b/d46/d63/da7/d67/fa5 [0,4194304] 0 2024-05-25T06:11:43.129 INFO:tasks.workunit.client.1.smithi203.stdout:7/941: chown d1/d9/d2c/d79/dd6/f12b 10786531 1 2024-05-25T06:11:43.130 INFO:tasks.workunit.client.1.smithi203.stdout:7/942: dread - d1/d9/d2c/d79/f12a zero size 2024-05-25T06:11:43.131 INFO:tasks.workunit.client.0.smithi160.stdout:3/544: dwrite d19/f99 [0,4194304] 0 2024-05-25T06:11:43.132 INFO:tasks.workunit.client.1.smithi203.stdout:7/943: dread - d1/d9/d2c/d79/df0/f133 zero size 2024-05-25T06:11:43.132 INFO:tasks.workunit.client.0.smithi160.stdout:0/707: write d2b/d31/d35/d36/f4a [4776512,44668] 0 2024-05-25T06:11:43.133 INFO:tasks.workunit.client.0.smithi160.stdout:8/659: rename d5/d44/d1f/da3/d28/d4a/f3a to d5/d7/dcb/fe1 0 2024-05-25T06:11:43.135 INFO:tasks.workunit.client.0.smithi160.stdout:6/608: symlink d5/de/d58/ld2 0 2024-05-25T06:11:43.135 INFO:tasks.workunit.client.0.smithi160.stdout:2/629: mkdir d12/d47/da0/dd3 0 2024-05-25T06:11:43.135 INFO:tasks.workunit.client.0.smithi160.stdout:3/545: rmdir d19/da6/d6a 39 2024-05-25T06:11:43.136 INFO:tasks.workunit.client.1.smithi203.stdout:7/944: creat d1/d9/d2c/d118/d127/f13c x:0 0 0 2024-05-25T06:11:43.140 INFO:tasks.workunit.client.0.smithi160.stdout:1/560: rename de/d21/d49/d1c/d63 to de/d21/d49/d35/db8 0 2024-05-25T06:11:43.141 INFO:tasks.workunit.client.1.smithi203.stdout:7/945: creat d1/d9/d2c/d62/d7a/d103/f13d x:0 0 0 2024-05-25T06:11:43.142 INFO:tasks.workunit.client.0.smithi160.stdout:0/708: symlink d2b/d31/d35/d36/d86/dd5/le1 0 2024-05-25T06:11:43.142 INFO:tasks.workunit.client.1.smithi203.stdout:7/946: stat d1/d11/d3d/f7f 0 2024-05-25T06:11:43.143 INFO:tasks.workunit.client.0.smithi160.stdout:8/660: mknod d5/d7/dcb/ce2 0 2024-05-25T06:11:43.143 INFO:tasks.workunit.client.0.smithi160.stdout:6/609: symlink d5/d45/d63/ld3 0 2024-05-25T06:11:43.145 INFO:tasks.workunit.client.0.smithi160.stdout:2/630: creat d12/d1c/d59/fd4 x:0 0 0 2024-05-25T06:11:43.145 INFO:tasks.workunit.client.0.smithi160.stdout:9/594: dread d4/d11/d12/d2e/d45/d90/fba [0,4194304] 0 2024-05-25T06:11:43.145 INFO:tasks.workunit.client.1.smithi203.stdout:7/947: symlink d1/d9/d2c/d79/l13e 0 2024-05-25T06:11:43.146 INFO:tasks.workunit.client.0.smithi160.stdout:8/661: chown d5/d44/d1f/da3/d28/d49/fc4 188 1 2024-05-25T06:11:43.147 INFO:tasks.workunit.client.1.smithi203.stdout:7/948: mknod d1/d9/d2c/d36/d3b/c13f 0 2024-05-25T06:11:43.147 INFO:tasks.workunit.client.0.smithi160.stdout:0/709: rename d2b/d31/d35/d43/dab/fac to d2b/d46/dcb/fe2 0 2024-05-25T06:11:43.149 INFO:tasks.workunit.client.0.smithi160.stdout:3/546: unlink d19/d2f/d38/d43/d5b/d65/d7e/c85 0 2024-05-25T06:11:43.149 INFO:tasks.workunit.client.0.smithi160.stdout:3/547: stat d19/c9a 0 2024-05-25T06:11:43.149 INFO:tasks.workunit.client.1.smithi203.stdout:7/949: mkdir d1/d11/d3d/d9a/d140 0 2024-05-25T06:11:43.150 INFO:tasks.workunit.client.0.smithi160.stdout:3/548: write d19/d22/f8f [1199870,17663] 0 2024-05-25T06:11:43.151 INFO:tasks.workunit.client.0.smithi160.stdout:3/549: readlink d19/d2f/d38/d43/d5b/d65/d7e/lb8 0 2024-05-25T06:11:43.152 INFO:tasks.workunit.client.0.smithi160.stdout:2/631: creat d12/d3c/fd5 x:0 0 0 2024-05-25T06:11:43.152 INFO:tasks.workunit.client.0.smithi160.stdout:2/632: chown f8 20086288 1 2024-05-25T06:11:43.153 INFO:tasks.workunit.client.1.smithi203.stdout:7/950: dread d1/d9/f29 [4194304,4194304] 0 2024-05-25T06:11:43.153 INFO:tasks.workunit.client.0.smithi160.stdout:6/610: symlink d5/de/d32/d74/d77/d89/ld4 0 2024-05-25T06:11:43.153 INFO:tasks.workunit.client.0.smithi160.stdout:2/633: dread - d12/d3c/f9e zero size 2024-05-25T06:11:43.154 INFO:tasks.workunit.client.0.smithi160.stdout:0/710: read d2b/d31/d35/f3e [3728015,25446] 0 2024-05-25T06:11:43.155 INFO:tasks.workunit.client.0.smithi160.stdout:0/711: chown d2b/c2f 58 1 2024-05-25T06:11:43.156 INFO:tasks.workunit.client.0.smithi160.stdout:0/712: truncate d2b/d46/faa 1462071 0 2024-05-25T06:11:43.156 INFO:tasks.workunit.client.0.smithi160.stdout:6/611: mkdir d5/de/d32/d9f/dd5 0 2024-05-25T06:11:43.157 INFO:tasks.workunit.client.0.smithi160.stdout:3/550: mkdir d19/d2f/d38/d43/d5b/d65/d83/dba 0 2024-05-25T06:11:43.158 INFO:tasks.workunit.client.0.smithi160.stdout:3/551: fsync d19/d22/f2d 0 2024-05-25T06:11:43.158 INFO:tasks.workunit.client.1.smithi203.stdout:7/951: dwrite d1/d9/d2c/d36/f9d [0,4194304] 0 2024-05-25T06:11:43.158 INFO:tasks.workunit.client.0.smithi160.stdout:8/662: dread d5/d7/dcb/d7a/f85 [0,4194304] 0 2024-05-25T06:11:43.159 INFO:tasks.workunit.client.0.smithi160.stdout:9/595: rename d4/d11/d1e/d3e/d2f/c43 to d4/d11/cbd 0 2024-05-25T06:11:43.165 INFO:tasks.workunit.client.0.smithi160.stdout:8/663: mkdir d5/d44/dba/ddb/de3 0 2024-05-25T06:11:43.165 INFO:tasks.workunit.client.0.smithi160.stdout:6/612: dread d5/d39/f51 [0,4194304] 0 2024-05-25T06:11:43.167 INFO:tasks.workunit.client.1.smithi203.stdout:7/952: dwrite d1/d9/d2c/d36/d54/fc3 [0,4194304] 0 2024-05-25T06:11:43.169 INFO:tasks.workunit.client.0.smithi160.stdout:0/713: dwrite d2b/d46/d63/da7/d67/db8/fd0 [0,4194304] 0 2024-05-25T06:11:43.169 INFO:tasks.workunit.client.0.smithi160.stdout:8/664: chown d5/d44/lc0 14279651 1 2024-05-25T06:11:43.169 INFO:tasks.workunit.client.0.smithi160.stdout:9/596: rename d4/d11/d1e/d3e/d25/d27/c5b to d4/d11/d1e/d24/cbe 0 2024-05-25T06:11:43.170 INFO:tasks.workunit.client.0.smithi160.stdout:3/552: dwrite d19/d2f/d38/d60/f82 [0,4194304] 0 2024-05-25T06:11:43.171 INFO:tasks.workunit.client.1.smithi203.stdout:7/953: fsync d1/d9/fe 0 2024-05-25T06:11:43.172 INFO:tasks.workunit.client.0.smithi160.stdout:8/665: write d5/d9/fc [4898718,18170] 0 2024-05-25T06:11:43.172 INFO:tasks.workunit.client.0.smithi160.stdout:9/597: chown d4/d11/d1e/l7f 4 1 2024-05-25T06:11:43.176 INFO:tasks.workunit.client.1.smithi203.stdout:7/954: creat d1/d9/d2c/d36/d54/f141 x:0 0 0 2024-05-25T06:11:43.177 INFO:tasks.workunit.client.0.smithi160.stdout:9/598: dread d4/d11/d1e/f4d [0,4194304] 0 2024-05-25T06:11:43.178 INFO:tasks.workunit.client.0.smithi160.stdout:4/599: sync 2024-05-25T06:11:43.179 INFO:tasks.workunit.client.0.smithi160.stdout:0/714: creat d2b/d31/d6d/fe3 x:0 0 0 2024-05-25T06:11:43.180 INFO:tasks.workunit.client.0.smithi160.stdout:9/599: dread - d4/d11/d1e/d24/d66/fa0 zero size 2024-05-25T06:11:43.183 INFO:tasks.workunit.client.0.smithi160.stdout:6/613: getdents d5/de/d32/d35/d40 0 2024-05-25T06:11:43.183 INFO:tasks.workunit.client.0.smithi160.stdout:4/600: chown d4/d6/d5b/d66/c6f 12443 1 2024-05-25T06:11:43.183 INFO:tasks.workunit.client.0.smithi160.stdout:3/553: dwrite d19/d22/f24 [0,4194304] 0 2024-05-25T06:11:43.186 INFO:tasks.workunit.client.0.smithi160.stdout:0/715: mknod d2b/d46/dbd/ce4 0 2024-05-25T06:11:43.188 INFO:tasks.workunit.client.0.smithi160.stdout:6/614: unlink d5/de/c1f 0 2024-05-25T06:11:43.190 INFO:tasks.workunit.client.0.smithi160.stdout:3/554: fsync d19/d2f/d38/d43/d5b/d65/d7f/f81 0 2024-05-25T06:11:43.191 INFO:tasks.workunit.client.0.smithi160.stdout:8/666: getdents d5/d44/d1f/da3/d28/d49 0 2024-05-25T06:11:43.193 INFO:tasks.workunit.client.0.smithi160.stdout:9/600: rmdir d4/d11/d1e/d3e/d25/d5e/da2 39 2024-05-25T06:11:43.195 INFO:tasks.workunit.client.0.smithi160.stdout:3/555: mknod d19/d2f/d38/d43/d5b/d65/d7e/cbb 0 2024-05-25T06:11:43.195 INFO:tasks.workunit.client.0.smithi160.stdout:0/716: truncate d2b/d31/d35/f55 634704 0 2024-05-25T06:11:43.196 INFO:tasks.workunit.client.0.smithi160.stdout:8/667: symlink d5/d9/d72/le4 0 2024-05-25T06:11:43.197 INFO:tasks.workunit.client.0.smithi160.stdout:3/556: chown d19/d2f/c4f 1 1 2024-05-25T06:11:43.198 INFO:tasks.workunit.client.0.smithi160.stdout:0/717: mknod d2b/d46/d63/da7/d67/ce5 0 2024-05-25T06:11:43.198 INFO:tasks.workunit.client.0.smithi160.stdout:3/557: stat d19/d2f/d38/fb0 0 2024-05-25T06:11:43.199 INFO:tasks.workunit.client.0.smithi160.stdout:3/558: stat d19/d2f/fa5 0 2024-05-25T06:11:43.204 INFO:tasks.workunit.client.0.smithi160.stdout:8/668: dwrite d5/d44/d1f/fb6 [0,4194304] 0 2024-05-25T06:11:43.207 INFO:tasks.workunit.client.0.smithi160.stdout:0/718: dwrite d2b/d31/d35/d36/fd3 [0,4194304] 0 2024-05-25T06:11:43.211 INFO:tasks.workunit.client.0.smithi160.stdout:0/719: rmdir d2b/d46/d63/db3 39 2024-05-25T06:11:43.211 INFO:tasks.workunit.client.0.smithi160.stdout:0/720: fdatasync f23 0 2024-05-25T06:11:43.213 INFO:tasks.workunit.client.0.smithi160.stdout:8/669: dwrite d5/d7/dcb/d75/fa5 [0,4194304] 0 2024-05-25T06:11:43.218 INFO:tasks.workunit.client.1.smithi203.stdout:7/955: dread d1/d9/d2c/d3a/d4e/f61 [0,4194304] 0 2024-05-25T06:11:43.220 INFO:tasks.workunit.client.0.smithi160.stdout:0/721: rename d2b/d46/d63/da7/f72 to d2b/d31/d35/d36/d86/dd5/fe6 0 2024-05-25T06:11:43.220 INFO:tasks.workunit.client.0.smithi160.stdout:8/670: fsync d5/d44/d1f/da3/d28/d4a/d73/f91 0 2024-05-25T06:11:43.221 INFO:tasks.workunit.client.0.smithi160.stdout:0/722: dread - d2b/d46/dcb/fd1 zero size 2024-05-25T06:11:43.221 INFO:tasks.workunit.client.1.smithi203.stdout:0/957: dwrite dd/d32/d37/f101 [0,4194304] 0 2024-05-25T06:11:43.222 INFO:tasks.workunit.client.0.smithi160.stdout:8/671: write d5/d44/f68 [756517,58086] 0 2024-05-25T06:11:43.223 INFO:tasks.workunit.client.1.smithi203.stdout:0/958: stat dd/d5c/d63/d106/df3/d10c/f12b 0 2024-05-25T06:11:43.227 INFO:tasks.workunit.client.1.smithi203.stdout:7/956: dread d1/d9/d2c/d36/d54/fc3 [0,4194304] 0 2024-05-25T06:11:43.228 INFO:tasks.workunit.client.0.smithi160.stdout:8/672: rename d5/d44/d1f/da3/d28/d4a/d73/f91 to d5/d7/d9a/db5/fe5 0 2024-05-25T06:11:43.228 INFO:tasks.workunit.client.1.smithi203.stdout:0/959: creat dd/d10/df5/f15a x:0 0 0 2024-05-25T06:11:43.231 INFO:tasks.workunit.client.1.smithi203.stdout:0/960: rmdir dd/d5c/d63/d106/d4f 39 2024-05-25T06:11:43.236 INFO:tasks.workunit.client.1.smithi203.stdout:0/961: write dd/d5c/d63/d106/da0/db9/df9/d86/fbf [2386263,77288] 0 2024-05-25T06:11:43.236 INFO:tasks.workunit.client.1.smithi203.stdout:0/962: mkdir dd/d5c/d63/d106/df3/de2/d15b 0 2024-05-25T06:11:43.239 INFO:tasks.workunit.client.1.smithi203.stdout:0/963: mknod dd/d5c/d63/d106/d75/d117/c15c 0 2024-05-25T06:11:43.239 INFO:tasks.workunit.client.1.smithi203.stdout:0/964: stat dd/d10/d34/d4c/d99 0 2024-05-25T06:11:43.242 INFO:tasks.workunit.client.1.smithi203.stdout:0/965: mknod dd/d5c/d63/d106/d48/db1/c15d 0 2024-05-25T06:11:43.244 INFO:tasks.workunit.client.1.smithi203.stdout:0/966: symlink dd/d32/d37/l15e 0 2024-05-25T06:11:43.246 INFO:tasks.workunit.client.1.smithi203.stdout:0/967: creat dd/d45/f15f x:0 0 0 2024-05-25T06:11:43.248 INFO:tasks.workunit.client.1.smithi203.stdout:0/968: truncate dd/d5c/d63/d106/df3/ffe 26378 0 2024-05-25T06:11:43.251 INFO:tasks.workunit.client.0.smithi160.stdout:0/723: read d2b/d46/d63/f7e [4070357,88268] 0 2024-05-25T06:11:43.252 INFO:tasks.workunit.client.1.smithi203.stdout:0/969: dread dd/d5c/d63/d106/d4f/db7/df0/f149 [0,4194304] 0 2024-05-25T06:11:43.253 INFO:tasks.workunit.client.0.smithi160.stdout:0/724: dread - d2b/d31/d35/d36/f84 zero size 2024-05-25T06:11:43.253 INFO:tasks.workunit.client.1.smithi203.stdout:0/970: chown dd/d5c/d63/d106/d75/d93/c9a 2 1 2024-05-25T06:11:43.253 INFO:tasks.workunit.client.1.smithi203.stdout:0/971: read dd/d5c/f98 [279334,65786] 0 2024-05-25T06:11:43.290 INFO:tasks.workunit.client.1.smithi203.stdout:1/866: dwrite d14/d57/db7/fc9 [0,4194304] 0 2024-05-25T06:11:43.291 INFO:tasks.workunit.client.1.smithi203.stdout:1/867: chown d14/d57/db7/dd2/c75 2002 1 2024-05-25T06:11:43.294 INFO:tasks.workunit.client.1.smithi203.stdout:1/868: symlink d14/d7c/l127 0 2024-05-25T06:11:43.295 INFO:tasks.workunit.client.1.smithi203.stdout:1/869: write d14/d57/d6b/da1/faa [4689710,30629] 0 2024-05-25T06:11:43.296 INFO:tasks.workunit.client.0.smithi160.stdout:1/561: write de/f13 [1178538,55354] 0 2024-05-25T06:11:43.298 INFO:tasks.workunit.client.1.smithi203.stdout:1/870: chown d14/d57/dbb/cd4 1168672122 1 2024-05-25T06:11:43.301 INFO:tasks.workunit.client.0.smithi160.stdout:5/737: sync 2024-05-25T06:11:43.302 INFO:tasks.workunit.client.1.smithi203.stdout:1/871: getdents d14/d57/db7/dd2/db3 0 2024-05-25T06:11:43.302 INFO:tasks.workunit.client.0.smithi160.stdout:1/562: rename de/d21/d49/d1c/d59/d7c/d93/f84 to de/d21/d49/d1c/d59/d67/fb9 0 2024-05-25T06:11:43.304 INFO:tasks.workunit.client.1.smithi203.stdout:1/872: mknod d14/d57/d97/da4/dad/c128 0 2024-05-25T06:11:43.305 INFO:tasks.workunit.client.0.smithi160.stdout:0/725: dread d2b/d31/f7a [0,4194304] 0 2024-05-25T06:11:43.306 INFO:tasks.workunit.client.1.smithi203.stdout:1/873: mknod d14/d23/c129 0 2024-05-25T06:11:43.306 INFO:tasks.workunit.client.0.smithi160.stdout:0/726: dread - d2b/d31/f7d zero size 2024-05-25T06:11:43.307 INFO:tasks.workunit.client.0.smithi160.stdout:1/563: rmdir de/d21/d8a 0 2024-05-25T06:11:43.309 INFO:tasks.workunit.client.0.smithi160.stdout:2/634: dwrite d12/d1c/f1e [0,4194304] 0 2024-05-25T06:11:43.309 INFO:tasks.workunit.client.0.smithi160.stdout:5/738: getdents db/dc/d13/d1d/d3f/d72/d98/da5/dae 0 2024-05-25T06:11:43.309 INFO:tasks.workunit.client.1.smithi203.stdout:1/874: unlink d14/d23/f78 0 2024-05-25T06:11:43.312 INFO:tasks.workunit.client.0.smithi160.stdout:1/564: chown de/d21/d49/d1c/d59/d67/fb9 733363532 1 2024-05-25T06:11:43.312 INFO:tasks.workunit.client.0.smithi160.stdout:2/635: write d12/d3c/fd5 [268933,60748] 0 2024-05-25T06:11:43.312 INFO:tasks.workunit.client.1.smithi203.stdout:1/875: symlink d14/d57/d6b/l12a 0 2024-05-25T06:11:43.314 INFO:tasks.workunit.client.0.smithi160.stdout:2/636: truncate d12/d3e/d5a/f93 840154 0 2024-05-25T06:11:43.315 INFO:tasks.workunit.client.0.smithi160.stdout:5/739: creat db/dc/d13/d1d/d3f/d72/fe2 x:0 0 0 2024-05-25T06:11:43.318 INFO:tasks.workunit.client.0.smithi160.stdout:4/601: truncate d4/d6/d21/f27 4938698 0 2024-05-25T06:11:43.320 INFO:tasks.workunit.client.1.smithi203.stdout:7/957: write d1/d9/f2e [869804,14411] 0 2024-05-25T06:11:43.320 INFO:tasks.workunit.client.0.smithi160.stdout:2/637: creat d12/d75/dca/fd6 x:0 0 0 2024-05-25T06:11:43.322 INFO:tasks.workunit.client.0.smithi160.stdout:6/615: stat d5/de/c3b 0 2024-05-25T06:11:43.323 INFO:tasks.workunit.client.0.smithi160.stdout:2/638: write f10 [1312683,35496] 0 2024-05-25T06:11:43.326 INFO:tasks.workunit.client.0.smithi160.stdout:2/639: chown d12/d3c/f61 13884 1 2024-05-25T06:11:43.328 INFO:tasks.workunit.client.1.smithi203.stdout:7/958: rename d1/d9/d2c/d62/d7a/d89/fec to d1/d9/d2c/d36/f142 0 2024-05-25T06:11:43.329 INFO:tasks.workunit.client.0.smithi160.stdout:1/565: dwrite de/f3a [0,4194304] 0 2024-05-25T06:11:43.330 INFO:tasks.workunit.client.1.smithi203.stdout:1/876: write d14/d23/d115/f80 [1776849,64958] 0 2024-05-25T06:11:43.331 INFO:tasks.workunit.client.0.smithi160.stdout:4/602: link d4/f26 d4/d63/d64/fb9 0 2024-05-25T06:11:43.334 INFO:tasks.workunit.client.0.smithi160.stdout:2/640: write d12/d1c/f27 [3871172,22491] 0 2024-05-25T06:11:43.336 INFO:tasks.workunit.client.1.smithi203.stdout:1/877: mknod d14/d57/d97/da4/dad/c12b 0 2024-05-25T06:11:43.336 INFO:tasks.workunit.client.1.smithi203.stdout:7/959: dwrite d1/d9/d2c/d36/d66/fba [0,4194304] 0 2024-05-25T06:11:43.338 INFO:tasks.workunit.client.1.smithi203.stdout:7/960: dread - d1/d9/d2c/d36/fdc zero size 2024-05-25T06:11:43.339 INFO:tasks.workunit.client.1.smithi203.stdout:1/878: symlink d14/d81/l12c 0 2024-05-25T06:11:43.339 INFO:tasks.workunit.client.1.smithi203.stdout:7/961: fsync d1/d9/d2c/d118/d127/d10b/f120 0 2024-05-25T06:11:43.341 INFO:tasks.workunit.client.0.smithi160.stdout:3/559: dwrite d19/f27 [0,4194304] 0 2024-05-25T06:11:43.341 INFO:tasks.workunit.client.1.smithi203.stdout:1/879: write d14/d7c/fd3 [300806,116515] 0 2024-05-25T06:11:43.342 INFO:tasks.workunit.client.0.smithi160.stdout:5/740: dwrite db/d47/d81/dc1/fda [0,4194304] 0 2024-05-25T06:11:43.344 INFO:tasks.workunit.client.0.smithi160.stdout:1/566: mkdir de/d21/d49/d1c/d41/d45/d7d/db3/dba 0 2024-05-25T06:11:43.346 INFO:tasks.workunit.client.0.smithi160.stdout:4/603: rename d4/d6/d28/l90 to d4/d63/lba 0 2024-05-25T06:11:43.349 INFO:tasks.workunit.client.0.smithi160.stdout:2/641: rmdir d12/d3e/d5a/d8d/db4/dcb 39 2024-05-25T06:11:43.350 INFO:tasks.workunit.client.1.smithi203.stdout:1/880: fsync d14/d57/d97/fa3 0 2024-05-25T06:11:43.353 INFO:tasks.workunit.client.0.smithi160.stdout:0/727: dread d2b/da8/fb0 [0,4194304] 0 2024-05-25T06:11:43.355 INFO:tasks.workunit.client.1.smithi203.stdout:1/881: symlink d14/d57/d97/dec/d126/l12d 0 2024-05-25T06:11:43.356 INFO:tasks.workunit.client.0.smithi160.stdout:4/604: creat d4/d6/d5b/d66/d82/fbb x:0 0 0 2024-05-25T06:11:43.359 INFO:tasks.workunit.client.0.smithi160.stdout:2/642: dwrite d12/d1c/f2c [0,4194304] 0 2024-05-25T06:11:43.362 INFO:tasks.workunit.client.0.smithi160.stdout:1/567: symlink de/d21/d49/d1c/d41/d45/dac/lbb 0 2024-05-25T06:11:43.364 INFO:tasks.workunit.client.0.smithi160.stdout:6/616: link d5/de/d32/d74/lc5 d5/de/d32/ld6 0 2024-05-25T06:11:43.365 INFO:tasks.workunit.client.0.smithi160.stdout:6/617: stat d5/d39/d8f/lc2 0 2024-05-25T06:11:43.366 INFO:tasks.workunit.client.0.smithi160.stdout:2/643: symlink d12/d1c/d37/d49/ld7 0 2024-05-25T06:11:43.369 INFO:tasks.workunit.client.0.smithi160.stdout:4/605: mkdir d4/d1c/d54/dbc 0 2024-05-25T06:11:43.369 INFO:tasks.workunit.client.0.smithi160.stdout:5/741: creat db/dc/d13/d1d/d23/d45/fe3 x:0 0 0 2024-05-25T06:11:43.369 INFO:tasks.workunit.client.0.smithi160.stdout:3/560: link d19/c51 d19/d2f/d38/d43/d5b/cbc 0 2024-05-25T06:11:43.371 INFO:tasks.workunit.client.0.smithi160.stdout:5/742: dread - db/dc/d13/fd0 zero size 2024-05-25T06:11:43.372 INFO:tasks.workunit.client.0.smithi160.stdout:2/644: rename d12/d1c/c33 to d12/d75/dae/cd8 0 2024-05-25T06:11:43.373 INFO:tasks.workunit.client.0.smithi160.stdout:0/728: getdents d2b/d31/d35 0 2024-05-25T06:11:43.374 INFO:tasks.workunit.client.0.smithi160.stdout:1/568: creat de/d39/d8f/fbc x:0 0 0 2024-05-25T06:11:43.375 INFO:tasks.workunit.client.0.smithi160.stdout:4/606: mkdir d4/d6/dbd 0 2024-05-25T06:11:43.376 INFO:tasks.workunit.client.0.smithi160.stdout:1/569: write de/d39/d8f/fbc [383071,26348] 0 2024-05-25T06:11:43.376 INFO:tasks.workunit.client.0.smithi160.stdout:3/561: rename d19/f8e to d19/d2f/d38/d43/d5b/d65/fbd 0 2024-05-25T06:11:43.377 INFO:tasks.workunit.client.0.smithi160.stdout:7/696: sync 2024-05-25T06:11:43.378 INFO:tasks.workunit.client.0.smithi160.stdout:0/729: fdatasync d2b/da8/fb9 0 2024-05-25T06:11:43.381 INFO:tasks.workunit.client.0.smithi160.stdout:4/607: creat d4/d63/fbe x:0 0 0 2024-05-25T06:11:43.381 INFO:tasks.workunit.client.0.smithi160.stdout:6/618: dread d5/d39/d8f/f55 [4194304,4194304] 0 2024-05-25T06:11:43.383 INFO:tasks.workunit.client.0.smithi160.stdout:4/608: chown d4/d63/d70/d94 310763069 1 2024-05-25T06:11:43.383 INFO:tasks.workunit.client.0.smithi160.stdout:3/562: mknod d19/d2f/d38/d43/cbe 0 2024-05-25T06:11:43.384 INFO:tasks.workunit.client.0.smithi160.stdout:2/645: dwrite d12/d75/dca/f55 [0,4194304] 0 2024-05-25T06:11:43.385 INFO:tasks.workunit.client.0.smithi160.stdout:0/730: truncate d2b/d31/d35/d36/f89 1865342 0 2024-05-25T06:11:43.387 INFO:tasks.workunit.client.0.smithi160.stdout:5/743: rename db/d47/d5e/d6e/la1 to db/dc/d13/d1d/le4 0 2024-05-25T06:11:43.388 INFO:tasks.workunit.client.0.smithi160.stdout:6/619: write d5/de/d7d/fcb [833585,46795] 0 2024-05-25T06:11:43.389 INFO:tasks.workunit.client.1.smithi203.stdout:1/882: dread d14/d57/d1d/da8/fe7 [0,4194304] 0 2024-05-25T06:11:43.390 INFO:tasks.workunit.client.0.smithi160.stdout:3/563: creat d19/d2f/d38/d43/d5b/d65/fbf x:0 0 0 2024-05-25T06:11:43.390 INFO:tasks.workunit.client.0.smithi160.stdout:2/646: rmdir d12/d1c/d59 39 2024-05-25T06:11:43.393 INFO:tasks.workunit.client.0.smithi160.stdout:1/570: rename de/d21/d49/d35/db8 to de/d21/d49/d1c/d59/d7f/da1/dbd 0 2024-05-25T06:11:43.394 INFO:tasks.workunit.client.0.smithi160.stdout:2/647: write d12/d3e/d5f/d74/d7a/fac [661292,90612] 0 2024-05-25T06:11:43.394 INFO:tasks.workunit.client.0.smithi160.stdout:6/620: readlink d5/de/l10 0 2024-05-25T06:11:43.396 INFO:tasks.workunit.client.0.smithi160.stdout:4/609: symlink d4/d6/d21/d42/dae/lbf 0 2024-05-25T06:11:43.396 INFO:tasks.workunit.client.0.smithi160.stdout:0/731: truncate d2b/d31/d35/f55 691824 0 2024-05-25T06:11:43.397 INFO:tasks.workunit.client.0.smithi160.stdout:0/732: chown d2b/d31/d35/fc9 42336163 1 2024-05-25T06:11:43.397 INFO:tasks.workunit.client.0.smithi160.stdout:3/564: truncate d19/d2f/d38/f3e 294871 0 2024-05-25T06:11:43.398 INFO:tasks.workunit.client.0.smithi160.stdout:3/565: readlink d19/d22/l25 0 2024-05-25T06:11:43.399 INFO:tasks.workunit.client.0.smithi160.stdout:3/566: chown d19/d2f/d38/f78 480861 1 2024-05-25T06:11:43.399 INFO:tasks.workunit.client.0.smithi160.stdout:4/610: mknod d4/d6/d21/cc0 0 2024-05-25T06:11:43.402 INFO:tasks.workunit.client.0.smithi160.stdout:9/601: sync 2024-05-25T06:11:43.402 INFO:tasks.workunit.client.0.smithi160.stdout:8/673: sync 2024-05-25T06:11:43.403 INFO:tasks.workunit.client.0.smithi160.stdout:2/648: link d12/d47/ca6 d12/d1c/d2a/d7e/cd9 0 2024-05-25T06:11:43.404 INFO:tasks.workunit.client.0.smithi160.stdout:8/674: write d5/d7/dcb/d7a/f92 [2571822,74080] 0 2024-05-25T06:11:43.405 INFO:tasks.workunit.client.0.smithi160.stdout:3/567: rename d19/l36 to d19/d2f/d34/lc0 0 2024-05-25T06:11:43.406 INFO:tasks.workunit.client.1.smithi203.stdout:7/962: write d1/d9/d2c/d3a/d4e/f6c [966679,89267] 0 2024-05-25T06:11:43.407 INFO:tasks.workunit.client.1.smithi203.stdout:1/883: write d14/dff/db8/fb0 [530498,119605] 0 2024-05-25T06:11:43.408 INFO:tasks.workunit.client.0.smithi160.stdout:4/611: getdents d4/d6/d7b 0 2024-05-25T06:11:43.410 INFO:tasks.workunit.client.1.smithi203.stdout:7/963: fsync d1/dbd/dc1/fe6 0 2024-05-25T06:11:43.411 INFO:tasks.workunit.client.0.smithi160.stdout:9/602: fsync d4/d11/d14/f61 0 2024-05-25T06:11:43.411 INFO:tasks.workunit.client.0.smithi160.stdout:7/697: dread d6/d3e/fca [0,4194304] 0 2024-05-25T06:11:43.413 INFO:tasks.workunit.client.1.smithi203.stdout:1/884: rename d14/d57/dbb/df8/l110 to d14/d57/d7e/l12e 0 2024-05-25T06:11:43.414 INFO:tasks.workunit.client.0.smithi160.stdout:8/675: rename d5/d44/d1f/da3/d28/d4a/d73 to d5/d44/d1f/da3/d28/de6 0 2024-05-25T06:11:43.415 INFO:tasks.workunit.client.0.smithi160.stdout:2/649: dwrite d12/d3c/f61 [0,4194304] 0 2024-05-25T06:11:43.417 INFO:tasks.workunit.client.0.smithi160.stdout:4/612: creat d4/d63/d64/fc1 x:0 0 0 2024-05-25T06:11:43.418 INFO:tasks.workunit.client.1.smithi203.stdout:7/964: dwrite d1/d9/dc2/fcb [0,4194304] 0 2024-05-25T06:11:43.419 INFO:tasks.workunit.client.1.smithi203.stdout:9/912: sync 2024-05-25T06:11:43.421 INFO:tasks.workunit.client.0.smithi160.stdout:9/603: creat d4/d11/d1e/d24/d66/fbf x:0 0 0 2024-05-25T06:11:43.422 INFO:tasks.workunit.client.1.smithi203.stdout:9/913: chown d0/df/d2d/ld3 3052204 1 2024-05-25T06:11:43.424 INFO:tasks.workunit.client.1.smithi203.stdout:1/885: dwrite d14/d7c/d9d/dc8/df3/f123 [0,4194304] 0 2024-05-25T06:11:43.425 INFO:tasks.workunit.client.0.smithi160.stdout:7/698: dwrite d6/d2b/d5d/d89/fa9 [0,4194304] 0 2024-05-25T06:11:43.427 INFO:tasks.workunit.client.0.smithi160.stdout:2/650: creat d12/d1c/d2a/fda x:0 0 0 2024-05-25T06:11:43.428 INFO:tasks.workunit.client.0.smithi160.stdout:2/651: dread - d12/d1c/d37/d4b/f89 zero size 2024-05-25T06:11:43.433 INFO:tasks.workunit.client.0.smithi160.stdout:8/676: rename d5/d9/d72/fbe to d5/d44/d1f/da3/d28/de6/dc2/dd6/fe7 0 2024-05-25T06:11:43.439 INFO:tasks.workunit.client.1.smithi203.stdout:7/965: dwrite d1/d9/d2c/d36/d54/ffb [0,4194304] 0 2024-05-25T06:11:43.441 INFO:tasks.workunit.client.1.smithi203.stdout:1/886: dwrite d14/d57/d1d/f40 [0,4194304] 0 2024-05-25T06:11:43.444 INFO:tasks.workunit.client.1.smithi203.stdout:9/914: dwrite d0/df/d37/f117 [0,4194304] 0 2024-05-25T06:11:43.446 INFO:tasks.workunit.client.1.smithi203.stdout:7/966: fdatasync d1/d9/d2c/d36/fbc 0 2024-05-25T06:11:43.448 INFO:tasks.workunit.client.1.smithi203.stdout:7/967: read - d1/d9/d2c/d79/df0/f133 zero size 2024-05-25T06:11:43.449 INFO:tasks.workunit.client.1.smithi203.stdout:1/887: symlink d14/d57/l12f 0 2024-05-25T06:11:43.450 INFO:tasks.workunit.client.0.smithi160.stdout:0/733: dread d2b/d31/d35/d36/d86/f66 [0,4194304] 0 2024-05-25T06:11:43.450 INFO:tasks.workunit.client.1.smithi203.stdout:9/915: creat d0/df/ddd/d113/f12d x:0 0 0 2024-05-25T06:11:43.453 INFO:tasks.workunit.client.1.smithi203.stdout:9/916: rmdir d0/d17/d20/d6e/d75 39 2024-05-25T06:11:43.454 INFO:tasks.workunit.client.0.smithi160.stdout:8/677: mkdir d5/d44/d1f/da3/d28/dc6/de8 0 2024-05-25T06:11:43.456 INFO:tasks.workunit.client.0.smithi160.stdout:8/678: fsync d5/d44/d1f/da3/d28/d4a/d93/fab 0 2024-05-25T06:11:43.457 INFO:tasks.workunit.client.1.smithi203.stdout:9/917: mkdir d0/d17/d23/def/d12e 0 2024-05-25T06:11:43.458 INFO:tasks.workunit.client.0.smithi160.stdout:1/571: write de/d21/d49/d1c/d41/d96/fa8 [1111756,27378] 0 2024-05-25T06:11:43.459 INFO:tasks.workunit.client.0.smithi160.stdout:7/699: mknod d6/dba/cec 0 2024-05-25T06:11:43.460 INFO:tasks.workunit.client.1.smithi203.stdout:9/918: creat d0/d17/f12f x:0 0 0 2024-05-25T06:11:43.460 INFO:tasks.workunit.client.0.smithi160.stdout:2/652: truncate d12/f45 1345028 0 2024-05-25T06:11:43.462 INFO:tasks.workunit.client.0.smithi160.stdout:9/604: dwrite d4/d11/d1e/d6c/f81 [0,4194304] 0 2024-05-25T06:11:43.464 INFO:tasks.workunit.client.0.smithi160.stdout:8/679: unlink d5/d44/d1f/da3/d28/dc6/f95 0 2024-05-25T06:11:43.466 INFO:tasks.workunit.client.0.smithi160.stdout:2/653: readlink d12/d3e/d5f/d74/d7a/lc4 0 2024-05-25T06:11:43.466 INFO:tasks.workunit.client.0.smithi160.stdout:8/680: chown d5/d44/d1f/da3/d28/f3f 103771930 1 2024-05-25T06:11:43.467 INFO:tasks.workunit.client.0.smithi160.stdout:2/654: fsync d12/f8f 0 2024-05-25T06:11:43.468 INFO:tasks.workunit.client.0.smithi160.stdout:7/700: dwrite d6/d3e/da1/fbf [0,4194304] 0 2024-05-25T06:11:43.468 INFO:tasks.workunit.client.0.smithi160.stdout:1/572: rename fb to de/d21/d49/d1c/d52/fbe 0 2024-05-25T06:11:43.470 INFO:tasks.workunit.client.0.smithi160.stdout:2/655: truncate d12/d1c/d37/fcc 179945 0 2024-05-25T06:11:43.471 INFO:tasks.workunit.client.0.smithi160.stdout:1/573: write de/d21/d49/d1c/d59/d67/fb9 [1608822,52893] 0 2024-05-25T06:11:43.472 INFO:tasks.workunit.client.0.smithi160.stdout:1/574: write f5 [4191564,34793] 0 2024-05-25T06:11:43.473 INFO:tasks.workunit.client.0.smithi160.stdout:2/656: readlink d12/d3e/d5f/lad 0 2024-05-25T06:11:43.476 INFO:tasks.workunit.client.0.smithi160.stdout:2/657: creat d12/d1c/d37/fdb x:0 0 0 2024-05-25T06:11:43.477 INFO:tasks.workunit.client.0.smithi160.stdout:2/658: read - d12/d1c/d37/fdb zero size 2024-05-25T06:11:43.477 INFO:tasks.workunit.client.0.smithi160.stdout:1/575: symlink de/d21/d49/d90/lbf 0 2024-05-25T06:11:43.478 INFO:tasks.workunit.client.0.smithi160.stdout:1/576: write de/f3a [3714196,38005] 0 2024-05-25T06:11:43.481 INFO:tasks.workunit.client.0.smithi160.stdout:1/577: creat de/d21/d49/d1c/d59/d71/fc0 x:0 0 0 2024-05-25T06:11:43.482 INFO:tasks.workunit.client.0.smithi160.stdout:1/578: dread - de/d21/d49/d1c/f9d zero size 2024-05-25T06:11:43.482 INFO:tasks.workunit.client.0.smithi160.stdout:2/659: creat d12/d3e/d5a/d8d/db4/fdc x:0 0 0 2024-05-25T06:11:43.482 INFO:tasks.workunit.client.0.smithi160.stdout:2/660: dread - d12/d3e/db5/fc3 zero size 2024-05-25T06:11:43.483 INFO:tasks.workunit.client.0.smithi160.stdout:1/579: creat de/d21/d49/d1c/d59/d67/fc1 x:0 0 0 2024-05-25T06:11:43.484 INFO:tasks.workunit.client.0.smithi160.stdout:2/661: creat d12/d1c/d2a/fdd x:0 0 0 2024-05-25T06:11:43.485 INFO:tasks.workunit.client.0.smithi160.stdout:1/580: write de/d21/d49/fa2 [1040470,115320] 0 2024-05-25T06:11:43.485 INFO:tasks.workunit.client.0.smithi160.stdout:2/662: write d12/d5c/f72 [294200,24460] 0 2024-05-25T06:11:43.488 INFO:tasks.workunit.client.0.smithi160.stdout:1/581: unlink de/d21/d49/d28/f65 0 2024-05-25T06:11:43.489 INFO:tasks.workunit.client.0.smithi160.stdout:1/582: stat de/c30 0 2024-05-25T06:11:43.489 INFO:tasks.workunit.client.0.smithi160.stdout:1/583: fdatasync de/d21/d49/d1c/f53 0 2024-05-25T06:11:43.490 INFO:tasks.workunit.client.0.smithi160.stdout:2/663: link d12/d3e/l51 d12/d5c/d85/lde 0 2024-05-25T06:11:43.491 INFO:tasks.workunit.client.0.smithi160.stdout:2/664: fsync d12/d3e/d5f/d74/d7a/fac 0 2024-05-25T06:11:43.493 INFO:tasks.workunit.client.0.smithi160.stdout:2/665: mknod d12/d1c/d2a/d7e/d90/cdf 0 2024-05-25T06:11:43.493 INFO:tasks.workunit.client.0.smithi160.stdout:2/666: chown f7 156696 1 2024-05-25T06:11:43.493 INFO:tasks.workunit.client.0.smithi160.stdout:2/667: read - d12/d97/faa zero size 2024-05-25T06:11:43.495 INFO:tasks.workunit.client.0.smithi160.stdout:2/668: creat d12/d3e/d5a/d8d/fe0 x:0 0 0 2024-05-25T06:11:43.496 INFO:tasks.workunit.client.0.smithi160.stdout:2/669: rmdir d12 39 2024-05-25T06:11:43.547 INFO:tasks.workunit.client.1.smithi203.stdout:0/972: sync 2024-05-25T06:11:43.548 INFO:tasks.workunit.client.1.smithi203.stdout:0/973: stat dd/d32/de5 0 2024-05-25T06:11:43.548 INFO:tasks.workunit.client.1.smithi203.stdout:0/974: fsync dd/d32/de5/f112 0 2024-05-25T06:11:43.551 INFO:tasks.workunit.client.1.smithi203.stdout:9/919: dread d0/df/f15 [0,4194304] 0 2024-05-25T06:11:43.552 INFO:tasks.workunit.client.1.smithi203.stdout:9/920: write d0/fa6 [1848142,95117] 0 2024-05-25T06:11:43.555 INFO:tasks.workunit.client.1.smithi203.stdout:0/975: getdents dd/d5c/d63/d106/d4f/d62/d78 0 2024-05-25T06:11:43.557 INFO:tasks.workunit.client.1.smithi203.stdout:9/921: creat d0/df/ddd/f130 x:0 0 0 2024-05-25T06:11:43.558 INFO:tasks.workunit.client.1.smithi203.stdout:0/976: mkdir dd/d5c/d63/d106/d75/d121/d160 0 2024-05-25T06:11:43.559 INFO:tasks.workunit.client.1.smithi203.stdout:0/977: chown dd/d5c/d63/d106/d4f 0 1 2024-05-25T06:11:43.560 INFO:tasks.workunit.client.1.smithi203.stdout:7/968: write d1/d9/d2c/d36/d54/fb8 [386212,45782] 0 2024-05-25T06:11:43.560 INFO:tasks.workunit.client.1.smithi203.stdout:1/888: write d14/d57/d1d/f30 [2391361,47702] 0 2024-05-25T06:11:43.561 INFO:tasks.workunit.client.0.smithi160.stdout:3/568: truncate d19/d2f/d38/d43/f53 1834372 0 2024-05-25T06:11:43.563 INFO:tasks.workunit.client.1.smithi203.stdout:1/889: mknod d14/dff/c130 0 2024-05-25T06:11:43.564 INFO:tasks.workunit.client.1.smithi203.stdout:7/969: write d1/d9/d2c/d36/d54/f8f [4061921,73527] 0 2024-05-25T06:11:43.565 INFO:tasks.workunit.client.1.smithi203.stdout:7/970: fsync d1/d9/d2c/d79/f81 0 2024-05-25T06:11:43.567 INFO:tasks.workunit.client.1.smithi203.stdout:1/890: rmdir d14/d57/d97/dec 39 2024-05-25T06:11:43.570 INFO:tasks.workunit.client.0.smithi160.stdout:0/734: dwrite d2b/f30 [4194304,4194304] 0 2024-05-25T06:11:43.570 INFO:tasks.workunit.client.0.smithi160.stdout:3/569: link d19/d2f/d34/d67/d96/l98 d19/d2f/d38/d43/d5b/lc1 0 2024-05-25T06:11:43.571 INFO:tasks.workunit.client.0.smithi160.stdout:3/570: write d19/da6/f76 [880580,41351] 0 2024-05-25T06:11:43.572 INFO:tasks.workunit.client.0.smithi160.stdout:0/735: write d2b/d46/d63/da7/d67/d6a/fcf [398325,56744] 0 2024-05-25T06:11:43.574 INFO:tasks.workunit.client.0.smithi160.stdout:3/571: read d19/d2f/d38/d47/f64 [107017,91034] 0 2024-05-25T06:11:43.580 INFO:tasks.workunit.client.0.smithi160.stdout:3/572: rename d19/d22/f8f to d19/d22/fc2 0 2024-05-25T06:11:43.583 INFO:tasks.workunit.client.0.smithi160.stdout:1/584: truncate de/f15 316876 0 2024-05-25T06:11:43.584 INFO:tasks.workunit.client.0.smithi160.stdout:4/613: dread d4/d6/d21/f27 [0,4194304] 0 2024-05-25T06:11:43.585 INFO:tasks.workunit.client.1.smithi203.stdout:0/978: truncate dd/d5c/d63/d106/da0/db9/df9/d86/f14b 2948468 0 2024-05-25T06:11:43.586 INFO:tasks.workunit.client.0.smithi160.stdout:3/573: rename d19/f1e to d19/db2/fc3 0 2024-05-25T06:11:43.586 INFO:tasks.workunit.client.0.smithi160.stdout:3/574: chown d19/d22/l25 89 1 2024-05-25T06:11:43.587 INFO:tasks.workunit.client.1.smithi203.stdout:7/971: truncate d1/d9/d2c/fe7 2262161 0 2024-05-25T06:11:43.587 INFO:tasks.workunit.client.0.smithi160.stdout:3/575: fdatasync d19/d2f/d38/f78 0 2024-05-25T06:11:43.587 INFO:tasks.workunit.client.0.smithi160.stdout:2/670: write d12/d3e/d5a/d8d/fb8 [5176593,76564] 0 2024-05-25T06:11:43.588 INFO:tasks.workunit.client.1.smithi203.stdout:7/972: chown d1/d9/d2c/d79/l13e 255131688 1 2024-05-25T06:11:43.589 INFO:tasks.workunit.client.1.smithi203.stdout:0/979: creat dd/d5c/d63/d106/d4f/f161 x:0 0 0 2024-05-25T06:11:43.590 INFO:tasks.workunit.client.0.smithi160.stdout:4/614: rename d4/d6/faa to d4/d6/d35/d3d/d9e/fc2 0 2024-05-25T06:11:43.594 INFO:tasks.workunit.client.1.smithi203.stdout:0/980: creat dd/d5c/d63/d106/d75/d121/d160/f162 x:0 0 0 2024-05-25T06:11:43.594 INFO:tasks.workunit.client.0.smithi160.stdout:3/576: creat d19/d2f/d38/d60/fc4 x:0 0 0 2024-05-25T06:11:43.594 INFO:tasks.workunit.client.0.smithi160.stdout:2/671: creat d12/d1c/d2a/fe1 x:0 0 0 2024-05-25T06:11:43.595 INFO:tasks.workunit.client.0.smithi160.stdout:4/615: dwrite d4/f2a [0,4194304] 0 2024-05-25T06:11:43.595 INFO:tasks.workunit.client.1.smithi203.stdout:7/973: dwrite d1/d11/f21 [0,4194304] 0 2024-05-25T06:11:43.596 INFO:tasks.workunit.client.0.smithi160.stdout:2/672: readlink d12/d1c/d37/d49/l8c 0 2024-05-25T06:11:43.600 INFO:tasks.workunit.client.0.smithi160.stdout:1/585: link de/f3d de/d39/d8f/fc2 0 2024-05-25T06:11:43.601 INFO:tasks.workunit.client.1.smithi203.stdout:7/974: mknod d1/d9/d2c/d36/c143 0 2024-05-25T06:11:43.601 INFO:tasks.workunit.client.0.smithi160.stdout:3/577: mknod d19/d2f/d38/d43/d5b/d65/d7e/cc5 0 2024-05-25T06:11:43.602 INFO:tasks.workunit.client.1.smithi203.stdout:1/891: dread d14/d57/d6b/f4f [0,4194304] 0 2024-05-25T06:11:43.602 INFO:tasks.workunit.client.0.smithi160.stdout:1/586: write de/d21/d49/d1c/d59/f95 [1912301,84370] 0 2024-05-25T06:11:43.603 INFO:tasks.workunit.client.0.smithi160.stdout:3/578: chown d19/d2f/d38/d43/d5b/d65/d7e/cc5 2271 1 2024-05-25T06:11:43.604 INFO:tasks.workunit.client.0.smithi160.stdout:1/587: stat de/d21/d49/d35/c4b 0 2024-05-25T06:11:43.604 INFO:tasks.workunit.client.1.smithi203.stdout:7/975: mkdir d1/dbd/dc1/df3/d144 0 2024-05-25T06:11:43.605 INFO:tasks.workunit.client.1.smithi203.stdout:1/892: fsync d14/d57/d1d/f2f 0 2024-05-25T06:11:43.606 INFO:tasks.workunit.client.0.smithi160.stdout:2/673: dread d12/d3e/f41 [0,4194304] 0 2024-05-25T06:11:43.608 INFO:tasks.workunit.client.0.smithi160.stdout:1/588: rmdir de/d21/d49/d1c/d41/d96 39 2024-05-25T06:11:43.612 INFO:tasks.workunit.client.0.smithi160.stdout:2/674: unlink d12/l14 0 2024-05-25T06:11:43.612 INFO:tasks.workunit.client.1.smithi203.stdout:7/976: symlink d1/d9/d2c/d62/db3/de5/l145 0 2024-05-25T06:11:43.614 INFO:tasks.workunit.client.1.smithi203.stdout:0/981: write dd/d10/d34/f3c [2696848,76334] 0 2024-05-25T06:11:43.614 INFO:tasks.workunit.client.1.smithi203.stdout:7/977: fdatasync d1/d9/d2c/d3a/dde/f10c 0 2024-05-25T06:11:43.615 INFO:tasks.workunit.client.1.smithi203.stdout:0/982: truncate dd/d45/f5b 5643238 0 2024-05-25T06:11:43.619 INFO:tasks.workunit.client.1.smithi203.stdout:0/983: creat dd/d5c/d63/d106/d48/de6/f163 x:0 0 0 2024-05-25T06:11:43.622 INFO:tasks.workunit.client.1.smithi203.stdout:0/984: symlink dd/d5c/d63/d106/d48/d91/dad/l164 0 2024-05-25T06:11:43.626 INFO:tasks.workunit.client.1.smithi203.stdout:0/985: mknod dd/d5c/d63/d106/d75/d145/c165 0 2024-05-25T06:11:43.628 INFO:tasks.workunit.client.1.smithi203.stdout:1/893: write d14/d23/d115/f6e [456903,38144] 0 2024-05-25T06:11:43.628 INFO:tasks.workunit.client.1.smithi203.stdout:0/986: creat dd/d5c/d63/f166 x:0 0 0 2024-05-25T06:11:43.633 INFO:tasks.workunit.client.1.smithi203.stdout:0/987: rmdir dd/d10/d34/d4c/d5d 39 2024-05-25T06:11:43.634 INFO:tasks.workunit.client.0.smithi160.stdout:4/616: dread d4/d1a/fa2 [4194304,4194304] 0 2024-05-25T06:11:43.635 INFO:tasks.workunit.client.1.smithi203.stdout:0/988: creat dd/d5c/d63/d106/da0/f167 x:0 0 0 2024-05-25T06:11:43.638 INFO:tasks.workunit.client.1.smithi203.stdout:1/894: rename d14/d7c/d9d/dc8/df3 to d14/d57/d97/dec/d131 0 2024-05-25T06:11:43.640 INFO:tasks.workunit.client.1.smithi203.stdout:7/978: dread d1/d9/d2c/d79/f81 [0,4194304] 0 2024-05-25T06:11:43.640 INFO:tasks.workunit.client.1.smithi203.stdout:0/989: creat dd/d5c/d63/d128/f168 x:0 0 0 2024-05-25T06:11:43.641 INFO:tasks.workunit.client.0.smithi160.stdout:3/579: rename d19/d2f/d38/d47/fa2 to d19/d2f/fc6 0 2024-05-25T06:11:43.642 INFO:tasks.workunit.client.1.smithi203.stdout:1/895: rename d14/dff/db8/dc2/f9a to d14/d57/d1d/d10f/f132 0 2024-05-25T06:11:43.644 INFO:tasks.workunit.client.1.smithi203.stdout:7/979: symlink d1/d9/d2c/d3a/dd1/dff/l146 0 2024-05-25T06:11:43.645 INFO:tasks.workunit.client.1.smithi203.stdout:0/990: unlink dd/d5c/d63/d106/da0/db9/df9/d86/fbf 0 2024-05-25T06:11:43.647 INFO:tasks.workunit.client.0.smithi160.stdout:3/580: dread d19/d22/f2a [0,4194304] 0 2024-05-25T06:11:43.647 INFO:tasks.workunit.client.1.smithi203.stdout:0/991: mkdir dd/d32/de5/d169 0 2024-05-25T06:11:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:43 smithi203 ceph-mon[36030]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:43.650 INFO:tasks.workunit.client.1.smithi203.stdout:7/980: creat d1/d9/d2c/d3a/f147 x:0 0 0 2024-05-25T06:11:43.652 INFO:tasks.workunit.client.1.smithi203.stdout:0/992: mknod dd/d5c/d63/d106/df3/d10c/c16a 0 2024-05-25T06:11:43.653 INFO:tasks.workunit.client.1.smithi203.stdout:1/896: dwrite d14/d57/db7/dd2/fcd [4194304,4194304] 0 2024-05-25T06:11:43.656 INFO:tasks.workunit.client.1.smithi203.stdout:0/993: mknod dd/d5c/dd5/c16b 0 2024-05-25T06:11:43.661 INFO:tasks.workunit.client.1.smithi203.stdout:0/994: creat dd/d5c/d63/d106/f16c x:0 0 0 2024-05-25T06:11:43.663 INFO:tasks.workunit.client.1.smithi203.stdout:1/897: dwrite d14/d57/db7/dd2/fce [0,4194304] 0 2024-05-25T06:11:43.668 INFO:tasks.workunit.client.1.smithi203.stdout:1/898: creat d14/d57/d97/d11d/f133 x:0 0 0 2024-05-25T06:11:43.670 INFO:tasks.workunit.client.0.smithi160.stdout:1/589: truncate de/d21/d49/d1c/f53 2959404 0 2024-05-25T06:11:43.671 INFO:tasks.workunit.client.0.smithi160.stdout:1/590: write de/d21/fa6 [1984238,117993] 0 2024-05-25T06:11:43.672 INFO:tasks.workunit.client.0.smithi160.stdout:4/617: truncate d4/d6/d21/f27 1408018 0 2024-05-25T06:11:43.678 INFO:tasks.workunit.client.1.smithi203.stdout:7/981: dread d1/d9/d2c/d36/f142 [0,4194304] 0 2024-05-25T06:11:43.682 INFO:tasks.workunit.client.0.smithi160.stdout:3/581: fsync d19/d22/f2c 0 2024-05-25T06:11:43.682 INFO:tasks.workunit.client.1.smithi203.stdout:7/982: truncate d1/d9/d2c/d3a/dd1/fda 462191 0 2024-05-25T06:11:43.683 INFO:tasks.workunit.client.0.smithi160.stdout:2/675: creat d12/d3e/d5a/d8d/dc9/fe2 x:0 0 0 2024-05-25T06:11:43.683 INFO:tasks.workunit.client.0.smithi160.stdout:1/591: creat de/d21/d49/d1c/d41/d45/d7d/fc3 x:0 0 0 2024-05-25T06:11:43.684 INFO:tasks.workunit.client.1.smithi203.stdout:7/983: chown d1/d9/d2c/d3a/le8 31 1 2024-05-25T06:11:43.686 INFO:tasks.workunit.client.0.smithi160.stdout:4/618: dwrite d4/d63/f81 [0,4194304] 0 2024-05-25T06:11:43.687 INFO:tasks.workunit.client.0.smithi160.stdout:4/619: fdatasync d4/d63/fbe 0 2024-05-25T06:11:43.688 INFO:tasks.workunit.client.1.smithi203.stdout:1/899: dwrite d14/d57/d97/fa3 [4194304,4194304] 0 2024-05-25T06:11:43.688 INFO:tasks.workunit.client.0.smithi160.stdout:4/620: dread - d4/d6/d35/d3d/fb8 zero size 2024-05-25T06:11:43.691 INFO:tasks.workunit.client.1.smithi203.stdout:1/900: rmdir d14/d57/d1d/d10f 39 2024-05-25T06:11:43.697 INFO:tasks.workunit.client.1.smithi203.stdout:7/984: dwrite d1/d9/d2c/d36/d54/fc3 [0,4194304] 0 2024-05-25T06:11:43.701 INFO:tasks.workunit.client.1.smithi203.stdout:0/995: dread dd/f5a [0,4194304] 0 2024-05-25T06:11:43.703 INFO:tasks.workunit.client.1.smithi203.stdout:7/985: creat d1/d9/d2c/d3a/dd1/f148 x:0 0 0 2024-05-25T06:11:43.707 INFO:tasks.workunit.client.1.smithi203.stdout:0/996: fdatasync dd/d5c/d63/d106/d4f/d62/f9c 0 2024-05-25T06:11:43.710 INFO:tasks.workunit.client.1.smithi203.stdout:0/997: mkdir dd/d5c/d63/d106/da0/db9/d16d 0 2024-05-25T06:11:43.714 INFO:tasks.workunit.client.1.smithi203.stdout:0/998: creat dd/d10/d34/d4c/d5d/f16e x:0 0 0 2024-05-25T06:11:43.715 INFO:tasks.workunit.client.1.smithi203.stdout:0/999: fsync dd/d32/d108/f110 0 2024-05-25T06:11:43.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:43 smithi160 ceph-mon[29041]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:11:43.721 INFO:tasks.workunit.client.0.smithi160.stdout:5/744: sync 2024-05-25T06:11:43.721 INFO:tasks.workunit.client.0.smithi160.stdout:9/605: sync 2024-05-25T06:11:43.721 INFO:tasks.workunit.client.0.smithi160.stdout:6/621: sync 2024-05-25T06:11:43.721 INFO:tasks.workunit.client.0.smithi160.stdout:5/745: readlink l2 0 2024-05-25T06:11:43.724 INFO:tasks.workunit.client.1.smithi203.stdout:1/901: dread d14/d57/d97/ff2 [0,4194304] 0 2024-05-25T06:11:43.726 INFO:tasks.workunit.client.1.smithi203.stdout:1/902: symlink d14/d57/d1d/da8/l134 0 2024-05-25T06:11:43.728 INFO:tasks.workunit.client.1.smithi203.stdout:7/986: dread d1/d9/fb [0,4194304] 0 2024-05-25T06:11:43.728 INFO:tasks.workunit.client.1.smithi203.stdout:1/903: unlink c4 0 2024-05-25T06:11:43.730 INFO:tasks.workunit.client.1.smithi203.stdout:1/904: creat d14/d81/d100/f135 x:0 0 0 2024-05-25T06:11:43.731 INFO:tasks.workunit.client.1.smithi203.stdout:7/987: truncate d1/d9/f76 1148521 0 2024-05-25T06:11:43.734 INFO:tasks.workunit.client.1.smithi203.stdout:1/905: rename d14/d57/db7/dd2/db3/f7a to d14/d57/dbb/f136 0 2024-05-25T06:11:43.735 INFO:tasks.workunit.client.0.smithi160.stdout:3/582: creat d19/d2f/d38/d47/fc7 x:0 0 0 2024-05-25T06:11:43.738 INFO:tasks.workunit.client.1.smithi203.stdout:1/906: mknod d14/d57/db7/dd2/dca/dcf/df7/c137 0 2024-05-25T06:11:43.739 INFO:tasks.workunit.client.1.smithi203.stdout:7/988: link d1/d9/fe d1/dbd/dc1/f149 0 2024-05-25T06:11:43.739 INFO:tasks.workunit.client.1.smithi203.stdout:1/907: dread - d14/d57/fe9 zero size 2024-05-25T06:11:43.741 INFO:tasks.workunit.client.0.smithi160.stdout:4/621: unlink d4/d6/d21/d42/cb0 0 2024-05-25T06:11:43.742 INFO:tasks.workunit.client.1.smithi203.stdout:1/908: mknod d14/d23/d115/de4/c138 0 2024-05-25T06:11:43.742 INFO:tasks.workunit.client.0.smithi160.stdout:4/622: fdatasync d4/d63/fbe 0 2024-05-25T06:11:43.743 INFO:tasks.workunit.client.1.smithi203.stdout:7/989: write d1/d11/d3d/d4c/f63 [2131738,116342] 0 2024-05-25T06:11:43.743 INFO:tasks.workunit.client.0.smithi160.stdout:9/606: creat d4/d11/d1e/d3e/d2f/d79/fc0 x:0 0 0 2024-05-25T06:11:43.745 INFO:tasks.workunit.client.0.smithi160.stdout:5/746: write db/dc/d13/d1d/d3f/d72/dc5/fdc [77508,59101] 0 2024-05-25T06:11:43.747 INFO:tasks.workunit.client.1.smithi203.stdout:1/909: write d14/d57/db7/dd2/d101/f117 [588291,107962] 0 2024-05-25T06:11:43.749 INFO:tasks.workunit.client.0.smithi160.stdout:3/583: creat d19/d2f/d38/d43/d5b/d65/d83/dba/fc8 x:0 0 0 2024-05-25T06:11:43.750 INFO:tasks.workunit.client.0.smithi160.stdout:9/607: rename d4/d11/d1e/d3e/d2f/d79/db7 to d4/d11/d12/d2e/d45/d90/dc1 0 2024-05-25T06:11:43.751 INFO:tasks.workunit.client.0.smithi160.stdout:5/747: fsync db/dc/d13/d1d/d31/d3b/d69/f79 0 2024-05-25T06:11:43.754 INFO:tasks.workunit.client.0.smithi160.stdout:4/623: dwrite d4/d6/d35/d3d/d9e/fc2 [0,4194304] 0 2024-05-25T06:11:43.767 INFO:tasks.workunit.client.0.smithi160.stdout:4/624: mkdir d4/d6/d21/dc3 0 2024-05-25T06:11:43.769 INFO:tasks.workunit.client.0.smithi160.stdout:8/681: sync 2024-05-25T06:11:43.769 INFO:tasks.workunit.client.0.smithi160.stdout:0/736: sync 2024-05-25T06:11:43.769 INFO:tasks.workunit.client.0.smithi160.stdout:7/701: sync 2024-05-25T06:11:43.770 INFO:tasks.workunit.client.0.smithi160.stdout:0/737: chown f5 1128 1 2024-05-25T06:11:43.772 INFO:tasks.workunit.client.0.smithi160.stdout:4/625: creat d4/d6/d5b/d66/fc4 x:0 0 0 2024-05-25T06:11:43.774 INFO:tasks.workunit.client.0.smithi160.stdout:9/608: rename d4/d11/d1e/d3e/d25/d27/f7d to d4/fc2 0 2024-05-25T06:11:43.775 INFO:tasks.workunit.client.0.smithi160.stdout:3/584: dread d19/d2f/d34/f5e [0,4194304] 0 2024-05-25T06:11:43.775 INFO:tasks.workunit.client.1.smithi203.stdout:1/910: dread d14/d23/f42 [0,4194304] 0 2024-05-25T06:11:43.775 INFO:tasks.workunit.client.0.smithi160.stdout:4/626: fsync d4/d6/d5b/d66/d82/f84 0 2024-05-25T06:11:43.778 INFO:tasks.workunit.client.1.smithi203.stdout:1/911: mkdir d14/d81/d100/d139 0 2024-05-25T06:11:43.780 INFO:tasks.workunit.client.0.smithi160.stdout:0/738: dwrite d2b/d31/d35/d36/f8b [4194304,4194304] 0 2024-05-25T06:11:43.782 INFO:tasks.workunit.client.0.smithi160.stdout:7/702: rename d6/d32/f42 to d6/d32/d5b/fed 0 2024-05-25T06:11:43.782 INFO:tasks.workunit.client.0.smithi160.stdout:5/748: dread db/dc/d13/d1d/d23/f46 [0,4194304] 0 2024-05-25T06:11:43.783 INFO:tasks.workunit.client.0.smithi160.stdout:9/609: creat d4/d11/d1e/d3e/d2f/d79/fc3 x:0 0 0 2024-05-25T06:11:43.785 INFO:tasks.workunit.client.0.smithi160.stdout:4/627: creat d4/d63/d64/d88/fc5 x:0 0 0 2024-05-25T06:11:43.786 INFO:tasks.workunit.client.0.smithi160.stdout:3/585: rmdir d19/da6/d6a 39 2024-05-25T06:11:43.787 INFO:tasks.workunit.client.0.smithi160.stdout:8/682: dwrite d5/d7/d9a/db5/fe5 [0,4194304] 0 2024-05-25T06:11:43.788 INFO:tasks.workunit.client.0.smithi160.stdout:4/628: fsync d4/d63/d64/d88/fa5 0 2024-05-25T06:11:43.791 INFO:tasks.workunit.client.0.smithi160.stdout:7/703: truncate d6/dc/f25 2017557 0 2024-05-25T06:11:43.792 INFO:tasks.workunit.client.1.smithi203.stdout:1/912: dwrite d14/d57/d7e/fa7 [4194304,4194304] 0 2024-05-25T06:11:43.792 INFO:tasks.workunit.client.0.smithi160.stdout:9/610: dwrite d4/fae [0,4194304] 0 2024-05-25T06:11:43.794 INFO:tasks.workunit.client.0.smithi160.stdout:9/611: chown d4/d11/d14/d8e/fb8 256649496 1 2024-05-25T06:11:43.795 INFO:tasks.workunit.client.0.smithi160.stdout:7/704: dread d6/d32/d5b/f96 [0,4194304] 0 2024-05-25T06:11:43.795 INFO:tasks.workunit.client.0.smithi160.stdout:8/683: creat d5/d44/dba/ddb/fe9 x:0 0 0 2024-05-25T06:11:43.796 INFO:tasks.workunit.client.1.smithi203.stdout:1/913: read d14/d57/d1d/f2a [5754463,12308] 0 2024-05-25T06:11:43.797 INFO:tasks.workunit.client.0.smithi160.stdout:7/705: chown d6/c26 220 1 2024-05-25T06:11:43.797 INFO:tasks.workunit.client.0.smithi160.stdout:7/706: read - d6/d2b/da0/dc7/fda zero size 2024-05-25T06:11:43.798 INFO:tasks.workunit.client.0.smithi160.stdout:7/707: chown d6/d32/f3f 40199878 1 2024-05-25T06:11:43.800 INFO:tasks.workunit.client.1.smithi203.stdout:1/914: dread d14/d57/db7/dd2/dca/ff9 [0,4194304] 0 2024-05-25T06:11:43.800 INFO:tasks.workunit.client.0.smithi160.stdout:7/708: fsync d6/dc/d64/f45 0 2024-05-25T06:11:43.800 INFO:tasks.workunit.client.0.smithi160.stdout:7/709: fdatasync d6/d32/d92/fd5 0 2024-05-25T06:11:43.803 INFO:tasks.workunit.client.1.smithi203.stdout:1/915: mkdir d14/d57/db7/d13a 0 2024-05-25T06:11:43.804 INFO:tasks.workunit.client.1.smithi203.stdout:1/916: fsync d14/dff/db8/dc2/f72 0 2024-05-25T06:11:43.806 INFO:tasks.workunit.client.0.smithi160.stdout:4/629: truncate d4/f26 6226067 0 2024-05-25T06:11:43.809 INFO:tasks.workunit.client.0.smithi160.stdout:3/586: getdents d19/d2f/d38/d43/d5b 0 2024-05-25T06:11:43.810 INFO:tasks.workunit.client.0.smithi160.stdout:8/684: dread d5/d44/d1f/da3/d28/d4a/d4e/d86/fa4 [0,4194304] 0 2024-05-25T06:11:43.811 INFO:tasks.workunit.client.1.smithi203.stdout:1/917: link d14/dff/db8/dc2/lc6 d14/d57/l13b 0 2024-05-25T06:11:43.811 INFO:tasks.workunit.client.0.smithi160.stdout:1/592: truncate de/f4d 708204 0 2024-05-25T06:11:43.812 INFO:tasks.workunit.client.0.smithi160.stdout:7/710: dwrite d6/d32/d92/fc9 [0,4194304] 0 2024-05-25T06:11:43.812 INFO:tasks.workunit.client.0.smithi160.stdout:3/587: creat d19/d22/fc9 x:0 0 0 2024-05-25T06:11:43.813 INFO:tasks.workunit.client.0.smithi160.stdout:7/711: fdatasync d6/dc/d64/f45 0 2024-05-25T06:11:43.815 INFO:tasks.workunit.client.0.smithi160.stdout:8/685: creat d5/d9/d72/daf/dbf/fea x:0 0 0 2024-05-25T06:11:43.815 INFO:tasks.workunit.client.0.smithi160.stdout:8/686: readlink d5/d7/l51 0 2024-05-25T06:11:43.816 INFO:tasks.workunit.client.1.smithi203.stdout:1/918: link d14/d57/d97/dec/c11c d14/d23/d115/c13c 0 2024-05-25T06:11:43.816 INFO:tasks.workunit.client.0.smithi160.stdout:7/712: mkdir d6/d3e/d6f/d73/dee 0 2024-05-25T06:11:43.817 INFO:tasks.workunit.client.0.smithi160.stdout:8/687: write d5/d9/fc [2245815,84619] 0 2024-05-25T06:11:43.818 INFO:tasks.workunit.client.1.smithi203.stdout:1/919: write d14/d57/db7/dd2/ddc/ffc [2624297,25680] 0 2024-05-25T06:11:43.818 INFO:tasks.workunit.client.0.smithi160.stdout:8/688: stat d5/d44/d1f/d58 0 2024-05-25T06:11:43.820 INFO:tasks.workunit.client.1.smithi203.stdout:1/920: mknod d14/dac/c13d 0 2024-05-25T06:11:43.822 INFO:tasks.workunit.client.0.smithi160.stdout:7/713: symlink d6/dc/d21/ddd/lef 0 2024-05-25T06:11:43.823 INFO:tasks.workunit.client.0.smithi160.stdout:7/714: chown d6/d3e/d6f/f76 3012598 1 2024-05-25T06:11:43.824 INFO:tasks.workunit.client.0.smithi160.stdout:7/715: chown d6/dc/d64/d5a 198141 1 2024-05-25T06:11:43.824 INFO:tasks.workunit.client.0.smithi160.stdout:8/689: unlink d5/d44/f16 0 2024-05-25T06:11:43.824 INFO:tasks.workunit.client.1.smithi203.stdout:1/921: rename d14/d57/d7e/fb5 to d14/d57/db7/dd2/dca/d11a/f13e 0 2024-05-25T06:11:43.825 INFO:tasks.workunit.client.0.smithi160.stdout:7/716: dread - d6/d2b/f5c zero size 2024-05-25T06:11:43.827 INFO:tasks.workunit.client.0.smithi160.stdout:7/717: write d6/d2b/da0/dc7/fda [899191,80754] 0 2024-05-25T06:11:43.827 INFO:tasks.workunit.client.1.smithi203.stdout:1/922: creat d14/d57/d97/dec/d131/f13f x:0 0 0 2024-05-25T06:11:43.828 INFO:tasks.workunit.client.0.smithi160.stdout:8/690: unlink d5/d44/d1f/da3/d28/de6/dc2/dd6/fe7 0 2024-05-25T06:11:43.829 INFO:tasks.workunit.client.0.smithi160.stdout:7/718: chown d6/d3e/da1/ddb/ce9 493143 1 2024-05-25T06:11:43.832 INFO:tasks.workunit.client.0.smithi160.stdout:8/691: chown d5/l64 129 1 2024-05-25T06:11:43.832 INFO:tasks.workunit.client.0.smithi160.stdout:2/676: dwrite d12/d1c/f6c [0,4194304] 0 2024-05-25T06:11:43.837 INFO:tasks.workunit.client.0.smithi160.stdout:8/692: unlink d5/d7/fdc 0 2024-05-25T06:11:43.837 INFO:tasks.workunit.client.0.smithi160.stdout:8/693: readlink d5/d7/l51 0 2024-05-25T06:11:43.838 INFO:tasks.workunit.client.1.smithi203.stdout:1/923: getdents d14/d57/d7e 0 2024-05-25T06:11:43.839 INFO:tasks.workunit.client.1.smithi203.stdout:1/924: chown d14/d57/d7e 242557957 1 2024-05-25T06:11:43.839 INFO:tasks.workunit.client.0.smithi160.stdout:7/719: dwrite d6/d2b/da0/dc7/fdc [0,4194304] 0 2024-05-25T06:11:43.841 INFO:tasks.workunit.client.0.smithi160.stdout:2/677: mkdir d12/d3e/d5f/d74/d7a/de3 0 2024-05-25T06:11:43.842 INFO:tasks.workunit.client.0.smithi160.stdout:2/678: fdatasync d12/d75/dca/f24 0 2024-05-25T06:11:43.843 INFO:tasks.workunit.client.0.smithi160.stdout:2/679: fsync d12/d3e/db5/fc3 0 2024-05-25T06:11:43.845 INFO:tasks.workunit.client.0.smithi160.stdout:3/588: dread d19/d2f/d38/d43/d5b/f6d [0,4194304] 0 2024-05-25T06:11:43.845 INFO:tasks.workunit.client.0.smithi160.stdout:7/720: fdatasync d6/dc/f82 0 2024-05-25T06:11:43.851 INFO:tasks.workunit.client.0.smithi160.stdout:8/694: link d5/d44/d1f/da3/fa8 d5/d7/dcb/d75/feb 0 2024-05-25T06:11:43.851 INFO:tasks.workunit.client.0.smithi160.stdout:8/695: stat d5/d44/dba 0 2024-05-25T06:11:43.852 INFO:tasks.workunit.client.0.smithi160.stdout:8/696: chown d5/d44/d1f/da3/d28/d4a/db8 60100537 1 2024-05-25T06:11:43.853 INFO:tasks.workunit.client.0.smithi160.stdout:7/721: fsync d6/d32/d92/f9c 0 2024-05-25T06:11:43.854 INFO:tasks.workunit.client.0.smithi160.stdout:3/589: symlink d19/d2f/d38/d43/d5b/d65/d83/dba/lca 0 2024-05-25T06:11:43.856 INFO:tasks.workunit.client.0.smithi160.stdout:2/680: creat d12/d5c/d85/d9b/fe4 x:0 0 0 2024-05-25T06:11:43.858 INFO:tasks.workunit.client.0.smithi160.stdout:3/590: read d19/d22/f2c [757683,25093] 0 2024-05-25T06:11:43.860 INFO:tasks.workunit.client.0.smithi160.stdout:2/681: chown d12/d3e/l51 0 1 2024-05-25T06:11:43.861 INFO:tasks.workunit.client.0.smithi160.stdout:3/591: dread - d19/d22/f2e zero size 2024-05-25T06:11:43.863 INFO:tasks.workunit.client.0.smithi160.stdout:0/739: creat d2b/d46/d63/fe7 x:0 0 0 2024-05-25T06:11:43.863 INFO:tasks.workunit.client.0.smithi160.stdout:4/630: dwrite d4/d6/d21/f27 [0,4194304] 0 2024-05-25T06:11:43.863 INFO:tasks.workunit.client.0.smithi160.stdout:2/682: rmdir d12/d47/da0 39 2024-05-25T06:11:43.863 INFO:tasks.workunit.client.0.smithi160.stdout:3/592: stat d19/d2f/d38/d47/f54 0 2024-05-25T06:11:43.865 INFO:tasks.workunit.client.0.smithi160.stdout:2/683: chown d12/d75/dca/l44 512 1 2024-05-25T06:11:43.865 INFO:tasks.workunit.client.0.smithi160.stdout:3/593: readlink d19/d2f/d38/d43/l63 0 2024-05-25T06:11:43.867 INFO:tasks.workunit.client.0.smithi160.stdout:3/594: write d19/d2f/d34/f55 [1291825,46221] 0 2024-05-25T06:11:43.867 INFO:tasks.workunit.client.0.smithi160.stdout:0/740: mkdir d2b/d46/dcb/de8 0 2024-05-25T06:11:43.874 INFO:tasks.workunit.client.0.smithi160.stdout:3/595: dwrite d19/f27 [4194304,4194304] 0 2024-05-25T06:11:43.875 INFO:tasks.workunit.client.0.smithi160.stdout:0/741: chown d2b/d31/f34 10962 1 2024-05-25T06:11:43.876 INFO:tasks.workunit.client.0.smithi160.stdout:7/722: dread f1 [0,4194304] 0 2024-05-25T06:11:43.877 INFO:tasks.workunit.client.0.smithi160.stdout:3/596: write d19/d2f/d38/d43/d5b/d65/f89 [1875697,116610] 0 2024-05-25T06:11:43.880 INFO:tasks.workunit.client.0.smithi160.stdout:4/631: getdents d4/d63 0 2024-05-25T06:11:43.881 INFO:tasks.workunit.client.0.smithi160.stdout:0/742: mknod d2b/d46/d63/da7/ce9 0 2024-05-25T06:11:43.888 INFO:tasks.workunit.client.0.smithi160.stdout:3/597: stat d19/d2f/d34/d67/d96/c9f 0 2024-05-25T06:11:43.889 INFO:tasks.workunit.client.0.smithi160.stdout:0/743: readlink d2b/l70 0 2024-05-25T06:11:43.890 INFO:tasks.workunit.client.0.smithi160.stdout:2/684: link d12/d75/dca/l40 d12/d47/le5 0 2024-05-25T06:11:43.892 INFO:tasks.workunit.client.0.smithi160.stdout:7/723: mkdir d6/dc/df0 0 2024-05-25T06:11:43.893 INFO:tasks.workunit.client.0.smithi160.stdout:4/632: symlink d4/d6/d7b/lc6 0 2024-05-25T06:11:43.895 INFO:tasks.workunit.client.0.smithi160.stdout:3/598: creat d19/d2f/d38/d43/d5b/d65/d83/fcb x:0 0 0 2024-05-25T06:11:43.897 INFO:tasks.workunit.client.0.smithi160.stdout:8/697: write d5/d7/dcb/d7a/f99 [801818,58319] 0 2024-05-25T06:11:43.899 INFO:tasks.workunit.client.0.smithi160.stdout:8/698: dread - d5/d7/dcb/d6a/fcf zero size 2024-05-25T06:11:43.899 INFO:tasks.workunit.client.0.smithi160.stdout:4/633: creat d4/d6/d28/fc7 x:0 0 0 2024-05-25T06:11:43.902 INFO:tasks.workunit.client.0.smithi160.stdout:4/634: write d4/d6/d21/f3f [4705428,116036] 0 2024-05-25T06:11:43.902 INFO:tasks.workunit.client.1.smithi203.stdout:1/925: rename d14/d57/d1d/da8/l134 to d14/d57/d1d/l140 0 2024-05-25T06:11:43.904 INFO:tasks.workunit.client.0.smithi160.stdout:3/599: mknod d19/da7/ccc 0 2024-05-25T06:11:43.904 INFO:tasks.workunit.client.1.smithi203.stdout:1/926: creat d14/d81/f141 x:0 0 0 2024-05-25T06:11:43.905 INFO:tasks.workunit.client.0.smithi160.stdout:7/724: dwrite d6/dc/d64/d5a/fbd [0,4194304] 0 2024-05-25T06:11:43.906 INFO:tasks.workunit.client.0.smithi160.stdout:0/744: dread d2b/d31/d35/d36/f8b [0,4194304] 0 2024-05-25T06:11:43.908 INFO:tasks.workunit.client.0.smithi160.stdout:3/600: read d19/d2f/d38/d47/f57 [3803815,105230] 0 2024-05-25T06:11:43.909 INFO:tasks.workunit.client.1.smithi203.stdout:1/927: rmdir d14/d57/dbb/df8 0 2024-05-25T06:11:43.909 INFO:tasks.workunit.client.0.smithi160.stdout:3/601: dread - d19/d2f/fa5 zero size 2024-05-25T06:11:43.912 INFO:tasks.workunit.client.1.smithi203.stdout:1/928: creat d14/f142 x:0 0 0 2024-05-25T06:11:43.915 INFO:tasks.workunit.client.1.smithi203.stdout:9/922: sync 2024-05-25T06:11:43.917 INFO:tasks.workunit.client.1.smithi203.stdout:1/929: getdents d14/d7c/d9d/dc8 0 2024-05-25T06:11:43.918 INFO:tasks.workunit.client.0.smithi160.stdout:7/725: fdatasync d6/d3e/d6f/f72 0 2024-05-25T06:11:43.919 INFO:tasks.workunit.client.0.smithi160.stdout:7/726: fdatasync d6/d2b/d5d/d89/f8a 0 2024-05-25T06:11:43.919 INFO:tasks.workunit.client.1.smithi203.stdout:1/930: write d14/d57/d6b/f5d [3993150,63529] 0 2024-05-25T06:11:43.920 INFO:tasks.workunit.client.1.smithi203.stdout:9/923: write d0/d17/d20/d6e/d75/df2/f111 [530656,47642] 0 2024-05-25T06:11:43.921 INFO:tasks.workunit.client.0.smithi160.stdout:7/727: dread d6/d32/f80 [0,4194304] 0 2024-05-25T06:11:43.921 INFO:tasks.workunit.client.1.smithi203.stdout:9/924: chown d0/d17/d20/d6e/d75/df2/d114/db2/lbb 1958 1 2024-05-25T06:11:43.922 INFO:tasks.workunit.client.1.smithi203.stdout:1/931: truncate d14/d57/db7/dd2/ddc/ffc 4577240 0 2024-05-25T06:11:43.922 INFO:tasks.workunit.client.0.smithi160.stdout:2/685: truncate d12/d75/f88 2842274 0 2024-05-25T06:11:43.923 INFO:tasks.workunit.client.0.smithi160.stdout:2/686: fdatasync d12/d3c/f61 0 2024-05-25T06:11:43.926 INFO:tasks.workunit.client.0.smithi160.stdout:3/602: mknod d19/d2f/d38/ccd 0 2024-05-25T06:11:43.930 INFO:tasks.workunit.client.1.smithi203.stdout:9/925: dwrite d0/df/d37/da8/dd6/f10f [0,4194304] 0 2024-05-25T06:11:43.930 INFO:tasks.workunit.client.0.smithi160.stdout:0/745: truncate d2b/d46/f5a 2440077 0 2024-05-25T06:11:43.932 INFO:tasks.workunit.client.1.smithi203.stdout:1/932: dwrite d14/d57/dbb/f136 [4194304,4194304] 0 2024-05-25T06:11:43.938 INFO:tasks.workunit.client.0.smithi160.stdout:3/603: dwrite d19/d2f/d38/d47/fc7 [0,4194304] 0 2024-05-25T06:11:43.939 INFO:tasks.workunit.client.1.smithi203.stdout:1/933: rename d14/d57/db7 to d14/d57/d6b/da1/de2/d143 0 2024-05-25T06:11:43.940 INFO:tasks.workunit.client.0.smithi160.stdout:4/635: dwrite d4/d6/d5b/d66/f74 [0,4194304] 0 2024-05-25T06:11:43.940 INFO:tasks.workunit.client.0.smithi160.stdout:0/746: creat d2b/d31/d35/d36/d37/fea x:0 0 0 2024-05-25T06:11:43.941 INFO:tasks.workunit.client.0.smithi160.stdout:2/687: dwrite d12/d75/fc8 [0,4194304] 0 2024-05-25T06:11:43.941 INFO:tasks.workunit.client.0.smithi160.stdout:6/622: sync 2024-05-25T06:11:43.942 INFO:tasks.workunit.client.0.smithi160.stdout:0/747: chown f24 1 1 2024-05-25T06:11:43.943 INFO:tasks.workunit.client.0.smithi160.stdout:2/688: stat d12/d1c/d2a/l66 0 2024-05-25T06:11:43.945 INFO:tasks.workunit.client.0.smithi160.stdout:0/748: dread - d2b/d46/d63/fc7 zero size 2024-05-25T06:11:43.956 INFO:tasks.workunit.client.0.smithi160.stdout:6/623: unlink d5/d45/d63/caa 0 2024-05-25T06:11:43.957 INFO:tasks.workunit.client.0.smithi160.stdout:3/604: dwrite d19/d2f/fc6 [0,4194304] 0 2024-05-25T06:11:43.957 INFO:tasks.workunit.client.0.smithi160.stdout:2/689: creat d12/d1c/d37/d4b/fe6 x:0 0 0 2024-05-25T06:11:43.959 INFO:tasks.workunit.client.0.smithi160.stdout:0/749: symlink d2b/d31/d35/d43/dab/dc6/leb 0 2024-05-25T06:11:43.964 INFO:tasks.workunit.client.0.smithi160.stdout:2/690: mknod d12/d3e/d5f/ce7 0 2024-05-25T06:11:43.964 INFO:tasks.workunit.client.0.smithi160.stdout:6/624: creat d5/de/d32/fd7 x:0 0 0 2024-05-25T06:11:43.964 INFO:tasks.workunit.client.0.smithi160.stdout:3/605: creat d19/d2f/d38/fce x:0 0 0 2024-05-25T06:11:43.967 INFO:tasks.workunit.client.0.smithi160.stdout:3/606: write d19/d2f/f94 [3964553,29466] 0 2024-05-25T06:11:43.971 INFO:tasks.workunit.client.0.smithi160.stdout:0/750: dwrite d2b/da8/fb0 [0,4194304] 0 2024-05-25T06:11:43.972 INFO:tasks.workunit.client.0.smithi160.stdout:0/751: chown c27 7060 1 2024-05-25T06:11:43.973 INFO:tasks.workunit.client.0.smithi160.stdout:2/691: dwrite d12/d1c/f1e [0,4194304] 0 2024-05-25T06:11:43.976 INFO:tasks.workunit.client.0.smithi160.stdout:2/692: mknod d12/d1c/d2a/d7e/d90/ce8 0 2024-05-25T06:11:43.980 INFO:tasks.workunit.client.0.smithi160.stdout:2/693: unlink d12/d1c/f2f 0 2024-05-25T06:11:43.988 INFO:tasks.workunit.client.0.smithi160.stdout:6/625: dread d5/d45/d4a/dbc/dc4/fce [0,4194304] 0 2024-05-25T06:11:43.991 INFO:tasks.workunit.client.0.smithi160.stdout:6/626: creat d5/de/d7d/fd8 x:0 0 0 2024-05-25T06:11:43.993 INFO:tasks.workunit.client.0.smithi160.stdout:4/636: dwrite d4/d6/f22 [0,4194304] 0 2024-05-25T06:11:43.994 INFO:tasks.workunit.client.0.smithi160.stdout:6/627: symlink d5/de/d7d/ld9 0 2024-05-25T06:11:43.994 INFO:tasks.workunit.client.0.smithi160.stdout:4/637: dread - d4/d63/d64/fc1 zero size 2024-05-25T06:11:43.996 INFO:tasks.workunit.client.0.smithi160.stdout:6/628: fsync d5/de/d7d/fcb 0 2024-05-25T06:11:43.999 INFO:tasks.workunit.client.0.smithi160.stdout:6/629: chown d5/d45/c70 4 1 2024-05-25T06:11:43.999 INFO:tasks.workunit.client.0.smithi160.stdout:3/607: dwrite d19/db2/fc3 [0,4194304] 0 2024-05-25T06:11:43.999 INFO:tasks.workunit.client.0.smithi160.stdout:4/638: fdatasync d4/d6/d35/f58 0 2024-05-25T06:11:44.002 INFO:tasks.workunit.client.0.smithi160.stdout:0/752: write d2b/d46/d63/fb1 [1743618,40351] 0 2024-05-25T06:11:44.005 INFO:tasks.workunit.client.0.smithi160.stdout:6/630: read d5/d45/d4a/f5d [110181,21570] 0 2024-05-25T06:11:44.006 INFO:tasks.workunit.client.0.smithi160.stdout:3/608: stat d19/d2f/d34/da8/lb5 0 2024-05-25T06:11:44.007 INFO:tasks.workunit.client.0.smithi160.stdout:0/753: mknod d2b/d46/dbd/cec 0 2024-05-25T06:11:44.007 INFO:tasks.workunit.client.0.smithi160.stdout:2/694: dwrite d12/d75/dca/f4c [0,4194304] 0 2024-05-25T06:11:44.009 INFO:tasks.workunit.client.0.smithi160.stdout:2/695: readlink d12/d1c/d2a/l46 0 2024-05-25T06:11:44.010 INFO:tasks.workunit.client.0.smithi160.stdout:3/609: chown d19/d2f/d38/d43/d5b/lc1 7 1 2024-05-25T06:11:44.011 INFO:tasks.workunit.client.0.smithi160.stdout:6/631: mknod d5/d39/da6/dac/cda 0 2024-05-25T06:11:44.011 INFO:tasks.workunit.client.0.smithi160.stdout:0/754: mknod d2b/d31/d35/d7b/ced 0 2024-05-25T06:11:44.018 INFO:tasks.workunit.client.0.smithi160.stdout:2/696: dwrite d12/d3e/d5a/f7f [0,4194304] 0 2024-05-25T06:11:44.019 INFO:tasks.workunit.client.0.smithi160.stdout:3/610: dwrite d19/db2/fc3 [0,4194304] 0 2024-05-25T06:11:44.026 INFO:tasks.workunit.client.0.smithi160.stdout:6/632: dread d5/d45/d63/f83 [0,4194304] 0 2024-05-25T06:11:44.028 INFO:tasks.workunit.client.0.smithi160.stdout:3/611: getdents d19/d2f/d38/d60 0 2024-05-25T06:11:44.030 INFO:tasks.workunit.client.0.smithi160.stdout:6/633: mknod d5/de/d32/d35/dc3/cdb 0 2024-05-25T06:11:44.031 INFO:tasks.workunit.client.0.smithi160.stdout:3/612: truncate d19/f88 1035306 0 2024-05-25T06:11:44.031 INFO:tasks.workunit.client.0.smithi160.stdout:6/634: chown d5/d39/f6c 35935281 1 2024-05-25T06:11:44.032 INFO:tasks.workunit.client.0.smithi160.stdout:2/697: truncate d12/d5c/f6b 3917039 0 2024-05-25T06:11:44.037 INFO:tasks.workunit.client.0.smithi160.stdout:3/613: rename d19/d2f/d34/d67/d96/cac to d19/ccf 0 2024-05-25T06:11:44.038 INFO:tasks.workunit.client.0.smithi160.stdout:6/635: dwrite d5/d45/f9a [0,4194304] 0 2024-05-25T06:11:44.041 INFO:tasks.workunit.client.0.smithi160.stdout:3/614: dread - d19/da6/f9c zero size 2024-05-25T06:11:44.041 INFO:tasks.workunit.client.0.smithi160.stdout:4/639: dread d4/d63/d64/fb9 [4194304,4194304] 0 2024-05-25T06:11:44.042 INFO:tasks.workunit.client.0.smithi160.stdout:0/755: dread d2b/d46/f5a [0,4194304] 0 2024-05-25T06:11:44.042 INFO:tasks.workunit.client.0.smithi160.stdout:2/698: dread d12/d75/dca/f24 [0,4194304] 0 2024-05-25T06:11:44.046 INFO:tasks.workunit.client.0.smithi160.stdout:4/640: symlink d4/d6/lc8 0 2024-05-25T06:11:44.048 INFO:tasks.workunit.client.0.smithi160.stdout:6/636: symlink d5/d30/d36/d73/d87/ldc 0 2024-05-25T06:11:44.050 INFO:tasks.workunit.client.0.smithi160.stdout:2/699: dwrite d12/d1c/f6c [0,4194304] 0 2024-05-25T06:11:44.051 INFO:tasks.workunit.client.0.smithi160.stdout:0/756: dwrite d2b/d46/d63/da7/f8e [0,4194304] 0 2024-05-25T06:11:44.052 INFO:tasks.workunit.client.0.smithi160.stdout:4/641: truncate d4/d6/d28/f9d 522033 0 2024-05-25T06:11:44.052 INFO:tasks.workunit.client.0.smithi160.stdout:2/700: fdatasync d12/d3e/d5a/d8d/fb9 0 2024-05-25T06:11:44.053 INFO:tasks.workunit.client.0.smithi160.stdout:6/637: chown d5/d45/c72 14 1 2024-05-25T06:11:44.057 INFO:tasks.workunit.client.0.smithi160.stdout:2/701: mknod d12/d75/ce9 0 2024-05-25T06:11:44.058 INFO:tasks.workunit.client.0.smithi160.stdout:2/702: chown d12/cb7 497 1 2024-05-25T06:11:44.062 INFO:tasks.workunit.client.0.smithi160.stdout:6/638: mkdir d5/d39/d8f/d2b/ddd 0 2024-05-25T06:11:44.063 INFO:tasks.workunit.client.0.smithi160.stdout:0/757: link d2b/d31/f7d d2b/d46/d63/db3/fee 0 2024-05-25T06:11:44.064 INFO:tasks.workunit.client.0.smithi160.stdout:2/703: mkdir d12/d5c/d85/dc1/dea 0 2024-05-25T06:11:44.065 INFO:tasks.workunit.client.0.smithi160.stdout:4/642: dwrite d4/d63/d64/fc1 [0,4194304] 0 2024-05-25T06:11:44.068 INFO:tasks.workunit.client.0.smithi160.stdout:6/639: symlink d5/d39/d8f/d2b/ddd/lde 0 2024-05-25T06:11:44.069 INFO:tasks.workunit.client.0.smithi160.stdout:0/758: symlink d2b/d46/d63/da7/da6/lef 0 2024-05-25T06:11:44.073 INFO:tasks.workunit.client.0.smithi160.stdout:0/759: creat d2b/d46/d63/da7/da6/ff0 x:0 0 0 2024-05-25T06:11:44.074 INFO:tasks.workunit.client.0.smithi160.stdout:2/704: getdents d12/d1c/d59 0 2024-05-25T06:11:44.075 INFO:tasks.workunit.client.0.smithi160.stdout:9/612: sync 2024-05-25T06:11:44.075 INFO:tasks.workunit.client.0.smithi160.stdout:5/749: sync 2024-05-25T06:11:44.076 INFO:tasks.workunit.client.0.smithi160.stdout:0/760: creat d2b/d46/dbd/ff1 x:0 0 0 2024-05-25T06:11:44.076 INFO:tasks.workunit.client.0.smithi160.stdout:0/761: chown l4 14901 1 2024-05-25T06:11:44.077 INFO:tasks.workunit.client.0.smithi160.stdout:3/615: dread d19/f99 [0,4194304] 0 2024-05-25T06:11:44.077 INFO:tasks.workunit.client.0.smithi160.stdout:2/705: truncate d12/d3e/d5a/d8d/fb9 794137 0 2024-05-25T06:11:44.079 INFO:tasks.workunit.client.0.smithi160.stdout:5/750: dread db/d47/fcb [0,4194304] 0 2024-05-25T06:11:44.080 INFO:tasks.workunit.client.0.smithi160.stdout:0/762: creat d2b/d31/d35/d43/d98/ff2 x:0 0 0 2024-05-25T06:11:44.080 INFO:tasks.workunit.client.0.smithi160.stdout:2/706: creat d12/d97/feb x:0 0 0 2024-05-25T06:11:44.129 INFO:tasks.workunit.client.0.smithi160.stdout:3/616: write d19/f44 [3185276,95958] 0 2024-05-25T06:11:44.131 INFO:tasks.workunit.client.0.smithi160.stdout:5/751: getdents db/dc/d13/d1d/d23/d24/daf 0 2024-05-25T06:11:44.132 INFO:tasks.workunit.client.0.smithi160.stdout:3/617: rmdir d19/da7 39 2024-05-25T06:11:44.133 INFO:tasks.workunit.client.0.smithi160.stdout:2/707: symlink d12/d47/da0/dd3/lec 0 2024-05-25T06:11:44.136 INFO:tasks.workunit.client.0.smithi160.stdout:0/763: dwrite d2b/d31/d35/d36/fb6 [0,4194304] 0 2024-05-25T06:11:44.137 INFO:tasks.workunit.client.0.smithi160.stdout:4/643: dwrite d4/d63/d64/fb9 [4194304,4194304] 0 2024-05-25T06:11:44.137 INFO:tasks.workunit.client.0.smithi160.stdout:2/708: chown d12/d1c/d2a/d7e/fa7 1816 1 2024-05-25T06:11:44.139 INFO:tasks.workunit.client.0.smithi160.stdout:6/640: dread d5/de/d9e/fa2 [0,4194304] 0 2024-05-25T06:11:44.139 INFO:tasks.workunit.client.0.smithi160.stdout:2/709: write d12/f9f [1121106,76630] 0 2024-05-25T06:11:44.141 INFO:tasks.workunit.client.0.smithi160.stdout:2/710: readlink d12/d3e/d5f/d74/d7a/l8e 0 2024-05-25T06:11:44.141 INFO:tasks.workunit.client.0.smithi160.stdout:2/711: truncate ff 2371203 0 2024-05-25T06:11:44.143 INFO:tasks.workunit.client.0.smithi160.stdout:4/644: mknod d4/d63/d70/cc9 0 2024-05-25T06:11:44.145 INFO:tasks.workunit.client.0.smithi160.stdout:0/764: rename d2b/d46/d63/fc7 to d2b/d31/d35/d7b/ff3 0 2024-05-25T06:11:44.146 INFO:tasks.workunit.client.0.smithi160.stdout:5/752: getdents db/d47/d5e/d6e 0 2024-05-25T06:11:44.148 INFO:tasks.workunit.client.0.smithi160.stdout:3/618: truncate d19/d22/f2d 3013786 0 2024-05-25T06:11:44.150 INFO:tasks.workunit.client.0.smithi160.stdout:5/753: stat db/dc/d13/d1d/d31/d3b/lb8 0 2024-05-25T06:11:44.150 INFO:tasks.workunit.client.0.smithi160.stdout:3/619: dread - d19/da6/f9c zero size 2024-05-25T06:11:44.151 INFO:tasks.workunit.client.0.smithi160.stdout:0/765: mknod d2b/d31/d35/d43/d98/cf4 0 2024-05-25T06:11:44.152 INFO:tasks.workunit.client.0.smithi160.stdout:2/712: mkdir d12/d3e/d5a/d8d/db4/dcb/ded 0 2024-05-25T06:11:44.154 INFO:tasks.workunit.client.0.smithi160.stdout:4/645: dwrite d4/d6/d28/f25 [0,4194304] 0 2024-05-25T06:11:44.156 INFO:tasks.workunit.client.0.smithi160.stdout:3/620: mknod d19/d2f/d38/d43/d5b/cd0 0 2024-05-25T06:11:44.157 INFO:tasks.workunit.client.0.smithi160.stdout:5/754: mknod db/dc/d13/d1d/d23/d24/ce5 0 2024-05-25T06:11:44.157 INFO:tasks.workunit.client.0.smithi160.stdout:0/766: write d2b/d46/d63/f7e [1157485,31596] 0 2024-05-25T06:11:44.160 INFO:tasks.workunit.client.0.smithi160.stdout:0/767: truncate d2b/d31/d35/d43/d98/f99 1122589 0 2024-05-25T06:11:44.160 INFO:tasks.workunit.client.0.smithi160.stdout:2/713: rename d12/l21 to d12/d75/dae/lee 0 2024-05-25T06:11:44.161 INFO:tasks.workunit.client.0.smithi160.stdout:3/621: fsync d19/d2f/d34/f5e 0 2024-05-25T06:11:44.162 INFO:tasks.workunit.client.0.smithi160.stdout:0/768: write d2b/d31/d35/d36/d37/f95 [471520,111070] 0 2024-05-25T06:11:44.162 INFO:tasks.workunit.client.0.smithi160.stdout:2/714: truncate d12/d3e/d5a/d8d/fb8 5688281 0 2024-05-25T06:11:44.163 INFO:tasks.workunit.client.0.smithi160.stdout:3/622: write d19/d2f/f94 [1293434,79852] 0 2024-05-25T06:11:44.163 INFO:tasks.workunit.client.0.smithi160.stdout:2/715: fsync d12/d1c/f6c 0 2024-05-25T06:11:44.165 INFO:tasks.workunit.client.0.smithi160.stdout:5/755: rename db/dc/d13/d1d/d31/d56/c73 to db/dc/d13/d1d/d3f/d72/d98/da5/ce6 0 2024-05-25T06:11:44.167 INFO:tasks.workunit.client.0.smithi160.stdout:4/646: getdents d4/d6/d28 0 2024-05-25T06:11:44.168 INFO:tasks.workunit.client.0.smithi160.stdout:0/769: read d2b/d46/d63/da7/da6/fd2 [438324,48463] 0 2024-05-25T06:11:44.168 INFO:tasks.workunit.client.0.smithi160.stdout:3/623: truncate d19/d2f/d38/d43/d5b/d65/fbf 259872 0 2024-05-25T06:11:44.169 INFO:tasks.workunit.client.0.smithi160.stdout:3/624: dread - d19/da6/f9c zero size 2024-05-25T06:11:44.170 INFO:tasks.workunit.client.1.smithi203.stdout:7/990: sync 2024-05-25T06:11:44.171 INFO:tasks.workunit.client.0.smithi160.stdout:4/647: dread d4/d63/d64/f80 [0,4194304] 0 2024-05-25T06:11:44.172 INFO:tasks.workunit.client.1.smithi203.stdout:7/991: chown d1/d9/d2c/d36/c143 15 1 2024-05-25T06:11:44.174 INFO:tasks.workunit.client.1.smithi203.stdout:7/992: creat d1/d11/d3d/d9a/f14a x:0 0 0 2024-05-25T06:11:44.175 INFO:tasks.workunit.client.0.smithi160.stdout:2/716: fsync d12/d75/f88 0 2024-05-25T06:11:44.175 INFO:tasks.workunit.client.0.smithi160.stdout:2/717: fsync f7 0 2024-05-25T06:11:44.176 INFO:tasks.workunit.client.0.smithi160.stdout:2/718: truncate d12/f8f 422040 0 2024-05-25T06:11:44.177 INFO:tasks.workunit.client.0.smithi160.stdout:3/625: symlink d19/d2f/d34/d67/ld1 0 2024-05-25T06:11:44.177 INFO:tasks.workunit.client.1.smithi203.stdout:7/993: mkdir d1/d9/d2c/d36/d3b/d14b 0 2024-05-25T06:11:44.179 INFO:tasks.workunit.client.0.smithi160.stdout:6/641: dread d5/d30/d36/d73/d6e/f84 [0,4194304] 0 2024-05-25T06:11:44.181 INFO:tasks.workunit.client.0.smithi160.stdout:0/770: mknod d2b/d46/cf5 0 2024-05-25T06:11:44.182 INFO:tasks.workunit.client.0.smithi160.stdout:0/771: truncate d2b/d46/dcb/fd1 156641 0 2024-05-25T06:11:44.184 INFO:tasks.workunit.client.0.smithi160.stdout:3/626: stat d19/d2f/d34/da8/lb1 0 2024-05-25T06:11:44.185 INFO:tasks.workunit.client.1.smithi203.stdout:7/994: write d1/d9/d2c/d3a/f49 [705716,64717] 0 2024-05-25T06:11:44.185 INFO:tasks.workunit.client.1.smithi203.stdout:7/995: stat d1/f4d 0 2024-05-25T06:11:44.185 INFO:tasks.workunit.client.0.smithi160.stdout:4/648: link d4/d1a/fa2 d4/d63/d64/d88/fca 0 2024-05-25T06:11:44.186 INFO:tasks.workunit.client.0.smithi160.stdout:4/649: chown d4/d6/d35/c4a 2461 1 2024-05-25T06:11:44.188 INFO:tasks.workunit.client.0.smithi160.stdout:8/699: sync 2024-05-25T06:11:44.188 INFO:tasks.workunit.client.0.smithi160.stdout:1/593: sync 2024-05-25T06:11:44.188 INFO:tasks.workunit.client.0.smithi160.stdout:7/728: sync 2024-05-25T06:11:44.191 INFO:tasks.workunit.client.0.smithi160.stdout:6/642: write d5/de/f17 [6609549,119062] 0 2024-05-25T06:11:44.191 INFO:tasks.workunit.client.0.smithi160.stdout:4/650: write d4/d6/f97 [28802,23299] 0 2024-05-25T06:11:44.191 INFO:tasks.workunit.client.1.smithi203.stdout:7/996: write d1/fd9 [398086,6956] 0 2024-05-25T06:11:44.192 INFO:tasks.workunit.client.0.smithi160.stdout:1/594: write de/f3a [4310580,92801] 0 2024-05-25T06:11:44.193 INFO:tasks.workunit.client.0.smithi160.stdout:0/772: symlink d2b/d46/d63/db3/lf6 0 2024-05-25T06:11:44.194 INFO:tasks.workunit.client.0.smithi160.stdout:0/773: readlink l7 0 2024-05-25T06:11:44.196 INFO:tasks.workunit.client.0.smithi160.stdout:2/719: creat d12/d3e/d5f/fef x:0 0 0 2024-05-25T06:11:44.197 INFO:tasks.workunit.client.0.smithi160.stdout:5/756: dwrite db/dc/d13/f87 [0,4194304] 0 2024-05-25T06:11:44.199 INFO:tasks.workunit.client.1.smithi203.stdout:7/997: dread d1/d9/d2c/d79/dc7/f11c [0,4194304] 0 2024-05-25T06:11:44.201 INFO:tasks.workunit.client.1.smithi203.stdout:7/998: chown d1/d9/d2c/d36/d3b/c97 5 1 2024-05-25T06:11:44.201 INFO:tasks.workunit.client.0.smithi160.stdout:3/627: mknod d19/d2f/d38/cd2 0 2024-05-25T06:11:44.202 INFO:tasks.workunit.client.0.smithi160.stdout:3/628: fdatasync d19/d2f/d38/d43/d5b/d65/d83/dba/fc8 0 2024-05-25T06:11:44.204 INFO:tasks.workunit.client.1.smithi203.stdout:7/999: fsync d1/d9/d2c/d36/d66/f8d 0 2024-05-25T06:11:44.204 INFO:tasks.workunit.client.0.smithi160.stdout:3/629: truncate d19/d2f/d38/fce 3551 0 2024-05-25T06:11:44.206 INFO:tasks.workunit.client.0.smithi160.stdout:3/630: write d19/d2f/d38/d60/fc4 [107287,71341] 0 2024-05-25T06:11:44.206 INFO:tasks.workunit.client.0.smithi160.stdout:3/631: stat d19/d22/f2a 0 2024-05-25T06:11:44.207 INFO:tasks.workunit.client.0.smithi160.stdout:2/720: mknod d12/d75/dca/cf0 0 2024-05-25T06:11:44.208 INFO:tasks.workunit.client.0.smithi160.stdout:5/757: mknod db/ce7 0 2024-05-25T06:11:44.211 INFO:tasks.workunit.client.0.smithi160.stdout:4/651: unlink d4/d6/d5b/d66/c91 0 2024-05-25T06:11:44.211 INFO:tasks.workunit.client.0.smithi160.stdout:2/721: dread - d12/d1c/d37/d4b/f79 zero size 2024-05-25T06:11:44.211 INFO:tasks.workunit.client.0.smithi160.stdout:3/632: fdatasync d19/d2f/d34/f40 0 2024-05-25T06:11:44.212 INFO:tasks.workunit.client.0.smithi160.stdout:1/595: symlink de/d21/d49/d1c/d41/d96/lc4 0 2024-05-25T06:11:44.214 INFO:tasks.workunit.client.0.smithi160.stdout:3/633: stat d19/l91 0 2024-05-25T06:11:44.215 INFO:tasks.workunit.client.0.smithi160.stdout:1/596: write f9 [6316326,97096] 0 2024-05-25T06:11:44.215 INFO:tasks.workunit.client.0.smithi160.stdout:8/700: getdents d5/d9/d72/dd9 0 2024-05-25T06:11:44.216 INFO:tasks.workunit.client.0.smithi160.stdout:5/758: mknod db/dc/d13/d9a/ce8 0 2024-05-25T06:11:44.217 INFO:tasks.workunit.client.0.smithi160.stdout:2/722: mkdir d12/d3c/df1 0 2024-05-25T06:11:44.217 INFO:tasks.workunit.client.0.smithi160.stdout:3/634: stat d19/d2f/d34/fa1 0 2024-05-25T06:11:44.218 INFO:tasks.workunit.client.0.smithi160.stdout:4/652: fsync d4/d6/d35/d3d/d55/f56 0 2024-05-25T06:11:44.218 INFO:tasks.workunit.client.0.smithi160.stdout:6/643: dread d5/d30/d36/d73/f95 [0,4194304] 0 2024-05-25T06:11:44.220 INFO:tasks.workunit.client.0.smithi160.stdout:7/729: dwrite d6/dc/d64/f3c [0,4194304] 0 2024-05-25T06:11:44.222 INFO:tasks.workunit.client.0.smithi160.stdout:0/774: dwrite d2b/d31/d35/f3e [0,4194304] 0 2024-05-25T06:11:44.222 INFO:tasks.workunit.client.0.smithi160.stdout:9/613: sync 2024-05-25T06:11:44.223 INFO:tasks.workunit.client.0.smithi160.stdout:8/701: write d5/d7/dcb/d7a/f92 [1263687,68336] 0 2024-05-25T06:11:44.224 INFO:tasks.workunit.client.0.smithi160.stdout:2/723: unlink d12/d1c/d2a/fdd 0 2024-05-25T06:11:44.232 INFO:tasks.workunit.client.0.smithi160.stdout:6/644: mkdir d5/d30/d36/d73/d87/da4/ddf 0 2024-05-25T06:11:44.234 INFO:tasks.workunit.client.0.smithi160.stdout:7/730: rename d6/dc/d21/f54 to d6/d3e/da1/ff1 0 2024-05-25T06:11:44.235 INFO:tasks.workunit.client.0.smithi160.stdout:4/653: dread d4/d1c/f9f [0,4194304] 0 2024-05-25T06:11:44.236 INFO:tasks.workunit.client.0.smithi160.stdout:9/614: unlink d4/d11/d1e/d24/c6b 0 2024-05-25T06:11:44.236 INFO:tasks.workunit.client.0.smithi160.stdout:9/615: chown d4/d11/d12/d2e/d45 218 1 2024-05-25T06:11:44.237 INFO:tasks.workunit.client.0.smithi160.stdout:3/635: symlink d19/d2f/d38/d43/d5b/d65/d7f/ld3 0 2024-05-25T06:11:44.238 INFO:tasks.workunit.client.0.smithi160.stdout:3/636: write d19/d2f/d34/f95 [1811070,50664] 0 2024-05-25T06:11:44.240 INFO:tasks.workunit.client.0.smithi160.stdout:8/702: creat d5/d7/dcb/fec x:0 0 0 2024-05-25T06:11:44.241 INFO:tasks.workunit.client.0.smithi160.stdout:5/759: creat db/dc/d13/d1d/d3f/fe9 x:0 0 0 2024-05-25T06:11:44.242 INFO:tasks.workunit.client.0.smithi160.stdout:0/775: truncate d2b/f78 2427441 0 2024-05-25T06:11:44.243 INFO:tasks.workunit.client.0.smithi160.stdout:2/724: mkdir d12/d1c/d2a/d7e/d90/df2 0 2024-05-25T06:11:44.243 INFO:tasks.workunit.client.0.smithi160.stdout:6/645: chown d5/de/f85 12241 1 2024-05-25T06:11:44.244 INFO:tasks.workunit.client.0.smithi160.stdout:2/725: write d12/d1c/d37/fdb [613272,64954] 0 2024-05-25T06:11:44.244 INFO:tasks.workunit.client.0.smithi160.stdout:1/597: dread de/d39/f85 [4194304,4194304] 0 2024-05-25T06:11:44.246 INFO:tasks.workunit.client.1.smithi203.stdout:1/934: sync 2024-05-25T06:11:44.246 INFO:tasks.workunit.client.1.smithi203.stdout:9/926: sync 2024-05-25T06:11:44.247 INFO:tasks.workunit.client.0.smithi160.stdout:2/726: write d12/d1c/d37/d4b/f89 [880362,123822] 0 2024-05-25T06:11:44.248 INFO:tasks.workunit.client.1.smithi203.stdout:1/935: dread - d14/d7c/ffb zero size 2024-05-25T06:11:44.248 INFO:tasks.workunit.client.0.smithi160.stdout:5/760: creat db/dc/d13/d1d/d3f/d72/fea x:0 0 0 2024-05-25T06:11:44.249 INFO:tasks.workunit.client.0.smithi160.stdout:2/727: dread - d12/d97/faa zero size 2024-05-25T06:11:44.250 INFO:tasks.workunit.client.0.smithi160.stdout:8/703: mkdir d5/d7/ded 0 2024-05-25T06:11:44.250 INFO:tasks.workunit.client.1.smithi203.stdout:1/936: symlink d14/d57/d97/dec/d131/l144 0 2024-05-25T06:11:44.251 INFO:tasks.workunit.client.1.smithi203.stdout:1/937: fdatasync d14/d23/f111 0 2024-05-25T06:11:44.251 INFO:tasks.workunit.client.0.smithi160.stdout:1/598: mkdir de/d21/d49/d1c/d59/d71/dc5 0 2024-05-25T06:11:44.253 INFO:tasks.workunit.client.1.smithi203.stdout:9/927: rename d0/d17/d20/d29/fe4 to d0/df/d37/d4f/d93/ddc/f131 0 2024-05-25T06:11:44.253 INFO:tasks.workunit.client.0.smithi160.stdout:4/654: creat d4/d6/d5b/d66/d78/fcb x:0 0 0 2024-05-25T06:11:44.253 INFO:tasks.workunit.client.1.smithi203.stdout:9/928: fsync d0/d17/d20/d29/ff4 0 2024-05-25T06:11:44.254 INFO:tasks.workunit.client.0.smithi160.stdout:3/637: link fe d19/d2f/d38/d43/d5b/d65/d7e/fd4 0 2024-05-25T06:11:44.254 INFO:tasks.workunit.client.0.smithi160.stdout:3/638: stat d19/f44 0 2024-05-25T06:11:44.255 INFO:tasks.workunit.client.1.smithi203.stdout:1/938: write d14/d57/d6b/da1/de2/d143/dd2/f62 [2373791,70860] 0 2024-05-25T06:11:44.255 INFO:tasks.workunit.client.0.smithi160.stdout:5/761: chown db/dc/d13/d1d/d3f/dc0/ldb 4146660 1 2024-05-25T06:11:44.256 INFO:tasks.workunit.client.1.smithi203.stdout:1/939: chown d14/d57/d6b/da1/de2/d143/dd2/f8d 13 1 2024-05-25T06:11:44.258 INFO:tasks.workunit.client.0.smithi160.stdout:3/639: truncate d19/d2f/d38/d43/d5b/d65/f93 67118 0 2024-05-25T06:11:44.260 INFO:tasks.workunit.client.0.smithi160.stdout:8/704: mkdir d5/d44/d1f/da3/d28/d4a/d4e/dee 0 2024-05-25T06:11:44.261 INFO:tasks.workunit.client.0.smithi160.stdout:8/705: read - d5/d7/d9a/fdd zero size 2024-05-25T06:11:44.263 INFO:tasks.workunit.client.0.smithi160.stdout:6/646: link d5/l19 d5/d39/dc1/le0 0 2024-05-25T06:11:44.264 INFO:tasks.workunit.client.0.smithi160.stdout:1/599: dread de/d39/d8f/fc2 [0,4194304] 0 2024-05-25T06:11:44.267 INFO:tasks.workunit.client.0.smithi160.stdout:3/640: mknod d19/d2f/d34/cd5 0 2024-05-25T06:11:44.270 INFO:tasks.workunit.client.0.smithi160.stdout:4/655: link d4/d6/d21/f3f d4/d1c/d54/d9b/fcc 0 2024-05-25T06:11:44.271 INFO:tasks.workunit.client.0.smithi160.stdout:4/656: fsync d4/f26 0 2024-05-25T06:11:44.271 INFO:tasks.workunit.client.0.smithi160.stdout:8/706: creat d5/d44/d1f/d58/d65/da2/fef x:0 0 0 2024-05-25T06:11:44.271 INFO:tasks.workunit.client.0.smithi160.stdout:7/731: write d6/dc/d64/f37 [2305518,89835] 0 2024-05-25T06:11:44.273 INFO:tasks.workunit.client.1.smithi203.stdout:1/940: dread d14/d57/d7e/fd0 [0,4194304] 0 2024-05-25T06:11:44.274 INFO:tasks.workunit.client.1.smithi203.stdout:1/941: chown d14/d57/d6b/da1/de2/l8c 49 1 2024-05-25T06:11:44.275 INFO:tasks.workunit.client.1.smithi203.stdout:1/942: fsync d14/d57/d7e/f10d 0 2024-05-25T06:11:44.276 INFO:tasks.workunit.client.1.smithi203.stdout:1/943: stat d14/f1a 0 2024-05-25T06:11:44.276 INFO:tasks.workunit.client.0.smithi160.stdout:5/762: rmdir db/dc/d13/d1d/d23/d45 39 2024-05-25T06:11:44.277 INFO:tasks.workunit.client.1.smithi203.stdout:1/944: chown c7 2068397558 1 2024-05-25T06:11:44.278 INFO:tasks.workunit.client.1.smithi203.stdout:1/945: read d14/d7c/fd3 [113990,53922] 0 2024-05-25T06:11:44.278 INFO:tasks.workunit.client.0.smithi160.stdout:4/657: mkdir d4/dcd 0 2024-05-25T06:11:44.279 INFO:tasks.workunit.client.0.smithi160.stdout:8/707: creat d5/d7/dcb/d6a/ff0 x:0 0 0 2024-05-25T06:11:44.280 INFO:tasks.workunit.client.0.smithi160.stdout:9/616: dwrite d4/d11/d14/f31 [0,4194304] 0 2024-05-25T06:11:44.281 INFO:tasks.workunit.client.0.smithi160.stdout:8/708: fsync d5/d44/d1f/da3/d28/d4a/d4e/d86/f66 0 2024-05-25T06:11:44.282 INFO:tasks.workunit.client.0.smithi160.stdout:1/600: mkdir de/d21/d49/dc6 0 2024-05-25T06:11:44.282 INFO:tasks.workunit.client.0.smithi160.stdout:8/709: stat d5/d44/f68 0 2024-05-25T06:11:44.284 INFO:tasks.workunit.client.0.smithi160.stdout:2/728: getdents d12/d3e/d5f/d74 0 2024-05-25T06:11:44.285 INFO:tasks.workunit.client.0.smithi160.stdout:3/641: symlink d19/ld6 0 2024-05-25T06:11:44.285 INFO:tasks.workunit.client.0.smithi160.stdout:1/601: readlink de/d21/d49/d1c/d59/d7c/d93/l4a 0 2024-05-25T06:11:44.285 INFO:tasks.workunit.client.1.smithi203.stdout:1/946: write d14/d23/f111 [177926,49727] 0 2024-05-25T06:11:44.287 INFO:tasks.workunit.client.0.smithi160.stdout:3/642: write d19/faf [48229,5635] 0 2024-05-25T06:11:44.288 INFO:tasks.workunit.client.0.smithi160.stdout:5/763: symlink db/dc/d13/d1d/d31/d56/leb 0 2024-05-25T06:11:44.289 INFO:tasks.workunit.client.1.smithi203.stdout:1/947: creat d14/d81/d100/f145 x:0 0 0 2024-05-25T06:11:44.290 INFO:tasks.workunit.client.0.smithi160.stdout:9/617: dwrite d4/d11/d1e/d3e/d25/f8b [0,4194304] 0 2024-05-25T06:11:44.294 INFO:tasks.workunit.client.0.smithi160.stdout:6/647: rename d5/de/d32/d35/d40 to d5/de1 0 2024-05-25T06:11:44.295 INFO:tasks.workunit.client.0.smithi160.stdout:8/710: symlink d5/d44/d1f/da3/d28/d4a/d4e/lf1 0 2024-05-25T06:11:44.296 INFO:tasks.workunit.client.0.smithi160.stdout:1/602: mkdir de/d21/d49/d1c/d52/dc7 0 2024-05-25T06:11:44.299 INFO:tasks.workunit.client.0.smithi160.stdout:6/648: fdatasync d5/de1/f7e 0 2024-05-25T06:11:44.302 INFO:tasks.workunit.client.0.smithi160.stdout:3/643: symlink d19/da7/ld7 0 2024-05-25T06:11:44.303 INFO:tasks.workunit.client.0.smithi160.stdout:3/644: chown d19/d2f/f94 939124 1 2024-05-25T06:11:44.304 INFO:tasks.workunit.client.0.smithi160.stdout:1/603: rename cd to de/d21/d49/d1c/d52/dc7/cc8 0 2024-05-25T06:11:44.308 INFO:tasks.workunit.client.0.smithi160.stdout:6/649: creat d5/d45/fe2 x:0 0 0 2024-05-25T06:11:44.308 INFO:tasks.workunit.client.0.smithi160.stdout:8/711: dread d5/d44/d1f/da3/d28/f5e [0,4194304] 0 2024-05-25T06:11:44.310 INFO:tasks.workunit.client.0.smithi160.stdout:2/729: getdents d12/d1c 0 2024-05-25T06:11:44.311 INFO:tasks.workunit.client.0.smithi160.stdout:1/604: unlink de/d21/d49/d1c/d59/d7f/da1/ca7 0 2024-05-25T06:11:44.313 INFO:tasks.workunit.client.0.smithi160.stdout:8/712: read d5/d44/d1f/da3/fa8 [315502,53231] 0 2024-05-25T06:11:44.313 INFO:tasks.workunit.client.0.smithi160.stdout:6/650: fdatasync d5/de/d32/d74/f92 0 2024-05-25T06:11:44.314 INFO:tasks.workunit.client.1.smithi203.stdout:1/948: dread d14/d57/d1d/f2f [0,4194304] 0 2024-05-25T06:11:44.318 INFO:tasks.workunit.client.0.smithi160.stdout:2/730: mkdir d12/d3e/d5f/df3 0 2024-05-25T06:11:44.318 INFO:tasks.workunit.client.0.smithi160.stdout:1/605: write de/d21/d49/d1c/d59/d71/f5a [268380,115938] 0 2024-05-25T06:11:44.321 INFO:tasks.workunit.client.0.smithi160.stdout:3/645: dread d19/d22/f39 [0,4194304] 0 2024-05-25T06:11:44.324 INFO:tasks.workunit.client.0.smithi160.stdout:5/764: write db/dc/d13/d1d/d23/f33 [5292865,78151] 0 2024-05-25T06:11:44.324 INFO:tasks.workunit.client.0.smithi160.stdout:7/732: dwrite d6/dc/d64/f78 [0,4194304] 0 2024-05-25T06:11:44.324 INFO:tasks.workunit.client.0.smithi160.stdout:8/713: rename d5/d44/d1f/d58/d65/l90 to d5/d7/dcb/d7a/dc9/lf2 0 2024-05-25T06:11:44.326 INFO:tasks.workunit.client.0.smithi160.stdout:7/733: chown d6/dc 7266986 1 2024-05-25T06:11:44.327 INFO:tasks.workunit.client.0.smithi160.stdout:9/618: dread d4/d11/f26 [0,4194304] 0 2024-05-25T06:11:44.327 INFO:tasks.workunit.client.0.smithi160.stdout:2/731: mkdir d12/d97/df4 0 2024-05-25T06:11:44.329 INFO:tasks.workunit.client.0.smithi160.stdout:4/658: dwrite d4/d1a/fa2 [0,4194304] 0 2024-05-25T06:11:44.329 INFO:tasks.workunit.client.1.smithi203.stdout:1/949: dwrite d14/d57/d97/da4/f121 [0,4194304] 0 2024-05-25T06:11:44.332 INFO:tasks.workunit.client.0.smithi160.stdout:8/714: mknod d5/d44/dba/ddb/cf3 0 2024-05-25T06:11:44.332 INFO:tasks.workunit.client.0.smithi160.stdout:6/651: write d5/d45/d4a/fa5 [2519011,96797] 0 2024-05-25T06:11:44.333 INFO:tasks.workunit.client.0.smithi160.stdout:8/715: readlink d5/d9/lbc 0 2024-05-25T06:11:44.337 INFO:tasks.workunit.client.0.smithi160.stdout:1/606: creat de/d21/d49/d1c/fc9 x:0 0 0 2024-05-25T06:11:44.339 INFO:tasks.workunit.client.0.smithi160.stdout:9/619: dwrite d4/fa [0,4194304] 0 2024-05-25T06:11:44.342 INFO:tasks.workunit.client.1.smithi203.stdout:1/950: dwrite d14/d57/d6b/da1/de2/f76 [0,4194304] 0 2024-05-25T06:11:44.343 INFO:tasks.workunit.client.1.smithi203.stdout:1/951: read - d14/d7c/ffb zero size 2024-05-25T06:11:44.345 INFO:tasks.workunit.client.1.smithi203.stdout:1/952: mknod d14/d57/d97/d11d/c146 0 2024-05-25T06:11:44.351 INFO:tasks.workunit.client.0.smithi160.stdout:5/765: creat db/dc/fec x:0 0 0 2024-05-25T06:11:44.351 INFO:tasks.workunit.client.0.smithi160.stdout:1/607: chown de/d21/d49/d1c/fa5 1 1 2024-05-25T06:11:44.351 INFO:tasks.workunit.client.0.smithi160.stdout:4/659: symlink d4/d6/lce 0 2024-05-25T06:11:44.351 INFO:tasks.workunit.client.0.smithi160.stdout:4/660: fdatasync d4/d6/d21/d42/fa4 0 2024-05-25T06:11:44.352 INFO:tasks.workunit.client.1.smithi203.stdout:1/953: dread d14/d23/f111 [0,4194304] 0 2024-05-25T06:11:44.355 INFO:tasks.workunit.client.0.smithi160.stdout:7/734: getdents d6/d32 0 2024-05-25T06:11:44.356 INFO:tasks.workunit.client.1.smithi203.stdout:1/954: mknod d14/d57/d6b/da1/de2/d143/dd2/dca/dcf/df7/c147 0 2024-05-25T06:11:44.356 INFO:tasks.workunit.client.0.smithi160.stdout:6/652: mknod d5/de/d9e/dcd/ce3 0 2024-05-25T06:11:44.356 INFO:tasks.workunit.client.0.smithi160.stdout:7/735: dread - d6/d3e/d59/fc8 zero size 2024-05-25T06:11:44.357 INFO:tasks.workunit.client.1.smithi203.stdout:1/955: fsync d14/d57/d1d/f40 0 2024-05-25T06:11:44.357 INFO:tasks.workunit.client.0.smithi160.stdout:6/653: readlink d5/de/d32/l24 0 2024-05-25T06:11:44.358 INFO:tasks.workunit.client.0.smithi160.stdout:3/646: dread fe [0,4194304] 0 2024-05-25T06:11:44.359 INFO:tasks.workunit.client.0.smithi160.stdout:9/620: rename d4/d11/d1e/d24/d66/fa0 to d4/d11/d1e/d3e/d25/d27/d95/fc4 0 2024-05-25T06:11:44.359 INFO:tasks.workunit.client.0.smithi160.stdout:1/608: mknod de/d21/d49/d1c/d52/d77/db5/cca 0 2024-05-25T06:11:44.360 INFO:tasks.workunit.client.0.smithi160.stdout:7/736: creat d6/d32/d92/ff2 x:0 0 0 2024-05-25T06:11:44.361 INFO:tasks.workunit.client.0.smithi160.stdout:3/647: chown d19/d2f/d34/d67/d96 1771 1 2024-05-25T06:11:44.361 INFO:tasks.workunit.client.1.smithi203.stdout:1/956: rename d14/d57/d6b/da1/de2/d143/dd2/f62 to d14/d57/d6b/da1/de2/d143/dd2/dca/dcf/f148 0 2024-05-25T06:11:44.362 INFO:tasks.workunit.client.1.smithi203.stdout:1/957: truncate d14/dff/f109 107028 0 2024-05-25T06:11:44.363 INFO:tasks.workunit.client.1.smithi203.stdout:1/958: write d14/d57/d97/ff2 [363410,55094] 0 2024-05-25T06:11:44.364 INFO:tasks.workunit.client.0.smithi160.stdout:6/654: fsync d5/de/ff 0 2024-05-25T06:11:44.365 INFO:tasks.workunit.client.0.smithi160.stdout:9/621: dread d4/d11/d12/f9f [0,4194304] 0 2024-05-25T06:11:44.365 INFO:tasks.workunit.client.0.smithi160.stdout:1/609: creat de/d39/d8f/fcb x:0 0 0 2024-05-25T06:11:44.366 INFO:tasks.workunit.client.0.smithi160.stdout:8/716: dwrite d5/d44/d1f/da3/d28/d49/fb4 [0,4194304] 0 2024-05-25T06:11:44.367 INFO:tasks.workunit.client.0.smithi160.stdout:7/737: write d6/d3e/d6f/f55 [625093,65279] 0 2024-05-25T06:11:44.367 INFO:tasks.workunit.client.0.smithi160.stdout:3/648: symlink d19/da6/ld8 0 2024-05-25T06:11:44.368 INFO:tasks.workunit.client.1.smithi203.stdout:1/959: unlink d14/d57/d6b/da1/de2/d143/cef 0 2024-05-25T06:11:44.369 INFO:tasks.workunit.client.0.smithi160.stdout:7/738: write d6/dc/d64/f37 [3070925,75360] 0 2024-05-25T06:11:44.370 INFO:tasks.workunit.client.1.smithi203.stdout:1/960: symlink d14/d23/d115/de4/l149 0 2024-05-25T06:11:44.371 INFO:tasks.workunit.client.0.smithi160.stdout:9/622: unlink d4/d11/d12/d2e/d45/d90/l9a 0 2024-05-25T06:11:44.371 INFO:tasks.workunit.client.0.smithi160.stdout:7/739: chown d6/dc/d64/d5a/d7c/le8 162 1 2024-05-25T06:11:44.373 INFO:tasks.workunit.client.1.smithi203.stdout:1/961: mkdir d14/d81/d14a 0 2024-05-25T06:11:44.373 INFO:tasks.workunit.client.0.smithi160.stdout:9/623: dread - d4/d11/d1e/d24/d66/fbf zero size 2024-05-25T06:11:44.375 INFO:tasks.workunit.client.1.smithi203.stdout:1/962: fsync d14/f1a 0 2024-05-25T06:11:44.376 INFO:tasks.workunit.client.0.smithi160.stdout:3/649: rename d19/d2f/d38/d43/d5b/d65/d83/dba to d19/d2f/d38/d43/d5b/d65/d7f/dd9 0 2024-05-25T06:11:44.377 INFO:tasks.workunit.client.0.smithi160.stdout:7/740: mknod d6/d3e/da1/ddb/cf3 0 2024-05-25T06:11:44.378 INFO:tasks.workunit.client.1.smithi203.stdout:1/963: mknod d14/d7c/d9d/dc8/c14b 0 2024-05-25T06:11:44.380 INFO:tasks.workunit.client.1.smithi203.stdout:1/964: creat d14/d57/d6b/da1/f14c x:0 0 0 2024-05-25T06:11:44.381 INFO:tasks.workunit.client.0.smithi160.stdout:3/650: dread d19/d2f/d38/f78 [0,4194304] 0 2024-05-25T06:11:44.381 INFO:tasks.workunit.client.0.smithi160.stdout:7/741: write d6/dc/d64/d5a/d7c/faa [880867,87996] 0 2024-05-25T06:11:44.381 INFO:tasks.workunit.client.1.smithi203.stdout:1/965: write d14/d57/f11f [933037,48322] 0 2024-05-25T06:11:44.382 INFO:tasks.workunit.client.0.smithi160.stdout:9/624: dwrite d4/d11/d1e/d3e/d2f/f98 [0,4194304] 0 2024-05-25T06:11:44.382 INFO:tasks.workunit.client.0.smithi160.stdout:7/742: read - d6/d2b/fc6 zero size 2024-05-25T06:11:44.384 INFO:tasks.workunit.client.0.smithi160.stdout:3/651: stat d19/d2f/d38/d43/d5b/d65/d83 0 2024-05-25T06:11:44.384 INFO:tasks.workunit.client.1.smithi203.stdout:1/966: chown d14/d57/d1d/d10f/f132 1592693 1 2024-05-25T06:11:44.385 INFO:tasks.workunit.client.0.smithi160.stdout:7/743: write d6/d2b/d5d/fc2 [690402,110556] 0 2024-05-25T06:11:44.386 INFO:tasks.workunit.client.1.smithi203.stdout:1/967: chown d14/d23/d115/f6e 1 1 2024-05-25T06:11:44.388 INFO:tasks.workunit.client.1.smithi203.stdout:1/968: creat d14/dff/f14d x:0 0 0 2024-05-25T06:11:44.392 INFO:tasks.workunit.client.0.smithi160.stdout:7/744: read d6/dc/ff [4402536,78928] 0 2024-05-25T06:11:44.392 INFO:tasks.workunit.client.1.smithi203.stdout:1/969: symlink d14/d57/d6b/da1/de2/d143/dd2/dca/l14e 0 2024-05-25T06:11:44.394 INFO:tasks.workunit.client.0.smithi160.stdout:5/766: rmdir db/dc 39 2024-05-25T06:11:44.396 INFO:tasks.workunit.client.0.smithi160.stdout:7/745: rename d6/dc/d64/d5a/d7c/ld7 to d6/d32/dce/lf4 0 2024-05-25T06:11:44.396 INFO:tasks.workunit.client.1.smithi203.stdout:1/970: rename d14/dac/c13d to d14/d57/d6b/da1/de2/d143/dd2/dca/dcf/c14f 0 2024-05-25T06:11:44.397 INFO:tasks.workunit.client.1.smithi203.stdout:1/971: chown d14/d57/dbb 7 1 2024-05-25T06:11:44.397 INFO:tasks.workunit.client.0.smithi160.stdout:5/767: stat db/dc/d13/d1d/l8e 0 2024-05-25T06:11:44.398 INFO:tasks.workunit.client.0.smithi160.stdout:7/746: mknod d6/d3e/d6f/d73/cf5 0 2024-05-25T06:11:44.399 INFO:tasks.workunit.client.0.smithi160.stdout:1/610: dread de/d21/d49/d35/f73 [0,4194304] 0 2024-05-25T06:11:44.401 INFO:tasks.workunit.client.0.smithi160.stdout:5/768: write db/dc/d13/d1d/d3f/d72/d98/da5/db5/fcd [412682,64783] 0 2024-05-25T06:11:44.402 INFO:tasks.workunit.client.0.smithi160.stdout:7/747: read d6/d3e/d6f/f6e [3335106,64899] 0 2024-05-25T06:11:44.404 INFO:tasks.workunit.client.0.smithi160.stdout:6/655: truncate d5/f1e 157366 0 2024-05-25T06:11:44.405 INFO:tasks.workunit.client.1.smithi203.stdout:1/972: dwrite d14/d57/d6b/da1/de2/d143/dd2/dca/dcf/df7/f11b [0,4194304] 0 2024-05-25T06:11:44.407 INFO:tasks.workunit.client.0.smithi160.stdout:1/611: rename fa to de/d21/d49/d1c/d52/dc7/fcc 0 2024-05-25T06:11:44.409 INFO:tasks.workunit.client.0.smithi160.stdout:7/748: dwrite f1 [0,4194304] 0 2024-05-25T06:11:44.410 INFO:tasks.workunit.client.0.smithi160.stdout:0/776: sync 2024-05-25T06:11:44.411 INFO:tasks.workunit.client.0.smithi160.stdout:9/625: dread d4/f5c [0,4194304] 0 2024-05-25T06:11:44.415 INFO:tasks.workunit.client.0.smithi160.stdout:5/769: creat db/dc/d13/d1d/d23/fed x:0 0 0 2024-05-25T06:11:44.415 INFO:tasks.workunit.client.0.smithi160.stdout:9/626: truncate d4/f69 9082974 0 2024-05-25T06:11:44.415 INFO:tasks.workunit.client.0.smithi160.stdout:3/652: write fe [1344487,62271] 0 2024-05-25T06:11:44.418 INFO:tasks.workunit.client.0.smithi160.stdout:0/777: mkdir d2b/d46/d63/da7/d67/d6a/df7 0 2024-05-25T06:11:44.424 INFO:tasks.workunit.client.0.smithi160.stdout:6/656: getdents d5/de/d32/d35/dc3 0 2024-05-25T06:11:44.424 INFO:tasks.workunit.client.0.smithi160.stdout:0/778: readlink d2b/d31/d35/d36/l53 0 2024-05-25T06:11:44.424 INFO:tasks.workunit.client.0.smithi160.stdout:7/749: getdents d6/d3e/d59 0 2024-05-25T06:11:44.425 INFO:tasks.workunit.client.0.smithi160.stdout:9/627: rmdir d4/d11/d1e/d24/d66 39 2024-05-25T06:11:44.426 INFO:tasks.workunit.client.0.smithi160.stdout:6/657: stat d5/de/d32/d74/d77/fb1 0 2024-05-25T06:11:44.426 INFO:tasks.workunit.client.1.smithi203.stdout:1/973: rename d14/d23/d115 to d14/d57/d1d/da8/d150 0 2024-05-25T06:11:44.427 INFO:tasks.workunit.client.0.smithi160.stdout:1/612: dwrite de/d21/d49/d35/f6e [0,4194304] 0 2024-05-25T06:11:44.429 INFO:tasks.workunit.client.0.smithi160.stdout:0/779: mkdir d2b/d46/d63/db3/dd6/df8 0 2024-05-25T06:11:44.429 INFO:tasks.workunit.client.0.smithi160.stdout:3/653: link d19/d2f/cb4 d19/d2f/cda 0 2024-05-25T06:11:44.431 INFO:tasks.workunit.client.0.smithi160.stdout:0/780: write d2b/d31/d35/d36/d37/f95 [4636753,105231] 0 2024-05-25T06:11:44.432 INFO:tasks.workunit.client.0.smithi160.stdout:6/658: chown d5/d30/d36/d73/d6e/lbe 27410162 1 2024-05-25T06:11:44.433 INFO:tasks.workunit.client.0.smithi160.stdout:0/781: stat d2b/d46/d63/da7/d67/db8 0 2024-05-25T06:11:44.434 INFO:tasks.workunit.client.0.smithi160.stdout:3/654: stat d19/d2f/d34/lc0 0 2024-05-25T06:11:44.434 INFO:tasks.workunit.client.0.smithi160.stdout:0/782: chown d2b 496307 1 2024-05-25T06:11:44.435 INFO:tasks.workunit.client.0.smithi160.stdout:9/628: mkdir d4/d11/d1e/d3e/d25/d27/dc5 0 2024-05-25T06:11:44.437 INFO:tasks.workunit.client.0.smithi160.stdout:1/613: unlink de/d21/d49/d1c/f8c 0 2024-05-25T06:11:44.437 INFO:tasks.workunit.client.1.smithi203.stdout:1/974: dwrite d14/d57/d7e/fd0 [4194304,4194304] 0 2024-05-25T06:11:44.437 INFO:tasks.workunit.client.0.smithi160.stdout:1/614: chown de/d21/d49 51 1 2024-05-25T06:11:44.439 INFO:tasks.workunit.client.0.smithi160.stdout:6/659: symlink d5/d45/d4a/le4 0 2024-05-25T06:11:44.443 INFO:tasks.workunit.client.0.smithi160.stdout:0/783: dread d2b/d46/faa [0,4194304] 0 2024-05-25T06:11:44.449 INFO:tasks.workunit.client.0.smithi160.stdout:6/660: write d5/d39/f6c [15246,98787] 0 2024-05-25T06:11:44.450 INFO:tasks.workunit.client.0.smithi160.stdout:3/655: unlink d19/d2f/d38/d43/l68 0 2024-05-25T06:11:44.450 INFO:tasks.workunit.client.0.smithi160.stdout:0/784: dwrite d2b/d46/d63/fa2 [4194304,4194304] 0 2024-05-25T06:11:44.452 INFO:tasks.workunit.client.0.smithi160.stdout:9/629: symlink d4/d11/d1e/d3e/d25/d5e/da2/lc6 0 2024-05-25T06:11:44.452 INFO:tasks.workunit.client.0.smithi160.stdout:1/615: fdatasync de/d21/d49/d1c/f53 0 2024-05-25T06:11:44.454 INFO:tasks.workunit.client.0.smithi160.stdout:6/661: rmdir d5/d39 39 2024-05-25T06:11:44.455 INFO:tasks.workunit.client.0.smithi160.stdout:9/630: dread - d4/d11/d1e/d24/d66/fbf zero size 2024-05-25T06:11:44.461 INFO:tasks.workunit.client.1.smithi203.stdout:1/975: dwrite d14/d57/d6b/da1/de2/d143/dd2/f39 [0,4194304] 0 2024-05-25T06:11:44.461 INFO:tasks.workunit.client.0.smithi160.stdout:6/662: creat d5/d45/d4a/fe5 x:0 0 0 2024-05-25T06:11:44.462 INFO:tasks.workunit.client.0.smithi160.stdout:0/785: getdents d2b/d46/d63/da7/db5 0 2024-05-25T06:11:44.464 INFO:tasks.workunit.client.0.smithi160.stdout:1/616: mknod de/d21/d49/d1c/d41/d45/dac/ccd 0 2024-05-25T06:11:44.470 INFO:tasks.workunit.client.0.smithi160.stdout:0/786: dread - d2b/d46/d63/db3/fee zero size 2024-05-25T06:11:44.470 INFO:tasks.workunit.client.0.smithi160.stdout:3/656: getdents d19/d2f/d38/d43/d5b/d65/d83 0 2024-05-25T06:11:44.470 INFO:tasks.workunit.client.0.smithi160.stdout:7/750: dwrite d6/d32/d5b/f96 [0,4194304] 0 2024-05-25T06:11:44.472 INFO:tasks.workunit.client.0.smithi160.stdout:6/663: mkdir d5/d39/da6/dac/de6 0 2024-05-25T06:11:44.473 INFO:tasks.workunit.client.0.smithi160.stdout:3/657: read - d19/d2f/d38/fb0 zero size 2024-05-25T06:11:44.475 INFO:tasks.workunit.client.0.smithi160.stdout:0/787: chown d2b/d46/d63/da7/l7c 86914 1 2024-05-25T06:11:44.475 INFO:tasks.workunit.client.0.smithi160.stdout:7/751: symlink d6/d2b/lf6 0 2024-05-25T06:11:44.478 INFO:tasks.workunit.client.0.smithi160.stdout:1/617: dwrite de/d21/f9a [0,4194304] 0 2024-05-25T06:11:44.481 INFO:tasks.workunit.client.0.smithi160.stdout:1/618: write de/d21/d49/d1c/d59/d67/fc1 [179530,95040] 0 2024-05-25T06:11:44.482 INFO:tasks.workunit.client.0.smithi160.stdout:3/658: creat d19/d2f/d38/d43/fdb x:0 0 0 2024-05-25T06:11:44.482 INFO:tasks.workunit.client.0.smithi160.stdout:0/788: rmdir d2b/d46/dcb/de8 0 2024-05-25T06:11:44.484 INFO:tasks.workunit.client.0.smithi160.stdout:6/664: dread d5/de/d32/f67 [0,4194304] 0 2024-05-25T06:11:44.487 INFO:tasks.workunit.client.0.smithi160.stdout:7/752: dwrite d6/d32/d92/fd5 [0,4194304] 0 2024-05-25T06:11:44.488 INFO:tasks.workunit.client.0.smithi160.stdout:3/659: creat d19/d2f/d38/d43/d5b/d65/d7f/dd9/fdc x:0 0 0 2024-05-25T06:11:44.490 INFO:tasks.workunit.client.0.smithi160.stdout:1/619: creat de/d21/d49/fce x:0 0 0 2024-05-25T06:11:44.491 INFO:tasks.workunit.client.0.smithi160.stdout:6/665: creat d5/d30/d36/d73/d87/da4/fe7 x:0 0 0 2024-05-25T06:11:44.491 INFO:tasks.workunit.client.0.smithi160.stdout:1/620: stat de/d39/d8f/fbc 0 2024-05-25T06:11:44.491 INFO:tasks.workunit.client.0.smithi160.stdout:0/789: dread d2b/d31/d35/d36/fb6 [0,4194304] 0 2024-05-25T06:11:44.492 INFO:tasks.workunit.client.0.smithi160.stdout:6/666: fsync d5/de/d32/d9f/fb7 0 2024-05-25T06:11:44.492 INFO:tasks.workunit.client.0.smithi160.stdout:1/621: chown de/d21/d49/d1c/d41/d45 2179717 1 2024-05-25T06:11:44.492 INFO:tasks.workunit.client.0.smithi160.stdout:7/753: unlink d6/d2b/d5d/f6a 0 2024-05-25T06:11:44.498 INFO:tasks.workunit.client.0.smithi160.stdout:3/660: rename d19/d22/cae to d19/d2f/d38/d43/d5b/d65/d7e/cdd 0 2024-05-25T06:11:44.499 INFO:tasks.workunit.client.0.smithi160.stdout:3/661: dread - d19/d2f/d34/fa1 zero size 2024-05-25T06:11:44.503 INFO:tasks.workunit.client.0.smithi160.stdout:0/790: rename l7 to d2b/d46/d63/lf9 0 2024-05-25T06:11:44.504 INFO:tasks.workunit.client.0.smithi160.stdout:6/667: dread d5/d30/f91 [0,4194304] 0 2024-05-25T06:11:44.506 INFO:tasks.workunit.client.0.smithi160.stdout:7/754: dread d6/dc/d64/f45 [0,4194304] 0 2024-05-25T06:11:44.508 INFO:tasks.workunit.client.0.smithi160.stdout:6/668: rename d5/de1/l62 to d5/d30/d36/d73/d6e/le8 0 2024-05-25T06:11:44.509 INFO:tasks.workunit.client.0.smithi160.stdout:7/755: symlink d6/d2b/da0/dc7/lf7 0 2024-05-25T06:11:44.510 INFO:tasks.workunit.client.0.smithi160.stdout:6/669: fsync d5/de/d32/d74/fbb 0 2024-05-25T06:11:44.512 INFO:tasks.workunit.client.0.smithi160.stdout:7/756: mknod d6/d2b/d5d/d89/dd6/cf8 0 2024-05-25T06:11:44.515 INFO:tasks.workunit.client.0.smithi160.stdout:7/757: creat d6/dc/d21/ff9 x:0 0 0 2024-05-25T06:11:44.516 INFO:tasks.workunit.client.0.smithi160.stdout:6/670: getdents d5/de/d32/d74 0 2024-05-25T06:11:44.520 INFO:tasks.workunit.client.0.smithi160.stdout:6/671: rename d5/d39/d8f/c4b to d5/d45/d4a/dbc/ce9 0 2024-05-25T06:11:44.526 INFO:tasks.workunit.client.0.smithi160.stdout:6/672: dwrite d5/d45/d4a/fbd [0,4194304] 0 2024-05-25T06:11:44.528 INFO:tasks.workunit.client.0.smithi160.stdout:6/673: mkdir d5/d45/d4a/dbc/dea 0 2024-05-25T06:11:44.530 INFO:tasks.workunit.client.1.smithi203.stdout:9/929: sync 2024-05-25T06:11:44.548 INFO:tasks.workunit.client.0.smithi160.stdout:0/791: dread d2b/d31/f7a [0,4194304] 0 2024-05-25T06:11:44.549 INFO:tasks.workunit.client.0.smithi160.stdout:0/792: chown d2b/d46/c96 12 1 2024-05-25T06:11:44.550 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:44 smithi203 ceph-mon[36030]: pgmap v22: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 147 MiB/s rd, 381 MiB/s wr, 754 op/s 2024-05-25T06:11:44.552 INFO:tasks.workunit.client.0.smithi160.stdout:0/793: mkdir d2b/da8/db2/dfa 0 2024-05-25T06:11:44.554 INFO:tasks.workunit.client.1.smithi203.stdout:9/930: link d0/df/d37/l85 d0/dd0/l132 0 2024-05-25T06:11:44.554 INFO:tasks.workunit.client.0.smithi160.stdout:0/794: mknod d2b/d31/d35/d36/d86/cfb 0 2024-05-25T06:11:44.557 INFO:tasks.workunit.client.1.smithi203.stdout:9/931: link d0/d57/dae/c11b d0/df/d2b/c133 0 2024-05-25T06:11:44.569 INFO:tasks.workunit.client.0.smithi160.stdout:0/795: dread f24 [4194304,4194304] 0 2024-05-25T06:11:44.569 INFO:tasks.workunit.client.0.smithi160.stdout:3/662: dread d19/d2f/d34/f40 [0,4194304] 0 2024-05-25T06:11:44.570 INFO:tasks.workunit.client.0.smithi160.stdout:0/796: truncate d2b/d31/d6d/fe3 137877 0 2024-05-25T06:11:44.572 INFO:tasks.workunit.client.0.smithi160.stdout:0/797: read d2b/d31/d35/d36/f89 [1845945,127160] 0 2024-05-25T06:11:44.572 INFO:tasks.workunit.client.0.smithi160.stdout:3/663: mknod d19/d2f/d38/d43/d5b/d65/d7f/cde 0 2024-05-25T06:11:44.577 INFO:tasks.workunit.client.0.smithi160.stdout:3/664: unlink d19/d2f/d38/d43/d5b/d65/fbf 0 2024-05-25T06:11:44.578 INFO:tasks.workunit.client.0.smithi160.stdout:0/798: symlink d2b/d46/d63/lfc 0 2024-05-25T06:11:44.582 INFO:tasks.workunit.client.0.smithi160.stdout:3/665: unlink d19/d22/f2d 0 2024-05-25T06:11:44.585 INFO:tasks.workunit.client.0.smithi160.stdout:3/666: readlink d19/d2f/d38/d43/d5b/d65/d7f/dd9/lca 0 2024-05-25T06:11:44.586 INFO:tasks.workunit.client.0.smithi160.stdout:0/799: dwrite d2b/d46/d63/da7/d67/fa5 [0,4194304] 0 2024-05-25T06:11:44.593 INFO:tasks.workunit.client.0.smithi160.stdout:6/674: write d5/d45/d4a/f90 [435060,30264] 0 2024-05-25T06:11:44.594 INFO:tasks.workunit.client.1.smithi203.stdout:9/932: write d0/d17/d20/d6e/d75/df2/d114/db2/d7a/fb4 [2349231,78061] 0 2024-05-25T06:11:44.594 INFO:tasks.workunit.client.0.smithi160.stdout:3/667: dread d19/d2f/d38/d47/f64 [0,4194304] 0 2024-05-25T06:11:44.595 INFO:tasks.workunit.client.0.smithi160.stdout:6/675: write d5/de1/fc7 [285459,95929] 0 2024-05-25T06:11:44.597 INFO:tasks.workunit.client.0.smithi160.stdout:6/676: write d5/d45/d4a/f53 [2368792,114394] 0 2024-05-25T06:11:44.597 INFO:tasks.workunit.client.1.smithi203.stdout:9/933: getdents d0/df/ddd/d128 0 2024-05-25T06:11:44.598 INFO:tasks.workunit.client.1.smithi203.stdout:9/934: chown d0/d17/d20/d6e/d75/df2/d114/d5c/ca2 622 1 2024-05-25T06:11:44.599 INFO:tasks.workunit.client.1.smithi203.stdout:9/935: stat d0/d17/d20/d6e/d75 0 2024-05-25T06:11:44.600 INFO:tasks.workunit.client.0.smithi160.stdout:6/677: creat d5/de/d9e/feb x:0 0 0 2024-05-25T06:11:44.601 INFO:tasks.workunit.client.0.smithi160.stdout:6/678: chown d5/d45/d63/d98 435777 1 2024-05-25T06:11:44.603 INFO:tasks.workunit.client.1.smithi203.stdout:9/936: read d0/df/d2b/f2f [2347707,61888] 0 2024-05-25T06:11:44.603 INFO:tasks.workunit.client.0.smithi160.stdout:7/758: dread d6/d2b/f6b [0,4194304] 0 2024-05-25T06:11:44.604 INFO:tasks.workunit.client.0.smithi160.stdout:4/661: sync 2024-05-25T06:11:44.604 INFO:tasks.workunit.client.0.smithi160.stdout:8/717: sync 2024-05-25T06:11:44.604 INFO:tasks.workunit.client.0.smithi160.stdout:5/770: sync 2024-05-25T06:11:44.604 INFO:tasks.workunit.client.0.smithi160.stdout:9/631: sync 2024-05-25T06:11:44.605 INFO:tasks.workunit.client.0.smithi160.stdout:2/732: sync 2024-05-25T06:11:44.605 INFO:tasks.workunit.client.1.smithi203.stdout:9/937: fsync d0/df/d2d/d92/fc2 0 2024-05-25T06:11:44.606 INFO:tasks.workunit.client.0.smithi160.stdout:0/800: write f20 [108823,93519] 0 2024-05-25T06:11:44.607 INFO:tasks.workunit.client.0.smithi160.stdout:8/718: fsync d5/d7/dcb/fa0 0 2024-05-25T06:11:44.608 INFO:tasks.workunit.client.1.smithi203.stdout:9/938: readlink d0/df/d37/d4f/l64 0 2024-05-25T06:11:44.609 INFO:tasks.workunit.client.0.smithi160.stdout:7/759: chown d6/da8 23715193 1 2024-05-25T06:11:44.609 INFO:tasks.workunit.client.1.smithi203.stdout:9/939: dread - d0/d17/d20/f126 zero size 2024-05-25T06:11:44.611 INFO:tasks.workunit.client.0.smithi160.stdout:5/771: symlink db/dc/d13/d1d/d3f/dc0/lee 0 2024-05-25T06:11:44.611 INFO:tasks.workunit.client.1.smithi203.stdout:9/940: read d0/d57/fd9 [2406795,28376] 0 2024-05-25T06:11:44.612 INFO:tasks.workunit.client.0.smithi160.stdout:9/632: mknod d4/d11/cc7 0 2024-05-25T06:11:44.614 INFO:tasks.workunit.client.1.smithi203.stdout:9/941: creat d0/df/ddd/d11c/f134 x:0 0 0 2024-05-25T06:11:44.616 INFO:tasks.workunit.client.0.smithi160.stdout:2/733: link d12/d3c/fd5 d12/d97/df4/ff5 0 2024-05-25T06:11:44.617 INFO:tasks.workunit.client.0.smithi160.stdout:0/801: truncate d2b/f42 2730311 0 2024-05-25T06:11:44.617 INFO:tasks.workunit.client.0.smithi160.stdout:2/734: write ff [2359708,40922] 0 2024-05-25T06:11:44.618 INFO:tasks.workunit.client.0.smithi160.stdout:8/719: fdatasync d5/f48 0 2024-05-25T06:11:44.618 INFO:tasks.workunit.client.0.smithi160.stdout:4/662: symlink d4/d6/d5b/d66/d78/da1/lcf 0 2024-05-25T06:11:44.619 INFO:tasks.workunit.client.0.smithi160.stdout:7/760: truncate d6/d3e/fca 4160488 0 2024-05-25T06:11:44.619 INFO:tasks.workunit.client.1.smithi203.stdout:9/942: write d0/df/d37/da8/db8/fb9 [1046256,81543] 0 2024-05-25T06:11:44.621 INFO:tasks.workunit.client.0.smithi160.stdout:3/668: dread d19/d2f/d34/f95 [0,4194304] 0 2024-05-25T06:11:44.622 INFO:tasks.workunit.client.0.smithi160.stdout:2/735: write d12/d1c/d37/fcc [1179453,54806] 0 2024-05-25T06:11:44.622 INFO:tasks.workunit.client.1.smithi203.stdout:9/943: creat d0/df/ddd/d128/f135 x:0 0 0 2024-05-25T06:11:44.624 INFO:tasks.workunit.client.0.smithi160.stdout:9/633: fdatasync d4/d11/f38 0 2024-05-25T06:11:44.624 INFO:tasks.workunit.client.1.smithi203.stdout:9/944: readlink d0/d17/d20/d6e/d75/df2/d114/db2/lbb 0 2024-05-25T06:11:44.624 INFO:tasks.workunit.client.0.smithi160.stdout:6/679: dread d5/d39/d8f/f9d [0,4194304] 0 2024-05-25T06:11:44.625 INFO:tasks.workunit.client.0.smithi160.stdout:3/669: stat d19/d2f/d38/d43/c72 0 2024-05-25T06:11:44.627 INFO:tasks.workunit.client.0.smithi160.stdout:0/802: write d2b/d31/d35/f49 [260622,104994] 0 2024-05-25T06:11:44.632 INFO:tasks.workunit.client.0.smithi160.stdout:3/670: dread - d19/d2f/d38/d43/d5b/d65/d7f/dd9/fc8 zero size 2024-05-25T06:11:44.633 INFO:tasks.workunit.client.1.smithi203.stdout:9/945: dwrite d0/dd0/fa0 [0,4194304] 0 2024-05-25T06:11:44.633 INFO:tasks.workunit.client.0.smithi160.stdout:7/761: dread f1 [0,4194304] 0 2024-05-25T06:11:44.635 INFO:tasks.workunit.client.0.smithi160.stdout:8/720: dwrite d5/d7/dcb/d7a/f92 [0,4194304] 0 2024-05-25T06:11:44.635 INFO:tasks.workunit.client.0.smithi160.stdout:7/762: stat d6/dc/d21/l48 0 2024-05-25T06:11:44.638 INFO:tasks.workunit.client.0.smithi160.stdout:0/803: symlink d2b/d31/d35/d43/lfd 0 2024-05-25T06:11:44.640 INFO:tasks.workunit.client.0.smithi160.stdout:0/804: chown d2b/d31/d35/d36/l3d 77 1 2024-05-25T06:11:44.640 INFO:tasks.workunit.client.0.smithi160.stdout:9/634: rename d4/d11/d1e/d3e/d25/l51 to d4/d11/d12/d2e/d45/d83/lc8 0 2024-05-25T06:11:44.640 INFO:tasks.workunit.client.0.smithi160.stdout:5/772: dread db/dc/d13/d1d/d23/d24/d2e/f4e [0,4194304] 0 2024-05-25T06:11:44.643 INFO:tasks.workunit.client.0.smithi160.stdout:3/671: unlink d19/d2f/d38/fab 0 2024-05-25T06:11:44.644 INFO:tasks.workunit.client.0.smithi160.stdout:3/672: read - d19/d22/f2e zero size 2024-05-25T06:11:44.644 INFO:tasks.workunit.client.0.smithi160.stdout:4/663: write d4/d1c/d54/d9b/fcc [2978491,46719] 0 2024-05-25T06:11:44.645 INFO:tasks.workunit.client.0.smithi160.stdout:5/773: read db/dc/d13/f65 [3940180,64448] 0 2024-05-25T06:11:44.646 INFO:tasks.workunit.client.0.smithi160.stdout:6/680: getdents d5/de/d32/d74/d77/d89 0 2024-05-25T06:11:44.646 INFO:tasks.workunit.client.0.smithi160.stdout:3/673: write d19/d2f/d38/d43/d5b/d65/d83/fcb [574974,42376] 0 2024-05-25T06:11:44.648 INFO:tasks.workunit.client.0.smithi160.stdout:9/635: symlink d4/d11/d1e/d3e/d2f/lc9 0 2024-05-25T06:11:44.648 INFO:tasks.workunit.client.0.smithi160.stdout:4/664: write d4/d6/d5b/d66/d78/fcb [660175,109865] 0 2024-05-25T06:11:44.650 INFO:tasks.workunit.client.0.smithi160.stdout:4/665: dread - d4/d6/d28/fc7 zero size 2024-05-25T06:11:44.651 INFO:tasks.workunit.client.0.smithi160.stdout:0/805: symlink d2b/d46/d63/da7/db5/lfe 0 2024-05-25T06:11:44.654 INFO:tasks.workunit.client.0.smithi160.stdout:3/674: mkdir d19/d2f/d38/d43/d5b/d65/d83/ddf 0 2024-05-25T06:11:44.655 INFO:tasks.workunit.client.0.smithi160.stdout:5/774: dwrite db/dc/d13/f6a [4194304,4194304] 0 2024-05-25T06:11:44.655 INFO:tasks.workunit.client.0.smithi160.stdout:3/675: readlink d19/d22/l73 0 2024-05-25T06:11:44.656 INFO:tasks.workunit.client.0.smithi160.stdout:1/622: sync 2024-05-25T06:11:44.657 INFO:tasks.workunit.client.0.smithi160.stdout:9/636: write d4/d11/d1e/d3e/d25/d27/f77 [5333531,20872] 0 2024-05-25T06:11:44.657 INFO:tasks.workunit.client.0.smithi160.stdout:4/666: rmdir d4/d6/d5b 39 2024-05-25T06:11:44.658 INFO:tasks.workunit.client.0.smithi160.stdout:0/806: fdatasync d2b/d31/d35/d36/d86/fae 0 2024-05-25T06:11:44.660 INFO:tasks.workunit.client.0.smithi160.stdout:2/736: truncate d12/d1c/d37/f71 3972189 0 2024-05-25T06:11:44.660 INFO:tasks.workunit.client.0.smithi160.stdout:2/737: chown d12/f7b 2020 1 2024-05-25T06:11:44.661 INFO:tasks.workunit.client.0.smithi160.stdout:8/721: link d5/l47 d5/d44/d1f/d58/d65/da2/lf4 0 2024-05-25T06:11:44.662 INFO:tasks.workunit.client.1.smithi203.stdout:9/946: dread d0/f4 [0,4194304] 0 2024-05-25T06:11:44.663 INFO:tasks.workunit.client.0.smithi160.stdout:3/676: mknod d19/d22/ce0 0 2024-05-25T06:11:44.663 INFO:tasks.workunit.client.0.smithi160.stdout:3/677: chown fe 9 1 2024-05-25T06:11:44.664 INFO:tasks.workunit.client.0.smithi160.stdout:2/738: dread d12/d1c/f6c [0,4194304] 0 2024-05-25T06:11:44.665 INFO:tasks.workunit.client.0.smithi160.stdout:1/623: write de/f54 [1485463,122278] 0 2024-05-25T06:11:44.666 INFO:tasks.workunit.client.1.smithi203.stdout:9/947: mkdir d0/df/d37/da8/de3/d136 0 2024-05-25T06:11:44.666 INFO:tasks.workunit.client.0.smithi160.stdout:3/678: dread d19/d2f/d38/d60/fc4 [0,4194304] 0 2024-05-25T06:11:44.667 INFO:tasks.workunit.client.1.smithi203.stdout:9/948: write d0/d17/dec/f122 [686384,51824] 0 2024-05-25T06:11:44.667 INFO:tasks.workunit.client.0.smithi160.stdout:5/775: creat db/dc/d13/d1d/d23/dbd/fef x:0 0 0 2024-05-25T06:11:44.668 INFO:tasks.workunit.client.0.smithi160.stdout:3/679: fsync d19/d2f/d34/fa1 0 2024-05-25T06:11:44.669 INFO:tasks.workunit.client.1.smithi203.stdout:9/949: creat d0/d57/de0/f137 x:0 0 0 2024-05-25T06:11:44.671 INFO:tasks.workunit.client.0.smithi160.stdout:4/667: creat d4/d6/d21/fd0 x:0 0 0 2024-05-25T06:11:44.671 INFO:tasks.workunit.client.1.smithi203.stdout:9/950: unlink d0/df/fd1 0 2024-05-25T06:11:44.672 INFO:tasks.workunit.client.0.smithi160.stdout:7/763: write d6/d3e/d6f/f56 [1518261,117661] 0 2024-05-25T06:11:44.673 INFO:tasks.workunit.client.1.smithi203.stdout:9/951: truncate d0/d17/d20/d6e/f10b 316625 0 2024-05-25T06:11:44.673 INFO:tasks.workunit.client.0.smithi160.stdout:8/722: symlink d5/d44/d1f/d58/d65/da2/dd1/lf5 0 2024-05-25T06:11:44.674 INFO:tasks.workunit.client.0.smithi160.stdout:8/723: fsync d5/d44/f68 0 2024-05-25T06:11:44.678 INFO:tasks.workunit.client.0.smithi160.stdout:5/776: creat db/dc/d13/d1d/d23/dbd/ff0 x:0 0 0 2024-05-25T06:11:44.679 INFO:tasks.workunit.client.0.smithi160.stdout:3/680: mkdir d19/d2f/d34/d67/d96/de1 0 2024-05-25T06:11:44.680 INFO:tasks.workunit.client.0.smithi160.stdout:5/777: fsync db/dc/d13/d1d/d3f/d72/d98/da5/db5/fdd 0 2024-05-25T06:11:44.681 INFO:tasks.workunit.client.0.smithi160.stdout:4/668: mknod d4/d63/d64/cd1 0 2024-05-25T06:11:44.687 INFO:tasks.workunit.client.0.smithi160.stdout:6/681: dwrite d5/de/d32/f37 [0,4194304] 0 2024-05-25T06:11:44.688 INFO:tasks.workunit.client.0.smithi160.stdout:7/764: mkdir d6/d3e/d6f/dfa 0 2024-05-25T06:11:44.688 INFO:tasks.workunit.client.0.smithi160.stdout:8/724: mknod d5/d44/d1f/da3/d28/d4a/d93/cf6 0 2024-05-25T06:11:44.688 INFO:tasks.workunit.client.0.smithi160.stdout:5/778: write db/dc/d13/f87 [3677361,44539] 0 2024-05-25T06:11:44.689 INFO:tasks.workunit.client.0.smithi160.stdout:4/669: chown d4/d6/d35/fb2 2881 1 2024-05-25T06:11:44.690 INFO:tasks.workunit.client.0.smithi160.stdout:8/725: chown d5/d7/dcb/d7a/c7c 0 1 2024-05-25T06:11:44.704 INFO:tasks.workunit.client.0.smithi160.stdout:5/779: dread db/dc/d13/d1d/d23/d24/d2e/fb7 [0,4194304] 0 2024-05-25T06:11:44.706 INFO:tasks.workunit.client.0.smithi160.stdout:5/780: chown db/dc/d13/d1d/d23/d24/daf/db3/cc4 1438372 1 2024-05-25T06:11:44.706 INFO:tasks.workunit.client.0.smithi160.stdout:3/681: symlink d19/d2f/d38/d43/d5b/d65/d7f/dd9/le2 0 2024-05-25T06:11:44.707 INFO:tasks.workunit.client.0.smithi160.stdout:3/682: readlink d19/d2f/d38/d47/l8d 0 2024-05-25T06:11:44.707 INFO:tasks.workunit.client.1.smithi203.stdout:9/952: creat d0/df/d37/da8/db8/f138 x:0 0 0 2024-05-25T06:11:44.708 INFO:tasks.workunit.client.0.smithi160.stdout:6/682: creat d5/d39/fec x:0 0 0 2024-05-25T06:11:44.709 INFO:tasks.workunit.client.1.smithi203.stdout:9/953: creat d0/d17/d20/d6e/d75/f139 x:0 0 0 2024-05-25T06:11:44.710 INFO:tasks.workunit.client.0.smithi160.stdout:1/624: rmdir de/d21/d49/d1c/d59/d71/dc5 0 2024-05-25T06:11:44.711 INFO:tasks.workunit.client.0.smithi160.stdout:1/625: chown de/f3d 1201027 1 2024-05-25T06:11:44.712 INFO:tasks.workunit.client.0.smithi160.stdout:8/726: mknod d5/d9/d72/dd9/cf7 0 2024-05-25T06:11:44.712 INFO:tasks.workunit.client.1.smithi203.stdout:9/954: symlink d0/df/d37/l13a 0 2024-05-25T06:11:44.712 INFO:tasks.workunit.client.1.smithi203.stdout:9/955: write d0/d57/de0/f124 [20497,30994] 0 2024-05-25T06:11:44.715 INFO:tasks.workunit.client.1.smithi203.stdout:9/956: read d0/d17/d20/d6e/d75/df2/d114/db2/fe7 [367279,117669] 0 2024-05-25T06:11:44.715 INFO:tasks.workunit.client.0.smithi160.stdout:9/637: rename d4/d11/c68 to d4/d11/d12/d2e/cca 0 2024-05-25T06:11:44.717 INFO:tasks.workunit.client.0.smithi160.stdout:2/739: getdents d12/d5c/d85/dc1 0 2024-05-25T06:11:44.719 INFO:tasks.workunit.client.0.smithi160.stdout:6/683: mknod d5/d30/d36/d73/d6e/ced 0 2024-05-25T06:11:44.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:44 smithi160 ceph-mon[29041]: pgmap v22: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 147 MiB/s rd, 381 MiB/s wr, 754 op/s 2024-05-25T06:11:44.723 INFO:tasks.workunit.client.0.smithi160.stdout:1/626: unlink de/d21/d49/la9 0 2024-05-25T06:11:44.726 INFO:tasks.workunit.client.0.smithi160.stdout:9/638: symlink d4/d11/d1e/d3e/d25/lcb 0 2024-05-25T06:11:44.726 INFO:tasks.workunit.client.0.smithi160.stdout:5/781: dread db/d47/f66 [0,4194304] 0 2024-05-25T06:11:44.727 INFO:tasks.workunit.client.0.smithi160.stdout:3/683: truncate d19/d2f/d38/f3e 952213 0 2024-05-25T06:11:44.729 INFO:tasks.workunit.client.0.smithi160.stdout:5/782: truncate db/dc/fb4 19628 0 2024-05-25T06:11:44.730 INFO:tasks.workunit.client.1.smithi203.stdout:9/957: rename d0/d17/d20/d6e/d75/df2/d114/db2/lbb to d0/d17/d20/d29/l13b 0 2024-05-25T06:11:44.730 INFO:tasks.workunit.client.0.smithi160.stdout:9/639: write d4/f5c [2806457,115125] 0 2024-05-25T06:11:44.732 INFO:tasks.workunit.client.0.smithi160.stdout:9/640: dread - d4/d11/d14/d8e/fad zero size 2024-05-25T06:11:44.732 INFO:tasks.workunit.client.0.smithi160.stdout:5/783: stat db/dc/d13/d1d/d23/d24/daf/db3/fd6 0 2024-05-25T06:11:44.733 INFO:tasks.workunit.client.0.smithi160.stdout:3/684: unlink d19/d2f/d38/d47/f70 0 2024-05-25T06:11:44.733 INFO:tasks.workunit.client.1.smithi203.stdout:9/958: mknod d0/d17/d20/d29/c13c 0 2024-05-25T06:11:44.734 INFO:tasks.workunit.client.0.smithi160.stdout:3/685: fsync d19/f27 0 2024-05-25T06:11:44.734 INFO:tasks.workunit.client.1.smithi203.stdout:9/959: write d0/dd0/f110 [857598,116734] 0 2024-05-25T06:11:44.735 INFO:tasks.workunit.client.0.smithi160.stdout:6/684: dwrite d5/d39/d8f/f9d [0,4194304] 0 2024-05-25T06:11:44.735 INFO:tasks.workunit.client.0.smithi160.stdout:2/740: getdents d12/d47 0 2024-05-25T06:11:44.736 INFO:tasks.workunit.client.0.smithi160.stdout:3/686: fdatasync d19/d2f/d38/d43/d5b/d65/d7f/dd9/fdc 0 2024-05-25T06:11:44.740 INFO:tasks.workunit.client.0.smithi160.stdout:9/641: creat d4/d11/d12/fcc x:0 0 0 2024-05-25T06:11:44.740 INFO:tasks.workunit.client.0.smithi160.stdout:5/784: dwrite f6 [0,4194304] 0 2024-05-25T06:11:44.742 INFO:tasks.workunit.client.0.smithi160.stdout:2/741: dread - d12/d3e/d5f/fa4 zero size 2024-05-25T06:11:44.746 INFO:tasks.workunit.client.1.smithi203.stdout:9/960: dread d0/d17/f81 [0,4194304] 0 2024-05-25T06:11:44.746 INFO:tasks.workunit.client.0.smithi160.stdout:9/642: creat d4/d11/d12/d2e/d45/d83/fcd x:0 0 0 2024-05-25T06:11:44.747 INFO:tasks.workunit.client.0.smithi160.stdout:6/685: dread d5/f1e [0,4194304] 0 2024-05-25T06:11:44.747 INFO:tasks.workunit.client.0.smithi160.stdout:5/785: dread db/dc/d13/d1d/d23/d24/daf/f90 [0,4194304] 0 2024-05-25T06:11:44.747 INFO:tasks.workunit.client.0.smithi160.stdout:9/643: fdatasync d4/d11/d1e/d3e/f44 0 2024-05-25T06:11:44.749 INFO:tasks.workunit.client.1.smithi203.stdout:9/961: mknod d0/df/ddd/d11c/c13d 0 2024-05-25T06:11:44.749 INFO:tasks.workunit.client.0.smithi160.stdout:3/687: symlink d19/d2f/d34/d67/d96/de1/le3 0 2024-05-25T06:11:44.753 INFO:tasks.workunit.client.0.smithi160.stdout:2/742: rename d12/d75/f88 to d12/d3e/d5f/d74/d7a/de3/ff6 0 2024-05-25T06:11:44.755 INFO:tasks.workunit.client.0.smithi160.stdout:6/686: mknod d5/d45/d4a/dbc/cee 0 2024-05-25T06:11:44.756 INFO:tasks.workunit.client.0.smithi160.stdout:5/786: dwrite db/dc/d13/d1d/d23/fdf [0,4194304] 0 2024-05-25T06:11:44.757 INFO:tasks.workunit.client.0.smithi160.stdout:2/743: read d12/f9f [1001223,71453] 0 2024-05-25T06:11:44.760 INFO:tasks.workunit.client.0.smithi160.stdout:7/765: write d6/dc/f25 [2846540,57431] 0 2024-05-25T06:11:44.760 INFO:tasks.workunit.client.1.smithi203.stdout:9/962: dread d0/f39 [0,4194304] 0 2024-05-25T06:11:44.761 INFO:tasks.workunit.client.0.smithi160.stdout:4/670: truncate d4/d1a/fa2 4196468 0 2024-05-25T06:11:44.761 INFO:tasks.workunit.client.0.smithi160.stdout:7/766: readlink d6/d3e/da1/lb9 0 2024-05-25T06:11:44.763 INFO:tasks.workunit.client.1.smithi203.stdout:9/963: creat d0/df/d37/da8/de3/f13e x:0 0 0 2024-05-25T06:11:44.765 INFO:tasks.workunit.client.0.smithi160.stdout:8/727: dwrite d5/d44/d1f/da3/d28/d4a/d4e/fa1 [0,4194304] 0 2024-05-25T06:11:44.765 INFO:tasks.workunit.client.0.smithi160.stdout:6/687: chown d5/de/d18 25468975 1 2024-05-25T06:11:44.766 INFO:tasks.workunit.client.0.smithi160.stdout:8/728: chown d5/d9/d72/le4 122941425 1 2024-05-25T06:11:44.766 INFO:tasks.workunit.client.1.smithi203.stdout:9/964: mknod d0/df/d37/da8/de3/d136/c13f 0 2024-05-25T06:11:44.767 INFO:tasks.workunit.client.0.smithi160.stdout:8/729: write d5/d7/dcb/d7a/f99 [713587,47548] 0 2024-05-25T06:11:44.771 INFO:tasks.workunit.client.1.smithi203.stdout:9/965: getdents d0/df/d37/d4f/d93/ddc 0 2024-05-25T06:11:44.771 INFO:tasks.workunit.client.0.smithi160.stdout:2/744: read d12/d1c/d37/d4b/f5e [47950,31340] 0 2024-05-25T06:11:44.773 INFO:tasks.workunit.client.1.smithi203.stdout:9/966: dread - d0/d17/d23/f6c zero size 2024-05-25T06:11:44.775 INFO:tasks.workunit.client.0.smithi160.stdout:4/671: rename d4/c6e to d4/d1c/d54/dbc/cd2 0 2024-05-25T06:11:44.775 INFO:tasks.workunit.client.0.smithi160.stdout:4/672: chown d4/dcd 1043806234 1 2024-05-25T06:11:44.777 INFO:tasks.workunit.client.0.smithi160.stdout:4/673: write d4/d63/d64/fc1 [2161390,88254] 0 2024-05-25T06:11:44.779 INFO:tasks.workunit.client.0.smithi160.stdout:1/627: write de/d21/d49/f82 [387349,29882] 0 2024-05-25T06:11:44.781 INFO:tasks.workunit.client.0.smithi160.stdout:1/628: write de/d21/d49/d1c/d59/d71/f5a [1092056,31020] 0 2024-05-25T06:11:44.783 INFO:tasks.workunit.client.0.smithi160.stdout:9/644: creat d4/d11/d1e/d3e/d25/d27/fce x:0 0 0 2024-05-25T06:11:44.783 INFO:tasks.workunit.client.0.smithi160.stdout:2/745: symlink d12/d47/da0/dd3/lf7 0 2024-05-25T06:11:44.788 INFO:tasks.workunit.client.1.smithi203.stdout:9/967: truncate d0/f39 1289636 0 2024-05-25T06:11:44.789 INFO:tasks.workunit.client.0.smithi160.stdout:6/688: truncate d5/de/fd 1879052 0 2024-05-25T06:11:44.790 INFO:tasks.workunit.client.0.smithi160.stdout:9/645: dwrite d4/d11/d1e/d3e/d25/d5e/da2/fb0 [0,4194304] 0 2024-05-25T06:11:44.790 INFO:tasks.workunit.client.0.smithi160.stdout:5/787: mkdir db/dc/d13/d1d/d23/d45/df1 0 2024-05-25T06:11:44.790 INFO:tasks.workunit.client.1.smithi203.stdout:9/968: dread d0/fe5 [0,4194304] 0 2024-05-25T06:11:44.793 INFO:tasks.workunit.client.0.smithi160.stdout:4/674: rename d4/d6/d21/f32 to d4/d63/fd3 0 2024-05-25T06:11:44.794 INFO:tasks.workunit.client.1.smithi203.stdout:9/969: symlink d0/d17/d20/d12a/l140 0 2024-05-25T06:11:44.794 INFO:tasks.workunit.client.0.smithi160.stdout:8/730: write d5/d44/d1f/da3/d28/d49/fc3 [8722266,26771] 0 2024-05-25T06:11:44.795 INFO:tasks.workunit.client.0.smithi160.stdout:4/675: read - d4/d6/d21/fd0 zero size 2024-05-25T06:11:44.795 INFO:tasks.workunit.client.0.smithi160.stdout:3/688: link d19/d2f/d38/ccd d19/da6/ce4 0 2024-05-25T06:11:44.797 INFO:tasks.workunit.client.0.smithi160.stdout:4/676: truncate d4/d1a/f50 4443853 0 2024-05-25T06:11:44.797 INFO:tasks.workunit.client.1.smithi203.stdout:9/970: mknod d0/c141 0 2024-05-25T06:11:44.797 INFO:tasks.workunit.client.0.smithi160.stdout:2/746: mknod d12/d1c/cf8 0 2024-05-25T06:11:44.798 INFO:tasks.workunit.client.0.smithi160.stdout:3/689: dread d19/d22/f39 [4194304,4194304] 0 2024-05-25T06:11:44.798 INFO:tasks.workunit.client.0.smithi160.stdout:7/767: creat d6/d2b/d5d/d89/ffb x:0 0 0 2024-05-25T06:11:44.801 INFO:tasks.workunit.client.0.smithi160.stdout:6/689: unlink d5/d45/c70 0 2024-05-25T06:11:44.803 INFO:tasks.workunit.client.1.smithi203.stdout:9/971: rename d0/d17/d20/d29/f8a to d0/df/d37/d4f/d93/ddc/f142 0 2024-05-25T06:11:44.804 INFO:tasks.workunit.client.0.smithi160.stdout:8/731: chown d5/d44/d1f/da3/d28/d49/lb0 59 1 2024-05-25T06:11:44.805 INFO:tasks.workunit.client.0.smithi160.stdout:8/732: chown d5/d44/cae 9 1 2024-05-25T06:11:44.808 INFO:tasks.workunit.client.0.smithi160.stdout:5/788: dwrite db/dc/d13/d1d/fc9 [0,4194304] 0 2024-05-25T06:11:44.809 INFO:tasks.workunit.client.0.smithi160.stdout:1/629: truncate de/d21/d49/d1c/d59/d71/f5c 1254652 0 2024-05-25T06:11:44.810 INFO:tasks.workunit.client.1.smithi203.stdout:9/972: write d0/df/fc1 [901739,91754] 0 2024-05-25T06:11:44.812 INFO:tasks.workunit.client.0.smithi160.stdout:3/690: unlink d19/db2/fc3 0 2024-05-25T06:11:44.813 INFO:tasks.workunit.client.0.smithi160.stdout:7/768: symlink d6/d2b/d5d/lfc 0 2024-05-25T06:11:44.815 INFO:tasks.workunit.client.1.smithi203.stdout:9/973: link d0/c10 d0/d17/d20/d6e/d75/df2/d114/d5c/c143 0 2024-05-25T06:11:44.817 INFO:tasks.workunit.client.1.smithi203.stdout:9/974: stat d0/df/d37/da8/db8/f138 0 2024-05-25T06:11:44.818 INFO:tasks.workunit.client.1.smithi203.stdout:9/975: write d0/d17/d20/d6e/d75/f139 [980917,103182] 0 2024-05-25T06:11:44.819 INFO:tasks.workunit.client.0.smithi160.stdout:4/677: creat d4/d6/dbd/fd4 x:0 0 0 2024-05-25T06:11:44.820 INFO:tasks.workunit.client.0.smithi160.stdout:8/733: dread d5/d44/d1f/fd8 [0,4194304] 0 2024-05-25T06:11:44.821 INFO:tasks.workunit.client.0.smithi160.stdout:5/789: creat db/dc/d13/d1d/d3f/d4a/ff2 x:0 0 0 2024-05-25T06:11:44.823 INFO:tasks.workunit.client.0.smithi160.stdout:5/790: truncate db/dc/fb4 452310 0 2024-05-25T06:11:44.823 INFO:tasks.workunit.client.0.smithi160.stdout:3/691: unlink d19/d2f/d38/d43/d5b/d65/f93 0 2024-05-25T06:11:44.824 INFO:tasks.workunit.client.0.smithi160.stdout:9/646: creat d4/d11/fcf x:0 0 0 2024-05-25T06:11:44.825 INFO:tasks.workunit.client.1.smithi203.stdout:9/976: dwrite d0/df/d37/fb7 [0,4194304] 0 2024-05-25T06:11:44.826 INFO:tasks.workunit.client.0.smithi160.stdout:4/678: symlink d4/d6/d35/d3d/d55/ld5 0 2024-05-25T06:11:44.827 INFO:tasks.workunit.client.1.smithi203.stdout:9/977: fsync d0/df/d37/da8/dd6/ffa 0 2024-05-25T06:11:44.827 INFO:tasks.workunit.client.0.smithi160.stdout:1/630: creat de/d21/d49/dc6/fcf x:0 0 0 2024-05-25T06:11:44.827 INFO:tasks.workunit.client.0.smithi160.stdout:2/747: dwrite d12/d1c/d37/d4b/fd1 [0,4194304] 0 2024-05-25T06:11:44.829 INFO:tasks.workunit.client.0.smithi160.stdout:4/679: write d4/d6/d21/f3f [5237532,7062] 0 2024-05-25T06:11:44.830 INFO:tasks.workunit.client.1.smithi203.stdout:9/978: read d0/d17/d20/d6e/d75/fc0 [1056831,6319] 0 2024-05-25T06:11:44.831 INFO:tasks.workunit.client.0.smithi160.stdout:8/734: creat d5/d9/d72/daf/db9/ff8 x:0 0 0 2024-05-25T06:11:44.831 INFO:tasks.workunit.client.1.smithi203.stdout:9/979: chown d0/df/f4a 101 1 2024-05-25T06:11:44.834 INFO:tasks.workunit.client.0.smithi160.stdout:4/680: truncate d4/d6/f97 53546 0 2024-05-25T06:11:44.835 INFO:tasks.workunit.client.0.smithi160.stdout:4/681: read d4/d1c/f9f [585938,59258] 0 2024-05-25T06:11:44.840 INFO:tasks.workunit.client.0.smithi160.stdout:7/769: rename d6/da8 to d6/d2b/dfd 0 2024-05-25T06:11:44.843 INFO:tasks.workunit.client.0.smithi160.stdout:7/770: dread - d6/d2b/d5d/d89/ffb zero size 2024-05-25T06:11:44.844 INFO:tasks.workunit.client.0.smithi160.stdout:9/647: creat d4/d11/d1e/d3e/d25/d5e/da2/fd0 x:0 0 0 2024-05-25T06:11:44.845 INFO:tasks.workunit.client.0.smithi160.stdout:9/648: write d4/d11/d1e/d6c/f81 [1934859,108700] 0 2024-05-25T06:11:44.846 INFO:tasks.workunit.client.0.smithi160.stdout:6/690: dwrite d5/de/d32/d35/f82 [0,4194304] 0 2024-05-25T06:11:44.848 INFO:tasks.workunit.client.0.smithi160.stdout:2/748: unlink d12/d47/cc7 0 2024-05-25T06:11:44.850 INFO:tasks.workunit.client.0.smithi160.stdout:7/771: dwrite d6/d2b/d5d/fc2 [0,4194304] 0 2024-05-25T06:11:44.851 INFO:tasks.workunit.client.1.smithi203.stdout:9/980: dwrite d0/df/faa [0,4194304] 0 2024-05-25T06:11:44.859 INFO:tasks.workunit.client.0.smithi160.stdout:0/807: sync 2024-05-25T06:11:44.861 INFO:tasks.workunit.client.1.smithi203.stdout:9/981: dwrite d0/d17/d20/d29/ff4 [0,4194304] 0 2024-05-25T06:11:44.862 INFO:tasks.workunit.client.0.smithi160.stdout:4/682: rename d4/d6/f97 to d4/d63/d64/fd6 0 2024-05-25T06:11:44.862 INFO:tasks.workunit.client.0.smithi160.stdout:0/808: chown d2b/d46/dbd/ff1 12092749 1 2024-05-25T06:11:44.864 INFO:tasks.workunit.client.0.smithi160.stdout:3/692: write d19/d2f/d38/d43/f4e [509122,34229] 0 2024-05-25T06:11:44.866 INFO:tasks.workunit.client.0.smithi160.stdout:2/749: mkdir d12/d3e/d5a/d8d/df9 0 2024-05-25T06:11:44.869 INFO:tasks.workunit.client.0.smithi160.stdout:5/791: write db/dc/d13/d1d/d23/d24/daf/f90 [704804,113120] 0 2024-05-25T06:11:44.870 INFO:tasks.workunit.client.0.smithi160.stdout:7/772: symlink d6/dc/d21/lfe 0 2024-05-25T06:11:44.871 INFO:tasks.workunit.client.0.smithi160.stdout:7/773: fdatasync d6/d32/dce/fe3 0 2024-05-25T06:11:44.875 INFO:tasks.workunit.client.0.smithi160.stdout:4/683: chown d4/d6/d5b/d66/d82 36652573 1 2024-05-25T06:11:44.877 INFO:tasks.workunit.client.0.smithi160.stdout:3/693: unlink d19/d2f/d38/d43/d5b/d65/d7f/dd9/lca 0 2024-05-25T06:11:44.878 INFO:tasks.workunit.client.0.smithi160.stdout:3/694: readlink d19/da6/ld8 0 2024-05-25T06:11:44.884 INFO:tasks.workunit.client.0.smithi160.stdout:1/631: getdents de/d39/d8f 0 2024-05-25T06:11:44.886 INFO:tasks.workunit.client.0.smithi160.stdout:4/684: rename d4/d63/d64/fb9 to d4/d1c/d54/d9b/fd7 0 2024-05-25T06:11:44.888 INFO:tasks.workunit.client.0.smithi160.stdout:0/809: link d2b/d46/d63/db3/cdf d2b/d46/d63/da7/d67/d6a/cff 0 2024-05-25T06:11:44.890 INFO:tasks.workunit.client.0.smithi160.stdout:0/810: mknod d2b/d46/d63/da7/da6/dcd/c100 0 2024-05-25T06:11:44.891 INFO:tasks.workunit.client.0.smithi160.stdout:6/691: dread d5/d39/f41 [0,4194304] 0 2024-05-25T06:11:44.892 INFO:tasks.workunit.client.0.smithi160.stdout:0/811: chown d2b/d46/d63/da7/d67/db8/fd4 7586 1 2024-05-25T06:11:44.892 INFO:tasks.workunit.client.0.smithi160.stdout:1/632: dread de/d21/d49/d1c/fa5 [0,4194304] 0 2024-05-25T06:11:44.893 INFO:tasks.workunit.client.1.smithi203.stdout:9/982: write d0/f39 [143596,1876] 0 2024-05-25T06:11:44.893 INFO:tasks.workunit.client.0.smithi160.stdout:0/812: stat d2b/d31/d35/c6f 0 2024-05-25T06:11:44.893 INFO:tasks.workunit.client.0.smithi160.stdout:1/633: dread - de/d21/d49/fce zero size 2024-05-25T06:11:44.894 INFO:tasks.workunit.client.0.smithi160.stdout:3/695: link d19/d2f/l33 d19/le5 0 2024-05-25T06:11:44.896 INFO:tasks.workunit.client.1.smithi203.stdout:9/983: creat d0/df/d37/da8/de3/f144 x:0 0 0 2024-05-25T06:11:44.897 INFO:tasks.workunit.client.1.smithi203.stdout:9/984: read d0/d17/d20/d6e/f10b [163190,116336] 0 2024-05-25T06:11:44.899 INFO:tasks.workunit.client.1.smithi203.stdout:9/985: dread - d0/d17/d20/d6e/d75/df2/d114/f5e zero size 2024-05-25T06:11:44.900 INFO:tasks.workunit.client.0.smithi160.stdout:1/634: write de/d21/d49/d1c/d52/fbe [5135937,105733] 0 2024-05-25T06:11:44.902 INFO:tasks.workunit.client.0.smithi160.stdout:3/696: read d19/d2f/d38/d43/f4e [683809,42654] 0 2024-05-25T06:11:44.902 INFO:tasks.workunit.client.1.smithi203.stdout:9/986: chown d0/d17/d20/d6e/d75/f139 1602 1 2024-05-25T06:11:44.902 INFO:tasks.workunit.client.0.smithi160.stdout:6/692: dwrite d5/de/d7d/fcb [0,4194304] 0 2024-05-25T06:11:44.904 INFO:tasks.workunit.client.0.smithi160.stdout:0/813: dwrite d2b/f30 [0,4194304] 0 2024-05-25T06:11:44.905 INFO:tasks.workunit.client.0.smithi160.stdout:1/635: unlink de/f92 0 2024-05-25T06:11:44.906 INFO:tasks.workunit.client.1.smithi203.stdout:9/987: getdents d0/d17/d20/d6e/d75/df2/d114/db2/d7a 0 2024-05-25T06:11:44.907 INFO:tasks.workunit.client.0.smithi160.stdout:6/693: chown d5/d30/d36/d73/d6e/l8c 3 1 2024-05-25T06:11:44.912 INFO:tasks.workunit.client.0.smithi160.stdout:5/792: dread db/dc/d13/d1d/d31/d56/f61 [0,4194304] 0 2024-05-25T06:11:44.912 INFO:tasks.workunit.client.0.smithi160.stdout:3/697: creat d19/d2f/d38/d43/d5b/d65/d7e/fe6 x:0 0 0 2024-05-25T06:11:44.912 INFO:tasks.workunit.client.0.smithi160.stdout:9/649: truncate d4/d11/d1e/d3e/d2f/f98 3837768 0 2024-05-25T06:11:44.913 INFO:tasks.workunit.client.1.smithi203.stdout:1/976: sync 2024-05-25T06:11:44.913 INFO:tasks.workunit.client.0.smithi160.stdout:7/774: write d6/d3e/d6f/f72 [747235,73716] 0 2024-05-25T06:11:44.914 INFO:tasks.workunit.client.1.smithi203.stdout:1/977: write d14/d57/d97/dec/d131/f123 [4982167,57616] 0 2024-05-25T06:11:44.917 INFO:tasks.workunit.client.0.smithi160.stdout:4/685: write d4/d6/d35/f58 [440211,16201] 0 2024-05-25T06:11:44.918 INFO:tasks.workunit.client.1.smithi203.stdout:9/988: rmdir d0 39 2024-05-25T06:11:44.920 INFO:tasks.workunit.client.0.smithi160.stdout:2/750: write d12/d1c/d2a/f3a [2068761,97916] 0 2024-05-25T06:11:44.920 INFO:tasks.workunit.client.0.smithi160.stdout:0/814: mknod d2b/d46/d63/da7/da6/dcd/dd8/c101 0 2024-05-25T06:11:44.920 INFO:tasks.workunit.client.0.smithi160.stdout:5/793: write db/dc/d13/d1d/d23/d24/daf/db3/fba [821624,119127] 0 2024-05-25T06:11:44.920 INFO:tasks.workunit.client.0.smithi160.stdout:4/686: fdatasync d4/d6/d21/f3f 0 2024-05-25T06:11:44.922 INFO:tasks.workunit.client.0.smithi160.stdout:8/735: dwrite d5/d44/d1f/da3/d28/d49/f97 [0,4194304] 0 2024-05-25T06:11:44.924 INFO:tasks.workunit.client.0.smithi160.stdout:7/775: rename d6/d32/c79 to d6/dc/d64/d5a/d7c/cff 0 2024-05-25T06:11:44.925 INFO:tasks.workunit.client.0.smithi160.stdout:8/736: dread - d5/d7/d9a/fdd zero size 2024-05-25T06:11:44.925 INFO:tasks.workunit.client.0.smithi160.stdout:2/751: fdatasync d12/d1c/f1e 0 2024-05-25T06:11:44.926 INFO:tasks.workunit.client.0.smithi160.stdout:4/687: creat d4/d6/d28/fd8 x:0 0 0 2024-05-25T06:11:44.930 INFO:tasks.workunit.client.0.smithi160.stdout:7/776: truncate d6/d3e/d59/fc8 821704 0 2024-05-25T06:11:44.931 INFO:tasks.workunit.client.0.smithi160.stdout:4/688: write d4/d6/d5b/d66/f74 [4653594,127763] 0 2024-05-25T06:11:44.932 INFO:tasks.workunit.client.0.smithi160.stdout:6/694: rename d5/d30/l69 to d5/d45/d4a/d99/lef 0 2024-05-25T06:11:44.934 INFO:tasks.workunit.client.0.smithi160.stdout:1/636: dread de/f54 [0,4194304] 0 2024-05-25T06:11:44.935 INFO:tasks.workunit.client.0.smithi160.stdout:3/698: dwrite d19/d22/fc2 [0,4194304] 0 2024-05-25T06:11:44.935 INFO:tasks.workunit.client.0.smithi160.stdout:6/695: stat d5/d45/d4a/f90 0 2024-05-25T06:11:44.938 INFO:tasks.workunit.client.1.smithi203.stdout:9/989: write d0/d17/d20/fbc [1193415,119030] 0 2024-05-25T06:11:44.938 INFO:tasks.workunit.client.0.smithi160.stdout:2/752: mkdir d12/d3e/db5/dfa 0 2024-05-25T06:11:44.939 INFO:tasks.workunit.client.0.smithi160.stdout:1/637: write de/d21/d49/f2b [372,103770] 0 2024-05-25T06:11:44.941 INFO:tasks.workunit.client.0.smithi160.stdout:8/737: dwrite d5/d7/dcb/d6a/fcf [0,4194304] 0 2024-05-25T06:11:44.942 INFO:tasks.workunit.client.0.smithi160.stdout:0/815: link d2b/d31/d35/d43/d98/ff2 d2b/d46/d63/da7/db5/f102 0 2024-05-25T06:11:44.942 INFO:tasks.workunit.client.0.smithi160.stdout:1/638: fsync de/d21/d49/d28/f5b 0 2024-05-25T06:11:44.945 INFO:tasks.workunit.client.1.smithi203.stdout:1/978: dwrite d14/d57/d6b/da1/de2/fb1 [0,4194304] 0 2024-05-25T06:11:44.946 INFO:tasks.workunit.client.1.smithi203.stdout:9/990: rename d0/df/d37/da8/db8 to d0/d74/d145 0 2024-05-25T06:11:44.948 INFO:tasks.workunit.client.1.smithi203.stdout:1/979: symlink d14/d23/l151 0 2024-05-25T06:11:44.949 INFO:tasks.workunit.client.0.smithi160.stdout:6/696: fdatasync d5/de/d32/d74/d77/fb1 0 2024-05-25T06:11:44.950 INFO:tasks.workunit.client.0.smithi160.stdout:8/738: creat d5/d9c/ff9 x:0 0 0 2024-05-25T06:11:44.952 INFO:tasks.workunit.client.1.smithi203.stdout:9/991: symlink d0/d17/d20/d6e/l146 0 2024-05-25T06:11:44.952 INFO:tasks.workunit.client.0.smithi160.stdout:4/689: mknod d4/d6/d5b/d66/d78/cd9 0 2024-05-25T06:11:44.953 INFO:tasks.workunit.client.0.smithi160.stdout:0/816: mkdir d2b/d31/d35/d36/d86/d103 0 2024-05-25T06:11:44.954 INFO:tasks.workunit.client.0.smithi160.stdout:0/817: write d2b/f30 [5270791,73278] 0 2024-05-25T06:11:44.955 INFO:tasks.workunit.client.1.smithi203.stdout:9/992: mknod d0/d17/d20/d6e/d75/c147 0 2024-05-25T06:11:44.956 INFO:tasks.workunit.client.0.smithi160.stdout:0/818: write d2b/d46/d63/fb1 [2721665,52336] 0 2024-05-25T06:11:44.956 INFO:tasks.workunit.client.1.smithi203.stdout:9/993: chown d0/f6f 941323956 1 2024-05-25T06:11:44.957 INFO:tasks.workunit.client.0.smithi160.stdout:3/699: rename d19/da6/d6a to d19/da7/de7 0 2024-05-25T06:11:44.958 INFO:tasks.workunit.client.1.smithi203.stdout:1/980: dwrite d14/d57/d6b/da1/de2/d143/dd2/fce [0,4194304] 0 2024-05-25T06:11:44.960 INFO:tasks.workunit.client.1.smithi203.stdout:9/994: symlink d0/df/d37/l148 0 2024-05-25T06:11:44.961 INFO:tasks.workunit.client.1.smithi203.stdout:1/981: chown d14/d57/d6b/da1/de2/d143/dd2 56434155 1 2024-05-25T06:11:44.961 INFO:tasks.workunit.client.0.smithi160.stdout:6/697: mknod d5/de/d9e/cf0 0 2024-05-25T06:11:44.962 INFO:tasks.workunit.client.0.smithi160.stdout:8/739: mknod d5/d44/d1f/d58/cfa 0 2024-05-25T06:11:44.964 INFO:tasks.workunit.client.1.smithi203.stdout:1/982: creat d14/d57/d1d/da8/f152 x:0 0 0 2024-05-25T06:11:44.964 INFO:tasks.workunit.client.0.smithi160.stdout:0/819: mknod d2b/d31/d35/d36/daf/c104 0 2024-05-25T06:11:44.965 INFO:tasks.workunit.client.0.smithi160.stdout:0/820: stat d2b/d46/d63/db3/lb4 0 2024-05-25T06:11:44.965 INFO:tasks.workunit.client.1.smithi203.stdout:9/995: symlink d0/d17/d20/d29/l149 0 2024-05-25T06:11:44.967 INFO:tasks.workunit.client.1.smithi203.stdout:1/983: truncate d14/dff/db8/fb0 1468362 0 2024-05-25T06:11:44.969 INFO:tasks.workunit.client.1.smithi203.stdout:9/996: symlink d0/df/d37/d4f/l14a 0 2024-05-25T06:11:44.970 INFO:tasks.workunit.client.0.smithi160.stdout:4/690: rename d4/d63/d70 to d4/d1c/d54/dbc/dda 0 2024-05-25T06:11:44.971 INFO:tasks.workunit.client.0.smithi160.stdout:4/691: write d4/d1a/fa7 [3013718,66501] 0 2024-05-25T06:11:44.975 INFO:tasks.workunit.client.0.smithi160.stdout:2/753: creat d12/d3e/d5a/d8d/db4/ffb x:0 0 0 2024-05-25T06:11:44.976 INFO:tasks.workunit.client.1.smithi203.stdout:1/984: getdents d14/dff 0 2024-05-25T06:11:44.977 INFO:tasks.workunit.client.0.smithi160.stdout:1/639: dread de/d21/d49/d28/f3c [0,4194304] 0 2024-05-25T06:11:44.978 INFO:tasks.workunit.client.0.smithi160.stdout:6/698: dread d5/d30/d36/d73/d6e/f96 [0,4194304] 0 2024-05-25T06:11:44.982 INFO:tasks.workunit.client.0.smithi160.stdout:4/692: creat d4/d6/d5b/d66/d82/fdb x:0 0 0 2024-05-25T06:11:44.982 INFO:tasks.workunit.client.0.smithi160.stdout:8/740: read d5/d44/d1f/d58/f60 [835882,7673] 0 2024-05-25T06:11:44.982 INFO:tasks.workunit.client.1.smithi203.stdout:1/985: link d14/d23/lc7 d14/d57/d6b/da1/de2/d143/dd2/dca/dcf/df7/l153 0 2024-05-25T06:11:44.983 INFO:tasks.workunit.client.0.smithi160.stdout:2/754: mknod d12/d1c/d37/cfc 0 2024-05-25T06:11:44.986 INFO:tasks.workunit.client.0.smithi160.stdout:2/755: read - d12/d3e/db5/fc3 zero size 2024-05-25T06:11:44.986 INFO:tasks.workunit.client.0.smithi160.stdout:7/777: dread d6/d32/f80 [0,4194304] 0 2024-05-25T06:11:44.986 INFO:tasks.workunit.client.0.smithi160.stdout:1/640: dwrite de/d21/d49/f82 [0,4194304] 0 2024-05-25T06:11:44.986 INFO:tasks.workunit.client.0.smithi160.stdout:6/699: dread d5/d39/f41 [0,4194304] 0 2024-05-25T06:11:44.987 INFO:tasks.workunit.client.0.smithi160.stdout:2/756: readlink d12/d5c/d85/lde 0 2024-05-25T06:11:44.988 INFO:tasks.workunit.client.0.smithi160.stdout:2/757: chown d12/d97/df4 251095191 1 2024-05-25T06:11:44.988 INFO:tasks.workunit.client.0.smithi160.stdout:1/641: dread - de/d21/d49/d1c/d52/d77/fb0 zero size 2024-05-25T06:11:44.990 INFO:tasks.workunit.client.0.smithi160.stdout:3/700: dread d19/f45 [0,4194304] 0 2024-05-25T06:11:44.990 INFO:tasks.workunit.client.0.smithi160.stdout:8/741: mknod d5/d44/d1f/cfb 0 2024-05-25T06:11:44.993 INFO:tasks.workunit.client.0.smithi160.stdout:7/778: fsync d6/d2b/f51 0 2024-05-25T06:11:44.996 INFO:tasks.workunit.client.0.smithi160.stdout:5/794: truncate db/dc/d13/d1d/d3f/d72/dc5/fdc 646461 0 2024-05-25T06:11:44.996 INFO:tasks.workunit.client.0.smithi160.stdout:6/700: dwrite d5/d30/f91 [0,4194304] 0 2024-05-25T06:11:44.998 INFO:tasks.workunit.client.0.smithi160.stdout:5/795: write db/dc/d13/d1d/d31/d56/f92 [330739,3212] 0 2024-05-25T06:11:44.999 INFO:tasks.workunit.client.0.smithi160.stdout:3/701: mkdir d19/d2f/d38/d43/d5b/d65/d7e/de8 0 2024-05-25T06:11:45.004 INFO:tasks.workunit.client.0.smithi160.stdout:6/701: read d5/de/f12 [3704399,53645] 0 2024-05-25T06:11:45.006 INFO:tasks.workunit.client.0.smithi160.stdout:5/796: write db/dc/d13/d1d/d23/d45/fe3 [372090,116300] 0 2024-05-25T06:11:45.007 INFO:tasks.workunit.client.0.smithi160.stdout:7/779: symlink d6/dc/d64/d5a/dc4/l100 0 2024-05-25T06:11:45.008 INFO:tasks.workunit.client.0.smithi160.stdout:3/702: creat d19/d2f/d38/d43/d5b/d65/d7e/fe9 x:0 0 0 2024-05-25T06:11:45.011 INFO:tasks.workunit.client.0.smithi160.stdout:1/642: getdents de/d21/d49/d1c/d52 0 2024-05-25T06:11:45.015 INFO:tasks.workunit.client.0.smithi160.stdout:6/702: dread d5/d30/d36/d73/d6e/f84 [0,4194304] 0 2024-05-25T06:11:45.016 INFO:tasks.workunit.client.0.smithi160.stdout:6/703: chown d5/d45/d4a/f5d 1 1 2024-05-25T06:11:45.016 INFO:tasks.workunit.client.0.smithi160.stdout:0/821: write d2b/d46/f5a [1226696,92826] 0 2024-05-25T06:11:45.016 INFO:tasks.workunit.client.0.smithi160.stdout:7/780: rmdir d6/d2b/da0/dc7 39 2024-05-25T06:11:45.018 INFO:tasks.workunit.client.0.smithi160.stdout:6/704: write d5/d39/fec [73169,115400] 0 2024-05-25T06:11:45.018 INFO:tasks.workunit.client.0.smithi160.stdout:0/822: dread - d2b/d31/d6d/fbf zero size 2024-05-25T06:11:45.020 INFO:tasks.workunit.client.0.smithi160.stdout:1/643: rename de/d21/d49/d1c/d52/d77 to de/d21/d49/d1c/d59/d7f/da1/dbd/dd0 0 2024-05-25T06:11:45.021 INFO:tasks.workunit.client.0.smithi160.stdout:8/742: dread d5/d44/d1f/da3/d28/f3f [0,4194304] 0 2024-05-25T06:11:45.023 INFO:tasks.workunit.client.0.smithi160.stdout:4/693: write d4/d63/d64/d88/fca [228235,20939] 0 2024-05-25T06:11:45.024 INFO:tasks.workunit.client.0.smithi160.stdout:0/823: creat d2b/d31/d35/d43/d98/f105 x:0 0 0 2024-05-25T06:11:45.025 INFO:tasks.workunit.client.0.smithi160.stdout:0/824: fdatasync d2b/d46/d63/da7/d67/fa5 0 2024-05-25T06:11:45.025 INFO:tasks.workunit.client.0.smithi160.stdout:1/644: mknod de/d21/d49/d1c/d59/cd1 0 2024-05-25T06:11:45.029 INFO:tasks.workunit.client.0.smithi160.stdout:6/705: symlink d5/de/lf1 0 2024-05-25T06:11:45.035 INFO:tasks.workunit.client.0.smithi160.stdout:1/645: symlink de/d21/d49/d1c/d41/d45/d7d/db3/ld2 0 2024-05-25T06:11:45.037 INFO:tasks.workunit.client.0.smithi160.stdout:5/797: dwrite db/d47/d5e/f99 [0,4194304] 0 2024-05-25T06:11:45.040 INFO:tasks.workunit.client.0.smithi160.stdout:3/703: write d19/d2f/d38/d43/d5b/d65/fb9 [90179,105549] 0 2024-05-25T06:11:45.045 INFO:tasks.workunit.client.0.smithi160.stdout:6/706: dwrite d5/d39/d8f/f55 [0,4194304] 0 2024-05-25T06:11:45.046 INFO:tasks.workunit.client.0.smithi160.stdout:3/704: creat d19/db2/fea x:0 0 0 2024-05-25T06:11:45.047 INFO:tasks.workunit.client.0.smithi160.stdout:7/781: truncate d6/d32/d5b/fbe 1823266 0 2024-05-25T06:11:45.049 INFO:tasks.workunit.client.0.smithi160.stdout:8/743: truncate d5/d44/d1f/da3/d28/de6/fd3 2689086 0 2024-05-25T06:11:45.050 INFO:tasks.workunit.client.0.smithi160.stdout:8/744: chown d5/d9/d72/dd9/l8c 9424712 1 2024-05-25T06:11:45.050 INFO:tasks.workunit.client.0.smithi160.stdout:5/798: symlink db/d47/lf3 0 2024-05-25T06:11:45.051 INFO:tasks.workunit.client.0.smithi160.stdout:4/694: getdents d4/d1c/d54/dbc/dda 0 2024-05-25T06:11:45.052 INFO:tasks.workunit.client.0.smithi160.stdout:6/707: read d5/de/d32/d35/f82 [581050,88835] 0 2024-05-25T06:11:45.052 INFO:tasks.workunit.client.0.smithi160.stdout:3/705: rename d19/da6/c90 to d19/d2f/d34/d67/ceb 0 2024-05-25T06:11:45.053 INFO:tasks.workunit.client.0.smithi160.stdout:4/695: chown d4/d6/d35/d3d 401 1 2024-05-25T06:11:45.054 INFO:tasks.workunit.client.0.smithi160.stdout:3/706: chown d19/d2f/d38/d43/d5b/d65/d7e/fe9 4848475 1 2024-05-25T06:11:45.054 INFO:tasks.workunit.client.0.smithi160.stdout:8/745: creat d5/d9/d72/daf/ffc x:0 0 0 2024-05-25T06:11:45.055 INFO:tasks.workunit.client.0.smithi160.stdout:4/696: write d4/d1a/f87 [2735329,122927] 0 2024-05-25T06:11:45.056 INFO:tasks.workunit.client.0.smithi160.stdout:1/646: getdents de/d21/d49/d1c/d59/d7f/da1 0 2024-05-25T06:11:45.058 INFO:tasks.workunit.client.0.smithi160.stdout:4/697: truncate d4/d6/d28/f25 4205304 0 2024-05-25T06:11:45.061 INFO:tasks.workunit.client.0.smithi160.stdout:4/698: write d4/d63/d64/d88/fa5 [1435643,129832] 0 2024-05-25T06:11:45.065 INFO:tasks.workunit.client.0.smithi160.stdout:0/825: dread d2b/d46/d63/fb1 [0,4194304] 0 2024-05-25T06:11:45.065 INFO:tasks.workunit.client.0.smithi160.stdout:5/799: dread db/dc/d13/d1d/d3f/d72/f7f [0,4194304] 0 2024-05-25T06:11:45.066 INFO:tasks.workunit.client.0.smithi160.stdout:8/746: dwrite d5/fc5 [0,4194304] 0 2024-05-25T06:11:45.067 INFO:tasks.workunit.client.0.smithi160.stdout:5/800: stat db/dc/d13/d1d/d23/f33 0 2024-05-25T06:11:45.067 INFO:tasks.workunit.client.0.smithi160.stdout:1/647: dwrite f5 [0,4194304] 0 2024-05-25T06:11:45.071 INFO:tasks.workunit.client.0.smithi160.stdout:7/782: link d6/d3e/d6f/d73/fc5 d6/d2b/da0/f101 0 2024-05-25T06:11:45.075 INFO:tasks.workunit.client.0.smithi160.stdout:0/826: mknod d2b/d46/d63/da7/d67/d6a/c106 0 2024-05-25T06:11:45.076 INFO:tasks.workunit.client.0.smithi160.stdout:8/747: fdatasync d5/d44/d1f/f8f 0 2024-05-25T06:11:45.076 INFO:tasks.workunit.client.0.smithi160.stdout:3/707: write d19/d2f/d38/d60/fc4 [142696,13169] 0 2024-05-25T06:11:45.079 INFO:tasks.workunit.client.0.smithi160.stdout:3/708: dread - d19/d22/f2e zero size 2024-05-25T06:11:45.080 INFO:tasks.workunit.client.0.smithi160.stdout:6/708: dwrite d5/d30/d36/f49 [0,4194304] 0 2024-05-25T06:11:45.081 INFO:tasks.workunit.client.0.smithi160.stdout:5/801: dwrite db/dc/d13/d1d/d23/d24/daf/f90 [0,4194304] 0 2024-05-25T06:11:45.082 INFO:tasks.workunit.client.0.smithi160.stdout:7/783: creat d6/d3e/d6f/f102 x:0 0 0 2024-05-25T06:11:45.087 INFO:tasks.workunit.client.0.smithi160.stdout:4/699: write d4/d6/fe [2477662,95883] 0 2024-05-25T06:11:45.089 INFO:tasks.workunit.client.0.smithi160.stdout:6/709: chown d5/de/d18 39 1 2024-05-25T06:11:45.090 INFO:tasks.workunit.client.0.smithi160.stdout:3/709: creat d19/d2f/d34/d67/d96/fec x:0 0 0 2024-05-25T06:11:45.090 INFO:tasks.workunit.client.0.smithi160.stdout:8/748: truncate d5/d7/dcb/d7a/f85 704594 0 2024-05-25T06:11:45.094 INFO:tasks.workunit.client.0.smithi160.stdout:4/700: rename d4/d6/d35/d3d/c51 to d4/d6/d5b/cdc 0 2024-05-25T06:11:45.095 INFO:tasks.workunit.client.0.smithi160.stdout:6/710: dwrite d5/d45/f8e [0,4194304] 0 2024-05-25T06:11:45.099 INFO:tasks.workunit.client.0.smithi160.stdout:8/749: mknod d5/d7/dcb/cfd 0 2024-05-25T06:11:45.100 INFO:tasks.workunit.client.0.smithi160.stdout:3/710: mkdir d19/d2f/d38/d43/d5b/d65/d7f/dd9/ded 0 2024-05-25T06:11:45.101 INFO:tasks.workunit.client.0.smithi160.stdout:0/827: creat d2b/d31/d35/d36/f107 x:0 0 0 2024-05-25T06:11:45.101 INFO:tasks.workunit.client.0.smithi160.stdout:4/701: creat d4/d1a/fdd x:0 0 0 2024-05-25T06:11:45.102 INFO:tasks.workunit.client.0.smithi160.stdout:6/711: truncate d5/d45/d63/f83 387471 0 2024-05-25T06:11:45.103 INFO:tasks.workunit.client.0.smithi160.stdout:5/802: link db/dc/f83 db/dc/d13/d9a/ff4 0 2024-05-25T06:11:45.104 INFO:tasks.workunit.client.0.smithi160.stdout:5/803: stat db/ce7 0 2024-05-25T06:11:45.104 INFO:tasks.workunit.client.0.smithi160.stdout:0/828: chown d2b/d31/d35/d43/fc4 1604 1 2024-05-25T06:11:45.104 INFO:tasks.workunit.client.0.smithi160.stdout:3/711: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee 0 2024-05-25T06:11:45.106 INFO:tasks.workunit.client.0.smithi160.stdout:8/750: mkdir d5/d44/d1f/da3/d28/dc6/de8/dfe 0 2024-05-25T06:11:45.108 INFO:tasks.workunit.client.0.smithi160.stdout:1/648: dwrite de/d21/d49/d1c/d59/d7f/da1/dbd/f7b [0,4194304] 0 2024-05-25T06:11:45.108 INFO:tasks.workunit.client.0.smithi160.stdout:0/829: dread - d2b/d46/d63/da7/db5/f102 zero size 2024-05-25T06:11:45.108 INFO:tasks.workunit.client.0.smithi160.stdout:5/804: write db/dc/d13/d1d/fbb [956217,104820] 0 2024-05-25T06:11:45.109 INFO:tasks.workunit.client.0.smithi160.stdout:3/712: mkdir d19/d2f/d34/def 0 2024-05-25T06:11:45.109 INFO:tasks.workunit.client.0.smithi160.stdout:6/712: chown d5/de/l27 362137135 1 2024-05-25T06:11:45.110 INFO:tasks.workunit.client.0.smithi160.stdout:8/751: read d5/d44/d1f/da3/d28/d4a/f5c [3262635,3236] 0 2024-05-25T06:11:45.113 INFO:tasks.workunit.client.0.smithi160.stdout:4/702: dwrite d4/f4f [0,4194304] 0 2024-05-25T06:11:45.116 INFO:tasks.workunit.client.0.smithi160.stdout:0/830: mkdir d2b/d31/d35/d36/d86/dd5/d108 0 2024-05-25T06:11:45.116 INFO:tasks.workunit.client.0.smithi160.stdout:7/784: write d6/dc/d64/f43 [1539407,95825] 0 2024-05-25T06:11:45.117 INFO:tasks.workunit.client.0.smithi160.stdout:1/649: mkdir de/d21/d49/d1c/d41/d45/dac/dd3 0 2024-05-25T06:11:45.119 INFO:tasks.workunit.client.0.smithi160.stdout:5/805: dwrite f9 [0,4194304] 0 2024-05-25T06:11:45.120 INFO:tasks.workunit.client.0.smithi160.stdout:6/713: creat d5/d39/d8f/d2b/ff2 x:0 0 0 2024-05-25T06:11:45.122 INFO:tasks.workunit.client.0.smithi160.stdout:0/831: read d2b/d46/d63/da7/d67/fa5 [3197386,122243] 0 2024-05-25T06:11:45.123 INFO:tasks.workunit.client.0.smithi160.stdout:6/714: stat d5/d45/d4a/le4 0 2024-05-25T06:11:45.128 INFO:tasks.workunit.client.0.smithi160.stdout:2/758: sync 2024-05-25T06:11:45.128 INFO:tasks.workunit.client.0.smithi160.stdout:9/650: sync 2024-05-25T06:11:45.128 INFO:tasks.workunit.client.0.smithi160.stdout:0/832: write f20 [794966,103036] 0 2024-05-25T06:11:45.133 INFO:tasks.workunit.client.0.smithi160.stdout:4/703: getdents d4/dcd 0 2024-05-25T06:11:45.135 INFO:tasks.workunit.client.0.smithi160.stdout:3/713: dread d19/d2f/d38/d47/f54 [0,4194304] 0 2024-05-25T06:11:45.135 INFO:tasks.workunit.client.0.smithi160.stdout:2/759: chown fe 215 1 2024-05-25T06:11:45.137 INFO:tasks.workunit.client.0.smithi160.stdout:0/833: creat d2b/d46/d63/da7/da6/f109 x:0 0 0 2024-05-25T06:11:45.137 INFO:tasks.workunit.client.0.smithi160.stdout:1/650: dwrite de/d21/d49/d1c/d59/f62 [0,4194304] 0 2024-05-25T06:11:45.142 INFO:tasks.workunit.client.0.smithi160.stdout:5/806: link db/dc/d13/d1d/d3f/c64 db/dc/d13/d1d/d23/cf5 0 2024-05-25T06:11:45.143 INFO:tasks.workunit.client.0.smithi160.stdout:2/760: creat d12/d3e/d5f/d74/ffd x:0 0 0 2024-05-25T06:11:45.143 INFO:tasks.workunit.client.0.smithi160.stdout:1/651: dread - de/d21/f7a zero size 2024-05-25T06:11:45.144 INFO:tasks.workunit.client.0.smithi160.stdout:0/834: mknod d2b/d31/d35/d43/dab/c10a 0 2024-05-25T06:11:45.145 INFO:tasks.workunit.client.0.smithi160.stdout:9/651: dwrite d4/d11/d1e/d24/d66/f94 [4194304,4194304] 0 2024-05-25T06:11:45.148 INFO:tasks.workunit.client.0.smithi160.stdout:4/704: link d4/d6/d21/f3f d4/d63/d64/d88/fde 0 2024-05-25T06:11:45.149 INFO:tasks.workunit.client.0.smithi160.stdout:4/705: chown d4/d6/d5b/d66/f71 6662 1 2024-05-25T06:11:45.151 INFO:tasks.workunit.client.0.smithi160.stdout:3/714: dwrite d19/d2f/d38/d43/d5b/d65/d7f/f81 [0,4194304] 0 2024-05-25T06:11:45.151 INFO:tasks.workunit.client.0.smithi160.stdout:1/652: creat de/d21/d49/d1c/d52/fd4 x:0 0 0 2024-05-25T06:11:45.152 INFO:tasks.workunit.client.0.smithi160.stdout:8/752: dread d5/d44/f68 [0,4194304] 0 2024-05-25T06:11:45.152 INFO:tasks.workunit.client.0.smithi160.stdout:4/706: write d4/d6/d5b/d66/d82/f84 [1283790,68620] 0 2024-05-25T06:11:45.154 INFO:tasks.workunit.client.0.smithi160.stdout:6/715: write d5/f3f [3633489,25037] 0 2024-05-25T06:11:45.156 INFO:tasks.workunit.client.0.smithi160.stdout:5/807: creat db/ff6 x:0 0 0 2024-05-25T06:11:45.157 INFO:tasks.workunit.client.0.smithi160.stdout:1/653: truncate de/d21/d49/d1c/fc9 1010973 0 2024-05-25T06:11:45.157 INFO:tasks.workunit.client.0.smithi160.stdout:7/785: truncate d6/d2b/d5d/d89/fa9 2914577 0 2024-05-25T06:11:45.160 INFO:tasks.workunit.client.0.smithi160.stdout:2/761: rename d12/d1c/d37/d4b/f5e to d12/d1c/d59/ffe 0 2024-05-25T06:11:45.162 INFO:tasks.workunit.client.0.smithi160.stdout:3/715: dread - d19/d2f/d38/d43/f66 zero size 2024-05-25T06:11:45.165 INFO:tasks.workunit.client.0.smithi160.stdout:6/716: symlink d5/d45/lf3 0 2024-05-25T06:11:45.166 INFO:tasks.workunit.client.0.smithi160.stdout:8/753: symlink d5/d7/lff 0 2024-05-25T06:11:45.166 INFO:tasks.workunit.client.0.smithi160.stdout:0/835: write d2b/d46/faa [1306571,52671] 0 2024-05-25T06:11:45.168 INFO:tasks.workunit.client.0.smithi160.stdout:5/808: rmdir db/dc/d13/d1d/d23 39 2024-05-25T06:11:45.170 INFO:tasks.workunit.client.0.smithi160.stdout:1/654: mknod de/d21/cd5 0 2024-05-25T06:11:45.170 INFO:tasks.workunit.client.0.smithi160.stdout:3/716: dwrite d19/d2f/d38/d43/d5b/d65/d7f/dd9/fc8 [0,4194304] 0 2024-05-25T06:11:45.172 INFO:tasks.workunit.client.0.smithi160.stdout:7/786: symlink d6/d2b/dfd/l103 0 2024-05-25T06:11:45.172 INFO:tasks.workunit.client.0.smithi160.stdout:2/762: unlink d12/d97/feb 0 2024-05-25T06:11:45.174 INFO:tasks.workunit.client.0.smithi160.stdout:9/652: creat d4/d11/d1e/d24/fd1 x:0 0 0 2024-05-25T06:11:45.175 INFO:tasks.workunit.client.0.smithi160.stdout:6/717: symlink d5/de/d32/d74/lf4 0 2024-05-25T06:11:45.176 INFO:tasks.workunit.client.0.smithi160.stdout:6/718: chown d5/d30/d36/d73/d87/da4/ddf 20774 1 2024-05-25T06:11:45.182 INFO:tasks.workunit.client.0.smithi160.stdout:8/754: symlink d5/d9/d72/daf/db9/l100 0 2024-05-25T06:11:45.183 INFO:tasks.workunit.client.0.smithi160.stdout:8/755: chown d5/d44/d1f/da3/d28/d49 3 1 2024-05-25T06:11:45.186 INFO:tasks.workunit.client.0.smithi160.stdout:3/717: mkdir d19/d2f/d38/d43/d5b/d65/d83/df0 0 2024-05-25T06:11:45.187 INFO:tasks.workunit.client.0.smithi160.stdout:3/718: chown d19/d2f/d38/d47/c75 22 1 2024-05-25T06:11:45.188 INFO:tasks.workunit.client.0.smithi160.stdout:7/787: mknod d6/dc/d64/c104 0 2024-05-25T06:11:45.189 INFO:tasks.workunit.client.0.smithi160.stdout:9/653: symlink d4/d11/d12/ld2 0 2024-05-25T06:11:45.190 INFO:tasks.workunit.client.0.smithi160.stdout:6/719: creat d5/d39/ff5 x:0 0 0 2024-05-25T06:11:45.191 INFO:tasks.workunit.client.0.smithi160.stdout:7/788: dread d6/dc/f82 [0,4194304] 0 2024-05-25T06:11:45.192 INFO:tasks.workunit.client.0.smithi160.stdout:6/720: stat d5/de/d32/l24 0 2024-05-25T06:11:45.193 INFO:tasks.workunit.client.0.smithi160.stdout:4/707: dread d4/d1a/f40 [0,4194304] 0 2024-05-25T06:11:45.194 INFO:tasks.workunit.client.0.smithi160.stdout:3/719: creat d19/d2f/d38/d60/ff1 x:0 0 0 2024-05-25T06:11:45.198 INFO:tasks.workunit.client.0.smithi160.stdout:4/708: creat d4/d6/d35/d3d/d55/fdf x:0 0 0 2024-05-25T06:11:45.199 INFO:tasks.workunit.client.0.smithi160.stdout:0/836: write d2b/d46/d63/db3/fee [708581,26975] 0 2024-05-25T06:11:45.201 INFO:tasks.workunit.client.0.smithi160.stdout:1/655: getdents de/d21/d49/d1c/d52/dc7 0 2024-05-25T06:11:45.204 INFO:tasks.workunit.client.0.smithi160.stdout:1/656: creat de/d21/d49/d35/fd6 x:0 0 0 2024-05-25T06:11:45.205 INFO:tasks.workunit.client.0.smithi160.stdout:0/837: symlink d2b/d31/d35/d7b/l10b 0 2024-05-25T06:11:45.206 INFO:tasks.workunit.client.0.smithi160.stdout:1/657: write de/d21/d49/d35/f6e [4967632,75038] 0 2024-05-25T06:11:45.208 INFO:tasks.workunit.client.0.smithi160.stdout:5/809: write db/dc/d13/d1d/d3f/d72/dc5/fdc [768946,60325] 0 2024-05-25T06:11:45.208 INFO:tasks.workunit.client.0.smithi160.stdout:0/838: rmdir d2b/d31/d35/d36/daf 39 2024-05-25T06:11:45.215 INFO:tasks.workunit.client.0.smithi160.stdout:0/839: rename d2b/d31/d35/d43/d98/cf4 to d2b/d46/d63/da7/db5/c10c 0 2024-05-25T06:11:45.219 INFO:tasks.workunit.client.0.smithi160.stdout:5/810: dwrite f6 [4194304,4194304] 0 2024-05-25T06:11:45.220 INFO:tasks.workunit.client.0.smithi160.stdout:6/721: dwrite d5/d39/f51 [4194304,4194304] 0 2024-05-25T06:11:45.222 INFO:tasks.workunit.client.0.smithi160.stdout:9/654: dwrite d4/d11/f67 [0,4194304] 0 2024-05-25T06:11:45.222 INFO:tasks.workunit.client.0.smithi160.stdout:0/840: mknod d2b/d46/d63/da7/db5/c10d 0 2024-05-25T06:11:45.224 INFO:tasks.workunit.client.0.smithi160.stdout:6/722: stat d5/d45/d4a/f76 0 2024-05-25T06:11:45.226 INFO:tasks.workunit.client.0.smithi160.stdout:4/709: dread d4/d6/fe [0,4194304] 0 2024-05-25T06:11:45.227 INFO:tasks.workunit.client.0.smithi160.stdout:9/655: write d4/d11/d1e/d24/fc [4548618,76014] 0 2024-05-25T06:11:45.228 INFO:tasks.workunit.client.0.smithi160.stdout:5/811: mknod db/d47/cf7 0 2024-05-25T06:11:45.229 INFO:tasks.workunit.client.0.smithi160.stdout:9/656: readlink d4/d11/d1e/d6c/l71 0 2024-05-25T06:11:45.230 INFO:tasks.workunit.client.0.smithi160.stdout:0/841: fdatasync d2b/d31/d35/d7b/fa0 0 2024-05-25T06:11:45.231 INFO:tasks.workunit.client.0.smithi160.stdout:9/657: write d4/d11/d1e/d3e/d2f/f9d [871871,73420] 0 2024-05-25T06:11:45.232 INFO:tasks.workunit.client.0.smithi160.stdout:6/723: creat d5/d30/d36/d73/d87/ff6 x:0 0 0 2024-05-25T06:11:45.232 INFO:tasks.workunit.client.0.smithi160.stdout:4/710: symlink d4/d6/d21/d42/dae/le0 0 2024-05-25T06:11:45.233 INFO:tasks.workunit.client.0.smithi160.stdout:5/812: symlink db/d47/d5e/d6e/lf8 0 2024-05-25T06:11:45.235 INFO:tasks.workunit.client.0.smithi160.stdout:9/658: fdatasync d4/d11/d1e/d3e/d25/fb3 0 2024-05-25T06:11:45.237 INFO:tasks.workunit.client.0.smithi160.stdout:4/711: unlink d4/d1a/f40 0 2024-05-25T06:11:45.239 INFO:tasks.workunit.client.0.smithi160.stdout:5/813: creat db/dc/d13/d1d/ff9 x:0 0 0 2024-05-25T06:11:45.240 INFO:tasks.workunit.client.0.smithi160.stdout:4/712: truncate d4/d6/d35/f58 4264500 0 2024-05-25T06:11:45.240 INFO:tasks.workunit.client.0.smithi160.stdout:9/659: creat d4/d11/d1e/fd3 x:0 0 0 2024-05-25T06:11:45.240 INFO:tasks.workunit.client.0.smithi160.stdout:1/658: dread de/d21/d49/d35/f73 [0,4194304] 0 2024-05-25T06:11:45.243 INFO:tasks.workunit.client.0.smithi160.stdout:6/724: dwrite d5/de/d9e/fa2 [0,4194304] 0 2024-05-25T06:11:45.243 INFO:tasks.workunit.client.0.smithi160.stdout:4/713: truncate d4/d6/d5b/d66/fc4 524301 0 2024-05-25T06:11:45.244 INFO:tasks.workunit.client.0.smithi160.stdout:5/814: mknod db/dc/d13/d1d/d23/d24/daf/cfa 0 2024-05-25T06:11:45.247 INFO:tasks.workunit.client.0.smithi160.stdout:9/660: symlink d4/d11/d1e/d3e/d25/d27/dc5/ld4 0 2024-05-25T06:11:45.248 INFO:tasks.workunit.client.0.smithi160.stdout:6/725: chown d5/de/d7d/f9b 50 1 2024-05-25T06:11:45.248 INFO:tasks.workunit.client.0.smithi160.stdout:5/815: write db/dc/d13/d1d/d31/d56/f7d [3826456,75919] 0 2024-05-25T06:11:45.250 INFO:tasks.workunit.client.0.smithi160.stdout:4/714: rename d4/d6/d5b/d66/d78/l89 to d4/d1c/d54/dbc/dda/le1 0 2024-05-25T06:11:45.251 INFO:tasks.workunit.client.0.smithi160.stdout:9/661: write d4/d11/fcf [48475,54164] 0 2024-05-25T06:11:45.252 INFO:tasks.workunit.client.0.smithi160.stdout:5/816: write db/dc/fd7 [721756,60622] 0 2024-05-25T06:11:45.254 INFO:tasks.workunit.client.0.smithi160.stdout:6/726: mkdir d5/d30/d36/d73/d6e/df7 0 2024-05-25T06:11:45.255 INFO:tasks.workunit.client.0.smithi160.stdout:1/659: dwrite de/d21/d49/d1c/d41/d96/fa8 [0,4194304] 0 2024-05-25T06:11:45.265 INFO:tasks.workunit.client.0.smithi160.stdout:4/715: link d4/d6/d7b/lc6 d4/d6/d5b/d66/d82/le2 0 2024-05-25T06:11:45.269 INFO:tasks.workunit.client.0.smithi160.stdout:5/817: truncate f1 1356640 0 2024-05-25T06:11:45.270 INFO:tasks.workunit.client.0.smithi160.stdout:6/727: dwrite d5/de/f17 [16777216,4194304] 0 2024-05-25T06:11:45.270 INFO:tasks.workunit.client.0.smithi160.stdout:5/818: write db/f63 [3832370,88150] 0 2024-05-25T06:11:45.276 INFO:tasks.workunit.client.0.smithi160.stdout:9/662: dwrite d4/d11/d14/f3a [0,4194304] 0 2024-05-25T06:11:45.279 INFO:tasks.workunit.client.0.smithi160.stdout:5/819: write db/dc/d13/d1d/d3f/d72/dc5/fdc [1246826,118805] 0 2024-05-25T06:11:45.281 INFO:tasks.workunit.client.0.smithi160.stdout:1/660: truncate de/d21/d49/d35/f6e 913421 0 2024-05-25T06:11:45.284 INFO:tasks.workunit.client.0.smithi160.stdout:6/728: dwrite d5/d45/f8e [0,4194304] 0 2024-05-25T06:11:45.284 INFO:tasks.workunit.client.0.smithi160.stdout:6/729: write d5/de1/fc7 [609348,27468] 0 2024-05-25T06:11:45.288 INFO:tasks.workunit.client.0.smithi160.stdout:5/820: mkdir db/dc/d13/d1d/d31/d3b/dfb 0 2024-05-25T06:11:45.290 INFO:tasks.workunit.client.0.smithi160.stdout:5/821: write db/f63 [1965278,48320] 0 2024-05-25T06:11:45.291 INFO:tasks.workunit.client.0.smithi160.stdout:6/730: dread d5/d45/d4a/fa5 [0,4194304] 0 2024-05-25T06:11:45.292 INFO:tasks.workunit.client.0.smithi160.stdout:6/731: chown d5/d45/f8e 2595092 1 2024-05-25T06:11:45.293 INFO:tasks.workunit.client.0.smithi160.stdout:4/716: rename d4/d6/d35/fb2 to d4/d1c/d54/dbc/dda/fe3 0 2024-05-25T06:11:45.298 INFO:tasks.workunit.client.0.smithi160.stdout:4/717: rename d4/d6/d35/d3d/d55 to d4/d6/d28/de4 0 2024-05-25T06:11:45.302 INFO:tasks.workunit.client.0.smithi160.stdout:4/718: dread d4/d6/d21/f77 [0,4194304] 0 2024-05-25T06:11:45.308 INFO:tasks.workunit.client.0.smithi160.stdout:1/661: write de/d21/d49/d28/f7e [221573,29308] 0 2024-05-25T06:11:45.310 INFO:tasks.workunit.client.0.smithi160.stdout:1/662: readlink de/d21/d49/d1c/l6a 0 2024-05-25T06:11:45.310 INFO:tasks.workunit.client.0.smithi160.stdout:6/732: write d5/d45/d63/f83 [366740,25682] 0 2024-05-25T06:11:45.312 INFO:tasks.workunit.client.0.smithi160.stdout:2/763: sync 2024-05-25T06:11:45.313 INFO:tasks.workunit.client.0.smithi160.stdout:1/663: mknod de/d21/d49/d1c/d59/d71/cd7 0 2024-05-25T06:11:45.319 INFO:tasks.workunit.client.0.smithi160.stdout:2/764: mknod d12/d3e/d5a/d8d/db4/dcb/ded/cff 0 2024-05-25T06:11:45.320 INFO:tasks.workunit.client.0.smithi160.stdout:9/663: dread d4/d11/d14/f31 [0,4194304] 0 2024-05-25T06:11:45.321 INFO:tasks.workunit.client.0.smithi160.stdout:1/664: mknod de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/cd8 0 2024-05-25T06:11:45.322 INFO:tasks.workunit.client.0.smithi160.stdout:9/664: chown d4/cb9 312001 1 2024-05-25T06:11:45.325 INFO:tasks.workunit.client.0.smithi160.stdout:6/733: dwrite d5/de/fd [0,4194304] 0 2024-05-25T06:11:45.326 INFO:tasks.workunit.client.0.smithi160.stdout:5/822: dread db/f63 [4194304,4194304] 0 2024-05-25T06:11:45.327 INFO:tasks.workunit.client.0.smithi160.stdout:5/823: write db/dc/d13/f50 [533800,73023] 0 2024-05-25T06:11:45.328 INFO:tasks.workunit.client.0.smithi160.stdout:9/665: truncate d4/d11/f38 610938 0 2024-05-25T06:11:45.329 INFO:tasks.workunit.client.0.smithi160.stdout:5/824: write db/ff6 [620767,101326] 0 2024-05-25T06:11:45.329 INFO:tasks.workunit.client.0.smithi160.stdout:9/666: readlink d4/d11/d1e/d3e/d25/l54 0 2024-05-25T06:11:45.335 INFO:tasks.workunit.client.0.smithi160.stdout:9/667: symlink d4/d11/d1e/d24/d66/ld5 0 2024-05-25T06:11:45.337 INFO:tasks.workunit.client.0.smithi160.stdout:1/665: write de/d21/d49/f6f [382906,63351] 0 2024-05-25T06:11:45.338 INFO:tasks.workunit.client.0.smithi160.stdout:9/668: mknod d4/d11/d1e/d3e/d25/cd6 0 2024-05-25T06:11:45.339 INFO:tasks.workunit.client.0.smithi160.stdout:2/765: dread d12/d75/dca/f24 [0,4194304] 0 2024-05-25T06:11:45.340 INFO:tasks.workunit.client.0.smithi160.stdout:1/666: chown de/d21/d49/d1c/d59/d67/f83 7781147 1 2024-05-25T06:11:45.341 INFO:tasks.workunit.client.0.smithi160.stdout:5/825: fdatasync db/dc/d13/d1d/d23/d45/f8b 0 2024-05-25T06:11:45.341 INFO:tasks.workunit.client.0.smithi160.stdout:1/667: readlink de/d21/l55 0 2024-05-25T06:11:45.342 INFO:tasks.workunit.client.0.smithi160.stdout:4/719: dread d4/d1c/d54/d9b/fd7 [4194304,4194304] 0 2024-05-25T06:11:45.342 INFO:tasks.workunit.client.0.smithi160.stdout:6/734: getdents d5/de1 0 2024-05-25T06:11:45.342 INFO:tasks.workunit.client.0.smithi160.stdout:5/826: fdatasync db/dc/fd7 0 2024-05-25T06:11:45.344 INFO:tasks.workunit.client.0.smithi160.stdout:9/669: rename d4/d11/d14/c63 to d4/d11/d12/d2e/d45/cd7 0 2024-05-25T06:11:45.345 INFO:tasks.workunit.client.0.smithi160.stdout:5/827: read db/dc/d13/d1d/d3f/d72/d98/da5/db5/fcd [62822,103135] 0 2024-05-25T06:11:45.346 INFO:tasks.workunit.client.0.smithi160.stdout:1/668: creat de/d21/d49/d28/fd9 x:0 0 0 2024-05-25T06:11:45.346 INFO:tasks.workunit.client.0.smithi160.stdout:4/720: rmdir d4/d6/d7b 39 2024-05-25T06:11:45.346 INFO:tasks.workunit.client.0.smithi160.stdout:5/828: chown db/dc/d13/d1d/d3f/d72/d98/da5/db5/fdd 2 1 2024-05-25T06:11:45.347 INFO:tasks.workunit.client.0.smithi160.stdout:2/766: mkdir d12/d3e/d5a/d8d/df9/d100 0 2024-05-25T06:11:45.349 INFO:tasks.workunit.client.0.smithi160.stdout:9/670: mknod d4/d11/d12/d2e/d45/d90/cd8 0 2024-05-25T06:11:45.350 INFO:tasks.workunit.client.0.smithi160.stdout:6/735: creat d5/de/d32/d9f/dd5/ff8 x:0 0 0 2024-05-25T06:11:45.351 INFO:tasks.workunit.client.0.smithi160.stdout:4/721: creat d4/d63/fe5 x:0 0 0 2024-05-25T06:11:45.351 INFO:tasks.workunit.client.0.smithi160.stdout:1/669: mknod de/d21/d49/d1c/d59/d71/cda 0 2024-05-25T06:11:45.352 INFO:tasks.workunit.client.0.smithi160.stdout:5/829: read db/dc/d13/d1d/d31/d3b/f84 [218997,58273] 0 2024-05-25T06:11:45.352 INFO:tasks.workunit.client.0.smithi160.stdout:2/767: symlink d12/d3e/d5a/d8d/df9/l101 0 2024-05-25T06:11:45.353 INFO:tasks.workunit.client.0.smithi160.stdout:5/830: chown db/dc/d13/d1d/d31/d56/f61 6249 1 2024-05-25T06:11:45.354 INFO:tasks.workunit.client.0.smithi160.stdout:6/736: unlink d5/de/d9e/dcd/ce3 0 2024-05-25T06:11:45.355 INFO:tasks.workunit.client.0.smithi160.stdout:9/671: symlink d4/d11/d1e/dbc/ld9 0 2024-05-25T06:11:45.359 INFO:tasks.workunit.client.0.smithi160.stdout:5/831: dread db/dc/d13/d1d/d3f/d72/d98/da5/db5/fcd [0,4194304] 0 2024-05-25T06:11:45.362 INFO:tasks.workunit.client.0.smithi160.stdout:6/737: creat d5/d30/d36/d73/d87/da4/ddf/ff9 x:0 0 0 2024-05-25T06:11:45.363 INFO:tasks.workunit.client.0.smithi160.stdout:6/738: write d5/de/fd [4732625,106549] 0 2024-05-25T06:11:45.364 INFO:tasks.workunit.client.0.smithi160.stdout:2/768: mknod d12/d47/c102 0 2024-05-25T06:11:45.365 INFO:tasks.workunit.client.0.smithi160.stdout:2/769: write d12/f8f [892479,27170] 0 2024-05-25T06:11:45.372 INFO:tasks.workunit.client.0.smithi160.stdout:2/770: unlink d12/d3e/d5f/d74/d7a/l8e 0 2024-05-25T06:11:45.372 INFO:tasks.workunit.client.0.smithi160.stdout:9/672: getdents d4/d11/d1e/d3e/d25/d5e 0 2024-05-25T06:11:45.373 INFO:tasks.workunit.client.0.smithi160.stdout:1/670: write de/f19 [3591018,32166] 0 2024-05-25T06:11:45.374 INFO:tasks.workunit.client.0.smithi160.stdout:1/671: write de/d21/d49/d1c/d59/d71/f5a [1505305,7616] 0 2024-05-25T06:11:45.374 INFO:tasks.workunit.client.0.smithi160.stdout:5/832: rmdir db/dc/d13/d1d/d23/d45/df1 0 2024-05-25T06:11:45.375 INFO:tasks.workunit.client.0.smithi160.stdout:5/833: fdatasync f6 0 2024-05-25T06:11:45.376 INFO:tasks.workunit.client.0.smithi160.stdout:5/834: fdatasync db/fa9 0 2024-05-25T06:11:45.376 INFO:tasks.workunit.client.0.smithi160.stdout:2/771: mkdir d12/d1c/d37/d4b/d103 0 2024-05-25T06:11:45.377 INFO:tasks.workunit.client.0.smithi160.stdout:5/835: write db/d47/d81/dc1/fde [680440,79168] 0 2024-05-25T06:11:45.377 INFO:tasks.workunit.client.0.smithi160.stdout:9/673: unlink d4/d11/d1e/d3e/d2f/l40 0 2024-05-25T06:11:45.379 INFO:tasks.workunit.client.0.smithi160.stdout:1/672: mkdir de/d21/d49/d1c/d52/dc7/ddb 0 2024-05-25T06:11:45.381 INFO:tasks.workunit.client.0.smithi160.stdout:9/674: creat d4/d11/d1e/d3e/d25/d5e/fda x:0 0 0 2024-05-25T06:11:45.383 INFO:tasks.workunit.client.0.smithi160.stdout:2/772: getdents d12/d3e/d5a 0 2024-05-25T06:11:45.383 INFO:tasks.workunit.client.0.smithi160.stdout:9/675: write d4/d11/d1e/fd3 [916451,96904] 0 2024-05-25T06:11:45.383 INFO:tasks.workunit.client.0.smithi160.stdout:6/739: dwrite d5/d39/d8f/d2b/f88 [0,4194304] 0 2024-05-25T06:11:45.384 INFO:tasks.workunit.client.0.smithi160.stdout:1/673: mknod de/d21/d49/d1c/d59/d7f/cdc 0 2024-05-25T06:11:45.385 INFO:tasks.workunit.client.0.smithi160.stdout:9/676: dread - d4/d11/f93 zero size 2024-05-25T06:11:45.387 INFO:tasks.workunit.client.0.smithi160.stdout:9/677: chown d4/d11/d1e/dbc/ld9 133 1 2024-05-25T06:11:45.388 INFO:tasks.workunit.client.0.smithi160.stdout:6/740: fsync d5/d39/f41 0 2024-05-25T06:11:45.392 INFO:tasks.workunit.client.0.smithi160.stdout:9/678: dread d4/fc2 [0,4194304] 0 2024-05-25T06:11:45.393 INFO:tasks.workunit.client.0.smithi160.stdout:9/679: chown d4/d11/d1e/d3e/d2f/d79/fc0 1508192 1 2024-05-25T06:11:45.393 INFO:tasks.workunit.client.0.smithi160.stdout:5/836: dread f1 [0,4194304] 0 2024-05-25T06:11:45.399 INFO:tasks.workunit.client.0.smithi160.stdout:2/773: truncate d12/d3c/f61 1559509 0 2024-05-25T06:11:45.399 INFO:tasks.workunit.client.1.smithi203.stdout:9/997: sync 2024-05-25T06:11:45.403 INFO:tasks.workunit.client.0.smithi160.stdout:6/741: rename d5/de/d32/d74/lc5 to d5/d30/d36/lfa 0 2024-05-25T06:11:45.403 INFO:tasks.workunit.client.1.smithi203.stdout:9/998: rename d0/d17/d20/d29/c13c to d0/d17/d20/d6e/d75/c14b 0 2024-05-25T06:11:45.406 INFO:tasks.workunit.client.0.smithi160.stdout:2/774: getdents d12/d3e/d5f 0 2024-05-25T06:11:45.407 INFO:tasks.workunit.client.0.smithi160.stdout:6/742: link d5/d45/fe2 d5/de/d32/d35/dab/ffb 0 2024-05-25T06:11:45.408 INFO:tasks.workunit.client.0.smithi160.stdout:2/775: mkdir d12/d3e/d5f/d74/d7a/d104 0 2024-05-25T06:11:45.410 INFO:tasks.workunit.client.0.smithi160.stdout:6/743: mknod d5/de/d32/d74/cfc 0 2024-05-25T06:11:45.410 INFO:tasks.workunit.client.0.smithi160.stdout:1/674: dwrite de/d21/d49/d35/f40 [0,4194304] 0 2024-05-25T06:11:45.412 INFO:tasks.workunit.client.0.smithi160.stdout:2/776: mknod d12/d75/dca/c105 0 2024-05-25T06:11:45.415 INFO:tasks.workunit.client.0.smithi160.stdout:6/744: truncate d5/de/d32/d35/f82 4402198 0 2024-05-25T06:11:45.416 INFO:tasks.workunit.client.0.smithi160.stdout:5/837: dwrite f8 [0,4194304] 0 2024-05-25T06:11:45.418 INFO:tasks.workunit.client.0.smithi160.stdout:2/777: dread d12/d75/dca/f24 [0,4194304] 0 2024-05-25T06:11:45.419 INFO:tasks.workunit.client.0.smithi160.stdout:2/778: fsync d12/d1c/d2a/fe1 0 2024-05-25T06:11:45.420 INFO:tasks.workunit.client.0.smithi160.stdout:5/838: read db/d47/d81/dc1/fde [422011,39280] 0 2024-05-25T06:11:45.420 INFO:tasks.workunit.client.0.smithi160.stdout:6/745: mknod d5/de/d58/cfd 0 2024-05-25T06:11:45.421 INFO:tasks.workunit.client.0.smithi160.stdout:2/779: chown d12/d3e/d5f/d74/d7a/d104 0 1 2024-05-25T06:11:45.421 INFO:tasks.workunit.client.0.smithi160.stdout:6/746: readlink d5/d45/d4a/la3 0 2024-05-25T06:11:45.423 INFO:tasks.workunit.client.0.smithi160.stdout:8/756: sync 2024-05-25T06:11:45.424 INFO:tasks.workunit.client.0.smithi160.stdout:3/720: sync 2024-05-25T06:11:45.424 INFO:tasks.workunit.client.0.smithi160.stdout:7/789: sync 2024-05-25T06:11:45.424 INFO:tasks.workunit.client.0.smithi160.stdout:6/747: write d5/de1/fc7 [365666,103675] 0 2024-05-25T06:11:45.427 INFO:tasks.workunit.client.0.smithi160.stdout:7/790: chown d6/d2b/d5d/de2/le6 246 1 2024-05-25T06:11:45.428 INFO:tasks.workunit.client.0.smithi160.stdout:5/839: truncate db/dc/d13/d1d/d31/d3b/f2b 747455 0 2024-05-25T06:11:45.430 INFO:tasks.workunit.client.0.smithi160.stdout:7/791: write d6/dc/d64/d5a/d7c/fd0 [919736,23785] 0 2024-05-25T06:11:45.430 INFO:tasks.workunit.client.0.smithi160.stdout:1/675: truncate de/d21/d49/d1c/fc9 231914 0 2024-05-25T06:11:45.431 INFO:tasks.workunit.client.0.smithi160.stdout:3/721: dwrite d19/d2f/d34/f9d [0,4194304] 0 2024-05-25T06:11:45.431 INFO:tasks.workunit.client.0.smithi160.stdout:2/780: mknod d12/d3e/d5a/d8d/df9/d100/c106 0 2024-05-25T06:11:45.431 INFO:tasks.workunit.client.1.smithi203.stdout:9/999: dread d0/df/d2b/f2f [0,4194304] 0 2024-05-25T06:11:45.433 INFO:tasks.workunit.client.0.smithi160.stdout:8/757: creat d5/d44/d1f/da3/d28/d4a/d4e/d86/f101 x:0 0 0 2024-05-25T06:11:45.434 INFO:tasks.workunit.client.0.smithi160.stdout:5/840: write db/f57 [811311,32820] 0 2024-05-25T06:11:45.436 INFO:tasks.workunit.client.0.smithi160.stdout:8/758: chown d5/d44/d1f/d58/c71 1015 1 2024-05-25T06:11:45.437 INFO:tasks.workunit.client.0.smithi160.stdout:1/676: creat de/d21/d49/d1c/d41/d45/d7d/fdd x:0 0 0 2024-05-25T06:11:45.442 INFO:tasks.workunit.client.0.smithi160.stdout:7/792: creat d6/d32/d7e/f105 x:0 0 0 2024-05-25T06:11:45.442 INFO:tasks.workunit.client.0.smithi160.stdout:6/748: getdents d5/d39/da6 0 2024-05-25T06:11:45.444 INFO:tasks.workunit.client.0.smithi160.stdout:3/722: dwrite d19/d2f/d34/d67/d96/fec [0,4194304] 0 2024-05-25T06:11:45.451 INFO:tasks.workunit.client.0.smithi160.stdout:3/723: truncate d19/d22/fc9 810190 0 2024-05-25T06:11:45.452 INFO:tasks.workunit.client.0.smithi160.stdout:6/749: symlink d5/de/d32/d35/lfe 0 2024-05-25T06:11:45.454 INFO:tasks.workunit.client.0.smithi160.stdout:8/759: truncate d5/d44/d1f/da3/d28/d4a/f5c 6316509 0 2024-05-25T06:11:45.458 INFO:tasks.workunit.client.0.smithi160.stdout:7/793: dwrite d6/d3e/d6f/f76 [0,4194304] 0 2024-05-25T06:11:45.461 INFO:tasks.workunit.client.0.smithi160.stdout:3/724: rename d19/da7/de7/f97 to d19/d2f/d38/d43/d5b/d65/ff2 0 2024-05-25T06:11:45.469 INFO:tasks.workunit.client.0.smithi160.stdout:8/760: fsync d5/d44/d1f/da3/d28/d49/f70 0 2024-05-25T06:11:45.469 INFO:tasks.workunit.client.0.smithi160.stdout:6/750: dwrite d5/d30/d36/d73/d87/da4/ddf/ff9 [0,4194304] 0 2024-05-25T06:11:45.474 INFO:tasks.workunit.client.0.smithi160.stdout:5/841: dread db/fa7 [0,4194304] 0 2024-05-25T06:11:45.477 INFO:tasks.workunit.client.0.smithi160.stdout:2/781: dread d12/d75/dca/f15 [0,4194304] 0 2024-05-25T06:11:45.478 INFO:tasks.workunit.client.0.smithi160.stdout:6/751: symlink d5/d39/lff 0 2024-05-25T06:11:45.479 INFO:tasks.workunit.client.0.smithi160.stdout:7/794: dread d6/d3e/d6f/f76 [0,4194304] 0 2024-05-25T06:11:45.480 INFO:tasks.workunit.client.0.smithi160.stdout:7/795: readlink d6/d3e/da1/lb9 0 2024-05-25T06:11:45.480 INFO:tasks.workunit.client.0.smithi160.stdout:7/796: chown d6/d3e/da1/ff1 232075752 1 2024-05-25T06:11:45.482 INFO:tasks.workunit.client.0.smithi160.stdout:3/725: truncate d19/d2f/d38/d47/fc7 2685580 0 2024-05-25T06:11:45.483 INFO:tasks.workunit.client.0.smithi160.stdout:4/722: sync 2024-05-25T06:11:45.483 INFO:tasks.workunit.client.0.smithi160.stdout:0/842: sync 2024-05-25T06:11:45.485 INFO:tasks.workunit.client.0.smithi160.stdout:2/782: creat d12/d3e/d5f/d74/f107 x:0 0 0 2024-05-25T06:11:45.485 INFO:tasks.workunit.client.0.smithi160.stdout:1/677: dread de/f13 [0,4194304] 0 2024-05-25T06:11:45.487 INFO:tasks.workunit.client.0.smithi160.stdout:5/842: write db/dc/f93 [226864,28429] 0 2024-05-25T06:11:45.488 INFO:tasks.workunit.client.1.smithi203.stdout:1/986: sync 2024-05-25T06:11:45.491 INFO:tasks.workunit.client.0.smithi160.stdout:1/678: mkdir de/d21/d49/d28/dde 0 2024-05-25T06:11:45.493 INFO:tasks.workunit.client.0.smithi160.stdout:7/797: link d6/dc/d64/d5a/d7c/faa d6/dc/d64/dae/f106 0 2024-05-25T06:11:45.495 INFO:tasks.workunit.client.0.smithi160.stdout:6/752: write d5/de/d32/f67 [475862,53937] 0 2024-05-25T06:11:45.496 INFO:tasks.workunit.client.1.smithi203.stdout:1/987: link d14/d81/cdf d14/d57/d6b/da1/de2/d143/dd2/c154 0 2024-05-25T06:11:45.497 INFO:tasks.workunit.client.1.smithi203.stdout:1/988: dread - d14/d57/d1d/da8/d150/de4/f124 zero size 2024-05-25T06:11:45.498 INFO:tasks.workunit.client.0.smithi160.stdout:0/843: creat d2b/d31/f10e x:0 0 0 2024-05-25T06:11:45.498 INFO:tasks.workunit.client.0.smithi160.stdout:2/783: rename d12/d5c/d85/l8b to d12/d3e/d5f/d74/d7a/d104/l108 0 2024-05-25T06:11:45.502 INFO:tasks.workunit.client.1.smithi203.stdout:1/989: symlink d14/d57/d6b/da1/de2/d143/dd2/l155 0 2024-05-25T06:11:45.502 INFO:tasks.workunit.client.0.smithi160.stdout:0/844: write d2b/d31/d35/fc9 [1024449,82368] 0 2024-05-25T06:11:45.504 INFO:tasks.workunit.client.1.smithi203.stdout:1/990: rmdir d14/d57/d97/da4/dad/dd9 39 2024-05-25T06:11:45.504 INFO:tasks.workunit.client.0.smithi160.stdout:8/761: dread d5/d44/d1f/da3/d28/d49/fc3 [4194304,4194304] 0 2024-05-25T06:11:45.506 INFO:tasks.workunit.client.1.smithi203.stdout:1/991: rmdir d14/d57/d1d/da8/d150/de4 39 2024-05-25T06:11:45.506 INFO:tasks.workunit.client.0.smithi160.stdout:5/843: creat db/dc/d13/d1d/d23/ffc x:0 0 0 2024-05-25T06:11:45.507 INFO:tasks.workunit.client.0.smithi160.stdout:2/784: dread d12/d1c/d59/ffe [0,4194304] 0 2024-05-25T06:11:45.509 INFO:tasks.workunit.client.1.smithi203.stdout:1/992: getdents d14/dff 0 2024-05-25T06:11:45.509 INFO:tasks.workunit.client.1.smithi203.stdout:1/993: read - d14/d7c/f10b zero size 2024-05-25T06:11:45.510 INFO:tasks.workunit.client.0.smithi160.stdout:2/785: read d12/d1c/d37/fdb [367224,108124] 0 2024-05-25T06:11:45.512 INFO:tasks.workunit.client.0.smithi160.stdout:8/762: fdatasync d5/d44/d1f/d58/f60 0 2024-05-25T06:11:45.512 INFO:tasks.workunit.client.0.smithi160.stdout:8/763: stat d5/d9/d72/dd9/ld0 0 2024-05-25T06:11:45.513 INFO:tasks.workunit.client.1.smithi203.stdout:1/994: unlink d14/d57/l88 0 2024-05-25T06:11:45.514 INFO:tasks.workunit.client.0.smithi160.stdout:5/844: dwrite db/dc/d13/d1d/d23/dbd/fcc [0,4194304] 0 2024-05-25T06:11:45.514 INFO:tasks.workunit.client.1.smithi203.stdout:1/995: fdatasync d14/d57/d7e/fa7 0 2024-05-25T06:11:45.517 INFO:tasks.workunit.client.0.smithi160.stdout:6/753: write d5/de/f22 [1056249,68887] 0 2024-05-25T06:11:45.521 INFO:tasks.workunit.client.0.smithi160.stdout:3/726: dread d19/d2f/d38/d43/d5b/d65/f89 [4194304,4194304] 0 2024-05-25T06:11:45.521 INFO:tasks.workunit.client.0.smithi160.stdout:1/679: truncate de/d21/d49/d1c/d59/d67/fb9 3973050 0 2024-05-25T06:11:45.524 INFO:tasks.workunit.client.0.smithi160.stdout:7/798: truncate d6/d2b/d5d/d89/f90 1585864 0 2024-05-25T06:11:45.527 INFO:tasks.workunit.client.0.smithi160.stdout:6/754: creat d5/d45/d4a/dbc/dc4/f100 x:0 0 0 2024-05-25T06:11:45.528 INFO:tasks.workunit.client.0.smithi160.stdout:1/680: mknod de/d21/d49/d35/cdf 0 2024-05-25T06:11:45.529 INFO:tasks.workunit.client.0.smithi160.stdout:5/845: dwrite db/dc/d13/fd0 [0,4194304] 0 2024-05-25T06:11:45.531 INFO:tasks.workunit.client.0.smithi160.stdout:3/727: write d19/d2f/d38/d47/f4b [630367,76353] 0 2024-05-25T06:11:45.535 INFO:tasks.workunit.client.0.smithi160.stdout:2/786: link d12/d1c/f6c d12/f109 0 2024-05-25T06:11:45.537 INFO:tasks.workunit.client.0.smithi160.stdout:0/845: dwrite d2b/d31/d35/d36/d37/f3b [0,4194304] 0 2024-05-25T06:11:45.538 INFO:tasks.workunit.client.0.smithi160.stdout:2/787: readlink d12/d75/dca/la1 0 2024-05-25T06:11:45.540 INFO:tasks.workunit.client.0.smithi160.stdout:6/755: symlink d5/d39/da6/dac/l101 0 2024-05-25T06:11:45.543 INFO:tasks.workunit.client.0.smithi160.stdout:2/788: dread d12/d3e/d5a/d8d/fb9 [0,4194304] 0 2024-05-25T06:11:45.548 INFO:tasks.workunit.client.0.smithi160.stdout:5/846: rename db/dc/d13/fd0 to db/ffd 0 2024-05-25T06:11:45.550 INFO:tasks.workunit.client.0.smithi160.stdout:6/756: creat d5/d30/d36/d73/d87/da4/f102 x:0 0 0 2024-05-25T06:11:45.553 INFO:tasks.workunit.client.0.smithi160.stdout:7/799: dread d6/d3e/f70 [0,4194304] 0 2024-05-25T06:11:45.554 INFO:tasks.workunit.client.0.smithi160.stdout:6/757: mkdir d5/d39/d8f/d2b/d103 0 2024-05-25T06:11:45.555 INFO:tasks.workunit.client.0.smithi160.stdout:7/800: chown d6/d3e/d6f/d73/dee 15 1 2024-05-25T06:11:45.556 INFO:tasks.workunit.client.0.smithi160.stdout:5/847: link f9 db/dc/d13/d1d/d23/ffe 0 2024-05-25T06:11:45.557 INFO:tasks.workunit.client.0.smithi160.stdout:6/758: mkdir d5/d45/d104 0 2024-05-25T06:11:45.559 INFO:tasks.workunit.client.0.smithi160.stdout:6/759: truncate d5/d45/d4a/fa5 4369339 0 2024-05-25T06:11:45.562 INFO:tasks.workunit.client.0.smithi160.stdout:5/848: mknod db/dc/d13/d1d/d23/cff 0 2024-05-25T06:11:45.563 INFO:tasks.workunit.client.0.smithi160.stdout:6/760: rmdir d5/de/d18 39 2024-05-25T06:11:45.564 INFO:tasks.workunit.client.1.smithi203.stdout:1/996: fdatasync d14/d57/d1d/f89 0 2024-05-25T06:11:45.565 INFO:tasks.workunit.client.0.smithi160.stdout:3/728: dread d19/d2f/d38/d43/d5b/f6d [0,4194304] 0 2024-05-25T06:11:45.566 INFO:tasks.workunit.client.0.smithi160.stdout:6/761: chown d5/de/d7d/f9b 57 1 2024-05-25T06:11:45.566 INFO:tasks.workunit.client.0.smithi160.stdout:7/801: write d6/d3e/d6f/d73/fc5 [396730,126072] 0 2024-05-25T06:11:45.566 INFO:tasks.workunit.client.0.smithi160.stdout:5/849: chown db/dc/d13/d1d/d3f/d72/d98/da5/db5/fdd 40748263 1 2024-05-25T06:11:45.570 INFO:tasks.workunit.client.0.smithi160.stdout:5/850: unlink db/dc/d13/d1d/d3f/d72/fa2 0 2024-05-25T06:11:45.572 INFO:tasks.workunit.client.0.smithi160.stdout:3/729: fsync d19/d2f/d38/d47/f6e 0 2024-05-25T06:11:45.573 INFO:tasks.workunit.client.0.smithi160.stdout:7/802: rename d6/d2b/d5d/c61 to d6/dc/d21/ddd/c107 0 2024-05-25T06:11:45.574 INFO:tasks.workunit.client.0.smithi160.stdout:6/762: creat d5/d39/d8f/f105 x:0 0 0 2024-05-25T06:11:45.574 INFO:tasks.workunit.client.0.smithi160.stdout:7/803: fsync d6/d2b/d5d/fc2 0 2024-05-25T06:11:45.574 INFO:tasks.workunit.client.0.smithi160.stdout:2/789: dread d12/d1c/f27 [0,4194304] 0 2024-05-25T06:11:45.576 INFO:tasks.workunit.client.0.smithi160.stdout:3/730: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/df3 0 2024-05-25T06:11:45.577 INFO:tasks.workunit.client.0.smithi160.stdout:5/851: rmdir db/dc/d13/d1d/d23/d24/daf 39 2024-05-25T06:11:45.580 INFO:tasks.workunit.client.0.smithi160.stdout:7/804: creat d6/d3e/f108 x:0 0 0 2024-05-25T06:11:45.581 INFO:tasks.workunit.client.0.smithi160.stdout:6/763: creat d5/d39/d8f/d2b/f106 x:0 0 0 2024-05-25T06:11:45.582 INFO:tasks.workunit.client.0.smithi160.stdout:6/764: fdatasync d5/d45/f9a 0 2024-05-25T06:11:45.583 INFO:tasks.workunit.client.0.smithi160.stdout:2/790: dread f10 [0,4194304] 0 2024-05-25T06:11:45.584 INFO:tasks.workunit.client.0.smithi160.stdout:6/765: write d5/de1/fc7 [957154,106729] 0 2024-05-25T06:11:45.586 INFO:tasks.workunit.client.0.smithi160.stdout:2/791: chown d12/d3e/d5f/d74/d7a/fac 1098 1 2024-05-25T06:11:45.587 INFO:tasks.workunit.client.0.smithi160.stdout:3/731: mknod d19/d2f/d38/d43/d5b/d65/d83/dee/df3/cf4 0 2024-05-25T06:11:45.588 INFO:tasks.workunit.client.0.smithi160.stdout:7/805: dwrite d6/dc/d21/f97 [4194304,4194304] 0 2024-05-25T06:11:45.593 INFO:tasks.workunit.client.0.smithi160.stdout:3/732: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/df5 0 2024-05-25T06:11:45.595 INFO:tasks.workunit.client.0.smithi160.stdout:5/852: truncate db/f5d 1457286 0 2024-05-25T06:11:45.596 INFO:tasks.workunit.client.0.smithi160.stdout:7/806: truncate d6/d32/d7e/de0/fe5 836447 0 2024-05-25T06:11:45.598 INFO:tasks.workunit.client.1.smithi203.stdout:1/997: truncate d14/d7c/d9d/dc8/f108 1492157 0 2024-05-25T06:11:45.598 INFO:tasks.workunit.client.0.smithi160.stdout:6/766: dwrite d5/d30/fc6 [0,4194304] 0 2024-05-25T06:11:45.600 INFO:tasks.workunit.client.0.smithi160.stdout:3/733: creat d19/d2f/d34/def/ff6 x:0 0 0 2024-05-25T06:11:45.602 INFO:tasks.workunit.client.0.smithi160.stdout:6/767: mkdir d5/d30/d36/d73/d87/d107 0 2024-05-25T06:11:45.603 INFO:tasks.workunit.client.0.smithi160.stdout:2/792: dwrite d12/f45 [0,4194304] 0 2024-05-25T06:11:45.605 INFO:tasks.workunit.client.0.smithi160.stdout:3/734: fsync d19/d2f/d38/d43/d5b/d65/d83/fcb 0 2024-05-25T06:11:45.610 INFO:tasks.workunit.client.0.smithi160.stdout:7/807: dwrite d6/dc/d64/f45 [4194304,4194304] 0 2024-05-25T06:11:45.616 INFO:tasks.workunit.client.0.smithi160.stdout:3/735: rmdir d19/d2f/d38/d43/d5b/d65/d83/dee/df5 0 2024-05-25T06:11:45.617 INFO:tasks.workunit.client.0.smithi160.stdout:2/793: dwrite d12/d1c/d37/fcc [0,4194304] 0 2024-05-25T06:11:45.618 INFO:tasks.workunit.client.0.smithi160.stdout:6/768: dread d5/d30/f5a [0,4194304] 0 2024-05-25T06:11:45.619 INFO:tasks.workunit.client.0.smithi160.stdout:3/736: symlink d19/db2/lf7 0 2024-05-25T06:11:45.620 INFO:tasks.workunit.client.0.smithi160.stdout:2/794: creat d12/d75/dca/f10a x:0 0 0 2024-05-25T06:11:45.621 INFO:tasks.workunit.client.0.smithi160.stdout:2/795: stat d12/d3c/fce 0 2024-05-25T06:11:45.622 INFO:tasks.workunit.client.0.smithi160.stdout:2/796: fdatasync d12/d3e/d5a/f7f 0 2024-05-25T06:11:45.624 INFO:tasks.workunit.client.1.smithi203.stdout:1/998: write d14/f54 [4000778,59066] 0 2024-05-25T06:11:45.625 INFO:tasks.workunit.client.0.smithi160.stdout:3/737: write d19/d2f/d38/d43/d5b/d65/fbd [2587503,61018] 0 2024-05-25T06:11:45.626 INFO:tasks.workunit.client.0.smithi160.stdout:7/808: truncate d6/d32/d92/fc9 3731583 0 2024-05-25T06:11:45.628 INFO:tasks.workunit.client.0.smithi160.stdout:3/738: read f13 [1764133,92654] 0 2024-05-25T06:11:45.631 INFO:tasks.workunit.client.0.smithi160.stdout:2/797: chown d12/d1c/d37/cc0 2827 1 2024-05-25T06:11:45.632 INFO:tasks.workunit.client.0.smithi160.stdout:7/809: creat d6/d32/d7e/f109 x:0 0 0 2024-05-25T06:11:45.633 INFO:tasks.workunit.client.0.smithi160.stdout:2/798: write d12/d97/f99 [1727606,82272] 0 2024-05-25T06:11:45.634 INFO:tasks.workunit.client.0.smithi160.stdout:3/739: mkdir d19/d2f/d34/da8/df8 0 2024-05-25T06:11:45.636 INFO:tasks.workunit.client.0.smithi160.stdout:3/740: read d19/d2f/d38/d43/d5b/d65/fb9 [220046,29708] 0 2024-05-25T06:11:45.637 INFO:tasks.workunit.client.0.smithi160.stdout:2/799: creat d12/d5c/d85/dc1/f10b x:0 0 0 2024-05-25T06:11:45.640 INFO:tasks.workunit.client.0.smithi160.stdout:7/810: rename d6/d2b/d8b/faf to d6/d2b/d5d/de2/f10a 0 2024-05-25T06:11:45.641 INFO:tasks.workunit.client.0.smithi160.stdout:3/741: mknod d19/d2f/d34/d67/d96/cf9 0 2024-05-25T06:11:45.641 INFO:tasks.workunit.client.1.smithi203.stdout:1/999: write d14/d7c/f7d [210831,82802] 0 2024-05-25T06:11:45.641 INFO:tasks.workunit.client.0.smithi160.stdout:3/742: fdatasync d19/d2f/d34/fa1 0 2024-05-25T06:11:45.642 INFO:tasks.workunit.client.0.smithi160.stdout:3/743: readlink d19/d2f/d38/d43/d5b/d65/d7e/lb8 0 2024-05-25T06:11:45.644 INFO:tasks.workunit.client.0.smithi160.stdout:2/800: mkdir d12/d5c/d10c 0 2024-05-25T06:11:45.644 INFO:tasks.workunit.client.0.smithi160.stdout:7/811: stat d6/d2b/d5d/d89/fa9 0 2024-05-25T06:11:45.645 INFO:tasks.workunit.client.0.smithi160.stdout:2/801: read d12/d1c/d59/ffe [184258,111819] 0 2024-05-25T06:11:45.645 INFO:tasks.workunit.client.0.smithi160.stdout:7/812: fdatasync d6/dc/d64/f37 0 2024-05-25T06:11:45.647 INFO:tasks.workunit.client.1.smithi203.stderr:+ rm -rf -- ./tmp.Gkhzq4SuAA 2024-05-25T06:11:45.649 INFO:tasks.workunit.client.0.smithi160.stdout:9/680: sync 2024-05-25T06:11:45.655 INFO:tasks.workunit.client.0.smithi160.stdout:7/813: chown d6/d2b/fc6 209 1 2024-05-25T06:11:45.658 INFO:tasks.workunit.client.0.smithi160.stdout:3/744: rename d19/le5 to d19/d2f/d38/d43/d5b/lfa 0 2024-05-25T06:11:45.658 INFO:tasks.workunit.client.0.smithi160.stdout:2/802: getdents d12/d3e/d5f/df3 0 2024-05-25T06:11:45.659 INFO:tasks.workunit.client.0.smithi160.stdout:2/803: chown d12/d47 777 1 2024-05-25T06:11:45.662 INFO:tasks.workunit.client.0.smithi160.stdout:7/814: dwrite d6/d3e/d59/fc8 [0,4194304] 0 2024-05-25T06:11:45.668 INFO:tasks.workunit.client.0.smithi160.stdout:7/815: symlink d6/d32/d5b/l10b 0 2024-05-25T06:11:45.668 INFO:tasks.workunit.client.0.smithi160.stdout:2/804: dread d12/d1c/d37/f3b [0,4194304] 0 2024-05-25T06:11:45.669 INFO:tasks.workunit.client.0.smithi160.stdout:3/745: creat d19/d2f/d38/ffb x:0 0 0 2024-05-25T06:11:45.671 INFO:tasks.workunit.client.0.smithi160.stdout:9/681: rename d4/d11/d1e/d6c/f81 to d4/d11/d1e/d3e/d25/d5e/fdb 0 2024-05-25T06:11:45.674 INFO:tasks.workunit.client.0.smithi160.stdout:2/805: creat d12/d3e/d5a/d8d/df9/f10d x:0 0 0 2024-05-25T06:11:45.676 INFO:tasks.workunit.client.0.smithi160.stdout:2/806: mkdir d12/d3e/d5f/df3/d10e 0 2024-05-25T06:11:45.677 INFO:tasks.workunit.client.0.smithi160.stdout:2/807: readlink d12/d3c/la3 0 2024-05-25T06:11:45.680 INFO:tasks.workunit.client.0.smithi160.stdout:3/746: dwrite d19/d2f/d38/d43/d5b/d65/f89 [4194304,4194304] 0 2024-05-25T06:11:45.681 INFO:tasks.workunit.client.0.smithi160.stdout:2/808: creat d12/d47/da0/dd3/f10f x:0 0 0 2024-05-25T06:11:45.683 INFO:tasks.workunit.client.0.smithi160.stdout:2/809: dread d12/d1c/d37/d4b/fab [4194304,4194304] 0 2024-05-25T06:11:45.684 INFO:tasks.workunit.client.0.smithi160.stdout:3/747: creat d19/d2f/d38/d43/d5b/d65/ffc x:0 0 0 2024-05-25T06:11:45.693 INFO:tasks.workunit.client.0.smithi160.stdout:3/748: unlink d19/d2f/d38/d43/d5b/d65/fb9 0 2024-05-25T06:11:45.695 INFO:tasks.workunit.client.0.smithi160.stdout:3/749: rename d19/d2f/d38 to d19/d2f/d38/d43/d5b/d65/dfd 22 2024-05-25T06:11:45.701 INFO:tasks.workunit.client.0.smithi160.stdout:7/816: dread d6/d2b/f6b [0,4194304] 0 2024-05-25T06:11:45.705 INFO:tasks.workunit.client.0.smithi160.stdout:3/750: fdatasync d19/f99 0 2024-05-25T06:11:45.707 INFO:tasks.workunit.client.0.smithi160.stdout:7/817: link d6/d3e/d6f/f56 d6/d3e/d6f/dfa/f10c 0 2024-05-25T06:11:45.710 INFO:tasks.workunit.client.0.smithi160.stdout:2/810: dread d12/d3e/d5f/d74/f77 [4194304,4194304] 0 2024-05-25T06:11:45.710 INFO:tasks.workunit.client.0.smithi160.stdout:7/818: unlink d6/d3e/f9d 0 2024-05-25T06:11:45.710 INFO:tasks.workunit.client.0.smithi160.stdout:3/751: dread d19/d2f/d38/d43/d5b/d65/d7f/f81 [0,4194304] 0 2024-05-25T06:11:45.711 INFO:tasks.workunit.client.0.smithi160.stdout:7/819: fsync d6/d32/d92/fab 0 2024-05-25T06:11:45.713 INFO:tasks.workunit.client.0.smithi160.stdout:3/752: symlink d19/d2f/d34/d67/d96/lfe 0 2024-05-25T06:11:45.717 INFO:tasks.workunit.client.0.smithi160.stdout:3/753: creat d19/d2f/d38/d43/d5b/d65/fff x:0 0 0 2024-05-25T06:11:45.717 INFO:tasks.workunit.client.0.smithi160.stdout:2/811: link d12/d3e/d5a/c70 d12/d47/da0/c110 0 2024-05-25T06:11:45.718 INFO:tasks.workunit.client.0.smithi160.stdout:3/754: chown d19/d2f/d34/cd5 233266 1 2024-05-25T06:11:45.719 INFO:tasks.workunit.client.0.smithi160.stdout:3/755: write d19/d2f/d38/d47/f6e [246451,67137] 0 2024-05-25T06:11:45.720 INFO:tasks.workunit.client.0.smithi160.stdout:7/820: rename d6/d32/d5b/f96 to d6/d32/f10d 0 2024-05-25T06:11:45.721 INFO:tasks.workunit.client.0.smithi160.stdout:2/812: mknod d12/d1c/d2a/c111 0 2024-05-25T06:11:45.723 INFO:tasks.workunit.client.0.smithi160.stdout:7/821: fdatasync d6/dc/d64/dae/f106 0 2024-05-25T06:11:45.725 INFO:tasks.workunit.client.0.smithi160.stdout:3/756: dwrite d19/f44 [4194304,4194304] 0 2024-05-25T06:11:45.726 INFO:tasks.workunit.client.0.smithi160.stdout:2/813: creat d12/d3c/df1/f112 x:0 0 0 2024-05-25T06:11:45.726 INFO:tasks.workunit.client.0.smithi160.stdout:3/757: readlink d19/d22/l31 0 2024-05-25T06:11:45.728 INFO:tasks.workunit.client.0.smithi160.stdout:3/758: dread - d19/d2f/d38/d43/d5b/d65/d7e/fe9 zero size 2024-05-25T06:11:45.729 INFO:tasks.workunit.client.0.smithi160.stdout:3/759: chown d19/d2f/d38/d43/d5b/d65/d7e/lb8 1753 1 2024-05-25T06:11:45.731 INFO:tasks.workunit.client.0.smithi160.stdout:3/760: chown d19/d2f/d38/fce 485430 1 2024-05-25T06:11:45.732 INFO:tasks.workunit.client.0.smithi160.stdout:2/814: rename d12/d97 to d12/d3e/d5f/d113 0 2024-05-25T06:11:45.733 INFO:tasks.workunit.client.0.smithi160.stdout:7/822: dwrite d6/d32/d5b/fbb [4194304,4194304] 0 2024-05-25T06:11:45.746 INFO:tasks.workunit.client.0.smithi160.stdout:7/823: mkdir d6/d32/d5b/d10e 0 2024-05-25T06:11:45.752 INFO:tasks.workunit.client.0.smithi160.stdout:7/824: dread d6/d32/d7e/de0/fe5 [0,4194304] 0 2024-05-25T06:11:45.753 INFO:tasks.workunit.client.0.smithi160.stdout:2/815: getdents d12/d3e/d5f/df3 0 2024-05-25T06:11:45.754 INFO:tasks.workunit.client.0.smithi160.stdout:2/816: chown d12/d3e/d5f/d74/d7a/de3 18936 1 2024-05-25T06:11:45.756 INFO:tasks.workunit.client.0.smithi160.stdout:7/825: creat d6/d3e/d6f/f10f x:0 0 0 2024-05-25T06:11:45.757 INFO:tasks.workunit.client.0.smithi160.stdout:2/817: mkdir d12/d3e/d5a/d8d/df9/d114 0 2024-05-25T06:11:45.759 INFO:tasks.workunit.client.0.smithi160.stdout:2/818: chown d12/d1c/f31 8229 1 2024-05-25T06:11:45.760 INFO:tasks.workunit.client.0.smithi160.stdout:7/826: symlink d6/d2b/l110 0 2024-05-25T06:11:45.766 INFO:tasks.workunit.client.0.smithi160.stdout:7/827: creat d6/d2b/da0/dc7/f111 x:0 0 0 2024-05-25T06:11:45.779 INFO:tasks.workunit.client.0.smithi160.stdout:3/761: write d19/d2f/d38/d47/f64 [614890,45513] 0 2024-05-25T06:11:45.781 INFO:tasks.workunit.client.0.smithi160.stdout:3/762: truncate d19/d2f/d38/d47/f4c 1928735 0 2024-05-25T06:11:45.782 INFO:tasks.workunit.client.0.smithi160.stdout:2/819: write d12/d1c/d37/fdb [414099,43794] 0 2024-05-25T06:11:45.783 INFO:tasks.workunit.client.0.smithi160.stdout:3/763: write d19/d2f/d38/d47/f4c [2796043,1892] 0 2024-05-25T06:11:45.783 INFO:tasks.workunit.client.0.smithi160.stdout:2/820: read d12/d1c/d37/d4b/fab [4533015,19020] 0 2024-05-25T06:11:45.784 INFO:tasks.workunit.client.0.smithi160.stdout:7/828: write d6/dc/f19 [4936133,59175] 0 2024-05-25T06:11:45.786 INFO:tasks.workunit.client.0.smithi160.stdout:7/829: dread - d6/d32/d92/ff2 zero size 2024-05-25T06:11:45.787 INFO:tasks.workunit.client.0.smithi160.stdout:7/830: truncate d6/d32/f10d 8526221 0 2024-05-25T06:11:45.788 INFO:tasks.workunit.client.0.smithi160.stdout:3/764: creat d19/d2f/d34/da8/f100 x:0 0 0 2024-05-25T06:11:45.788 INFO:tasks.workunit.client.0.smithi160.stdout:7/831: fsync d6/d2b/d5d/fc2 0 2024-05-25T06:11:45.789 INFO:tasks.workunit.client.0.smithi160.stdout:7/832: read d6/d3e/f70 [3066999,79743] 0 2024-05-25T06:11:45.791 INFO:tasks.workunit.client.0.smithi160.stdout:7/833: readlink d6/d32/lad 0 2024-05-25T06:11:45.792 INFO:tasks.workunit.client.0.smithi160.stdout:7/834: write d6/dc/d64/d5a/fa6 [816112,127737] 0 2024-05-25T06:11:45.792 INFO:tasks.workunit.client.0.smithi160.stdout:3/765: creat d19/d2f/d38/d43/d5b/d65/d83/df0/f101 x:0 0 0 2024-05-25T06:11:45.792 INFO:tasks.workunit.client.0.smithi160.stdout:3/766: fdatasync d19/f99 0 2024-05-25T06:11:45.793 INFO:tasks.workunit.client.0.smithi160.stdout:2/821: truncate fe 1627358 0 2024-05-25T06:11:45.795 INFO:tasks.workunit.client.0.smithi160.stdout:7/835: fdatasync d6/d32/dce/fe3 0 2024-05-25T06:11:45.795 INFO:tasks.workunit.client.0.smithi160.stdout:2/822: chown d12/d75/dca/l40 7 1 2024-05-25T06:11:45.796 INFO:tasks.workunit.client.0.smithi160.stdout:3/767: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/d102 0 2024-05-25T06:11:45.799 INFO:tasks.workunit.client.0.smithi160.stdout:7/836: mknod d6/d32/d5b/d10e/c112 0 2024-05-25T06:11:45.800 INFO:tasks.workunit.client.0.smithi160.stdout:3/768: fsync f13 0 2024-05-25T06:11:45.803 INFO:tasks.workunit.client.0.smithi160.stdout:7/837: symlink d6/dc/d64/dac/l113 0 2024-05-25T06:11:45.810 INFO:tasks.workunit.client.0.smithi160.stdout:2/823: write d12/d1c/d59/ffe [732468,75170] 0 2024-05-25T06:11:45.813 INFO:tasks.workunit.client.0.smithi160.stdout:7/838: write d6/d2b/d5d/d89/fa9 [132022,121708] 0 2024-05-25T06:11:45.814 INFO:tasks.workunit.client.0.smithi160.stdout:2/824: creat d12/d3e/d5f/df3/d10e/f115 x:0 0 0 2024-05-25T06:11:45.815 INFO:tasks.workunit.client.0.smithi160.stdout:4/723: sync 2024-05-25T06:11:45.818 INFO:tasks.workunit.client.0.smithi160.stdout:3/769: dwrite d19/f88 [0,4194304] 0 2024-05-25T06:11:45.821 INFO:tasks.workunit.client.0.smithi160.stdout:2/825: creat d12/d3e/d5a/d8d/df9/d114/f116 x:0 0 0 2024-05-25T06:11:45.823 INFO:tasks.workunit.client.0.smithi160.stdout:3/770: creat d19/d22/f103 x:0 0 0 2024-05-25T06:11:45.827 INFO:tasks.workunit.client.0.smithi160.stdout:2/826: write d12/d3e/d5f/d74/f77 [4084189,22153] 0 2024-05-25T06:11:45.828 INFO:tasks.workunit.client.0.smithi160.stdout:7/839: getdents d6/d3e/da1 0 2024-05-25T06:11:45.829 INFO:tasks.workunit.client.0.smithi160.stdout:2/827: creat d12/d1c/d37/d4b/f117 x:0 0 0 2024-05-25T06:11:45.830 INFO:tasks.workunit.client.0.smithi160.stdout:7/840: truncate d6/d2b/da0/dc7/f111 395674 0 2024-05-25T06:11:45.830 INFO:tasks.workunit.client.0.smithi160.stdout:3/771: getdents d19/d2f/dad 0 2024-05-25T06:11:45.832 INFO:tasks.workunit.client.0.smithi160.stdout:7/841: mkdir d6/dc/d21/d114 0 2024-05-25T06:11:45.832 INFO:tasks.workunit.client.0.smithi160.stdout:3/772: mkdir d19/db2/d104 0 2024-05-25T06:11:45.832 INFO:tasks.workunit.client.0.smithi160.stdout:7/842: rmdir d6/dba 39 2024-05-25T06:11:45.833 INFO:tasks.workunit.client.0.smithi160.stdout:7/843: write d6/dc/d64/fb0 [736519,78515] 0 2024-05-25T06:11:45.837 INFO:tasks.workunit.client.0.smithi160.stdout:3/773: rename d19/d2f/d34/f5e to d19/d2f/f105 0 2024-05-25T06:11:45.838 INFO:tasks.workunit.client.0.smithi160.stdout:3/774: write d19/d2f/d38/d47/f6e [2951545,29116] 0 2024-05-25T06:11:45.839 INFO:tasks.workunit.client.0.smithi160.stdout:3/775: write d19/d2f/d38/d43/fdb [106467,31429] 0 2024-05-25T06:11:45.842 INFO:tasks.workunit.client.0.smithi160.stdout:7/844: rename d6/d32/l68 to d6/dc/d21/ddd/l115 0 2024-05-25T06:11:45.845 INFO:tasks.workunit.client.0.smithi160.stdout:7/845: symlink d6/d3e/da1/de7/l116 0 2024-05-25T06:11:45.850 INFO:tasks.workunit.client.0.smithi160.stdout:7/846: chown d6/dc/d21/ddd/l115 22310528 1 2024-05-25T06:11:45.851 INFO:tasks.workunit.client.0.smithi160.stdout:3/776: write d19/d22/f2e [42920,110530] 0 2024-05-25T06:11:45.852 INFO:tasks.workunit.client.0.smithi160.stdout:7/847: mkdir d6/d3e/d117 0 2024-05-25T06:11:45.856 INFO:tasks.workunit.client.0.smithi160.stdout:3/777: getdents d19/d2f/d34/d67/d96 0 2024-05-25T06:11:45.859 INFO:tasks.workunit.client.0.smithi160.stdout:7/848: dwrite f1 [0,4194304] 0 2024-05-25T06:11:45.860 INFO:tasks.workunit.client.0.smithi160.stdout:3/778: unlink d19/d2f/d38/d43/d5b/d65/ffc 0 2024-05-25T06:11:45.861 INFO:tasks.workunit.client.0.smithi160.stdout:7/849: rmdir d6/dc/d64/d5a 39 2024-05-25T06:11:45.862 INFO:tasks.workunit.client.0.smithi160.stdout:3/779: dread - d19/d2f/d38/d60/ff1 zero size 2024-05-25T06:11:45.864 INFO:tasks.workunit.client.0.smithi160.stdout:7/850: creat d6/d2b/d5d/d89/dd6/f118 x:0 0 0 2024-05-25T06:11:45.865 INFO:tasks.workunit.client.0.smithi160.stdout:3/780: creat d19/d2f/d38/d43/d5b/d65/d7e/f106 x:0 0 0 2024-05-25T06:11:45.866 INFO:tasks.workunit.client.0.smithi160.stdout:7/851: write d6/fb [361614,93780] 0 2024-05-25T06:11:45.871 INFO:tasks.workunit.client.0.smithi160.stdout:7/852: creat d6/dc/d64/d5a/d7c/f119 x:0 0 0 2024-05-25T06:11:45.873 INFO:tasks.workunit.client.0.smithi160.stdout:3/781: dwrite d19/d2f/d38/d43/d5b/d65/d7e/fe6 [0,4194304] 0 2024-05-25T06:11:45.876 INFO:tasks.workunit.client.0.smithi160.stdout:3/782: fdatasync d19/d2f/d38/d43/d5b/d65/d7e/f106 0 2024-05-25T06:11:45.881 INFO:tasks.workunit.client.0.smithi160.stdout:7/853: dwrite d6/dc/fd1 [0,4194304] 0 2024-05-25T06:11:45.884 INFO:tasks.workunit.client.0.smithi160.stdout:3/783: rename d19/da7/ld7 to d19/d2f/d38/d43/d5b/d65/d83/dee/d102/l107 0 2024-05-25T06:11:45.893 INFO:tasks.workunit.client.0.smithi160.stdout:7/854: getdents d6/d2b 0 2024-05-25T06:11:45.894 INFO:tasks.workunit.client.0.smithi160.stdout:3/784: rmdir d19/d2f/d38/d43/d5b/d65 39 2024-05-25T06:11:45.895 INFO:tasks.workunit.client.0.smithi160.stdout:3/785: write d19/d2f/d38/fb0 [712886,124434] 0 2024-05-25T06:11:45.895 INFO:tasks.workunit.client.0.smithi160.stdout:7/855: mknod d6/d3e/d59/c11a 0 2024-05-25T06:11:45.897 INFO:tasks.workunit.client.0.smithi160.stdout:3/786: fsync d19/d2f/d38/d43/d5b/d65/ff2 0 2024-05-25T06:11:45.900 INFO:tasks.workunit.client.0.smithi160.stdout:3/787: rmdir d19/d2f/d38/d43/d5b/d65/d7f 39 2024-05-25T06:11:45.905 INFO:tasks.workunit.client.0.smithi160.stdout:7/856: dwrite d6/d32/f3f [0,4194304] 0 2024-05-25T06:11:45.907 INFO:tasks.workunit.client.0.smithi160.stdout:7/857: fsync d6/d3e/d6f/f6e 0 2024-05-25T06:11:45.929 INFO:tasks.workunit.client.0.smithi160.stdout:7/858: dread d6/dc/d64/d5a/fbd [0,4194304] 0 2024-05-25T06:11:45.932 INFO:tasks.workunit.client.0.smithi160.stdout:7/859: getdents d6/dc/d64/d5a 0 2024-05-25T06:11:45.938 INFO:tasks.workunit.client.0.smithi160.stdout:7/860: dread d6/d3e/d6f/f55 [0,4194304] 0 2024-05-25T06:11:45.942 INFO:tasks.workunit.client.0.smithi160.stdout:7/861: write d6/dc/f82 [723382,4681] 0 2024-05-25T06:11:45.950 INFO:tasks.workunit.client.0.smithi160.stdout:7/862: dread d6/d2b/d5d/d89/f90 [0,4194304] 0 2024-05-25T06:11:45.990 INFO:tasks.workunit.client.0.smithi160.stdout:7/863: dwrite d6/d2b/da0/f101 [0,4194304] 0 2024-05-25T06:11:45.991 INFO:tasks.workunit.client.0.smithi160.stdout:7/864: mknod d6/d2b/d5d/d89/c11b 0 2024-05-25T06:11:45.999 INFO:tasks.workunit.client.0.smithi160.stdout:8/764: sync 2024-05-25T06:11:45.999 INFO:tasks.workunit.client.0.smithi160.stdout:1/681: sync 2024-05-25T06:11:46.001 INFO:tasks.workunit.client.0.smithi160.stdout:8/765: fdatasync d5/d44/d1f/da3/d28/d49/d61/f9f 0 2024-05-25T06:11:46.002 INFO:tasks.workunit.client.0.smithi160.stdout:1/682: symlink de/d21/d49/d1c/d52/dc7/le0 0 2024-05-25T06:11:46.010 INFO:tasks.workunit.client.0.smithi160.stdout:1/683: dwrite de/d21/d49/d1c/d59/d67/fa0 [0,4194304] 0 2024-05-25T06:11:46.011 INFO:tasks.workunit.client.0.smithi160.stdout:7/865: dread d6/d3e/d6f/f6e [0,4194304] 0 2024-05-25T06:11:46.021 INFO:tasks.workunit.client.0.smithi160.stdout:7/866: dwrite d6/d3e/d6f/f102 [0,4194304] 0 2024-05-25T06:11:46.022 INFO:tasks.workunit.client.0.smithi160.stdout:8/766: dread d5/d9/fbb [0,4194304] 0 2024-05-25T06:11:46.028 INFO:tasks.workunit.client.0.smithi160.stdout:8/767: link d5/d44/d1f/d58/cfa d5/d44/d1f/da3/d28/dc6/c102 0 2024-05-25T06:11:46.031 INFO:tasks.workunit.client.0.smithi160.stdout:8/768: mkdir d5/d7/ded/d103 0 2024-05-25T06:11:46.033 INFO:tasks.workunit.client.0.smithi160.stdout:8/769: mkdir d5/d9/d72/d104 0 2024-05-25T06:11:46.034 INFO:tasks.workunit.client.0.smithi160.stdout:1/684: dread de/d21/fa6 [0,4194304] 0 2024-05-25T06:11:46.035 INFO:tasks.workunit.client.0.smithi160.stdout:8/770: fdatasync d5/d44/d1f/f8f 0 2024-05-25T06:11:46.039 INFO:tasks.workunit.client.0.smithi160.stdout:8/771: truncate d5/d44/d1f/da3/d28/f5e 1239992 0 2024-05-25T06:11:46.039 INFO:tasks.workunit.client.0.smithi160.stdout:1/685: read de/d21/d49/d1c/d59/d71/f74 [1800547,51066] 0 2024-05-25T06:11:46.050 INFO:tasks.workunit.client.0.smithi160.stdout:7/867: dread d6/dc/d64/f38 [0,4194304] 0 2024-05-25T06:11:46.051 INFO:tasks.workunit.client.0.smithi160.stdout:1/686: dwrite de/f15 [0,4194304] 0 2024-05-25T06:11:46.054 INFO:tasks.workunit.client.0.smithi160.stdout:7/868: mknod d6/d3e/da1/de7/c11c 0 2024-05-25T06:11:46.055 INFO:tasks.workunit.client.0.smithi160.stdout:1/687: mknod de/d21/d49/dc6/ce1 0 2024-05-25T06:11:46.059 INFO:tasks.workunit.client.0.smithi160.stdout:1/688: mkdir de/d21/d49/d35/de2 0 2024-05-25T06:11:46.061 INFO:tasks.workunit.client.0.smithi160.stdout:7/869: getdents d6/d3e/d6f/d73 0 2024-05-25T06:11:46.063 INFO:tasks.workunit.client.0.smithi160.stdout:1/689: symlink de/d21/d49/d1c/d41/le3 0 2024-05-25T06:11:46.065 INFO:tasks.workunit.client.0.smithi160.stdout:1/690: fdatasync de/d21/d49/d35/f24 0 2024-05-25T06:11:46.065 INFO:tasks.workunit.client.0.smithi160.stdout:7/870: dread d6/dc/f82 [0,4194304] 0 2024-05-25T06:11:46.068 INFO:tasks.workunit.client.0.smithi160.stdout:7/871: rmdir d6/dc/d64/dae 39 2024-05-25T06:11:46.071 INFO:tasks.workunit.client.0.smithi160.stdout:7/872: rename d6/d3e/da1/ddb/cf3 to d6/d2b/d5d/d89/dd6/c11d 0 2024-05-25T06:11:46.073 INFO:tasks.workunit.client.0.smithi160.stdout:7/873: write d6/d32/f10d [2803660,70947] 0 2024-05-25T06:11:46.074 INFO:tasks.workunit.client.0.smithi160.stdout:1/691: dwrite de/d21/d49/d1c/d59/d67/fa0 [4194304,4194304] 0 2024-05-25T06:11:46.076 INFO:tasks.workunit.client.0.smithi160.stdout:1/692: truncate de/d21/d49/d35/fd6 834656 0 2024-05-25T06:11:46.079 INFO:tasks.workunit.client.0.smithi160.stdout:7/874: dread d6/d2b/f7a [0,4194304] 0 2024-05-25T06:11:46.081 INFO:tasks.workunit.client.0.smithi160.stdout:1/693: getdents de/d21/d49/d1c/d59/d67 0 2024-05-25T06:11:46.085 INFO:tasks.workunit.client.0.smithi160.stdout:1/694: creat de/d21/d49/d1c/d52/dc7/ddb/fe4 x:0 0 0 2024-05-25T06:11:46.086 INFO:tasks.workunit.client.0.smithi160.stdout:7/875: dwrite d6/dc/f25 [0,4194304] 0 2024-05-25T06:11:46.087 INFO:tasks.workunit.client.0.smithi160.stdout:1/695: write de/d21/d49/d1c/d59/d67/f83 [45355,19456] 0 2024-05-25T06:11:46.089 INFO:tasks.workunit.client.0.smithi160.stdout:7/876: mknod d6/d2b/dcc/c11e 0 2024-05-25T06:11:46.091 INFO:tasks.workunit.client.0.smithi160.stdout:1/696: truncate de/d21/d49/d28/f3c 4923108 0 2024-05-25T06:11:46.091 INFO:tasks.workunit.client.0.smithi160.stdout:7/877: chown d6/d2b/da0/dc7/lf7 25857 1 2024-05-25T06:11:46.095 INFO:tasks.workunit.client.0.smithi160.stdout:7/878: mknod d6/d32/c11f 0 2024-05-25T06:11:46.096 INFO:tasks.workunit.client.0.smithi160.stdout:1/697: symlink de/d21/d49/d1c/d41/d45/dac/dd3/le5 0 2024-05-25T06:11:46.096 INFO:tasks.workunit.client.0.smithi160.stdout:7/879: chown d6/d3e/d59/l94 3288506 1 2024-05-25T06:11:46.098 INFO:tasks.workunit.client.0.smithi160.stdout:7/880: chown d6/dc/c14 2835 1 2024-05-25T06:11:46.101 INFO:tasks.workunit.client.0.smithi160.stdout:1/698: dread de/d21/d49/d1c/fa5 [0,4194304] 0 2024-05-25T06:11:46.102 INFO:tasks.workunit.client.0.smithi160.stdout:1/699: write de/d21/d49/d35/f24 [1676453,91387] 0 2024-05-25T06:11:46.103 INFO:tasks.workunit.client.0.smithi160.stdout:1/700: readlink de/d21/d49/d1c/d41/d45/l78 0 2024-05-25T06:11:46.107 INFO:tasks.workunit.client.0.smithi160.stdout:1/701: creat de/d21/d49/d1c/d41/fe6 x:0 0 0 2024-05-25T06:11:46.109 INFO:tasks.workunit.client.0.smithi160.stdout:7/881: dwrite d6/d2b/f6b [0,4194304] 0 2024-05-25T06:11:46.110 INFO:tasks.workunit.client.0.smithi160.stdout:1/702: mknod de/d21/d49/d1c/d41/ce7 0 2024-05-25T06:11:46.114 INFO:tasks.workunit.client.0.smithi160.stdout:7/882: rename d6/d3e/d6f/d73/l74 to d6/d3e/d59/d65/l120 0 2024-05-25T06:11:46.115 INFO:tasks.workunit.client.0.smithi160.stdout:1/703: mkdir de/d21/d49/d1c/d59/d7c/d93/de8 0 2024-05-25T06:11:46.120 INFO:tasks.workunit.client.0.smithi160.stdout:1/704: dwrite f9 [0,4194304] 0 2024-05-25T06:11:46.123 INFO:tasks.workunit.client.0.smithi160.stdout:1/705: fsync de/f13 0 2024-05-25T06:11:46.134 INFO:tasks.workunit.client.0.smithi160.stdout:7/883: dread d6/d2b/d5d/fc2 [0,4194304] 0 2024-05-25T06:11:46.135 INFO:tasks.workunit.client.0.smithi160.stdout:7/884: stat d6/d2b/dfd 0 2024-05-25T06:11:46.135 INFO:tasks.workunit.client.0.smithi160.stdout:1/706: write de/d21/d49/d1c/d41/d96/fa8 [1520493,92453] 0 2024-05-25T06:11:46.137 INFO:tasks.workunit.client.0.smithi160.stdout:1/707: fsync de/d21/d49/d35/fd6 0 2024-05-25T06:11:46.137 INFO:tasks.workunit.client.0.smithi160.stdout:7/885: creat d6/d2b/dfd/f121 x:0 0 0 2024-05-25T06:11:46.139 INFO:tasks.workunit.client.0.smithi160.stdout:1/708: symlink de/d21/d49/d1c/d41/d45/d7d/le9 0 2024-05-25T06:11:46.141 INFO:tasks.workunit.client.0.smithi160.stdout:1/709: chown de/d21/d49/d1c/d59/d7f/faa 26048 1 2024-05-25T06:11:46.144 INFO:tasks.workunit.client.0.smithi160.stdout:1/710: creat de/d21/fea x:0 0 0 2024-05-25T06:11:46.145 INFO:tasks.workunit.client.0.smithi160.stdout:1/711: read - de/d21/f7a zero size 2024-05-25T06:11:46.145 INFO:tasks.workunit.client.0.smithi160.stdout:7/886: dwrite d6/d2b/d5d/fc2 [0,4194304] 0 2024-05-25T06:11:46.146 INFO:tasks.workunit.client.0.smithi160.stdout:7/887: write d6/dc/d21/f4f [3190431,68152] 0 2024-05-25T06:11:46.151 INFO:tasks.workunit.client.0.smithi160.stdout:0/846: sync 2024-05-25T06:11:46.152 INFO:tasks.workunit.client.0.smithi160.stdout:7/888: rename d6/d3e/d6f/d73/c7d to d6/c122 0 2024-05-25T06:11:46.155 INFO:tasks.workunit.client.0.smithi160.stdout:7/889: unlink d6/d32/d5b/cb5 0 2024-05-25T06:11:46.156 INFO:tasks.workunit.client.0.smithi160.stdout:1/712: write de/d21/d49/f82 [4498532,114973] 0 2024-05-25T06:11:46.160 INFO:tasks.workunit.client.0.smithi160.stdout:1/713: rename de/d21/d49/d35/fd6 to de/d21/d49/d90/feb 0 2024-05-25T06:11:46.164 INFO:tasks.workunit.client.0.smithi160.stdout:1/714: mkdir de/dec 0 2024-05-25T06:11:46.166 INFO:tasks.workunit.client.0.smithi160.stdout:1/715: dread de/d21/d49/d28/f7e [0,4194304] 0 2024-05-25T06:11:46.167 INFO:tasks.workunit.client.0.smithi160.stdout:0/847: truncate d2b/d31/d35/d36/d37/f3b 1019022 0 2024-05-25T06:11:46.171 INFO:tasks.workunit.client.0.smithi160.stdout:0/848: dread f1f [0,4194304] 0 2024-05-25T06:11:46.173 INFO:tasks.workunit.client.0.smithi160.stdout:0/849: write d2b/d46/fdc [174705,104470] 0 2024-05-25T06:11:46.174 INFO:tasks.workunit.client.0.smithi160.stdout:1/716: dread de/f10 [0,4194304] 0 2024-05-25T06:11:46.175 INFO:tasks.workunit.client.0.smithi160.stdout:7/890: dwrite d6/d32/d5b/fbe [0,4194304] 0 2024-05-25T06:11:46.176 INFO:tasks.workunit.client.0.smithi160.stdout:0/850: truncate d2b/d31/d35/fc9 1313648 0 2024-05-25T06:11:46.182 INFO:tasks.workunit.client.0.smithi160.stdout:0/851: write d2b/d31/fe0 [2755492,31137] 0 2024-05-25T06:11:46.184 INFO:tasks.workunit.client.0.smithi160.stdout:7/891: rmdir d6/d3e/d117 0 2024-05-25T06:11:46.187 INFO:tasks.workunit.client.0.smithi160.stdout:1/717: dwrite de/d21/d49/d28/f7e [0,4194304] 0 2024-05-25T06:11:46.197 INFO:tasks.workunit.client.0.smithi160.stdout:7/892: write d6/d32/d7e/de0/fe5 [1374090,69847] 0 2024-05-25T06:11:46.200 INFO:tasks.workunit.client.0.smithi160.stdout:1/718: dwrite de/d21/d49/f82 [4194304,4194304] 0 2024-05-25T06:11:46.204 INFO:tasks.workunit.client.0.smithi160.stdout:0/852: dwrite d2b/d46/f91 [0,4194304] 0 2024-05-25T06:11:46.209 INFO:tasks.workunit.client.0.smithi160.stdout:1/719: creat de/d21/d49/d1c/d41/d45/d7d/db3/fed x:0 0 0 2024-05-25T06:11:46.212 INFO:tasks.workunit.client.0.smithi160.stdout:1/720: dread - de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/fb0 zero size 2024-05-25T06:11:46.217 INFO:tasks.workunit.client.0.smithi160.stdout:1/721: rename de/d21/d49/d35/f42 to de/d21/d49/d1c/d41/fee 0 2024-05-25T06:11:46.218 INFO:tasks.workunit.client.0.smithi160.stdout:1/722: readlink de/d21/d49/d1c/d41/d45/d7d/db3/ld2 0 2024-05-25T06:11:46.219 INFO:tasks.workunit.client.0.smithi160.stdout:1/723: stat de/d21/d49/l6b 0 2024-05-25T06:11:46.220 INFO:tasks.workunit.client.0.smithi160.stdout:1/724: write de/f54 [642637,96779] 0 2024-05-25T06:11:46.223 INFO:tasks.workunit.client.0.smithi160.stdout:7/893: dread d6/fb [4194304,4194304] 0 2024-05-25T06:11:46.223 INFO:tasks.workunit.client.0.smithi160.stdout:1/725: creat de/d21/d49/d1c/d41/d45/d7d/db3/fef x:0 0 0 2024-05-25T06:11:46.224 INFO:tasks.workunit.client.0.smithi160.stdout:7/894: chown d6/dc/d64/f45 208018693 1 2024-05-25T06:11:46.226 INFO:tasks.workunit.client.0.smithi160.stdout:7/895: write d6/d2b/f6b [1074902,120957] 0 2024-05-25T06:11:46.226 INFO:tasks.workunit.client.0.smithi160.stdout:1/726: rename de/dec to de/d21/d49/d28/df0 0 2024-05-25T06:11:46.227 INFO:tasks.workunit.client.0.smithi160.stdout:7/896: dread - d6/dc/d21/ff9 zero size 2024-05-25T06:11:46.229 INFO:tasks.workunit.client.0.smithi160.stdout:1/727: chown de/d21/d49/d1c/d52/f56 63869508 1 2024-05-25T06:11:46.229 INFO:tasks.workunit.client.0.smithi160.stdout:0/853: dread d2b/f78 [0,4194304] 0 2024-05-25T06:11:46.231 INFO:tasks.workunit.client.0.smithi160.stdout:7/897: rename d6/dc/d64/d5a/d7c/l69 to d6/dc/d64/d5a/d7c/l123 0 2024-05-25T06:11:46.233 INFO:tasks.workunit.client.0.smithi160.stdout:0/854: creat d2b/d46/d63/db3/dd6/f10f x:0 0 0 2024-05-25T06:11:46.233 INFO:tasks.workunit.client.0.smithi160.stdout:7/898: chown d6/d3e/d59/f66 53755354 1 2024-05-25T06:11:46.234 INFO:tasks.workunit.client.0.smithi160.stdout:0/855: truncate d2b/da8/fc3 127238 0 2024-05-25T06:11:46.236 INFO:tasks.workunit.client.0.smithi160.stdout:1/728: dwrite de/d21/d49/d28/fad [0,4194304] 0 2024-05-25T06:11:46.238 INFO:tasks.workunit.client.0.smithi160.stdout:0/856: mkdir d2b/d46/d63/da7/da6/d110 0 2024-05-25T06:11:46.240 INFO:tasks.workunit.client.0.smithi160.stdout:0/857: symlink d2b/da8/db2/l111 0 2024-05-25T06:11:46.241 INFO:tasks.workunit.client.0.smithi160.stdout:0/858: fsync d2b/da8/fb0 0 2024-05-25T06:11:46.242 INFO:tasks.workunit.client.0.smithi160.stdout:1/729: mknod de/d21/d49/d1c/d41/cf1 0 2024-05-25T06:11:46.246 INFO:tasks.workunit.client.0.smithi160.stdout:0/859: creat d2b/d46/d63/da7/da6/f112 x:0 0 0 2024-05-25T06:11:46.253 INFO:tasks.workunit.client.0.smithi160.stdout:1/730: dwrite de/d21/d49/f2b [4194304,4194304] 0 2024-05-25T06:11:46.254 INFO:tasks.workunit.client.0.smithi160.stdout:1/731: write de/d21/d49/d1c/d59/d7c/f97 [1373649,80024] 0 2024-05-25T06:11:46.258 INFO:tasks.workunit.client.0.smithi160.stdout:0/860: truncate d2b/d31/d35/d36/d86/f66 712628 0 2024-05-25T06:11:46.260 INFO:tasks.workunit.client.0.smithi160.stdout:1/732: creat de/d21/d49/ff2 x:0 0 0 2024-05-25T06:11:46.261 INFO:tasks.workunit.client.0.smithi160.stdout:1/733: fsync de/d21/d49/d1c/d59/f95 0 2024-05-25T06:11:46.265 INFO:tasks.workunit.client.0.smithi160.stdout:1/734: dread de/d21/d49/d1c/d59/d67/fa0 [4194304,4194304] 0 2024-05-25T06:11:46.266 INFO:tasks.workunit.client.0.smithi160.stdout:1/735: chown de/d21/d49/d1c/d41/fe6 111803077 1 2024-05-25T06:11:46.267 INFO:tasks.workunit.client.0.smithi160.stdout:0/861: getdents d2b/d46/d63/da7/da6 0 2024-05-25T06:11:46.267 INFO:tasks.workunit.client.0.smithi160.stdout:0/862: chown c28 49 1 2024-05-25T06:11:46.268 INFO:tasks.workunit.client.0.smithi160.stdout:0/863: readlink d2b/d46/d63/da7/db5/lfe 0 2024-05-25T06:11:46.268 INFO:tasks.workunit.client.0.smithi160.stdout:1/736: symlink de/d21/d49/d1c/d52/dc7/ddb/lf3 0 2024-05-25T06:11:46.269 INFO:tasks.workunit.client.0.smithi160.stdout:0/864: readlink d2b/d46/d63/lf9 0 2024-05-25T06:11:46.270 INFO:tasks.workunit.client.0.smithi160.stdout:0/865: chown f12 35 1 2024-05-25T06:11:46.273 INFO:tasks.workunit.client.0.smithi160.stdout:0/866: creat d2b/d46/d63/da7/d67/d6a/df7/f113 x:0 0 0 2024-05-25T06:11:46.275 INFO:tasks.workunit.client.0.smithi160.stdout:0/867: read d2b/d31/d35/d43/d98/f99 [1042496,30402] 0 2024-05-25T06:11:46.279 INFO:tasks.workunit.client.0.smithi160.stdout:1/737: write de/f4d [1629847,114106] 0 2024-05-25T06:11:46.284 INFO:tasks.workunit.client.0.smithi160.stdout:1/738: rename de/d21/d49/l5e to de/d39/lf4 0 2024-05-25T06:11:46.295 INFO:tasks.workunit.client.0.smithi160.stdout:1/739: dwrite de/d21/d49/d1c/d41/d96/fa8 [0,4194304] 0 2024-05-25T06:11:46.297 INFO:tasks.workunit.client.0.smithi160.stdout:1/740: creat de/d21/d49/d1c/d41/d45/d7d/db3/dba/ff5 x:0 0 0 2024-05-25T06:11:46.298 INFO:tasks.workunit.client.0.smithi160.stdout:1/741: write de/d21/d49/ff2 [967636,85171] 0 2024-05-25T06:11:46.301 INFO:tasks.workunit.client.0.smithi160.stdout:0/868: dread d2b/d46/d63/da7/f8e [0,4194304] 0 2024-05-25T06:11:46.302 INFO:tasks.workunit.client.0.smithi160.stdout:1/742: mkdir de/d21/d49/d1c/d52/dc7/df6 0 2024-05-25T06:11:46.303 INFO:tasks.workunit.client.0.smithi160.stdout:0/869: mknod d2b/d46/d63/da7/da6/d110/c114 0 2024-05-25T06:11:46.306 INFO:tasks.workunit.client.0.smithi160.stdout:1/743: link f7 de/d21/d49/d1c/d41/d45/dac/ff7 0 2024-05-25T06:11:46.307 INFO:tasks.workunit.client.0.smithi160.stdout:0/870: symlink d2b/d46/d63/da7/d67/d6a/df7/l115 0 2024-05-25T06:11:46.308 INFO:tasks.workunit.client.0.smithi160.stdout:1/744: creat de/d21/d49/d1c/d52/dc7/ddb/ff8 x:0 0 0 2024-05-25T06:11:46.308 INFO:tasks.workunit.client.0.smithi160.stdout:0/871: write d2b/d46/d63/da7/d67/fca [521114,81959] 0 2024-05-25T06:11:46.310 INFO:tasks.workunit.client.0.smithi160.stdout:1/745: mknod de/d21/d49/d28/df0/cf9 0 2024-05-25T06:11:46.312 INFO:tasks.workunit.client.0.smithi160.stdout:0/872: getdents d2b/da8 0 2024-05-25T06:11:46.312 INFO:tasks.workunit.client.0.smithi160.stdout:1/746: dread de/d21/d49/d90/feb [0,4194304] 0 2024-05-25T06:11:46.316 INFO:tasks.workunit.client.0.smithi160.stdout:0/873: dread d2b/d46/d63/da7/da6/fbe [0,4194304] 0 2024-05-25T06:11:46.317 INFO:tasks.workunit.client.0.smithi160.stdout:1/747: symlink de/d21/d49/d1c/lfa 0 2024-05-25T06:11:46.328 INFO:tasks.workunit.client.0.smithi160.stdout:5/853: sync 2024-05-25T06:11:46.334 INFO:tasks.workunit.client.0.smithi160.stdout:0/874: link d2b/d46/d63/da7/d67/d6a/fcf d2b/d46/d63/da7/d67/d6a/df7/f116 0 2024-05-25T06:11:46.335 INFO:tasks.workunit.client.0.smithi160.stdout:1/748: link de/d21/d49/d1c/d59/d7f/f8e de/d21/d49/d1c/d41/d45/ffb 0 2024-05-25T06:11:46.336 INFO:tasks.workunit.client.0.smithi160.stdout:0/875: creat d2b/d46/dcb/f117 x:0 0 0 2024-05-25T06:11:46.338 INFO:tasks.workunit.client.0.smithi160.stdout:5/854: dwrite db/dc/d13/d1d/d23/d24/daf/db3/fd6 [0,4194304] 0 2024-05-25T06:11:46.339 INFO:tasks.workunit.client.0.smithi160.stdout:5/855: dread - db/dc/d13/d1d/d3f/fe9 zero size 2024-05-25T06:11:46.339 INFO:tasks.workunit.client.0.smithi160.stdout:0/876: mknod d2b/d46/d63/da7/da6/c118 0 2024-05-25T06:11:46.342 INFO:tasks.workunit.client.0.smithi160.stdout:1/749: creat de/d21/d49/d1c/d41/ffc x:0 0 0 2024-05-25T06:11:46.342 INFO:tasks.workunit.client.0.smithi160.stdout:0/877: rmdir d2b/d46/d63/da7 39 2024-05-25T06:11:46.345 INFO:tasks.workunit.client.0.smithi160.stdout:0/878: dread - d2b/d31/d35/d36/d86/f5b zero size 2024-05-25T06:11:46.347 INFO:tasks.workunit.client.0.smithi160.stdout:5/856: dread db/dc/d13/d1d/d23/d45/fe3 [0,4194304] 0 2024-05-25T06:11:46.348 INFO:tasks.workunit.client.0.smithi160.stdout:1/750: rename de/d21/d49/f2b to de/d21/d49/d28/df0/ffd 0 2024-05-25T06:11:46.351 INFO:tasks.workunit.client.0.smithi160.stdout:5/857: mknod db/d47/d5e/d6e/c100 0 2024-05-25T06:11:46.352 INFO:tasks.workunit.client.0.smithi160.stdout:1/751: mkdir de/d21/d49/d1c/d59/d71/dfe 0 2024-05-25T06:11:46.353 INFO:tasks.workunit.client.0.smithi160.stdout:1/752: fsync de/f13 0 2024-05-25T06:11:46.355 INFO:tasks.workunit.client.0.smithi160.stdout:0/879: link d2b/d46/d63/da7/da6/ff0 d2b/d31/d35/d43/d98/f119 0 2024-05-25T06:11:46.356 INFO:tasks.workunit.client.0.smithi160.stdout:0/880: chown d2b/d46/lbb 417 1 2024-05-25T06:11:46.358 INFO:tasks.workunit.client.0.smithi160.stdout:5/858: dwrite db/dc/d13/d1d/d23/dbd/ff0 [0,4194304] 0 2024-05-25T06:11:46.363 INFO:tasks.workunit.client.0.smithi160.stdout:0/881: symlink d2b/d31/d35/d43/dab/l11a 0 2024-05-25T06:11:46.368 INFO:tasks.workunit.client.0.smithi160.stdout:5/859: dwrite db/dc/d13/d1d/d3f/d4a/ff2 [0,4194304] 0 2024-05-25T06:11:46.371 INFO:tasks.workunit.client.0.smithi160.stdout:0/882: dwrite d2b/d31/d35/d36/fd3 [4194304,4194304] 0 2024-05-25T06:11:46.375 INFO:tasks.workunit.client.0.smithi160.stdout:5/860: mkdir db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101 0 2024-05-25T06:11:46.378 INFO:tasks.workunit.client.0.smithi160.stdout:0/883: chown d2b/f57 80556 1 2024-05-25T06:11:46.379 INFO:tasks.workunit.client.0.smithi160.stdout:1/753: truncate de/d21/d49/d1c/d59/d7f/da1/dbd/f7b 155259 0 2024-05-25T06:11:46.381 INFO:tasks.workunit.client.0.smithi160.stdout:0/884: mknod d2b/d46/d63/da7/da6/dcd/dd8/c11b 0 2024-05-25T06:11:46.382 INFO:tasks.workunit.client.0.smithi160.stdout:1/754: creat de/d39/d8f/fff x:0 0 0 2024-05-25T06:11:46.383 INFO:tasks.workunit.client.0.smithi160.stdout:5/861: creat db/dc/f102 x:0 0 0 2024-05-25T06:11:46.384 INFO:tasks.workunit.client.0.smithi160.stdout:0/885: creat d2b/d46/dbd/f11c x:0 0 0 2024-05-25T06:11:46.385 INFO:tasks.workunit.client.0.smithi160.stdout:1/755: mknod de/d21/d49/d1c/d59/d71/dfe/c100 0 2024-05-25T06:11:46.386 INFO:tasks.workunit.client.0.smithi160.stdout:1/756: rename de/d21 to de/d21/d49/d1c/d59/d7c/d93/d101 22 2024-05-25T06:11:46.389 INFO:tasks.workunit.client.0.smithi160.stdout:1/757: write de/d21/d49/d1c/d52/dc7/ddb/fe4 [762811,105660] 0 2024-05-25T06:11:46.391 INFO:tasks.workunit.client.0.smithi160.stdout:0/886: dwrite d2b/d31/d35/d36/fd3 [0,4194304] 0 2024-05-25T06:11:46.394 INFO:tasks.workunit.client.0.smithi160.stdout:1/758: rename de/f10 to de/d21/d49/d28/f102 0 2024-05-25T06:11:46.397 INFO:tasks.workunit.client.0.smithi160.stdout:1/759: creat de/d21/d49/d28/df0/f103 x:0 0 0 2024-05-25T06:11:46.402 INFO:tasks.workunit.client.0.smithi160.stdout:1/760: unlink de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/cd8 0 2024-05-25T06:11:46.403 INFO:tasks.workunit.client.0.smithi160.stdout:5/862: dwrite db/dc/d13/d1d/d23/ffe [0,4194304] 0 2024-05-25T06:11:46.403 INFO:tasks.workunit.client.0.smithi160.stdout:1/761: write de/d21/d49/f6f [405414,2157] 0 2024-05-25T06:11:46.405 INFO:tasks.workunit.client.0.smithi160.stdout:5/863: write db/dc/fd7 [1053332,9809] 0 2024-05-25T06:11:46.408 INFO:tasks.workunit.client.0.smithi160.stdout:1/762: creat de/d21/d49/f104 x:0 0 0 2024-05-25T06:11:46.413 INFO:tasks.workunit.client.0.smithi160.stdout:5/864: dread db/d47/fcb [0,4194304] 0 2024-05-25T06:11:46.415 INFO:tasks.workunit.client.0.smithi160.stdout:5/865: stat db/dc/d13/d1d/d31/d56/f61 0 2024-05-25T06:11:46.417 INFO:tasks.workunit.client.0.smithi160.stdout:0/887: dread f2 [0,4194304] 0 2024-05-25T06:11:46.417 INFO:tasks.workunit.client.0.smithi160.stdout:5/866: chown db/dc/d13/d1d/d23/d24/daf/ld4 2123094 1 2024-05-25T06:11:46.419 INFO:tasks.workunit.client.0.smithi160.stdout:5/867: write db/d47/f68 [7492561,91770] 0 2024-05-25T06:11:46.421 INFO:tasks.workunit.client.0.smithi160.stdout:5/868: stat db/dc/d13/d1d/d3f/d4a/ff2 0 2024-05-25T06:11:46.426 INFO:tasks.workunit.client.0.smithi160.stdout:5/869: unlink db/f22 0 2024-05-25T06:11:46.426 INFO:tasks.workunit.client.0.smithi160.stdout:1/763: dwrite de/d21/d49/d1c/d41/d45/ffb [0,4194304] 0 2024-05-25T06:11:46.426 INFO:tasks.workunit.client.0.smithi160.stdout:0/888: dwrite d2b/d31/d35/d36/d37/f95 [0,4194304] 0 2024-05-25T06:11:46.428 INFO:tasks.workunit.client.0.smithi160.stdout:5/870: readlink db/dc/d13/d1d/d23/d24/d2e/lb9 0 2024-05-25T06:11:46.429 INFO:tasks.workunit.client.0.smithi160.stdout:0/889: write d2b/d46/dbd/f11c [950847,82732] 0 2024-05-25T06:11:46.435 INFO:tasks.workunit.client.0.smithi160.stdout:5/871: getdents db/dc/d13/d1d/d3f/d72/dc5 0 2024-05-25T06:11:46.439 INFO:tasks.workunit.client.0.smithi160.stdout:6/769: sync 2024-05-25T06:11:46.444 INFO:tasks.workunit.client.0.smithi160.stdout:5/872: dwrite db/dc/d13/d1d/d23/d24/daf/f90 [0,4194304] 0 2024-05-25T06:11:46.470 INFO:tasks.workunit.client.0.smithi160.stdout:6/770: dwrite d5/de1/fc7 [0,4194304] 0 2024-05-25T06:11:46.471 INFO:tasks.workunit.client.0.smithi160.stdout:0/890: dread d2b/da8/fd9 [0,4194304] 0 2024-05-25T06:11:46.472 INFO:tasks.workunit.client.0.smithi160.stdout:5/873: creat db/dc/d13/d9a/f103 x:0 0 0 2024-05-25T06:11:46.473 INFO:tasks.workunit.client.0.smithi160.stdout:6/771: mknod d5/d30/d36/d73/d87/da4/c108 0 2024-05-25T06:11:46.474 INFO:tasks.workunit.client.0.smithi160.stdout:0/891: mknod d2b/d46/d63/db3/c11d 0 2024-05-25T06:11:46.476 INFO:tasks.workunit.client.0.smithi160.stdout:5/874: mkdir db/dc/d13/d1d/d3f/dc0/d104 0 2024-05-25T06:11:46.476 INFO:tasks.workunit.client.0.smithi160.stdout:0/892: write d2b/d46/dbd/f11c [916516,123881] 0 2024-05-25T06:11:46.477 INFO:tasks.workunit.client.0.smithi160.stdout:6/772: write d5/d30/fc6 [1869278,39971] 0 2024-05-25T06:11:46.483 INFO:tasks.workunit.client.0.smithi160.stdout:0/893: rename d2b/d31/d35/d36/d86/dd5/ldd to d2b/da8/db2/l11e 0 2024-05-25T06:11:46.486 INFO:tasks.workunit.client.0.smithi160.stdout:6/773: mkdir d5/de/d32/d35/dab/d109 0 2024-05-25T06:11:46.486 INFO:tasks.workunit.client.0.smithi160.stdout:5/875: rename db/d47/lf3 to db/dc/d13/d1d/d31/l105 0 2024-05-25T06:11:46.487 INFO:tasks.workunit.client.0.smithi160.stdout:0/894: creat d2b/d46/d63/da7/d67/f11f x:0 0 0 2024-05-25T06:11:46.489 INFO:tasks.workunit.client.0.smithi160.stdout:5/876: creat db/dc/d13/d1d/d31/dc6/f106 x:0 0 0 2024-05-25T06:11:46.490 INFO:tasks.workunit.client.0.smithi160.stdout:6/774: unlink d5/d30/c4f 0 2024-05-25T06:11:46.491 INFO:tasks.workunit.client.0.smithi160.stdout:6/775: chown d5/d45/d4a/lad 11 1 2024-05-25T06:11:46.492 INFO:tasks.workunit.client.0.smithi160.stdout:0/895: creat d2b/da8/db2/f120 x:0 0 0 2024-05-25T06:11:46.494 INFO:tasks.workunit.client.0.smithi160.stdout:6/776: readlink d5/d39/d8f/d2b/l7c 0 2024-05-25T06:11:46.496 INFO:tasks.workunit.client.0.smithi160.stdout:6/777: creat d5/d39/da6/dac/f10a x:0 0 0 2024-05-25T06:11:46.505 INFO:tasks.workunit.client.0.smithi160.stdout:6/778: truncate d5/d45/d4a/fbd 4179749 0 2024-05-25T06:11:46.509 INFO:tasks.workunit.client.0.smithi160.stdout:0/896: dwrite d2b/d31/f8f [0,4194304] 0 2024-05-25T06:11:46.514 INFO:tasks.workunit.client.0.smithi160.stdout:6/779: mkdir d5/d39/da6/dac/de6/d10b 0 2024-05-25T06:11:46.514 INFO:tasks.workunit.client.0.smithi160.stdout:5/877: dread f6 [4194304,4194304] 0 2024-05-25T06:11:46.515 INFO:tasks.workunit.client.0.smithi160.stdout:0/897: dread d2b/f78 [0,4194304] 0 2024-05-25T06:11:46.516 INFO:tasks.workunit.client.0.smithi160.stdout:0/898: readlink lf 0 2024-05-25T06:11:46.520 INFO:tasks.workunit.client.0.smithi160.stdout:6/780: creat d5/d39/da6/f10c x:0 0 0 2024-05-25T06:11:46.520 INFO:tasks.workunit.client.0.smithi160.stdout:0/899: fdatasync d2b/d46/d63/da7/f60 0 2024-05-25T06:11:46.538 INFO:tasks.workunit.client.0.smithi160.stdout:0/900: dwrite d2b/d31/d35/d36/d86/fae [0,4194304] 0 2024-05-25T06:11:46.544 INFO:tasks.workunit.client.0.smithi160.stdout:6/781: dread d5/de/d7d/fcb [0,4194304] 0 2024-05-25T06:11:46.550 INFO:tasks.workunit.client.0.smithi160.stdout:0/901: truncate d2b/d31/d35/f3e 209479 0 2024-05-25T06:11:46.552 INFO:tasks.workunit.client.0.smithi160.stdout:6/782: dwrite d5/de/d32/d35/f4e [0,4194304] 0 2024-05-25T06:11:46.554 INFO:tasks.workunit.client.0.smithi160.stdout:0/902: mknod d2b/d46/d63/da7/db5/c121 0 2024-05-25T06:11:46.559 INFO:tasks.workunit.client.0.smithi160.stdout:0/903: dread d2b/d31/d35/d36/d37/f95 [4194304,4194304] 0 2024-05-25T06:11:46.560 INFO:tasks.workunit.client.0.smithi160.stdout:6/783: link d5/de/f22 d5/de/d9e/dcd/f10d 0 2024-05-25T06:11:46.565 INFO:tasks.workunit.client.0.smithi160.stdout:6/784: creat d5/de/d32/f10e x:0 0 0 2024-05-25T06:11:46.567 INFO:tasks.workunit.client.0.smithi160.stdout:6/785: creat d5/d39/f10f x:0 0 0 2024-05-25T06:11:46.569 INFO:tasks.workunit.client.0.smithi160.stdout:6/786: readlink d5/de/l9c 0 2024-05-25T06:11:46.577 INFO:tasks.workunit.client.0.smithi160.stdout:6/787: dwrite d5/d30/d36/d73/d87/ff6 [0,4194304] 0 2024-05-25T06:11:46.581 INFO:tasks.workunit.client.0.smithi160.stdout:0/904: dread d2b/d31/d35/d7b/fa0 [0,4194304] 0 2024-05-25T06:11:46.582 INFO:tasks.workunit.client.0.smithi160.stdout:6/788: symlink d5/d39/l110 0 2024-05-25T06:11:46.583 INFO:tasks.workunit.client.0.smithi160.stdout:6/789: chown d5/de/fd 7114813 1 2024-05-25T06:11:46.587 INFO:tasks.workunit.client.0.smithi160.stdout:6/790: creat d5/d30/d36/f111 x:0 0 0 2024-05-25T06:11:46.593 INFO:tasks.workunit.client.0.smithi160.stdout:0/905: dwrite d2b/d31/d35/d43/d98/f105 [0,4194304] 0 2024-05-25T06:11:46.597 INFO:tasks.workunit.client.0.smithi160.stdout:6/791: dwrite d5/de1/fc7 [0,4194304] 0 2024-05-25T06:11:46.598 INFO:tasks.workunit.client.0.smithi160.stdout:0/906: creat d2b/d46/d63/da7/da6/da9/f122 x:0 0 0 2024-05-25T06:11:46.602 INFO:tasks.workunit.client.0.smithi160.stdout:0/907: mknod d2b/d46/d63/db3/dd6/c123 0 2024-05-25T06:11:46.607 INFO:tasks.workunit.client.0.smithi160.stdout:6/792: dwrite d5/de/f17 [20971520,4194304] 0 2024-05-25T06:11:46.607 INFO:tasks.workunit.client.0.smithi160.stdout:0/908: symlink d2b/d31/l124 0 2024-05-25T06:11:46.614 INFO:tasks.workunit.client.0.smithi160.stdout:0/909: rename d2b/da8/fc3 to d2b/d46/d63/da7/d67/f125 0 2024-05-25T06:11:46.618 INFO:tasks.workunit.client.0.smithi160.stdout:6/793: dwrite d5/de/d32/d35/dc3/fd0 [0,4194304] 0 2024-05-25T06:11:46.618 INFO:tasks.workunit.client.0.smithi160.stdout:0/910: mkdir d2b/d46/d63/da7/da6/da9/d126 0 2024-05-25T06:11:46.621 INFO:tasks.workunit.client.0.smithi160.stdout:9/682: sync 2024-05-25T06:11:46.624 INFO:tasks.workunit.client.0.smithi160.stdout:9/683: mknod d4/db2/cdc 0 2024-05-25T06:11:46.625 INFO:tasks.workunit.client.0.smithi160.stdout:0/911: getdents d2b/d46/dcb 0 2024-05-25T06:11:46.627 INFO:tasks.workunit.client.0.smithi160.stdout:9/684: chown d4/d11/d1e/d3e/d25/d5e/da2/fd0 230 1 2024-05-25T06:11:46.629 INFO:tasks.workunit.client.0.smithi160.stdout:0/912: truncate d2b/d46/d63/da7/d67/d6a/fcf 983477 0 2024-05-25T06:11:46.632 INFO:tasks.workunit.client.0.smithi160.stdout:0/913: mknod d2b/d31/d35/d43/dab/dc6/c127 0 2024-05-25T06:11:46.640 INFO:tasks.workunit.client.0.smithi160.stdout:9/685: dwrite d4/d11/d14/f53 [0,4194304] 0 2024-05-25T06:11:46.645 INFO:tasks.workunit.client.0.smithi160.stdout:0/914: dwrite d2b/d31/d35/d36/f89 [0,4194304] 0 2024-05-25T06:11:46.647 INFO:tasks.workunit.client.0.smithi160.stdout:6/794: dread d5/d30/f91 [0,4194304] 0 2024-05-25T06:11:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:46 smithi203 ceph-mon[36030]: pgmap v23: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 109 MiB/s rd, 279 MiB/s wr, 551 op/s 2024-05-25T06:11:46.650 INFO:tasks.workunit.client.0.smithi160.stdout:0/915: unlink d2b/f30 0 2024-05-25T06:11:46.652 INFO:tasks.workunit.client.0.smithi160.stdout:9/686: write d4/d11/d7a/f7b [185746,124181] 0 2024-05-25T06:11:46.654 INFO:tasks.workunit.client.0.smithi160.stdout:2/828: sync 2024-05-25T06:11:46.654 INFO:tasks.workunit.client.0.smithi160.stdout:4/724: sync 2024-05-25T06:11:46.656 INFO:tasks.workunit.client.0.smithi160.stdout:2/829: write d12/d75/dca/f60 [2542502,62803] 0 2024-05-25T06:11:46.658 INFO:tasks.workunit.client.0.smithi160.stdout:4/725: stat d4/d6/d5b/d66/d78/da1/dab 0 2024-05-25T06:11:46.659 INFO:tasks.workunit.client.0.smithi160.stdout:6/795: write d5/de/d7d/f9b [580714,57226] 0 2024-05-25T06:11:46.659 INFO:tasks.workunit.client.0.smithi160.stdout:0/916: link d2b/d46/d63/da7/db5/f102 d2b/d46/d63/da7/db5/f128 0 2024-05-25T06:11:46.662 INFO:tasks.workunit.client.0.smithi160.stdout:9/687: getdents d4/d11/d1e/d3e/d2f 0 2024-05-25T06:11:46.663 INFO:tasks.workunit.client.0.smithi160.stdout:9/688: chown d4/d11/d1e/d3e/d25/d5e/da2/lc6 11137 1 2024-05-25T06:11:46.664 INFO:tasks.workunit.client.0.smithi160.stdout:2/830: rename d12/d3e/db5/lcd to d12/d3c/df1/l118 0 2024-05-25T06:11:46.665 INFO:tasks.workunit.client.0.smithi160.stdout:0/917: creat d2b/d31/d35/d43/dab/dc6/f129 x:0 0 0 2024-05-25T06:11:46.665 INFO:tasks.workunit.client.0.smithi160.stdout:2/831: readlink d12/d3e/d5a/l67 0 2024-05-25T06:11:46.666 INFO:tasks.workunit.client.0.smithi160.stdout:0/918: read d2b/d31/d35/d36/fd3 [2053243,8550] 0 2024-05-25T06:11:46.668 INFO:tasks.workunit.client.0.smithi160.stdout:4/726: rename d4/d6/d5b/d66/c8a to d4/d1c/d54/dbc/dda/ce6 0 2024-05-25T06:11:46.671 INFO:tasks.workunit.client.0.smithi160.stdout:9/689: creat d4/d11/d12/d2e/d45/d83/fdd x:0 0 0 2024-05-25T06:11:46.672 INFO:tasks.workunit.client.0.smithi160.stdout:2/832: mknod d12/d5c/d85/dc1/c119 0 2024-05-25T06:11:46.676 INFO:tasks.workunit.client.0.smithi160.stdout:9/690: creat d4/fde x:0 0 0 2024-05-25T06:11:46.676 INFO:tasks.workunit.client.0.smithi160.stdout:0/919: dwrite d2b/d31/d35/d36/fd3 [0,4194304] 0 2024-05-25T06:11:46.682 INFO:tasks.workunit.client.0.smithi160.stdout:4/727: dwrite d4/d63/d64/fd6 [0,4194304] 0 2024-05-25T06:11:46.687 INFO:tasks.workunit.client.0.smithi160.stdout:9/691: truncate d4/d11/d1e/d3e/d25/d5e/da2/faa 978506 0 2024-05-25T06:11:46.690 INFO:tasks.workunit.client.0.smithi160.stdout:4/728: creat d4/d6/d5b/d66/d82/fe7 x:0 0 0 2024-05-25T06:11:46.693 INFO:tasks.workunit.client.0.smithi160.stdout:0/920: rename d2b/d31/d35/f49 to d2b/d46/d63/db3/dd6/df8/f12a 0 2024-05-25T06:11:46.700 INFO:tasks.workunit.client.0.smithi160.stdout:4/729: fsync d4/d6/d21/f3f 0 2024-05-25T06:11:46.702 INFO:tasks.workunit.client.0.smithi160.stdout:0/921: dread - d2b/d31/d6d/fbf zero size 2024-05-25T06:11:46.704 INFO:tasks.workunit.client.0.smithi160.stdout:2/833: dread f8 [0,4194304] 0 2024-05-25T06:11:46.704 INFO:tasks.workunit.client.0.smithi160.stdout:9/692: dwrite d4/d11/d14/f3a [0,4194304] 0 2024-05-25T06:11:46.705 INFO:tasks.workunit.client.0.smithi160.stdout:4/730: read d4/d6/d28/f17 [7466941,83170] 0 2024-05-25T06:11:46.706 INFO:tasks.workunit.client.0.smithi160.stdout:2/834: chown d12/f9f 4007290 1 2024-05-25T06:11:46.706 INFO:tasks.workunit.client.0.smithi160.stdout:0/922: chown d2b/d31/d35/c6c 10952715 1 2024-05-25T06:11:46.707 INFO:tasks.workunit.client.0.smithi160.stdout:4/731: write d4/d6/d5b/d66/f74 [1209937,18543] 0 2024-05-25T06:11:46.708 INFO:tasks.workunit.client.0.smithi160.stdout:2/835: fdatasync d12/d3e/d5f/d113/f99 0 2024-05-25T06:11:46.710 INFO:tasks.workunit.client.0.smithi160.stdout:2/836: read d12/d3e/d5a/f93 [97538,109191] 0 2024-05-25T06:11:46.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:46 smithi160 ceph-mon[29041]: pgmap v23: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 109 MiB/s rd, 279 MiB/s wr, 551 op/s 2024-05-25T06:11:46.723 INFO:tasks.workunit.client.0.smithi160.stdout:9/693: rename d4/d11/d14/l56 to d4/d11/d1e/d3e/d25/ldf 0 2024-05-25T06:11:46.725 INFO:tasks.workunit.client.0.smithi160.stdout:0/923: creat d2b/d46/d63/da7/da6/f12b x:0 0 0 2024-05-25T06:11:46.728 INFO:tasks.workunit.client.0.smithi160.stdout:4/732: creat d4/d1c/d54/dbc/dda/d94/fe8 x:0 0 0 2024-05-25T06:11:46.729 INFO:tasks.workunit.client.0.smithi160.stdout:2/837: creat d12/d5c/d85/dc1/f11a x:0 0 0 2024-05-25T06:11:46.730 INFO:tasks.workunit.client.0.smithi160.stdout:2/838: read - d12/d3e/d5f/fa4 zero size 2024-05-25T06:11:46.731 INFO:tasks.workunit.client.0.smithi160.stdout:9/694: creat d4/d11/d1e/d3e/d2f/d79/fe0 x:0 0 0 2024-05-25T06:11:46.733 INFO:tasks.workunit.client.0.smithi160.stdout:9/695: stat d4/d11/d1e/d3e/d25/d27/dc5 0 2024-05-25T06:11:46.733 INFO:tasks.workunit.client.0.smithi160.stdout:0/924: symlink d2b/d46/d63/da7/l12c 0 2024-05-25T06:11:46.734 INFO:tasks.workunit.client.0.smithi160.stdout:9/696: readlink d4/d11/d12/ld2 0 2024-05-25T06:11:46.735 INFO:tasks.workunit.client.0.smithi160.stdout:4/733: dread d4/d6/d5b/d66/d82/f84 [0,4194304] 0 2024-05-25T06:11:46.736 INFO:tasks.workunit.client.0.smithi160.stdout:9/697: write d4/d11/d1e/d3e/d25/d5e/fda [782657,51046] 0 2024-05-25T06:11:46.739 INFO:tasks.workunit.client.0.smithi160.stdout:0/925: rename d2b/d31/d35/c64 to d2b/d31/d35/c12d 0 2024-05-25T06:11:46.740 INFO:tasks.workunit.client.0.smithi160.stdout:4/734: rmdir d4/d6 39 2024-05-25T06:11:46.742 INFO:tasks.workunit.client.0.smithi160.stdout:9/698: mknod d4/d11/d1e/dbc/ce1 0 2024-05-25T06:11:46.742 INFO:tasks.workunit.client.0.smithi160.stdout:2/839: dwrite d12/d1c/f56 [0,4194304] 0 2024-05-25T06:11:46.745 INFO:tasks.workunit.client.0.smithi160.stdout:4/735: stat d4/d6/d28/l93 0 2024-05-25T06:11:46.747 INFO:tasks.workunit.client.0.smithi160.stdout:2/840: truncate d12/d1c/d37/f71 452217 0 2024-05-25T06:11:46.748 INFO:tasks.workunit.client.0.smithi160.stdout:4/736: mknod d4/d1c/ce9 0 2024-05-25T06:11:46.748 INFO:tasks.workunit.client.0.smithi160.stdout:2/841: read - d12/d1c/d37/d4b/f79 zero size 2024-05-25T06:11:46.750 INFO:tasks.workunit.client.0.smithi160.stdout:4/737: chown d4/d6/d28/de4/f56 0 1 2024-05-25T06:11:46.751 INFO:tasks.workunit.client.0.smithi160.stdout:4/738: write d4/d6/d21/fd0 [45078,89349] 0 2024-05-25T06:11:46.754 INFO:tasks.workunit.client.0.smithi160.stdout:0/926: write d2b/d31/d35/d7b/fa0 [2507458,76190] 0 2024-05-25T06:11:46.755 INFO:tasks.workunit.client.0.smithi160.stdout:4/739: mkdir d4/d6/d35/d3d/d9e/dea 0 2024-05-25T06:11:46.756 INFO:tasks.workunit.client.0.smithi160.stdout:2/842: dread ff [0,4194304] 0 2024-05-25T06:11:46.756 INFO:tasks.workunit.client.0.smithi160.stdout:0/927: mkdir d2b/da8/d12e 0 2024-05-25T06:11:46.758 INFO:tasks.workunit.client.0.smithi160.stdout:0/928: fdatasync d2b/d31/fe0 0 2024-05-25T06:11:46.758 INFO:tasks.workunit.client.0.smithi160.stdout:2/843: rmdir d12/d3e/d5f 39 2024-05-25T06:11:46.758 INFO:tasks.workunit.client.0.smithi160.stdout:4/740: creat d4/d63/d64/d88/feb x:0 0 0 2024-05-25T06:11:46.760 INFO:tasks.workunit.client.0.smithi160.stdout:0/929: getdents d2b/d31/d35/d36/d86/d103 0 2024-05-25T06:11:46.761 INFO:tasks.workunit.client.0.smithi160.stdout:2/844: symlink d12/d3e/db5/dfa/l11b 0 2024-05-25T06:11:46.762 INFO:tasks.workunit.client.0.smithi160.stdout:0/930: mknod d2b/d31/d35/d7b/c12f 0 2024-05-25T06:11:46.763 INFO:tasks.workunit.client.0.smithi160.stdout:2/845: rmdir d12/d1c/d2a/d7e/d90 39 2024-05-25T06:11:46.765 INFO:tasks.workunit.client.0.smithi160.stdout:9/699: dread d4/d11/d1e/d3e/d25/fb3 [0,4194304] 0 2024-05-25T06:11:46.765 INFO:tasks.workunit.client.0.smithi160.stdout:0/931: truncate d2b/d31/d35/d36/d37/f75 618493 0 2024-05-25T06:11:46.768 INFO:tasks.workunit.client.0.smithi160.stdout:0/932: truncate d2b/d46/d63/da7/da6/fce 968854 0 2024-05-25T06:11:46.772 INFO:tasks.workunit.client.0.smithi160.stdout:0/933: link d2b/d31/d6d/c9e d2b/da8/db2/c130 0 2024-05-25T06:11:46.781 INFO:tasks.workunit.client.0.smithi160.stdout:2/846: dwrite d12/d75/dca/f15 [0,4194304] 0 2024-05-25T06:11:46.781 INFO:tasks.workunit.client.0.smithi160.stdout:2/847: chown d12/d47/da0/dd3/f10f 2633633 1 2024-05-25T06:11:46.783 INFO:tasks.workunit.client.0.smithi160.stdout:2/848: stat d12/f45 0 2024-05-25T06:11:46.789 INFO:tasks.workunit.client.0.smithi160.stdout:2/849: dread d12/f8f [0,4194304] 0 2024-05-25T06:11:46.798 INFO:tasks.workunit.client.0.smithi160.stdout:2/850: truncate ff 1582526 0 2024-05-25T06:11:46.804 INFO:tasks.workunit.client.0.smithi160.stdout:2/851: write d12/f45 [2180804,54289] 0 2024-05-25T06:11:46.805 INFO:tasks.workunit.client.0.smithi160.stdout:2/852: dread - d12/d5c/d85/dc1/f10b zero size 2024-05-25T06:11:46.820 INFO:tasks.workunit.client.0.smithi160.stdout:2/853: dread d12/d1c/f6c [0,4194304] 0 2024-05-25T06:11:46.822 INFO:tasks.workunit.client.0.smithi160.stdout:2/854: symlink d12/d3c/l11c 0 2024-05-25T06:11:46.825 INFO:tasks.workunit.client.0.smithi160.stdout:2/855: creat d12/d3e/d5f/df3/d10e/f11d x:0 0 0 2024-05-25T06:11:46.827 INFO:tasks.workunit.client.0.smithi160.stdout:2/856: unlink d12/d1c/d2a/l39 0 2024-05-25T06:11:46.829 INFO:tasks.workunit.client.0.smithi160.stdout:2/857: symlink d12/d3e/d5a/d8d/db4/dcb/l11e 0 2024-05-25T06:11:46.829 INFO:tasks.workunit.client.0.smithi160.stdout:2/858: chown d12/d3e/db5 1230667 1 2024-05-25T06:11:46.830 INFO:tasks.workunit.client.0.smithi160.stdout:2/859: mkdir d12/d75/dca/d11f 0 2024-05-25T06:11:46.834 INFO:tasks.workunit.client.0.smithi160.stdout:2/860: getdents d12/d47/da0/dd3 0 2024-05-25T06:11:46.836 INFO:tasks.workunit.client.0.smithi160.stdout:1/764: sync 2024-05-25T06:11:46.836 INFO:tasks.workunit.client.0.smithi160.stdout:3/788: sync 2024-05-25T06:11:46.836 INFO:tasks.workunit.client.0.smithi160.stdout:7/899: sync 2024-05-25T06:11:46.836 INFO:tasks.workunit.client.0.smithi160.stdout:8/772: sync 2024-05-25T06:11:46.836 INFO:tasks.workunit.client.0.smithi160.stdout:5/878: sync 2024-05-25T06:11:46.837 INFO:tasks.workunit.client.0.smithi160.stdout:3/789: readlink d19/d2f/d38/d60/l9b 0 2024-05-25T06:11:46.838 INFO:tasks.workunit.client.0.smithi160.stdout:2/861: getdents d12/d3e 0 2024-05-25T06:11:46.841 INFO:tasks.workunit.client.0.smithi160.stdout:8/773: unlink d5/d7/dcb/d7a/dc9/cd7 0 2024-05-25T06:11:46.843 INFO:tasks.workunit.client.0.smithi160.stdout:8/774: chown d5/d44/d1f/d58/f60 176 1 2024-05-25T06:11:46.843 INFO:tasks.workunit.client.0.smithi160.stdout:5/879: dread f1 [0,4194304] 0 2024-05-25T06:11:46.843 INFO:tasks.workunit.client.0.smithi160.stdout:1/765: rename de/f54 to de/d21/d49/d90/f105 0 2024-05-25T06:11:46.844 INFO:tasks.workunit.client.0.smithi160.stdout:7/900: symlink d6/d32/l124 0 2024-05-25T06:11:46.845 INFO:tasks.workunit.client.0.smithi160.stdout:5/880: stat db/d47/cf7 0 2024-05-25T06:11:46.846 INFO:tasks.workunit.client.0.smithi160.stdout:3/790: creat d19/db2/d104/f108 x:0 0 0 2024-05-25T06:11:46.846 INFO:tasks.workunit.client.0.smithi160.stdout:1/766: dread - de/d21/d49/d1c/d41/d45/d7d/db3/dba/ff5 zero size 2024-05-25T06:11:46.850 INFO:tasks.workunit.client.0.smithi160.stdout:3/791: symlink d19/d2f/d34/da8/df8/l109 0 2024-05-25T06:11:46.851 INFO:tasks.workunit.client.0.smithi160.stdout:5/881: dread db/f57 [0,4194304] 0 2024-05-25T06:11:46.853 INFO:tasks.workunit.client.0.smithi160.stdout:2/862: dread d12/d3e/d5f/d74/f77 [4194304,4194304] 0 2024-05-25T06:11:46.853 INFO:tasks.workunit.client.0.smithi160.stdout:5/882: fdatasync db/dc/d13/d1d/d23/dbd/fef 0 2024-05-25T06:11:46.854 INFO:tasks.workunit.client.0.smithi160.stdout:3/792: read d19/d2f/d38/d43/d5b/d65/f89 [7028594,26554] 0 2024-05-25T06:11:46.861 INFO:tasks.workunit.client.0.smithi160.stdout:3/793: mkdir d19/d2f/d34/d67/d96/de1/d10a 0 2024-05-25T06:11:46.861 INFO:tasks.workunit.client.0.smithi160.stdout:7/901: dread d6/d3e/d6f/dfa/f10c [0,4194304] 0 2024-05-25T06:11:46.862 INFO:tasks.workunit.client.0.smithi160.stdout:3/794: truncate d19/faf 708401 0 2024-05-25T06:11:46.862 INFO:tasks.workunit.client.0.smithi160.stdout:5/883: symlink db/l107 0 2024-05-25T06:11:46.863 INFO:tasks.workunit.client.0.smithi160.stdout:1/767: truncate de/d21/d49/d1c/fa5 881789 0 2024-05-25T06:11:46.863 INFO:tasks.workunit.client.0.smithi160.stdout:5/884: write db/dc/d13/d1d/fbb [233755,90540] 0 2024-05-25T06:11:46.864 INFO:tasks.workunit.client.0.smithi160.stdout:1/768: write de/d21/d49/f6f [182983,108946] 0 2024-05-25T06:11:46.865 INFO:tasks.workunit.client.0.smithi160.stdout:7/902: unlink c4 0 2024-05-25T06:11:46.866 INFO:tasks.workunit.client.0.smithi160.stdout:1/769: stat de/d21/d49/d1c/d59/d67 0 2024-05-25T06:11:46.866 INFO:tasks.workunit.client.0.smithi160.stdout:2/863: rename d12/d3e/db5 to d12/d1c/d120 0 2024-05-25T06:11:46.867 INFO:tasks.workunit.client.0.smithi160.stdout:1/770: fdatasync de/d21/d49/d1c/d52/fd4 0 2024-05-25T06:11:46.868 INFO:tasks.workunit.client.0.smithi160.stdout:3/795: mknod d19/d2f/d38/d43/d5b/d65/d83/dee/d102/c10b 0 2024-05-25T06:11:46.872 INFO:tasks.workunit.client.0.smithi160.stdout:1/771: unlink de/d21/d49/d1c/d41/d45/d7d/db3/fef 0 2024-05-25T06:11:46.872 INFO:tasks.workunit.client.0.smithi160.stdout:2/864: dread d12/d5c/f72 [0,4194304] 0 2024-05-25T06:11:46.875 INFO:tasks.workunit.client.0.smithi160.stdout:5/885: rename db/dc/d13/d1d/d23/d24/d2e to db/d108 0 2024-05-25T06:11:46.878 INFO:tasks.workunit.client.0.smithi160.stdout:3/796: truncate d19/d2f/d38/d43/d5b/d65/d83/fcb 123157 0 2024-05-25T06:11:46.878 INFO:tasks.workunit.client.0.smithi160.stdout:7/903: getdents d6/d3e/da1/ddb 0 2024-05-25T06:11:46.916 INFO:tasks.workunit.client.0.smithi160.stdout:1/772: mkdir de/d21/d49/db1/d106 0 2024-05-25T06:11:46.916 INFO:tasks.workunit.client.0.smithi160.stdout:5/886: creat db/d108/f109 x:0 0 0 2024-05-25T06:11:46.920 INFO:tasks.workunit.client.0.smithi160.stdout:3/797: write f0 [5635197,61792] 0 2024-05-25T06:11:46.920 INFO:tasks.workunit.client.0.smithi160.stdout:1/773: unlink de/f13 0 2024-05-25T06:11:46.923 INFO:tasks.workunit.client.0.smithi160.stdout:5/887: write db/dc/d13/d1d/d3f/d72/f7f [91301,106767] 0 2024-05-25T06:11:46.924 INFO:tasks.workunit.client.0.smithi160.stdout:7/904: dwrite d6/d2b/da0/dc7/f111 [0,4194304] 0 2024-05-25T06:11:46.925 INFO:tasks.workunit.client.0.smithi160.stdout:2/865: creat d12/d3e/d5f/d74/d7a/d104/f121 x:0 0 0 2024-05-25T06:11:46.926 INFO:tasks.workunit.client.0.smithi160.stdout:3/798: mknod d19/d2f/d34/da8/c10c 0 2024-05-25T06:11:46.931 INFO:tasks.workunit.client.0.smithi160.stdout:3/799: write d19/da6/fb6 [94127,46015] 0 2024-05-25T06:11:46.936 INFO:tasks.workunit.client.0.smithi160.stdout:2/866: mknod d12/d3e/c122 0 2024-05-25T06:11:46.940 INFO:tasks.workunit.client.0.smithi160.stdout:1/774: rename de/d21/d49/d1c/d41/l8b to de/l107 0 2024-05-25T06:11:46.944 INFO:tasks.workunit.client.0.smithi160.stdout:3/800: dwrite d19/d22/f24 [0,4194304] 0 2024-05-25T06:11:46.944 INFO:tasks.workunit.client.0.smithi160.stdout:7/905: dread d6/d3e/d6f/f6e [0,4194304] 0 2024-05-25T06:11:46.945 INFO:tasks.workunit.client.0.smithi160.stdout:5/888: dwrite db/d47/d81/dc1/fde [0,4194304] 0 2024-05-25T06:11:46.947 INFO:tasks.workunit.client.0.smithi160.stdout:2/867: creat d12/d3e/d5a/d8d/db4/f123 x:0 0 0 2024-05-25T06:11:46.948 INFO:tasks.workunit.client.0.smithi160.stdout:1/775: write de/d21/d49/f104 [1009411,128781] 0 2024-05-25T06:11:46.950 INFO:tasks.workunit.client.0.smithi160.stdout:1/776: stat de/d21/d49/d35 0 2024-05-25T06:11:46.954 INFO:tasks.workunit.client.0.smithi160.stdout:7/906: read d6/dc/fd1 [3351026,34029] 0 2024-05-25T06:11:46.954 INFO:tasks.workunit.client.0.smithi160.stdout:3/801: mkdir d19/d2f/d34/d10d 0 2024-05-25T06:11:46.954 INFO:tasks.workunit.client.0.smithi160.stdout:2/868: chown d12/d3e/d5a/f93 50209858 1 2024-05-25T06:11:46.956 INFO:tasks.workunit.client.0.smithi160.stdout:5/889: truncate db/dc/d13/d1d/d23/d24/f3e 3594375 0 2024-05-25T06:11:46.959 INFO:tasks.workunit.client.0.smithi160.stdout:7/907: write d6/dc/d64/d5a/fbd [2749756,84960] 0 2024-05-25T06:11:46.960 INFO:tasks.workunit.client.0.smithi160.stdout:2/869: chown d12/d3e/d5f/d74/d7a/lc4 51190 1 2024-05-25T06:11:46.961 INFO:tasks.workunit.client.0.smithi160.stdout:7/908: fdatasync d6/dc/d64/f78 0 2024-05-25T06:11:46.963 INFO:tasks.workunit.client.0.smithi160.stdout:1/777: dwrite de/d21/d49/d1c/d41/fe6 [0,4194304] 0 2024-05-25T06:11:46.965 INFO:tasks.workunit.client.0.smithi160.stdout:5/890: creat db/dc/d13/d1d/d3f/dc0/d104/f10a x:0 0 0 2024-05-25T06:11:46.967 INFO:tasks.workunit.client.0.smithi160.stdout:2/870: mknod d12/d3e/d5f/d74/d7a/d104/c124 0 2024-05-25T06:11:46.968 INFO:tasks.workunit.client.0.smithi160.stdout:5/891: fsync db/dc/d13/d1d/d3f/d4a/ff2 0 2024-05-25T06:11:46.969 INFO:tasks.workunit.client.0.smithi160.stdout:2/871: read d12/f8f [280978,23451] 0 2024-05-25T06:11:46.969 INFO:tasks.workunit.client.0.smithi160.stdout:3/802: creat d19/d2f/d38/d43/d5b/d65/d7f/f10e x:0 0 0 2024-05-25T06:11:46.970 INFO:tasks.workunit.client.0.smithi160.stdout:7/909: unlink d6/dc/d64/f38 0 2024-05-25T06:11:46.974 INFO:tasks.workunit.client.0.smithi160.stdout:1/778: dread de/d21/d49/d28/fad [0,4194304] 0 2024-05-25T06:11:46.975 INFO:tasks.workunit.client.0.smithi160.stdout:5/892: creat db/dc/d13/d1d/d3f/d72/f10b x:0 0 0 2024-05-25T06:11:46.979 INFO:tasks.workunit.client.0.smithi160.stdout:3/803: rename d19/d2f/d38/d43/d5b/d65/d7e/de8 to d19/d2f/d38/d60/d10f 0 2024-05-25T06:11:46.981 INFO:tasks.workunit.client.0.smithi160.stdout:1/779: mknod de/d21/d49/d1c/d41/d45/dac/c108 0 2024-05-25T06:11:46.983 INFO:tasks.workunit.client.0.smithi160.stdout:5/893: creat db/d47/d81/f10c x:0 0 0 2024-05-25T06:11:46.983 INFO:tasks.workunit.client.0.smithi160.stdout:3/804: read d19/d2f/d38/d43/d5b/d65/d7e/fd4 [2166025,96100] 0 2024-05-25T06:11:46.984 INFO:tasks.workunit.client.0.smithi160.stdout:5/894: chown db/dc/d13/d1d/d23/f46 94614012 1 2024-05-25T06:11:46.985 INFO:tasks.workunit.client.0.smithi160.stdout:5/895: chown db/d47/cf7 269659 1 2024-05-25T06:11:46.986 INFO:tasks.workunit.client.0.smithi160.stdout:2/872: truncate d12/d1c/f1e 3257823 0 2024-05-25T06:11:46.987 INFO:tasks.workunit.client.0.smithi160.stdout:5/896: dread db/dc/d13/d1d/faa [0,4194304] 0 2024-05-25T06:11:46.988 INFO:tasks.workunit.client.0.smithi160.stdout:1/780: truncate de/d21/d49/d1c/f53 3338448 0 2024-05-25T06:11:46.988 INFO:tasks.workunit.client.0.smithi160.stdout:7/910: link d6/d3e/d6f/d73/l87 d6/dc/d64/dac/l125 0 2024-05-25T06:11:46.988 INFO:tasks.workunit.client.0.smithi160.stdout:5/897: chown db/dc/d13/f50 10 1 2024-05-25T06:11:46.989 INFO:tasks.workunit.client.0.smithi160.stdout:7/911: chown d6/d2b/da0/dc7 1 1 2024-05-25T06:11:46.993 INFO:tasks.workunit.client.0.smithi160.stdout:3/805: rmdir d19/d2f/d38 39 2024-05-25T06:11:46.998 INFO:tasks.workunit.client.0.smithi160.stdout:1/781: rename de/f4d to de/d21/d49/d35/f109 0 2024-05-25T06:11:46.999 INFO:tasks.workunit.client.0.smithi160.stdout:2/873: dwrite d12/d1c/d37/fdb [0,4194304] 0 2024-05-25T06:11:47.000 INFO:tasks.workunit.client.0.smithi160.stdout:5/898: unlink f8 0 2024-05-25T06:11:47.010 INFO:tasks.workunit.client.0.smithi160.stdout:2/874: dwrite d12/d3e/d5a/d8d/fb8 [0,4194304] 0 2024-05-25T06:11:47.013 INFO:tasks.workunit.client.0.smithi160.stdout:5/899: rename db/dc/d13/d1d/d3f/d72/d98/da5/fac to db/dc/d13/d1d/d3f/d72/d98/da5/f10d 0 2024-05-25T06:11:47.021 INFO:tasks.workunit.client.0.smithi160.stdout:7/912: dwrite d6/d2b/d5d/d89/fa7 [0,4194304] 0 2024-05-25T06:11:47.022 INFO:tasks.workunit.client.0.smithi160.stdout:1/782: dread de/d21/d49/d1c/d59/d67/fa0 [0,4194304] 0 2024-05-25T06:11:47.027 INFO:tasks.workunit.client.0.smithi160.stdout:3/806: getdents d19/d2f/d38/d60 0 2024-05-25T06:11:47.029 INFO:tasks.workunit.client.0.smithi160.stdout:1/783: getdents de/d21/d49/d1c/d41 0 2024-05-25T06:11:47.031 INFO:tasks.workunit.client.0.smithi160.stdout:3/807: getdents d19/d2f/d34/d67/d96/de1/d10a 0 2024-05-25T06:11:47.032 INFO:tasks.workunit.client.0.smithi160.stdout:3/808: dread - d19/d2f/fa5 zero size 2024-05-25T06:11:47.033 INFO:tasks.workunit.client.0.smithi160.stdout:1/784: unlink de/d21/d49/d1c/d41/le3 0 2024-05-25T06:11:47.036 INFO:tasks.workunit.client.0.smithi160.stdout:5/900: rmdir db/dc/d13/d1d 39 2024-05-25T06:11:47.037 INFO:tasks.workunit.client.0.smithi160.stdout:3/809: write d19/da6/f76 [1817773,115931] 0 2024-05-25T06:11:47.037 INFO:tasks.workunit.client.0.smithi160.stdout:1/785: symlink de/d21/d49/d1c/d59/d71/dfe/l10a 0 2024-05-25T06:11:47.038 INFO:tasks.workunit.client.0.smithi160.stdout:2/875: dread d12/d1c/f27 [0,4194304] 0 2024-05-25T06:11:47.040 INFO:tasks.workunit.client.0.smithi160.stdout:2/876: write d12/d3e/d5a/f7f [4488510,84839] 0 2024-05-25T06:11:47.041 INFO:tasks.workunit.client.0.smithi160.stdout:3/810: unlink d19/d22/f2e 0 2024-05-25T06:11:47.044 INFO:tasks.workunit.client.0.smithi160.stdout:5/901: dread db/dc/d13/d1d/d23/d24/daf/f90 [0,4194304] 0 2024-05-25T06:11:47.044 INFO:tasks.workunit.client.0.smithi160.stdout:3/811: mkdir d19/d2f/d34/da8/df8/d110 0 2024-05-25T06:11:47.047 INFO:tasks.workunit.client.0.smithi160.stdout:2/877: rename d12/d3e/d5a/d8d/fa8 to d12/d3e/d5f/df3/f125 0 2024-05-25T06:11:47.052 INFO:tasks.workunit.client.0.smithi160.stdout:3/812: mkdir d19/d2f/d34/d111 0 2024-05-25T06:11:47.053 INFO:tasks.workunit.client.0.smithi160.stdout:3/813: chown d19/d2f/d38/d43/d5b/d65/d7e/lb8 1 1 2024-05-25T06:11:47.053 INFO:tasks.workunit.client.0.smithi160.stdout:2/878: creat d12/d3c/f126 x:0 0 0 2024-05-25T06:11:47.054 INFO:tasks.workunit.client.0.smithi160.stdout:3/814: chown d19/d2f/d38/d60/d10f 0 1 2024-05-25T06:11:47.055 INFO:tasks.workunit.client.0.smithi160.stdout:3/815: readlink d19/d2f/d34/d67/ld1 0 2024-05-25T06:11:47.058 INFO:tasks.workunit.client.0.smithi160.stdout:1/786: truncate de/d21/d49/d1c/d41/d45/dac/ff7 2416385 0 2024-05-25T06:11:47.059 INFO:tasks.workunit.client.0.smithi160.stdout:2/879: dread d12/d3e/d5f/d74/d7a/fac [0,4194304] 0 2024-05-25T06:11:47.059 INFO:tasks.workunit.client.0.smithi160.stdout:3/816: creat d19/d2f/d38/d43/d5b/d65/d83/dee/df3/f112 x:0 0 0 2024-05-25T06:11:47.060 INFO:tasks.workunit.client.0.smithi160.stdout:2/880: dread - d12/d1c/d37/f9d zero size 2024-05-25T06:11:47.062 INFO:tasks.workunit.client.0.smithi160.stdout:3/817: rmdir d19/d2f/d34/def 39 2024-05-25T06:11:47.064 INFO:tasks.workunit.client.0.smithi160.stdout:2/881: symlink d12/d5c/d85/d9b/l127 0 2024-05-25T06:11:47.064 INFO:tasks.workunit.client.0.smithi160.stdout:5/902: write db/f5d [2437693,1338] 0 2024-05-25T06:11:47.066 INFO:tasks.workunit.client.0.smithi160.stdout:2/882: write d12/d3e/d5a/d8d/fe0 [1017275,87426] 0 2024-05-25T06:11:47.066 INFO:tasks.workunit.client.0.smithi160.stdout:1/787: rename de/d21/d49/d1c/d59/d7c/d93/l4a to de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/l10b 0 2024-05-25T06:11:47.067 INFO:tasks.workunit.client.0.smithi160.stdout:2/883: readlink d12/d1c/d120/dfa/l11b 0 2024-05-25T06:11:47.068 INFO:tasks.workunit.client.0.smithi160.stdout:2/884: fdatasync d12/d5c/fbd 0 2024-05-25T06:11:47.069 INFO:tasks.workunit.client.0.smithi160.stdout:5/903: rmdir db/dc/d13/d1d/d3f/d72/d98/da5/db5 39 2024-05-25T06:11:47.074 INFO:tasks.workunit.client.0.smithi160.stdout:2/885: rename d12/d3e/d5a/d8d/df9/d114 to d12/d3e/d5a/d8d/db4/dcb/d128 0 2024-05-25T06:11:47.076 INFO:tasks.workunit.client.0.smithi160.stdout:5/904: symlink db/dc/d13/d1d/d23/d24/l10e 0 2024-05-25T06:11:47.076 INFO:tasks.workunit.client.0.smithi160.stdout:5/905: stat db/dc/f97 0 2024-05-25T06:11:47.079 INFO:tasks.workunit.client.0.smithi160.stdout:5/906: mknod db/d47/d5e/d6e/c10f 0 2024-05-25T06:11:47.081 INFO:tasks.workunit.client.0.smithi160.stdout:3/818: dread d19/d2f/d34/d67/d96/fec [0,4194304] 0 2024-05-25T06:11:47.082 INFO:tasks.workunit.client.0.smithi160.stdout:2/886: getdents d12/d1c/d37/d4b 0 2024-05-25T06:11:47.083 INFO:tasks.workunit.client.0.smithi160.stdout:2/887: read - d12/d3e/d5a/d8d/db4/dcb/d128/f116 zero size 2024-05-25T06:11:47.085 INFO:tasks.workunit.client.0.smithi160.stdout:2/888: write d12/d75/dca/f15 [4476964,91160] 0 2024-05-25T06:11:47.087 INFO:tasks.workunit.client.0.smithi160.stdout:5/907: rename db/dc/d13/d1d/d31/d3b/d69/f7e to db/dc/d13/d1d/d23/d24/daf/f110 0 2024-05-25T06:11:47.089 INFO:tasks.workunit.client.0.smithi160.stdout:3/819: dwrite d19/d2f/d38/d43/fdb [0,4194304] 0 2024-05-25T06:11:47.089 INFO:tasks.workunit.client.0.smithi160.stdout:2/889: chown d12/d3e/f41 0 1 2024-05-25T06:11:47.094 INFO:tasks.workunit.client.0.smithi160.stdout:2/890: creat d12/d3e/d5f/d74/d7a/d104/f129 x:0 0 0 2024-05-25T06:11:47.095 INFO:tasks.workunit.client.0.smithi160.stdout:3/820: unlink d19/d2f/d38/d43/d5b/cbc 0 2024-05-25T06:11:47.096 INFO:tasks.workunit.client.0.smithi160.stdout:5/908: rename db/dc/d13/d1d/d31/l105 to db/dc/d13/d1d/d23/d45/l111 0 2024-05-25T06:11:47.097 INFO:tasks.workunit.client.0.smithi160.stdout:3/821: fsync d19/d2f/d38/d47/f57 0 2024-05-25T06:11:47.098 INFO:tasks.workunit.client.0.smithi160.stdout:5/909: write db/dc/fb4 [551411,48481] 0 2024-05-25T06:11:47.099 INFO:tasks.workunit.client.0.smithi160.stdout:3/822: chown d19/d2f/d38/d47/f6e 1990829 1 2024-05-25T06:11:47.100 INFO:tasks.workunit.client.0.smithi160.stdout:5/910: write db/d47/d81/f10c [216051,27924] 0 2024-05-25T06:11:47.103 INFO:tasks.workunit.client.0.smithi160.stdout:5/911: mknod db/dc/d13/d1d/d3f/d72/dc5/c112 0 2024-05-25T06:11:47.104 INFO:tasks.workunit.client.0.smithi160.stdout:5/912: chown db/dc/d13/d1d/d31/d56/leb 381 1 2024-05-25T06:11:47.105 INFO:tasks.workunit.client.0.smithi160.stdout:3/823: rmdir d19/da7/de7 0 2024-05-25T06:11:47.105 INFO:tasks.workunit.client.0.smithi160.stdout:3/824: chown d19/d2f/d34/da8/df8/d110 89 1 2024-05-25T06:11:47.109 INFO:tasks.workunit.client.0.smithi160.stdout:3/825: symlink d19/d2f/d38/d43/d5b/d65/d7f/l113 0 2024-05-25T06:11:47.109 INFO:tasks.workunit.client.0.smithi160.stdout:3/826: read d19/d2f/d34/f4d [3583210,28973] 0 2024-05-25T06:11:47.116 INFO:tasks.workunit.client.0.smithi160.stdout:3/827: write d19/d2f/f105 [1266787,2179] 0 2024-05-25T06:11:47.116 INFO:tasks.workunit.client.0.smithi160.stdout:3/828: readlink d19/d2f/d34/d67/ld1 0 2024-05-25T06:11:47.121 INFO:tasks.workunit.client.0.smithi160.stdout:5/913: dread f6 [0,4194304] 0 2024-05-25T06:11:47.123 INFO:tasks.workunit.client.0.smithi160.stdout:5/914: dread - db/dc/d13/d1d/d3f/d72/d98/f9b zero size 2024-05-25T06:11:47.124 INFO:tasks.workunit.client.0.smithi160.stdout:3/829: dwrite d19/d2f/d38/d43/d5b/d65/f89 [4194304,4194304] 0 2024-05-25T06:11:47.125 INFO:tasks.workunit.client.0.smithi160.stdout:5/915: chown db/dc/f97 70662 1 2024-05-25T06:11:47.126 INFO:tasks.workunit.client.0.smithi160.stdout:5/916: truncate db/d108/f109 248546 0 2024-05-25T06:11:47.133 INFO:tasks.workunit.client.0.smithi160.stdout:5/917: rename db/dc/d13/d1d/d23/d24/daf to db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/d113 0 2024-05-25T06:11:47.136 INFO:tasks.workunit.client.0.smithi160.stdout:5/918: unlink db/dc/f93 0 2024-05-25T06:11:47.139 INFO:tasks.workunit.client.0.smithi160.stdout:5/919: mknod db/d47/c114 0 2024-05-25T06:11:47.140 INFO:tasks.workunit.client.0.smithi160.stdout:5/920: stat db/dc/d13/d1d/d23/dbd 0 2024-05-25T06:11:47.145 INFO:tasks.workunit.client.0.smithi160.stdout:3/830: dread d19/d2f/d38/d43/d5b/d65/d7e/fe6 [0,4194304] 0 2024-05-25T06:11:47.148 INFO:tasks.workunit.client.0.smithi160.stdout:3/831: unlink d19/d2f/d38/d43/d5b/d65/d7e/cc5 0 2024-05-25T06:11:47.150 INFO:tasks.workunit.client.0.smithi160.stdout:5/921: truncate db/d47/f68 8198484 0 2024-05-25T06:11:47.152 INFO:tasks.workunit.client.0.smithi160.stdout:3/832: dread d19/d2f/d38/d43/fdb [0,4194304] 0 2024-05-25T06:11:47.163 INFO:tasks.workunit.client.0.smithi160.stdout:5/922: mknod db/dc/d13/d1d/c115 0 2024-05-25T06:11:47.169 INFO:tasks.workunit.client.0.smithi160.stdout:5/923: truncate db/dc/d13/f50 1538396 0 2024-05-25T06:11:47.180 INFO:tasks.workunit.client.0.smithi160.stdout:5/924: dread db/dc/d13/d1d/d23/f46 [0,4194304] 0 2024-05-25T06:11:47.182 INFO:tasks.workunit.client.0.smithi160.stdout:5/925: write db/ffd [442995,92137] 0 2024-05-25T06:11:47.184 INFO:tasks.workunit.client.0.smithi160.stdout:5/926: mknod db/d47/c116 0 2024-05-25T06:11:47.190 INFO:tasks.workunit.client.0.smithi160.stdout:5/927: dwrite db/dc/fec [0,4194304] 0 2024-05-25T06:11:47.199 INFO:tasks.workunit.client.0.smithi160.stdout:6/796: sync 2024-05-25T06:11:47.200 INFO:tasks.workunit.client.0.smithi160.stdout:5/928: dwrite db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/d113/db3/fd6 [0,4194304] 0 2024-05-25T06:11:47.206 INFO:tasks.workunit.client.0.smithi160.stdout:6/797: rename d5/d30/d36/d73/d87 to d5/d39/d112 0 2024-05-25T06:11:47.207 INFO:tasks.workunit.client.0.smithi160.stdout:5/929: stat db/dc/d13/d1d/d31/d56/lca 0 2024-05-25T06:11:47.208 INFO:tasks.workunit.client.0.smithi160.stdout:6/798: mkdir d5/d39/dc1/d113 0 2024-05-25T06:11:47.209 INFO:tasks.workunit.client.0.smithi160.stdout:5/930: truncate db/dc/d13/d1d/d23/ffc 726493 0 2024-05-25T06:11:47.211 INFO:tasks.workunit.client.0.smithi160.stdout:6/799: symlink d5/d30/d36/d73/d6e/l114 0 2024-05-25T06:11:47.212 INFO:tasks.workunit.client.0.smithi160.stdout:6/800: truncate d5/d45/d63/fc8 802369 0 2024-05-25T06:11:47.229 INFO:tasks.workunit.client.0.smithi160.stdout:5/931: dread db/f2a [0,4194304] 0 2024-05-25T06:11:47.232 INFO:tasks.workunit.client.0.smithi160.stdout:5/932: creat db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/f117 x:0 0 0 2024-05-25T06:11:47.235 INFO:tasks.workunit.client.0.smithi160.stdout:5/933: mknod db/dc/d13/d1d/d3f/d72/d98/da5/c118 0 2024-05-25T06:11:47.247 INFO:tasks.workunit.client.0.smithi160.stdout:5/934: dwrite db/d108/f32 [0,4194304] 0 2024-05-25T06:11:47.256 INFO:tasks.workunit.client.0.smithi160.stdout:5/935: dwrite db/dc/d13/d1d/d23/d45/fe3 [0,4194304] 0 2024-05-25T06:11:47.259 INFO:tasks.workunit.client.0.smithi160.stdout:5/936: mknod db/d47/d5e/d6e/c119 0 2024-05-25T06:11:47.275 INFO:tasks.workunit.client.0.smithi160.stdout:5/937: dwrite db/d47/d5e/fc3 [0,4194304] 0 2024-05-25T06:11:47.285 INFO:tasks.workunit.client.0.smithi160.stdout:5/938: dread db/ff6 [0,4194304] 0 2024-05-25T06:11:47.288 INFO:tasks.workunit.client.0.smithi160.stdout:5/939: fdatasync db/dc/d13/d1d/d23/fdf 0 2024-05-25T06:11:47.550 INFO:tasks.workunit.client.0.smithi160.stdout:9/700: sync 2024-05-25T06:11:47.550 INFO:tasks.workunit.client.0.smithi160.stdout:4/741: sync 2024-05-25T06:11:47.552 INFO:tasks.workunit.client.0.smithi160.stdout:4/742: unlink d4/d1a/f87 0 2024-05-25T06:11:47.561 INFO:tasks.workunit.client.0.smithi160.stdout:4/743: dwrite d4/d1c/d54/d9b/fcc [4194304,4194304] 0 2024-05-25T06:11:47.562 INFO:tasks.workunit.client.0.smithi160.stdout:4/744: readlink d4/l14 0 2024-05-25T06:11:47.565 INFO:tasks.workunit.client.0.smithi160.stdout:4/745: rmdir d4/d1c/d54/dbc/dda/d94 39 2024-05-25T06:11:47.567 INFO:tasks.workunit.client.0.smithi160.stdout:4/746: truncate d4/d6/d28/f17 5579738 0 2024-05-25T06:11:47.571 INFO:tasks.workunit.client.0.smithi160.stdout:4/747: truncate d4/d6/d28/de4/f79 81685 0 2024-05-25T06:11:47.574 INFO:tasks.workunit.client.0.smithi160.stdout:4/748: mkdir d4/d1c/d54/dbc/dda/d94/dec 0 2024-05-25T06:11:47.586 INFO:tasks.workunit.client.0.smithi160.stdout:4/749: dwrite d4/d6/d21/f77 [0,4194304] 0 2024-05-25T06:11:47.602 INFO:tasks.workunit.client.0.smithi160.stdout:4/750: dread d4/d63/d64/fc1 [0,4194304] 0 2024-05-25T06:11:47.606 INFO:tasks.workunit.client.0.smithi160.stdout:4/751: creat d4/d63/fed x:0 0 0 2024-05-25T06:11:47.611 INFO:tasks.workunit.client.0.smithi160.stdout:4/752: read d4/d6/d35/f53 [4726011,65477] 0 2024-05-25T06:11:47.612 INFO:tasks.workunit.client.0.smithi160.stdout:4/753: truncate d4/d6/d5b/d66/d82/fdb 693066 0 2024-05-25T06:11:47.614 INFO:tasks.workunit.client.0.smithi160.stdout:4/754: mkdir d4/d6/d35/dee 0 2024-05-25T06:11:47.617 INFO:tasks.workunit.client.0.smithi160.stdout:4/755: rename d4/d6/d5b/d66/d82/fbb to d4/d6/d5b/fef 0 2024-05-25T06:11:47.619 INFO:tasks.workunit.client.0.smithi160.stdout:4/756: truncate d4/f76 9687 0 2024-05-25T06:11:47.621 INFO:tasks.workunit.client.0.smithi160.stdout:4/757: truncate d4/d63/fb7 6388647 0 2024-05-25T06:11:47.622 INFO:tasks.workunit.client.0.smithi160.stdout:4/758: mkdir d4/d6/dbd/df0 0 2024-05-25T06:11:47.634 INFO:tasks.workunit.client.0.smithi160.stdout:4/759: truncate d4/d6/d35/f58 2686394 0 2024-05-25T06:11:47.635 INFO:tasks.workunit.client.0.smithi160.stdout:4/760: mkdir d4/d1a/df1 0 2024-05-25T06:11:47.648 INFO:tasks.workunit.client.0.smithi160.stdout:4/761: dread d4/f2a [0,4194304] 0 2024-05-25T06:11:47.662 INFO:tasks.workunit.client.0.smithi160.stdout:4/762: dread d4/d6/d21/f3f [0,4194304] 0 2024-05-25T06:11:47.664 INFO:tasks.workunit.client.0.smithi160.stdout:4/763: write d4/d1c/d54/f65 [715671,97731] 0 2024-05-25T06:11:47.665 INFO:tasks.workunit.client.0.smithi160.stdout:4/764: chown d4/d6/d5b/d66/fc4 33750 1 2024-05-25T06:11:47.666 INFO:tasks.workunit.client.0.smithi160.stdout:4/765: chown d4/d6/d5b/d66/d78/fcb 4224 1 2024-05-25T06:11:47.669 INFO:tasks.workunit.client.0.smithi160.stdout:4/766: creat d4/d1c/d54/dbc/ff2 x:0 0 0 2024-05-25T06:11:47.682 INFO:tasks.workunit.client.0.smithi160.stdout:4/767: dread d4/d6/d21/d42/f7c [0,4194304] 0 2024-05-25T06:11:47.687 INFO:tasks.workunit.client.0.smithi160.stdout:4/768: write d4/f76 [799237,110049] 0 2024-05-25T06:11:47.689 INFO:tasks.workunit.client.0.smithi160.stdout:4/769: symlink d4/d1c/d54/dbc/lf3 0 2024-05-25T06:11:47.692 INFO:tasks.workunit.client.0.smithi160.stdout:4/770: creat d4/d6/d35/dee/ff4 x:0 0 0 2024-05-25T06:11:47.698 INFO:tasks.workunit.client.0.smithi160.stdout:4/771: dwrite d4/d1c/d54/dbc/dda/d94/fe8 [0,4194304] 0 2024-05-25T06:11:47.715 INFO:tasks.workunit.client.0.smithi160.stdout:0/934: sync 2024-05-25T06:11:47.716 INFO:tasks.workunit.client.0.smithi160.stdout:0/935: chown d2b/d46/d63/da7/da6/da9/f122 0 1 2024-05-25T06:11:47.723 INFO:tasks.workunit.client.0.smithi160.stdout:4/772: mkdir d4/d6/d35/d3d/d9e/dea/df5 0 2024-05-25T06:11:47.725 INFO:tasks.workunit.client.0.smithi160.stdout:4/773: symlink d4/d1c/d54/dbc/dda/d94/lf6 0 2024-05-25T06:11:47.729 INFO:tasks.workunit.client.0.smithi160.stdout:0/936: dwrite d2b/d46/d63/da7/fc8 [0,4194304] 0 2024-05-25T06:11:47.729 INFO:tasks.workunit.client.0.smithi160.stdout:4/774: symlink d4/d6/d35/dee/lf7 0 2024-05-25T06:11:47.734 INFO:tasks.workunit.client.0.smithi160.stdout:0/937: unlink d2b/d46/d63/da7/da6/dcd/dd8/c11b 0 2024-05-25T06:11:47.736 INFO:tasks.workunit.client.0.smithi160.stdout:4/775: fdatasync d4/d6/d5b/d66/d82/fa8 0 2024-05-25T06:11:47.742 INFO:tasks.workunit.client.0.smithi160.stdout:4/776: mknod d4/d6/cf8 0 2024-05-25T06:11:47.743 INFO:tasks.workunit.client.0.smithi160.stdout:4/777: chown d4/d6/d28/c6b 1010301023 1 2024-05-25T06:11:47.745 INFO:tasks.workunit.client.0.smithi160.stdout:4/778: creat d4/d1c/ff9 x:0 0 0 2024-05-25T06:11:47.751 INFO:tasks.workunit.client.0.smithi160.stdout:4/779: dwrite d4/d6/d28/fd8 [0,4194304] 0 2024-05-25T06:11:47.764 INFO:tasks.workunit.client.0.smithi160.stdout:0/938: dread d2b/d31/d35/d43/d98/f105 [0,4194304] 0 2024-05-25T06:11:47.767 INFO:tasks.workunit.client.0.smithi160.stdout:0/939: creat d2b/d31/d35/d43/dab/f131 x:0 0 0 2024-05-25T06:11:47.769 INFO:tasks.workunit.client.0.smithi160.stdout:4/780: dread d4/d6/d5b/d66/f74 [0,4194304] 0 2024-05-25T06:11:47.771 INFO:tasks.workunit.client.0.smithi160.stdout:0/940: getdents d2b/d46/d63/db3/dd6/df8 0 2024-05-25T06:11:47.772 INFO:tasks.workunit.client.0.smithi160.stdout:0/941: stat d2b/d46/d63/da7/da6/f12b 0 2024-05-25T06:11:47.773 INFO:tasks.workunit.client.0.smithi160.stdout:4/781: dread d4/d6/d5b/d66/f74 [0,4194304] 0 2024-05-25T06:11:47.776 INFO:tasks.workunit.client.0.smithi160.stdout:8/775: sync 2024-05-25T06:11:47.777 INFO:tasks.workunit.client.0.smithi160.stdout:0/942: rename d2b/d46/dcb/fe2 to d2b/d31/d35/f132 0 2024-05-25T06:11:47.780 INFO:tasks.workunit.client.0.smithi160.stdout:4/782: mkdir d4/d6/d5b/d66/d78/da1/dab/dfa 0 2024-05-25T06:11:47.782 INFO:tasks.workunit.client.0.smithi160.stdout:4/783: truncate d4/d6/d28/f17 1688694 0 2024-05-25T06:11:47.783 INFO:tasks.workunit.client.0.smithi160.stdout:8/776: write d5/d7/d9a/db5/fbd [340119,112116] 0 2024-05-25T06:11:47.787 INFO:tasks.workunit.client.0.smithi160.stdout:4/784: creat d4/d6/d35/d3d/ffb x:0 0 0 2024-05-25T06:11:47.790 INFO:tasks.workunit.client.0.smithi160.stdout:0/943: dwrite d2b/d46/d63/db3/fee [0,4194304] 0 2024-05-25T06:11:47.797 INFO:tasks.workunit.client.0.smithi160.stdout:0/944: read f24 [1348405,10269] 0 2024-05-25T06:11:47.798 INFO:tasks.workunit.client.0.smithi160.stdout:8/777: dwrite d5/d44/d1f/da3/d28/f38 [0,4194304] 0 2024-05-25T06:11:47.802 INFO:tasks.workunit.client.0.smithi160.stdout:0/945: rmdir d2b/d31/d35/d43/dab/dc6 39 2024-05-25T06:11:47.802 INFO:tasks.workunit.client.0.smithi160.stdout:8/778: rmdir d5/d44/d1f/da3/d28/d4a/d93 39 2024-05-25T06:11:47.804 INFO:tasks.workunit.client.0.smithi160.stdout:8/779: chown d5/d44/d1f/da3/d28/d49/d61/c94 102 1 2024-05-25T06:11:47.808 INFO:tasks.workunit.client.0.smithi160.stdout:8/780: truncate d5/d44/d1f/d58/d65/da2/fef 183730 0 2024-05-25T06:11:47.809 INFO:tasks.workunit.client.0.smithi160.stdout:0/946: dwrite d2b/d46/d63/da7/fc8 [0,4194304] 0 2024-05-25T06:11:47.812 INFO:tasks.workunit.client.0.smithi160.stdout:8/781: symlink d5/d44/d1f/da3/l105 0 2024-05-25T06:11:47.815 INFO:tasks.workunit.client.0.smithi160.stdout:0/947: rename d2b/d46/fd7 to d2b/d46/d63/da7/da6/dcd/dd8/f133 0 2024-05-25T06:11:47.815 INFO:tasks.workunit.client.0.smithi160.stdout:4/785: dread d4/f26 [4194304,4194304] 0 2024-05-25T06:11:47.818 INFO:tasks.workunit.client.0.smithi160.stdout:0/948: chown d2b/d46/l80 12534 1 2024-05-25T06:11:47.819 INFO:tasks.workunit.client.0.smithi160.stdout:4/786: creat d4/d6/dbd/df0/ffc x:0 0 0 2024-05-25T06:11:47.820 INFO:tasks.workunit.client.0.smithi160.stdout:4/787: fsync d4/d1a/f50 0 2024-05-25T06:11:47.824 INFO:tasks.workunit.client.0.smithi160.stdout:0/949: unlink d2b/d31/d35/d36/f84 0 2024-05-25T06:11:47.826 INFO:tasks.workunit.client.0.smithi160.stdout:4/788: write d4/d6/d5b/d66/f71 [1182070,74885] 0 2024-05-25T06:11:47.830 INFO:tasks.workunit.client.0.smithi160.stdout:4/789: chown d4/d6/d5b/d66/c8f 10015054 1 2024-05-25T06:11:47.830 INFO:tasks.workunit.client.0.smithi160.stdout:8/782: dwrite d5/d7/dcb/d7a/fcd [0,4194304] 0 2024-05-25T06:11:47.833 INFO:tasks.workunit.client.0.smithi160.stdout:4/790: truncate d4/d6/d21/d42/fa4 963233 0 2024-05-25T06:11:47.838 INFO:tasks.workunit.client.0.smithi160.stdout:0/950: write d2b/d31/d35/f55 [1142087,897] 0 2024-05-25T06:11:47.840 INFO:tasks.workunit.client.0.smithi160.stdout:0/951: fsync d2b/d46/d63/f9f 0 2024-05-25T06:11:47.842 INFO:tasks.workunit.client.0.smithi160.stdout:0/952: truncate d2b/d46/d63/da7/d67/f11f 111166 0 2024-05-25T06:11:47.843 INFO:tasks.workunit.client.0.smithi160.stdout:0/953: chown d2b/f57 410995 1 2024-05-25T06:11:47.846 INFO:tasks.workunit.client.0.smithi160.stdout:0/954: read f16 [2891903,39721] 0 2024-05-25T06:11:47.851 INFO:tasks.workunit.client.0.smithi160.stdout:4/791: dread d4/d6/f7 [0,4194304] 0 2024-05-25T06:11:47.853 INFO:tasks.workunit.client.0.smithi160.stdout:4/792: chown d4/d1c/d54/dbc/dda/d94/c9c 15475 1 2024-05-25T06:11:47.860 INFO:tasks.workunit.client.0.smithi160.stdout:0/955: dwrite d2b/d31/d35/f3e [0,4194304] 0 2024-05-25T06:11:47.862 INFO:tasks.workunit.client.0.smithi160.stdout:4/793: dwrite d4/d6/d5b/d66/d82/f84 [0,4194304] 0 2024-05-25T06:11:47.863 INFO:tasks.workunit.client.0.smithi160.stdout:4/794: fdatasync d4/f76 0 2024-05-25T06:11:47.863 INFO:tasks.workunit.client.0.smithi160.stdout:0/956: truncate d2b/d46/d63/da7/d67/f125 569559 0 2024-05-25T06:11:47.869 INFO:tasks.workunit.client.0.smithi160.stdout:0/957: mkdir d2b/d31/d35/d43/d134 0 2024-05-25T06:11:47.874 INFO:tasks.workunit.client.0.smithi160.stdout:4/795: link d4/d6/d5b/c7f d4/d6/d35/dee/cfd 0 2024-05-25T06:11:47.875 INFO:tasks.workunit.client.0.smithi160.stdout:4/796: readlink d4/d1c/d54/d9b/lb6 0 2024-05-25T06:11:47.876 INFO:tasks.workunit.client.0.smithi160.stdout:4/797: chown d4/d6/d5b/d66/f74 10 1 2024-05-25T06:11:47.877 INFO:tasks.workunit.client.0.smithi160.stdout:0/958: getdents d2b/d46/d63/da7/db5 0 2024-05-25T06:11:47.878 INFO:tasks.workunit.client.0.smithi160.stdout:4/798: rmdir d4/d1c/d54 39 2024-05-25T06:11:47.878 INFO:tasks.workunit.client.0.smithi160.stdout:0/959: dread - d2b/d46/d63/da7/da6/f112 zero size 2024-05-25T06:11:47.882 INFO:tasks.workunit.client.0.smithi160.stdout:4/799: link d4/d6/d21/f3f d4/d1a/df1/ffe 0 2024-05-25T06:11:47.883 INFO:tasks.workunit.client.0.smithi160.stdout:4/800: stat d4/d1c/d54/d9b/caf 0 2024-05-25T06:11:47.886 INFO:tasks.workunit.client.0.smithi160.stdout:4/801: mknod d4/d6/dbd/df0/cff 0 2024-05-25T06:11:47.890 INFO:tasks.workunit.client.0.smithi160.stdout:0/960: dread d2b/d46/d63/db3/dd6/df8/f12a [0,4194304] 0 2024-05-25T06:11:47.892 INFO:tasks.workunit.client.0.smithi160.stdout:0/961: rmdir d2b/d46/d63/db3/dd6/df8 39 2024-05-25T06:11:47.893 INFO:tasks.workunit.client.0.smithi160.stdout:0/962: fdatasync d2b/d31/d35/d36/f4a 0 2024-05-25T06:11:47.898 INFO:tasks.workunit.client.0.smithi160.stdout:0/963: mkdir d2b/d31/d135 0 2024-05-25T06:11:47.902 INFO:tasks.workunit.client.0.smithi160.stdout:4/802: dwrite d4/d6/d35/f58 [0,4194304] 0 2024-05-25T06:11:47.903 INFO:tasks.workunit.client.0.smithi160.stdout:0/964: write f1b [1376300,103190] 0 2024-05-25T06:11:47.905 INFO:tasks.workunit.client.0.smithi160.stdout:4/803: read d4/d6/f22 [450578,85483] 0 2024-05-25T06:11:47.907 INFO:tasks.workunit.client.0.smithi160.stdout:4/804: chown d4/d6/d21/d42/c44 13861359 1 2024-05-25T06:11:47.911 INFO:tasks.workunit.client.0.smithi160.stdout:4/805: unlink d4/d6/d21/lad 0 2024-05-25T06:11:47.914 INFO:tasks.workunit.client.0.smithi160.stdout:4/806: creat d4/f100 x:0 0 0 2024-05-25T06:11:47.915 INFO:tasks.workunit.client.0.smithi160.stdout:4/807: write d4/f76 [1743,12078] 0 2024-05-25T06:11:47.926 INFO:tasks.workunit.client.0.smithi160.stdout:4/808: write d4/d6/d28/f17 [2306111,88981] 0 2024-05-25T06:11:47.928 INFO:tasks.workunit.client.0.smithi160.stdout:4/809: mknod d4/dcd/c101 0 2024-05-25T06:11:47.928 INFO:tasks.workunit.client.0.smithi160.stdout:4/810: readlink d4/d6/lce 0 2024-05-25T06:11:47.929 INFO:tasks.workunit.client.0.smithi160.stdout:4/811: chown d4/d6/d28/fd8 39899719 1 2024-05-25T06:11:47.929 INFO:tasks.workunit.client.0.smithi160.stdout:4/812: chown d4/d6/d21 24728 1 2024-05-25T06:11:47.932 INFO:tasks.workunit.client.0.smithi160.stdout:4/813: dread d4/d1c/d54/dbc/dda/d94/fe8 [0,4194304] 0 2024-05-25T06:11:47.933 INFO:tasks.workunit.client.0.smithi160.stdout:4/814: chown d4/d6/d21/d42/dae 827 1 2024-05-25T06:11:47.934 INFO:tasks.workunit.client.0.smithi160.stdout:4/815: chown d4/d6/d35/d3d/d9e/fc2 880 1 2024-05-25T06:11:47.936 INFO:tasks.workunit.client.0.smithi160.stdout:4/816: symlink d4/d63/d64/d88/l102 0 2024-05-25T06:11:47.936 INFO:tasks.workunit.client.0.smithi160.stdout:4/817: stat d4/d1c/d54/dbc/ff2 0 2024-05-25T06:11:47.939 INFO:tasks.workunit.client.0.smithi160.stdout:4/818: rename d4/d6/lc8 to d4/d6/d5b/d66/l103 0 2024-05-25T06:11:47.946 INFO:tasks.workunit.client.0.smithi160.stdout:4/819: dwrite d4/d6/d28/de4/fa9 [0,4194304] 0 2024-05-25T06:11:47.947 INFO:tasks.workunit.client.0.smithi160.stdout:4/820: readlink d4/d1c/d54/dbc/lf3 0 2024-05-25T06:11:47.956 INFO:tasks.workunit.client.0.smithi160.stdout:4/821: dwrite d4/d63/d64/d88/fc5 [0,4194304] 0 2024-05-25T06:11:47.962 INFO:tasks.workunit.client.0.smithi160.stdout:4/822: rename d4/d63/d64/fc1 to d4/d1c/d54/dbc/dda/f104 0 2024-05-25T06:11:47.963 INFO:tasks.workunit.client.0.smithi160.stdout:4/823: fsync d4/d6/d28/f25 0 2024-05-25T06:11:47.966 INFO:tasks.workunit.client.0.smithi160.stdout:4/824: mknod d4/d63/d64/d88/c105 0 2024-05-25T06:11:47.969 INFO:tasks.workunit.client.0.smithi160.stdout:4/825: creat d4/d1c/d54/d9b/f106 x:0 0 0 2024-05-25T06:11:47.969 INFO:tasks.workunit.client.0.smithi160.stdout:4/826: chown d4/d63/d64 1829684131 1 2024-05-25T06:11:47.970 INFO:tasks.workunit.client.0.smithi160.stdout:4/827: read d4/d6/d5b/d66/fc4 [137184,118269] 0 2024-05-25T06:11:47.981 INFO:tasks.workunit.client.0.smithi160.stdout:4/828: dwrite d4/d63/d64/d88/fde [0,4194304] 0 2024-05-25T06:11:47.986 INFO:tasks.workunit.client.0.smithi160.stdout:4/829: rename d4/d63/fe5 to d4/d6/f107 0 2024-05-25T06:11:47.989 INFO:tasks.workunit.client.0.smithi160.stdout:4/830: rename d4/d6/lce to d4/d6/d35/d3d/l108 0 2024-05-25T06:11:48.003 INFO:tasks.workunit.client.0.smithi160.stdout:4/831: write d4/d1c/d54/f6c [9298607,46697] 0 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:3/833: sync 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:2/891: sync 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:5/940: sync 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:9/701: sync 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:6/801: sync 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:7/913: sync 2024-05-25T06:11:48.023 INFO:tasks.workunit.client.0.smithi160.stdout:1/788: sync 2024-05-25T06:11:48.025 INFO:tasks.workunit.client.0.smithi160.stdout:6/802: fdatasync d5/d45/f8e 0 2024-05-25T06:11:48.026 INFO:tasks.workunit.client.0.smithi160.stdout:5/941: dread - db/dc/d13/d1d/ff9 zero size 2024-05-25T06:11:48.026 INFO:tasks.workunit.client.0.smithi160.stdout:9/702: write d4/d11/d1e/d3e/d2f/d79/fc0 [497995,129488] 0 2024-05-25T06:11:48.048 INFO:tasks.workunit.client.0.smithi160.stdout:7/914: creat d6/d2b/d5d/f126 x:0 0 0 2024-05-25T06:11:48.049 INFO:tasks.workunit.client.0.smithi160.stdout:1/789: creat de/f10c x:0 0 0 2024-05-25T06:11:48.050 INFO:tasks.workunit.client.0.smithi160.stdout:3/834: fsync d19/d2f/d34/f4d 0 2024-05-25T06:11:48.052 INFO:tasks.workunit.client.0.smithi160.stdout:6/803: rename d5/d45/d4a/d99 to d5/d39/dc1/d113/d115 0 2024-05-25T06:11:48.054 INFO:tasks.workunit.client.0.smithi160.stdout:7/915: mkdir d6/d2b/d127 0 2024-05-25T06:11:48.055 INFO:tasks.workunit.client.0.smithi160.stdout:6/804: read - d5/d39/d8f/d2b/ff2 zero size 2024-05-25T06:11:48.056 INFO:tasks.workunit.client.0.smithi160.stdout:2/892: dwrite d12/d3e/d5f/df3/d10e/f11d [0,4194304] 0 2024-05-25T06:11:48.056 INFO:tasks.workunit.client.0.smithi160.stdout:7/916: chown d6/d32/d5b/d10e 5265 1 2024-05-25T06:11:48.059 INFO:tasks.workunit.client.0.smithi160.stdout:2/893: write d12/d3e/d5f/d74/d7a/d104/f121 [240913,90226] 0 2024-05-25T06:11:48.061 INFO:tasks.workunit.client.0.smithi160.stdout:2/894: dread d12/f8f [0,4194304] 0 2024-05-25T06:11:48.061 INFO:tasks.workunit.client.0.smithi160.stdout:7/917: mkdir d6/d32/d5b/d10e/d128 0 2024-05-25T06:11:48.062 INFO:tasks.workunit.client.0.smithi160.stdout:2/895: read - d12/d3c/f9e zero size 2024-05-25T06:11:48.063 INFO:tasks.workunit.client.0.smithi160.stdout:6/805: unlink d5/d30/d36/d73/d6e/f96 0 2024-05-25T06:11:48.064 INFO:tasks.workunit.client.0.smithi160.stdout:2/896: chown d12/d3e/d5a/d8d/df9/l101 53 1 2024-05-25T06:11:48.072 INFO:tasks.workunit.client.0.smithi160.stdout:1/790: write f7 [1852728,63599] 0 2024-05-25T06:11:48.073 INFO:tasks.workunit.client.0.smithi160.stdout:3/835: dwrite f11 [4194304,4194304] 0 2024-05-25T06:11:48.074 INFO:tasks.workunit.client.0.smithi160.stdout:7/918: mkdir d6/dba/d129 0 2024-05-25T06:11:48.077 INFO:tasks.workunit.client.0.smithi160.stdout:1/791: fdatasync de/d39/d8f/fbc 0 2024-05-25T06:11:48.077 INFO:tasks.workunit.client.0.smithi160.stdout:7/919: dread - d6/d32/dce/fe3 zero size 2024-05-25T06:11:48.077 INFO:tasks.workunit.client.0.smithi160.stdout:3/836: stat d19/d2f/d38/d43/d5b/d65/l7a 0 2024-05-25T06:11:48.077 INFO:tasks.workunit.client.0.smithi160.stdout:5/942: rename db/dc/d13/d1d/d23/d45/f5f to db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/d113/f11a 0 2024-05-25T06:11:48.080 INFO:tasks.workunit.client.0.smithi160.stdout:1/792: write de/d21/d49/d1c/d59/d71/f5a [667868,44676] 0 2024-05-25T06:11:48.084 INFO:tasks.workunit.client.0.smithi160.stdout:9/703: dread d4/d11/d12/d58/f64 [0,4194304] 0 2024-05-25T06:11:48.085 INFO:tasks.workunit.client.0.smithi160.stdout:3/837: creat d19/d2f/d38/d43/d5b/d65/d83/dee/f114 x:0 0 0 2024-05-25T06:11:48.086 INFO:tasks.workunit.client.0.smithi160.stdout:7/920: getdents d6/d2b/d127 0 2024-05-25T06:11:48.088 INFO:tasks.workunit.client.0.smithi160.stdout:5/943: rename db/dc/f83 to db/dc/d13/d9a/f11b 0 2024-05-25T06:11:48.093 INFO:tasks.workunit.client.0.smithi160.stdout:6/806: dwrite d5/d45/d4a/dbc/dc4/fce [0,4194304] 0 2024-05-25T06:11:48.094 INFO:tasks.workunit.client.0.smithi160.stdout:3/838: creat d19/d2f/d38/d43/d5b/d65/d83/dee/f115 x:0 0 0 2024-05-25T06:11:48.095 INFO:tasks.workunit.client.0.smithi160.stdout:5/944: readlink db/dc/d13/d1d/d31/d3b/d69/l71 0 2024-05-25T06:11:48.095 INFO:tasks.workunit.client.0.smithi160.stdout:2/897: dwrite d12/d3c/fc2 [4194304,4194304] 0 2024-05-25T06:11:48.097 INFO:tasks.workunit.client.0.smithi160.stdout:6/807: chown d5/de1/c59 2920 1 2024-05-25T06:11:48.104 INFO:tasks.workunit.client.0.smithi160.stdout:7/921: truncate d6/d32/d92/f9c 214982 0 2024-05-25T06:11:48.105 INFO:tasks.workunit.client.0.smithi160.stdout:3/839: symlink d19/d2f/d38/d60/d10f/l116 0 2024-05-25T06:11:48.106 INFO:tasks.workunit.client.0.smithi160.stdout:6/808: rmdir d5/de1 39 2024-05-25T06:11:48.107 INFO:tasks.workunit.client.0.smithi160.stdout:2/898: creat d12/d3e/d5a/d8d/dc9/f12a x:0 0 0 2024-05-25T06:11:48.109 INFO:tasks.workunit.client.0.smithi160.stdout:9/704: dwrite d4/fc2 [0,4194304] 0 2024-05-25T06:11:48.111 INFO:tasks.workunit.client.0.smithi160.stdout:7/922: rmdir d6/d32/d5b 39 2024-05-25T06:11:48.112 INFO:tasks.workunit.client.0.smithi160.stdout:5/945: creat db/d47/f11c x:0 0 0 2024-05-25T06:11:48.113 INFO:tasks.workunit.client.0.smithi160.stdout:1/793: dwrite de/d39/f72 [0,4194304] 0 2024-05-25T06:11:48.116 INFO:tasks.workunit.client.0.smithi160.stdout:9/705: fsync d4/d11/d1e/d3e/d25/d27/f77 0 2024-05-25T06:11:48.116 INFO:tasks.workunit.client.0.smithi160.stdout:1/794: chown de/f3a 3867 1 2024-05-25T06:11:48.120 INFO:tasks.workunit.client.0.smithi160.stdout:7/923: mknod d6/d32/dce/c12a 0 2024-05-25T06:11:48.122 INFO:tasks.workunit.client.0.smithi160.stdout:3/840: dwrite d19/d2f/d38/d47/f4c [0,4194304] 0 2024-05-25T06:11:48.122 INFO:tasks.workunit.client.0.smithi160.stdout:7/924: chown d6/d3e/d59/c11a 64771 1 2024-05-25T06:11:48.124 INFO:tasks.workunit.client.0.smithi160.stdout:5/946: creat db/d47/d5e/f11d x:0 0 0 2024-05-25T06:11:48.124 INFO:tasks.workunit.client.0.smithi160.stdout:2/899: truncate d12/d3e/d5a/d8d/fb9 494639 0 2024-05-25T06:11:48.126 INFO:tasks.workunit.client.0.smithi160.stdout:5/947: fsync db/dc/d13/d1d/d23/dbd/ff0 0 2024-05-25T06:11:48.127 INFO:tasks.workunit.client.0.smithi160.stdout:9/706: chown d4/d11/d1e/d3e/d25/d27/l87 441043135 1 2024-05-25T06:11:48.129 INFO:tasks.workunit.client.0.smithi160.stdout:6/809: creat d5/de/d32/d74/f116 x:0 0 0 2024-05-25T06:11:48.132 INFO:tasks.workunit.client.0.smithi160.stdout:1/795: rename de/d21/d49/d28/l48 to de/d21/d49/d1c/d59/d71/l10d 0 2024-05-25T06:11:48.137 INFO:tasks.workunit.client.0.smithi160.stdout:2/900: creat d12/d5c/d85/dc1/dea/f12b x:0 0 0 2024-05-25T06:11:48.139 INFO:tasks.workunit.client.0.smithi160.stdout:6/810: rename d5/de/d58/f5b to d5/d39/d112/d107/f117 0 2024-05-25T06:11:48.141 INFO:tasks.workunit.client.0.smithi160.stdout:2/901: chown d12/d1c/d2a/c58 6491 1 2024-05-25T06:11:48.146 INFO:tasks.workunit.client.0.smithi160.stdout:2/902: rmdir d12/d1c/d2a 39 2024-05-25T06:11:48.146 INFO:tasks.workunit.client.0.smithi160.stdout:1/796: dwrite de/d21/d49/d1c/d41/ffc [0,4194304] 0 2024-05-25T06:11:48.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: pgmap v24: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 158 MiB/s rd, 390 MiB/s wr, 800 op/s 2024-05-25T06:11:48.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:48.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: Adjusting osd_memory_target on smithi203 to 2581M 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:11:48.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:11:48.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:47 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:11:48.151 INFO:tasks.workunit.client.0.smithi160.stdout:1/797: write de/d21/d49/d1c/d41/d45/d7d/db3/fed [912904,9757] 0 2024-05-25T06:11:48.152 INFO:tasks.workunit.client.0.smithi160.stdout:2/903: creat d12/d3c/df1/f12c x:0 0 0 2024-05-25T06:11:48.154 INFO:tasks.workunit.client.0.smithi160.stdout:6/811: link d5/de1/fc7 d5/d39/d8f/f118 0 2024-05-25T06:11:48.158 INFO:tasks.workunit.client.0.smithi160.stdout:2/904: symlink d12/d1c/d37/d4b/d103/l12d 0 2024-05-25T06:11:48.163 INFO:tasks.workunit.client.0.smithi160.stdout:6/812: dread - d5/d39/d112/da4/fb2 zero size 2024-05-25T06:11:48.164 INFO:tasks.workunit.client.0.smithi160.stdout:1/798: getdents de/d39 0 2024-05-25T06:11:48.166 INFO:tasks.workunit.client.0.smithi160.stdout:5/948: dwrite db/d108/f2f [0,4194304] 0 2024-05-25T06:11:48.167 INFO:tasks.workunit.client.0.smithi160.stdout:2/905: mkdir d12/d3e/d5a/d8d/dc9/d12e 0 2024-05-25T06:11:48.169 INFO:tasks.workunit.client.0.smithi160.stdout:3/841: dwrite d19/d2f/d38/d43/f53 [0,4194304] 0 2024-05-25T06:11:48.170 INFO:tasks.workunit.client.0.smithi160.stdout:9/707: dwrite d4/d11/d12/d2e/d45/f9e [0,4194304] 0 2024-05-25T06:11:48.172 INFO:tasks.workunit.client.0.smithi160.stdout:6/813: truncate d5/d45/d4a/fa8 383703 0 2024-05-25T06:11:48.174 INFO:tasks.workunit.client.0.smithi160.stdout:6/814: truncate d5/d45/d63/fc8 1164696 0 2024-05-25T06:11:48.182 INFO:tasks.workunit.client.0.smithi160.stdout:1/799: symlink de/d21/d49/d1c/d52/dc7/df6/l10e 0 2024-05-25T06:11:48.183 INFO:tasks.workunit.client.0.smithi160.stdout:6/815: dwrite d5/de/d32/fd7 [0,4194304] 0 2024-05-25T06:11:48.183 INFO:tasks.workunit.client.0.smithi160.stdout:3/842: mkdir d19/d2f/d38/d43/d5b/d65/d83/ddf/d117 0 2024-05-25T06:11:48.186 INFO:tasks.workunit.client.0.smithi160.stdout:3/843: write d19/d2f/d34/fa1 [495196,87154] 0 2024-05-25T06:11:48.191 INFO:tasks.workunit.client.0.smithi160.stdout:3/844: read d19/d2f/f94 [131246,55552] 0 2024-05-25T06:11:48.192 INFO:tasks.workunit.client.0.smithi160.stdout:6/816: rename d5/d45/d4a/dba to d5/d39/d112/d119 0 2024-05-25T06:11:48.193 INFO:tasks.workunit.client.0.smithi160.stdout:2/906: dwrite d12/d1c/f2c [4194304,4194304] 0 2024-05-25T06:11:48.194 INFO:tasks.workunit.client.0.smithi160.stdout:1/800: creat de/d21/d49/d1c/d59/d7c/d93/de8/f10f x:0 0 0 2024-05-25T06:11:48.194 INFO:tasks.workunit.client.0.smithi160.stdout:3/845: chown d19/d2f/d34/f55 15341 1 2024-05-25T06:11:48.194 INFO:tasks.workunit.client.0.smithi160.stdout:6/817: chown d5/de/d32/d35/dab/ffb 0 1 2024-05-25T06:11:48.196 INFO:tasks.workunit.client.0.smithi160.stdout:6/818: readlink d5/d39/d112/ldc 0 2024-05-25T06:11:48.197 INFO:tasks.workunit.client.0.smithi160.stdout:9/708: dread d4/d11/d1e/d3e/d25/fb3 [0,4194304] 0 2024-05-25T06:11:48.197 INFO:tasks.workunit.client.0.smithi160.stdout:3/846: symlink d19/d2f/d38/d47/l118 0 2024-05-25T06:11:48.198 INFO:tasks.workunit.client.0.smithi160.stdout:2/907: fdatasync d12/d1c/d37/f3b 0 2024-05-25T06:11:48.201 INFO:tasks.workunit.client.0.smithi160.stdout:9/709: creat d4/d11/d1e/d3e/d25/d5e/da2/fe2 x:0 0 0 2024-05-25T06:11:48.204 INFO:tasks.workunit.client.0.smithi160.stdout:6/819: getdents d5/de/d9e 0 2024-05-25T06:11:48.205 INFO:tasks.workunit.client.0.smithi160.stdout:6/820: write d5/de/d32/f10e [976349,8516] 0 2024-05-25T06:11:48.206 INFO:tasks.workunit.client.0.smithi160.stdout:9/710: unlink d4/d11/d12/f48 0 2024-05-25T06:11:48.208 INFO:tasks.workunit.client.0.smithi160.stdout:3/847: rename d19/d2f/d38/d43/d5b/d65/d7e/f106 to d19/d2f/d34/def/f119 0 2024-05-25T06:11:48.209 INFO:tasks.workunit.client.0.smithi160.stdout:3/848: fsync d19/d2f/d38/d47/f54 0 2024-05-25T06:11:48.210 INFO:tasks.workunit.client.0.smithi160.stdout:9/711: mknod d4/d11/d1e/d3e/d25/d27/ce3 0 2024-05-25T06:11:48.211 INFO:tasks.workunit.client.0.smithi160.stdout:3/849: fsync d19/d2f/d38/d47/f8b 0 2024-05-25T06:11:48.211 INFO:tasks.workunit.client.0.smithi160.stdout:6/821: link d5/de/d7d/ld9 d5/d45/d4a/l11a 0 2024-05-25T06:11:48.214 INFO:tasks.workunit.client.0.smithi160.stdout:3/850: symlink d19/d2f/d38/d60/d10f/l11a 0 2024-05-25T06:11:48.216 INFO:tasks.workunit.client.0.smithi160.stdout:3/851: write d19/d2f/d38/d47/f4c [3149862,29710] 0 2024-05-25T06:11:48.216 INFO:tasks.workunit.client.0.smithi160.stdout:9/712: creat d4/d11/d1e/d3e/d25/d5e/fe4 x:0 0 0 2024-05-25T06:11:48.217 INFO:tasks.workunit.client.0.smithi160.stdout:3/852: readlink d19/d2f/d34/d67/d96/de1/le3 0 2024-05-25T06:11:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: pgmap v24: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 158 MiB/s rd, 390 MiB/s wr, 800 op/s 2024-05-25T06:11:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: Adjusting osd_memory_target on smithi203 to 2581M 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:11:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:11:48.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:47 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:11:48.224 INFO:tasks.workunit.client.0.smithi160.stdout:3/853: mknod d19/d2f/d34/d111/c11b 0 2024-05-25T06:11:48.224 INFO:tasks.workunit.client.0.smithi160.stdout:9/713: dwrite d4/d11/d1e/d3e/d2f/d79/fc0 [0,4194304] 0 2024-05-25T06:11:48.224 INFO:tasks.workunit.client.0.smithi160.stdout:2/908: write d12/d3e/d5a/f93 [712827,40666] 0 2024-05-25T06:11:48.228 INFO:tasks.workunit.client.0.smithi160.stdout:9/714: chown d4/d11/d1e/d3e/d2f/d79/fc3 3746871 1 2024-05-25T06:11:48.230 INFO:tasks.workunit.client.0.smithi160.stdout:6/822: truncate d5/d45/d63/f83 10884 0 2024-05-25T06:11:48.235 INFO:tasks.workunit.client.0.smithi160.stdout:3/854: dwrite d19/d2f/f105 [0,4194304] 0 2024-05-25T06:11:48.236 INFO:tasks.workunit.client.0.smithi160.stdout:9/715: truncate d4/d11/d12/f35 2098493 0 2024-05-25T06:11:48.236 INFO:tasks.workunit.client.0.smithi160.stdout:6/823: mknod d5/d45/d4a/dbc/dea/c11b 0 2024-05-25T06:11:48.237 INFO:tasks.workunit.client.0.smithi160.stdout:2/909: dwrite d12/d75/dca/f55 [0,4194304] 0 2024-05-25T06:11:48.237 INFO:tasks.workunit.client.0.smithi160.stdout:6/824: read d5/de/d32/fd7 [3852808,14014] 0 2024-05-25T06:11:48.239 INFO:tasks.workunit.client.0.smithi160.stdout:6/825: fdatasync d5/f11 0 2024-05-25T06:11:48.241 INFO:tasks.workunit.client.0.smithi160.stdout:9/716: dread - d4/d11/d1e/d3e/d25/d27/fce zero size 2024-05-25T06:11:48.243 INFO:tasks.workunit.client.0.smithi160.stdout:9/717: chown d4/d11/d14/f31 4480 1 2024-05-25T06:11:48.245 INFO:tasks.workunit.client.0.smithi160.stdout:2/910: symlink d12/d1c/l12f 0 2024-05-25T06:11:48.248 INFO:tasks.workunit.client.0.smithi160.stdout:6/826: chown d5/de/l1c 3 1 2024-05-25T06:11:48.250 INFO:tasks.workunit.client.0.smithi160.stdout:3/855: dwrite d19/d2f/d38/fce [0,4194304] 0 2024-05-25T06:11:48.251 INFO:tasks.workunit.client.0.smithi160.stdout:9/718: rmdir d4/d11/d1e/d3e/d25/d27/d95 39 2024-05-25T06:11:48.252 INFO:tasks.workunit.client.0.smithi160.stdout:2/911: rename d12/d1c to d12/d1c/d59/d130 22 2024-05-25T06:11:48.252 INFO:tasks.workunit.client.0.smithi160.stdout:9/719: dread - d4/d11/d14/d8e/fad zero size 2024-05-25T06:11:48.253 INFO:tasks.workunit.client.0.smithi160.stdout:3/856: creat d19/da7/f11c x:0 0 0 2024-05-25T06:11:48.255 INFO:tasks.workunit.client.0.smithi160.stdout:9/720: mkdir d4/db2/de5 0 2024-05-25T06:11:48.258 INFO:tasks.workunit.client.0.smithi160.stdout:6/827: rename d5/d45/d4a/fbd to d5/d30/d36/d73/d6e/f11c 0 2024-05-25T06:11:48.260 INFO:tasks.workunit.client.0.smithi160.stdout:9/721: fdatasync d4/d11/d1e/d24/f82 0 2024-05-25T06:11:48.266 INFO:tasks.workunit.client.0.smithi160.stdout:9/722: creat d4/d11/d12/d2e/d45/fe6 x:0 0 0 2024-05-25T06:11:48.268 INFO:tasks.workunit.client.0.smithi160.stdout:3/857: link cc d19/d2f/d38/d43/d5b/d65/d7e/c11d 0 2024-05-25T06:11:48.274 INFO:tasks.workunit.client.0.smithi160.stdout:3/858: unlink d19/d2f/d38/d60/fc4 0 2024-05-25T06:11:48.275 INFO:tasks.workunit.client.0.smithi160.stdout:6/828: link d5/de/d32/d74/lf4 d5/d39/d8f/d2b/l11d 0 2024-05-25T06:11:48.277 INFO:tasks.workunit.client.0.smithi160.stdout:9/723: rename d4/d11/d1e/d3e/d25/ldf to d4/d11/d1e/d6c/le7 0 2024-05-25T06:11:48.278 INFO:tasks.workunit.client.0.smithi160.stdout:3/859: fsync d19/d2f/d38/d43/f4e 0 2024-05-25T06:11:48.279 INFO:tasks.workunit.client.0.smithi160.stdout:3/860: fsync d19/d2f/f94 0 2024-05-25T06:11:48.280 INFO:tasks.workunit.client.0.smithi160.stdout:6/829: creat d5/d39/d112/d107/f11e x:0 0 0 2024-05-25T06:11:48.282 INFO:tasks.workunit.client.0.smithi160.stdout:6/830: chown d5/d39/d112/da4/fb2 15 1 2024-05-25T06:11:48.282 INFO:tasks.workunit.client.0.smithi160.stdout:2/912: dread d12/d1c/f56 [0,4194304] 0 2024-05-25T06:11:48.284 INFO:tasks.workunit.client.0.smithi160.stdout:3/861: mknod d19/d22/c11e 0 2024-05-25T06:11:48.290 INFO:tasks.workunit.client.0.smithi160.stdout:3/862: creat d19/d2f/d38/d60/f11f x:0 0 0 2024-05-25T06:11:48.290 INFO:tasks.workunit.client.0.smithi160.stdout:9/724: getdents d4/d11/d12/d58 0 2024-05-25T06:11:48.293 INFO:tasks.workunit.client.0.smithi160.stdout:9/725: mknod d4/d11/d1e/d3e/d25/d5e/da2/ce8 0 2024-05-25T06:11:48.299 INFO:tasks.workunit.client.0.smithi160.stdout:2/913: write d12/d3c/fd5 [1359700,216] 0 2024-05-25T06:11:48.300 INFO:tasks.workunit.client.0.smithi160.stdout:3/863: truncate f0 5110712 0 2024-05-25T06:11:48.301 INFO:tasks.workunit.client.0.smithi160.stdout:2/914: fsync d12/d75/dca/fd6 0 2024-05-25T06:11:48.303 INFO:tasks.workunit.client.0.smithi160.stdout:3/864: mkdir d19/d2f/d38/d43/d5b/d65/d83/d120 0 2024-05-25T06:11:48.303 INFO:tasks.workunit.client.0.smithi160.stdout:6/831: dread d5/f11 [0,4194304] 0 2024-05-25T06:11:48.306 INFO:tasks.workunit.client.0.smithi160.stdout:3/865: unlink d19/d2f/d38/f78 0 2024-05-25T06:11:48.307 INFO:tasks.workunit.client.0.smithi160.stdout:9/726: dread d4/d11/d1e/d3e/d25/f8b [0,4194304] 0 2024-05-25T06:11:48.307 INFO:tasks.workunit.client.0.smithi160.stdout:2/915: rename d12/d1c/c38 to d12/d1c/d2a/c131 0 2024-05-25T06:11:48.308 INFO:tasks.workunit.client.0.smithi160.stdout:6/832: symlink d5/de/d32/d9f/dd5/l11f 0 2024-05-25T06:11:48.314 INFO:tasks.workunit.client.0.smithi160.stdout:2/916: dwrite d12/d1c/f56 [0,4194304] 0 2024-05-25T06:11:48.316 INFO:tasks.workunit.client.0.smithi160.stdout:3/866: getdents d19/d2f/d38/d43/d5b/d65/d83/ddf 0 2024-05-25T06:11:48.317 INFO:tasks.workunit.client.0.smithi160.stdout:2/917: rmdir d12/d3e/d5a 39 2024-05-25T06:11:48.319 INFO:tasks.workunit.client.0.smithi160.stdout:9/727: write d4/d11/d14/fb6 [30321,93018] 0 2024-05-25T06:11:48.321 INFO:tasks.workunit.client.0.smithi160.stdout:2/918: rmdir d12/d1c/d120/dfa 39 2024-05-25T06:11:48.324 INFO:tasks.workunit.client.0.smithi160.stdout:3/867: creat d19/d2f/d38/f121 x:0 0 0 2024-05-25T06:11:48.325 INFO:tasks.workunit.client.0.smithi160.stdout:6/833: dwrite d5/d30/d36/d73/d6e/f11c [0,4194304] 0 2024-05-25T06:11:48.327 INFO:tasks.workunit.client.0.smithi160.stdout:6/834: truncate d5/d45/d4a/dbc/dc4/f100 832781 0 2024-05-25T06:11:48.329 INFO:tasks.workunit.client.0.smithi160.stdout:9/728: creat d4/d11/fe9 x:0 0 0 2024-05-25T06:11:48.330 INFO:tasks.workunit.client.0.smithi160.stdout:3/868: creat d19/d2f/d34/da8/df8/d110/f122 x:0 0 0 2024-05-25T06:11:48.334 INFO:tasks.workunit.client.0.smithi160.stdout:6/835: creat d5/d30/d36/d73/d6e/df7/f120 x:0 0 0 2024-05-25T06:11:48.336 INFO:tasks.workunit.client.0.smithi160.stdout:3/869: mknod d19/d2f/d34/d67/d96/de1/c123 0 2024-05-25T06:11:48.336 INFO:tasks.workunit.client.0.smithi160.stdout:2/919: write f11 [4533556,23199] 0 2024-05-25T06:11:48.339 INFO:tasks.workunit.client.0.smithi160.stdout:3/870: stat d19/d2f/d34/d67/d96/lfe 0 2024-05-25T06:11:48.340 INFO:tasks.workunit.client.0.smithi160.stdout:6/836: truncate d5/de/d32/d74/d77/fb1 646941 0 2024-05-25T06:11:48.341 INFO:tasks.workunit.client.0.smithi160.stdout:3/871: readlink d19/d2f/d38/d43/d5b/d65/d7e/lb8 0 2024-05-25T06:11:48.342 INFO:tasks.workunit.client.0.smithi160.stdout:6/837: write d5/de/d32/f10e [1532199,97303] 0 2024-05-25T06:11:48.344 INFO:tasks.workunit.client.0.smithi160.stdout:6/838: readlink d5/d39/dc1/le0 0 2024-05-25T06:11:48.350 INFO:tasks.workunit.client.0.smithi160.stdout:9/729: dread d4/f6e [0,4194304] 0 2024-05-25T06:11:48.355 INFO:tasks.workunit.client.0.smithi160.stdout:9/730: write d4/d11/d7a/f72 [856582,107724] 0 2024-05-25T06:11:48.359 INFO:tasks.workunit.client.0.smithi160.stdout:2/920: dread d12/d1c/d2a/d7e/fa2 [0,4194304] 0 2024-05-25T06:11:48.360 INFO:tasks.workunit.client.0.smithi160.stdout:9/731: symlink d4/d11/d1e/d3e/d2f/d79/lea 0 2024-05-25T06:11:48.361 INFO:tasks.workunit.client.0.smithi160.stdout:2/921: chown d12/d3e/d5a/d8d/dc9/d12e 30501 1 2024-05-25T06:11:48.362 INFO:tasks.workunit.client.0.smithi160.stdout:6/839: dread d5/d39/d8f/d2b/f88 [0,4194304] 0 2024-05-25T06:11:48.362 INFO:tasks.workunit.client.0.smithi160.stdout:9/732: unlink d4/d11/f67 0 2024-05-25T06:11:48.365 INFO:tasks.workunit.client.0.smithi160.stdout:3/872: dread d19/d2f/d38/d43/d5b/d65/d7f/f81 [0,4194304] 0 2024-05-25T06:11:48.365 INFO:tasks.workunit.client.0.smithi160.stdout:2/922: creat d12/d1c/d120/f132 x:0 0 0 2024-05-25T06:11:48.365 INFO:tasks.workunit.client.0.smithi160.stdout:9/733: truncate d4/d11/d1e/f1f 1695130 0 2024-05-25T06:11:48.367 INFO:tasks.workunit.client.0.smithi160.stdout:2/923: fdatasync d12/d5c/f72 0 2024-05-25T06:11:48.368 INFO:tasks.workunit.client.0.smithi160.stdout:3/873: mknod d19/c124 0 2024-05-25T06:11:48.368 INFO:tasks.workunit.client.0.smithi160.stdout:9/734: dread - d4/d11/d1e/d3e/d2f/d79/fe0 zero size 2024-05-25T06:11:48.370 INFO:tasks.workunit.client.0.smithi160.stdout:3/874: chown d19/d2f/d38/d43/f53 95193746 1 2024-05-25T06:11:48.371 INFO:tasks.workunit.client.0.smithi160.stdout:6/840: dwrite d5/de/d32/d74/f116 [0,4194304] 0 2024-05-25T06:11:48.376 INFO:tasks.workunit.client.0.smithi160.stdout:6/841: creat d5/d39/d8f/d2b/f121 x:0 0 0 2024-05-25T06:11:48.378 INFO:tasks.workunit.client.0.smithi160.stdout:2/924: rmdir d12/d1c/d120 39 2024-05-25T06:11:48.378 INFO:tasks.workunit.client.0.smithi160.stdout:6/842: chown d5/d39/d112/da4/fb2 4597 1 2024-05-25T06:11:48.381 INFO:tasks.workunit.client.0.smithi160.stdout:3/875: creat d19/d2f/d38/d43/d5b/d65/d83/ddf/d117/f125 x:0 0 0 2024-05-25T06:11:48.384 INFO:tasks.workunit.client.0.smithi160.stdout:3/876: rmdir d19/db2 39 2024-05-25T06:11:48.387 INFO:tasks.workunit.client.0.smithi160.stdout:6/843: dwrite d5/d39/d112/da4/fe7 [0,4194304] 0 2024-05-25T06:11:48.387 INFO:tasks.workunit.client.0.smithi160.stdout:2/925: mknod d12/c133 0 2024-05-25T06:11:48.390 INFO:tasks.workunit.client.0.smithi160.stdout:3/877: read d19/d22/f24 [3943413,98866] 0 2024-05-25T06:11:48.390 INFO:tasks.workunit.client.0.smithi160.stdout:2/926: chown d12/d3e/d5a/l67 326022 1 2024-05-25T06:11:48.392 INFO:tasks.workunit.client.0.smithi160.stdout:9/735: dwrite d4/d11/f38 [0,4194304] 0 2024-05-25T06:11:48.394 INFO:tasks.workunit.client.0.smithi160.stdout:9/736: write d4/d11/f26 [4460853,85886] 0 2024-05-25T06:11:48.395 INFO:tasks.workunit.client.0.smithi160.stdout:2/927: read - d12/d3c/fce zero size 2024-05-25T06:11:48.399 INFO:tasks.workunit.client.0.smithi160.stdout:3/878: creat d19/d2f/dad/f126 x:0 0 0 2024-05-25T06:11:48.399 INFO:tasks.workunit.client.0.smithi160.stdout:6/844: creat d5/d45/d4a/dbc/dea/f122 x:0 0 0 2024-05-25T06:11:48.399 INFO:tasks.workunit.client.0.smithi160.stdout:9/737: rename d4/f6e to d4/d11/d7a/feb 0 2024-05-25T06:11:48.400 INFO:tasks.workunit.client.0.smithi160.stdout:2/928: mknod d12/d3e/d5a/d8d/df9/d100/c134 0 2024-05-25T06:11:48.401 INFO:tasks.workunit.client.0.smithi160.stdout:9/738: write d4/d11/d1e/d3e/d25/d27/fce [652172,91462] 0 2024-05-25T06:11:48.406 INFO:tasks.workunit.client.0.smithi160.stdout:9/739: dwrite d4/d11/d1e/d24/d66/f94 [4194304,4194304] 0 2024-05-25T06:11:48.407 INFO:tasks.workunit.client.0.smithi160.stdout:2/929: rename d12/d3e/d5f to d12/d75/dae/d135 0 2024-05-25T06:11:48.408 INFO:tasks.workunit.client.0.smithi160.stdout:9/740: dread - d4/d11/d1e/d3e/d25/d5e/da2/fe2 zero size 2024-05-25T06:11:48.411 INFO:tasks.workunit.client.0.smithi160.stdout:2/930: mknod d12/d1c/d37/d4b/c136 0 2024-05-25T06:11:48.457 INFO:tasks.workunit.client.0.smithi160.stdout:8/783: sync 2024-05-25T06:11:48.460 INFO:tasks.workunit.client.0.smithi160.stdout:6/845: link d5/de/d58/l6a d5/d39/dc1/d113/l123 0 2024-05-25T06:11:48.461 INFO:tasks.workunit.client.0.smithi160.stdout:2/931: creat d12/d3e/d5a/d8d/db4/dcb/f137 x:0 0 0 2024-05-25T06:11:48.462 INFO:tasks.workunit.client.0.smithi160.stdout:8/784: read - d5/d7/d9a/fca zero size 2024-05-25T06:11:48.463 INFO:tasks.workunit.client.0.smithi160.stdout:9/741: rmdir d4/d11/d12/d2e/d45/d90/dc1 0 2024-05-25T06:11:48.463 INFO:tasks.workunit.client.0.smithi160.stdout:8/785: chown d5/d7/dcb/fe1 0 1 2024-05-25T06:11:48.463 INFO:tasks.workunit.client.0.smithi160.stdout:2/932: read d12/d75/dae/d135/d113/df4/ff5 [489379,62430] 0 2024-05-25T06:11:48.464 INFO:tasks.workunit.client.0.smithi160.stdout:6/846: rename d5/d39/d8f/d2b/d103 to d5/de/d58/d124 0 2024-05-25T06:11:48.465 INFO:tasks.workunit.client.0.smithi160.stdout:6/847: write d5/f3f [5195657,21797] 0 2024-05-25T06:11:48.466 INFO:tasks.workunit.client.0.smithi160.stdout:8/786: read - d5/d44/d1f/da3/d28/d4a/d93/fab zero size 2024-05-25T06:11:48.467 INFO:tasks.workunit.client.0.smithi160.stdout:6/848: write d5/d39/f51 [3366084,110464] 0 2024-05-25T06:11:48.468 INFO:tasks.workunit.client.0.smithi160.stdout:2/933: creat d12/d1c/d37/d4b/f138 x:0 0 0 2024-05-25T06:11:48.471 INFO:tasks.workunit.client.0.smithi160.stdout:8/787: mkdir d5/d44/d1f/da3/d28/de6/dc2/dd6/d106 0 2024-05-25T06:11:48.471 INFO:tasks.workunit.client.0.smithi160.stdout:8/788: chown d5/d44/dba/ddb/de3 21 1 2024-05-25T06:11:48.472 INFO:tasks.workunit.client.0.smithi160.stdout:6/849: creat d5/d39/d112/da4/f125 x:0 0 0 2024-05-25T06:11:48.479 INFO:tasks.workunit.client.0.smithi160.stdout:8/789: unlink d5/ld 0 2024-05-25T06:11:48.480 INFO:tasks.workunit.client.0.smithi160.stdout:6/850: mkdir d5/d126 0 2024-05-25T06:11:48.485 INFO:tasks.workunit.client.0.smithi160.stdout:2/934: write d12/d3e/f41 [323545,29171] 0 2024-05-25T06:11:48.488 INFO:tasks.workunit.client.0.smithi160.stdout:9/742: dwrite d4/d11/d1e/f1f [0,4194304] 0 2024-05-25T06:11:48.488 INFO:tasks.workunit.client.0.smithi160.stdout:2/935: write d12/d3c/f126 [830675,28486] 0 2024-05-25T06:11:48.490 INFO:tasks.workunit.client.0.smithi160.stdout:8/790: truncate d5/d7/dcb/d7a/fcd 3868016 0 2024-05-25T06:11:48.491 INFO:tasks.workunit.client.0.smithi160.stdout:6/851: dwrite d5/d39/d112/d107/f11e [0,4194304] 0 2024-05-25T06:11:48.493 INFO:tasks.workunit.client.0.smithi160.stdout:2/936: write d12/d75/dae/d135/fef [954329,54763] 0 2024-05-25T06:11:48.496 INFO:tasks.workunit.client.0.smithi160.stdout:6/852: fdatasync d5/d39/d8f/d2b/f106 0 2024-05-25T06:11:48.498 INFO:tasks.workunit.client.0.smithi160.stdout:2/937: write d12/d75/dae/d135/d113/faa [388288,11050] 0 2024-05-25T06:11:48.503 INFO:tasks.workunit.client.0.smithi160.stdout:6/853: dwrite d5/de/d32/f10e [0,4194304] 0 2024-05-25T06:11:48.505 INFO:tasks.workunit.client.0.smithi160.stdout:2/938: mknod d12/d1c/d37/d4b/d103/c139 0 2024-05-25T06:11:48.505 INFO:tasks.workunit.client.0.smithi160.stdout:9/743: link d4/d11/d1e/d3e/d2f/d79/f9b d4/d11/d12/d2e/fec 0 2024-05-25T06:11:48.507 INFO:tasks.workunit.client.0.smithi160.stdout:6/854: write d5/d39/d8f/f105 [696995,86632] 0 2024-05-25T06:11:48.509 INFO:tasks.workunit.client.0.smithi160.stdout:6/855: readlink d5/d39/l66 0 2024-05-25T06:11:48.509 INFO:tasks.workunit.client.0.smithi160.stdout:2/939: fdatasync d12/d1c/d37/d4b/f89 0 2024-05-25T06:11:48.511 INFO:tasks.workunit.client.0.smithi160.stdout:8/791: dwrite d5/d7/dcb/fa0 [0,4194304] 0 2024-05-25T06:11:48.512 INFO:tasks.workunit.client.0.smithi160.stdout:2/940: chown d12/d75/dca/la1 0 1 2024-05-25T06:11:48.513 INFO:tasks.workunit.client.0.smithi160.stdout:9/744: symlink d4/d11/d14/d8e/led 0 2024-05-25T06:11:48.513 INFO:tasks.workunit.client.0.smithi160.stdout:6/856: truncate d5/d39/da6/f10c 19346 0 2024-05-25T06:11:48.514 INFO:tasks.workunit.client.0.smithi160.stdout:6/857: stat d5/d39/f51 0 2024-05-25T06:11:48.514 INFO:tasks.workunit.client.0.smithi160.stdout:2/941: write d12/d75/dae/d135/df3/d10e/f11d [1686953,10225] 0 2024-05-25T06:11:48.516 INFO:tasks.workunit.client.0.smithi160.stdout:8/792: mkdir d5/d44/d1f/da3/d28/dc6/de8/d107 0 2024-05-25T06:11:48.516 INFO:tasks.workunit.client.0.smithi160.stdout:9/745: dread d4/d11/fcf [0,4194304] 0 2024-05-25T06:11:48.517 INFO:tasks.workunit.client.0.smithi160.stdout:6/858: truncate d5/d30/d36/f111 809249 0 2024-05-25T06:11:48.518 INFO:tasks.workunit.client.0.smithi160.stdout:9/746: readlink d4/d11/d1e/d3e/d25/d27/l7c 0 2024-05-25T06:11:48.521 INFO:tasks.workunit.client.0.smithi160.stdout:8/793: unlink d5/d44/d1f/l6e 0 2024-05-25T06:11:48.523 INFO:tasks.workunit.client.0.smithi160.stdout:9/747: mknod d4/d11/d1e/d24/d66/cee 0 2024-05-25T06:11:48.527 INFO:tasks.workunit.client.0.smithi160.stdout:8/794: truncate d5/d7/dcb/d75/f76 130417 0 2024-05-25T06:11:48.528 INFO:tasks.workunit.client.0.smithi160.stdout:9/748: read d4/f69 [760825,72157] 0 2024-05-25T06:11:48.528 INFO:tasks.workunit.client.0.smithi160.stdout:6/859: dwrite d5/de/d7d/fcb [0,4194304] 0 2024-05-25T06:11:48.534 INFO:tasks.workunit.client.0.smithi160.stdout:6/860: rename d5/d39/dc1/d113/d115/la1 to d5/d30/d36/d73/d6e/l127 0 2024-05-25T06:11:48.537 INFO:tasks.workunit.client.0.smithi160.stdout:9/749: dwrite d4/d11/d7a/f72 [0,4194304] 0 2024-05-25T06:11:48.539 INFO:tasks.workunit.client.0.smithi160.stdout:6/861: mknod d5/d45/d4a/dbc/dc4/c128 0 2024-05-25T06:11:48.548 INFO:tasks.workunit.client.0.smithi160.stdout:2/942: dwrite d12/d5c/d85/fb6 [0,4194304] 0 2024-05-25T06:11:48.550 INFO:tasks.workunit.client.0.smithi160.stdout:6/862: write d5/de/d32/fd7 [857214,124254] 0 2024-05-25T06:11:48.551 INFO:tasks.workunit.client.0.smithi160.stdout:2/943: creat d12/d5c/d85/d9b/f13a x:0 0 0 2024-05-25T06:11:48.552 INFO:tasks.workunit.client.0.smithi160.stdout:2/944: chown d12/d3e/d5a/d8d/df9 753 1 2024-05-25T06:11:48.552 INFO:tasks.workunit.client.0.smithi160.stdout:6/863: mknod d5/de/d58/c129 0 2024-05-25T06:11:48.553 INFO:tasks.workunit.client.0.smithi160.stdout:6/864: chown d5/d30/f5a 25321783 1 2024-05-25T06:11:48.554 INFO:tasks.workunit.client.0.smithi160.stdout:6/865: readlink d5/d39/d8f/lb5 0 2024-05-25T06:11:48.556 INFO:tasks.workunit.client.0.smithi160.stdout:6/866: write d5/d39/d8f/d2b/f121 [316871,46151] 0 2024-05-25T06:11:48.557 INFO:tasks.workunit.client.0.smithi160.stdout:6/867: readlink d5/d39/d8f/lc2 0 2024-05-25T06:11:48.559 INFO:tasks.workunit.client.0.smithi160.stdout:6/868: dread - d5/d45/fe2 zero size 2024-05-25T06:11:48.559 INFO:tasks.workunit.client.0.smithi160.stdout:2/945: dwrite d12/d3c/df1/f12c [0,4194304] 0 2024-05-25T06:11:48.563 INFO:tasks.workunit.client.0.smithi160.stdout:2/946: truncate d12/d1c/d2a/d7e/fa2 4472972 0 2024-05-25T06:11:48.564 INFO:tasks.workunit.client.0.smithi160.stdout:6/869: truncate d5/f3f 3324301 0 2024-05-25T06:11:48.566 INFO:tasks.workunit.client.0.smithi160.stdout:6/870: mkdir d5/d39/dc1/d113/d12a 0 2024-05-25T06:11:48.567 INFO:tasks.workunit.client.0.smithi160.stdout:6/871: fdatasync d5/d45/d4a/dbc/dc4/fce 0 2024-05-25T06:11:48.567 INFO:tasks.workunit.client.0.smithi160.stdout:2/947: rename d12/d1c/d2a/d7e/d90/df2 to d12/d1c/d37/d4b/d13b 0 2024-05-25T06:11:48.568 INFO:tasks.workunit.client.0.smithi160.stdout:6/872: fdatasync d5/d39/f10f 0 2024-05-25T06:11:48.571 INFO:tasks.workunit.client.0.smithi160.stdout:6/873: creat d5/d30/d36/d73/d6e/f12b x:0 0 0 2024-05-25T06:11:48.578 INFO:tasks.workunit.client.0.smithi160.stdout:2/948: write d12/d5c/f6b [1173780,21861] 0 2024-05-25T06:11:48.580 INFO:tasks.workunit.client.0.smithi160.stdout:6/874: dread d5/d45/d63/f83 [0,4194304] 0 2024-05-25T06:11:48.580 INFO:tasks.workunit.client.0.smithi160.stdout:2/949: dread - d12/d75/dae/d135/d74/ffd zero size 2024-05-25T06:11:48.581 INFO:tasks.workunit.client.0.smithi160.stdout:6/875: chown d5/d39/d8f/f55 244 1 2024-05-25T06:11:48.582 INFO:tasks.workunit.client.0.smithi160.stdout:2/950: symlink d12/d75/dae/d135/l13c 0 2024-05-25T06:11:48.584 INFO:tasks.workunit.client.0.smithi160.stdout:6/876: mkdir d5/d39/da6/dac/de6/d12c 0 2024-05-25T06:11:48.586 INFO:tasks.workunit.client.0.smithi160.stdout:6/877: dread - d5/de/d7d/fd8 zero size 2024-05-25T06:11:48.587 INFO:tasks.workunit.client.0.smithi160.stdout:6/878: readlink d5/d39/l110 0 2024-05-25T06:11:48.588 INFO:tasks.workunit.client.0.smithi160.stdout:6/879: readlink d5/d45/d4a/lad 0 2024-05-25T06:11:48.590 INFO:tasks.workunit.client.0.smithi160.stdout:2/951: dwrite d12/d1c/d2a/f3a [0,4194304] 0 2024-05-25T06:11:48.592 INFO:tasks.workunit.client.0.smithi160.stdout:6/880: symlink d5/de/d32/d74/d77/l12d 0 2024-05-25T06:11:48.595 INFO:tasks.workunit.client.0.smithi160.stdout:2/952: truncate d12/d75/dae/d135/d74/fb2 1131362 0 2024-05-25T06:11:48.598 INFO:tasks.workunit.client.0.smithi160.stdout:6/881: rename d5/d45/d63 to d5/de/d32/d35/d12e 0 2024-05-25T06:11:48.602 INFO:tasks.workunit.client.0.smithi160.stdout:6/882: creat d5/d39/da6/f12f x:0 0 0 2024-05-25T06:11:48.604 INFO:tasks.workunit.client.0.smithi160.stdout:6/883: symlink d5/d45/d4a/l130 0 2024-05-25T06:11:48.611 INFO:tasks.workunit.client.0.smithi160.stdout:2/953: dwrite d12/d75/dae/d135/d113/df4/ff5 [0,4194304] 0 2024-05-25T06:11:48.617 INFO:tasks.workunit.client.0.smithi160.stdout:2/954: symlink d12/d1c/d37/d4b/d103/l13d 0 2024-05-25T06:11:48.775 INFO:tasks.workunit.client.0.smithi160.stdout:4/832: sync 2024-05-25T06:11:48.775 INFO:tasks.workunit.client.0.smithi160.stdout:0/965: sync 2024-05-25T06:11:48.775 INFO:tasks.workunit.client.0.smithi160.stdout:7/925: sync 2024-05-25T06:11:48.776 INFO:tasks.workunit.client.0.smithi160.stdout:0/966: write d2b/d46/d63/f9f [1586098,61132] 0 2024-05-25T06:11:48.779 INFO:tasks.workunit.client.0.smithi160.stdout:7/926: truncate d6/d3e/da1/ff1 6962340 0 2024-05-25T06:11:48.793 INFO:tasks.workunit.client.0.smithi160.stdout:4/833: rename d4/d6/d28/f4e to d4/d1c/f109 0 2024-05-25T06:11:48.796 INFO:tasks.workunit.client.0.smithi160.stdout:0/967: rename d2b/d31/d35/d43/dab/f131 to d2b/d46/d63/da7/d67/f136 0 2024-05-25T06:11:48.799 INFO:tasks.workunit.client.0.smithi160.stdout:4/834: rename d4/l34 to d4/d6/dbd/df0/l10a 0 2024-05-25T06:11:48.800 INFO:tasks.workunit.client.0.smithi160.stdout:7/927: write d6/d32/d92/fc9 [2989178,125050] 0 2024-05-25T06:11:48.803 INFO:tasks.workunit.client.0.smithi160.stdout:7/928: fsync d6/d2b/d5d/fc2 0 2024-05-25T06:11:48.804 INFO:tasks.workunit.client.0.smithi160.stdout:4/835: creat d4/d6/d7b/f10b x:0 0 0 2024-05-25T06:11:48.804 INFO:tasks.workunit.client.0.smithi160.stdout:7/929: chown d6/d3e/d59/d65/ccb 845784 1 2024-05-25T06:11:48.806 INFO:tasks.workunit.client.0.smithi160.stdout:4/836: creat d4/d1a/df1/f10c x:0 0 0 2024-05-25T06:11:48.807 INFO:tasks.workunit.client.0.smithi160.stdout:4/837: creat d4/d1c/d54/d9b/f10d x:0 0 0 2024-05-25T06:11:48.809 INFO:tasks.workunit.client.0.smithi160.stdout:4/838: creat d4/d6/d21/f10e x:0 0 0 2024-05-25T06:11:48.812 INFO:tasks.workunit.client.0.smithi160.stdout:4/839: creat d4/d6/d5b/d66/d78/da1/dab/f10f x:0 0 0 2024-05-25T06:11:48.812 INFO:tasks.workunit.client.0.smithi160.stdout:4/840: fsync d4/d63/fed 0 2024-05-25T06:11:48.814 INFO:tasks.workunit.client.0.smithi160.stdout:4/841: creat d4/d6/d5b/d66/f110 x:0 0 0 2024-05-25T06:11:48.816 INFO:tasks.workunit.client.0.smithi160.stdout:4/842: creat d4/d63/d64/f111 x:0 0 0 2024-05-25T06:11:48.819 INFO:tasks.workunit.client.0.smithi160.stdout:4/843: rename d4/d1c/f9f to d4/d6/d35/d3d/d9e/f112 0 2024-05-25T06:11:48.825 INFO:tasks.workunit.client.0.smithi160.stdout:4/844: dwrite d4/f2a [0,4194304] 0 2024-05-25T06:11:48.827 INFO:tasks.workunit.client.0.smithi160.stdout:4/845: fdatasync d4/d63/d64/d88/fc5 0 2024-05-25T06:11:48.836 INFO:tasks.workunit.client.0.smithi160.stdout:4/846: dwrite d4/d6/d5b/d66/f71 [0,4194304] 0 2024-05-25T06:11:48.846 INFO:tasks.workunit.client.0.smithi160.stdout:4/847: dwrite d4/d6/d5b/d66/d82/fdb [0,4194304] 0 2024-05-25T06:11:48.854 INFO:tasks.workunit.client.0.smithi160.stdout:4/848: dwrite d4/d6/d35/d3d/d9e/fc2 [0,4194304] 0 2024-05-25T06:11:48.858 INFO:tasks.workunit.client.0.smithi160.stdout:4/849: symlink d4/d6/d35/d3d/d9e/l113 0 2024-05-25T06:11:48.869 INFO:tasks.workunit.client.0.smithi160.stdout:4/850: truncate d4/d6/d35/f58 1315783 0 2024-05-25T06:11:48.871 INFO:tasks.workunit.client.0.smithi160.stdout:4/851: creat d4/d6/d5b/d66/d78/da1/dab/f114 x:0 0 0 2024-05-25T06:11:48.872 INFO:tasks.workunit.client.0.smithi160.stdout:4/852: dread - d4/d6/d7b/f10b zero size 2024-05-25T06:11:48.875 INFO:tasks.workunit.client.0.smithi160.stdout:4/853: rmdir d4/d6/d21/dc3 0 2024-05-25T06:11:48.877 INFO:tasks.workunit.client.0.smithi160.stdout:4/854: creat d4/d6/d5b/d66/d78/da1/dab/dfa/f115 x:0 0 0 2024-05-25T06:11:48.878 INFO:tasks.workunit.client.0.smithi160.stdout:4/855: mknod d4/d63/c116 0 2024-05-25T06:11:48.879 INFO:tasks.workunit.client.0.smithi160.stdout:4/856: dread - d4/d6/d5b/d66/d78/da1/dab/dfa/f115 zero size 2024-05-25T06:11:48.920 INFO:tasks.workunit.client.0.smithi160.stdout:3/879: sync 2024-05-25T06:11:48.920 INFO:tasks.workunit.client.0.smithi160.stdout:1/801: sync 2024-05-25T06:11:48.920 INFO:tasks.workunit.client.0.smithi160.stdout:5/949: sync 2024-05-25T06:11:48.921 INFO:tasks.workunit.client.0.smithi160.stdout:4/857: mkdir d4/d6/d21/d117 0 2024-05-25T06:11:48.924 INFO:tasks.workunit.client.0.smithi160.stdout:1/802: read de/d21/d49/d35/f6e [671503,10608] 0 2024-05-25T06:11:48.926 INFO:tasks.workunit.client.0.smithi160.stdout:5/950: rename db/d47/fcb to db/dc/d13/d1d/d3f/d72/d98/f11e 0 2024-05-25T06:11:48.926 INFO:tasks.workunit.client.0.smithi160.stdout:4/858: dread d4/d6/d5b/d66/f71 [0,4194304] 0 2024-05-25T06:11:48.930 INFO:tasks.workunit.client.0.smithi160.stdout:3/880: rename d19/d2f/d34 to d19/d2f/d38/d43/d5b/d65/d7f/d127 0 2024-05-25T06:11:48.932 INFO:tasks.workunit.client.0.smithi160.stdout:5/951: rmdir db/dc/d13/d1d/d3f/dc0/d104 39 2024-05-25T06:11:48.936 INFO:tasks.workunit.client.0.smithi160.stdout:1/803: rename de/d21/d49/d1c/d59/d71/dfe/l10a to de/d21/d49/d1c/d59/d67/l110 0 2024-05-25T06:11:48.938 INFO:tasks.workunit.client.0.smithi160.stdout:5/952: mknod db/dc/d13/d1d/d3f/d72/c11f 0 2024-05-25T06:11:48.939 INFO:tasks.workunit.client.0.smithi160.stdout:3/881: fdatasync f0 0 2024-05-25T06:11:48.946 INFO:tasks.workunit.client.0.smithi160.stdout:5/953: dwrite db/dc/d13/d1d/d23/dbd/fcc [0,4194304] 0 2024-05-25T06:11:48.946 INFO:tasks.workunit.client.0.smithi160.stdout:5/954: chown db/d108/f109 142252228 1 2024-05-25T06:11:48.947 INFO:tasks.workunit.client.0.smithi160.stdout:4/859: getdents d4/d6/d35 0 2024-05-25T06:11:48.947 INFO:tasks.workunit.client.0.smithi160.stdout:5/955: write db/dc/d13/d1d/d23/dbd/fcc [5073503,100488] 0 2024-05-25T06:11:48.949 INFO:tasks.workunit.client.0.smithi160.stdout:5/956: write db/d47/d5e/fc3 [3862689,126113] 0 2024-05-25T06:11:48.949 INFO:tasks.workunit.client.0.smithi160.stdout:3/882: fdatasync d19/d2f/d38/d47/f57 0 2024-05-25T06:11:48.950 INFO:tasks.workunit.client.0.smithi160.stdout:1/804: mkdir de/d21/d49/d1c/d111 0 2024-05-25T06:11:48.956 INFO:tasks.workunit.client.0.smithi160.stdout:5/957: rmdir db/dc/d13/d1d/d3f 39 2024-05-25T06:11:48.957 INFO:tasks.workunit.client.0.smithi160.stdout:4/860: dwrite d4/d6/d35/d3d/ffb [0,4194304] 0 2024-05-25T06:11:48.957 INFO:tasks.workunit.client.0.smithi160.stdout:1/805: stat de/d21/d49/d1c/d41/cf1 0 2024-05-25T06:11:48.964 INFO:tasks.workunit.client.0.smithi160.stdout:1/806: symlink de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/l112 0 2024-05-25T06:11:48.967 INFO:tasks.workunit.client.0.smithi160.stdout:3/883: link d19/d2f/d38/d47/fc7 d19/d2f/d38/d43/d5b/d65/d7e/f128 0 2024-05-25T06:11:48.967 INFO:tasks.workunit.client.0.smithi160.stdout:3/884: chown d19/da7/f11c 132554 1 2024-05-25T06:11:48.969 INFO:tasks.workunit.client.0.smithi160.stdout:3/885: fsync d19/da6/f76 0 2024-05-25T06:11:48.972 INFO:tasks.workunit.client.0.smithi160.stdout:1/807: rename de/d21/d49/f1d to de/d21/d49/d90/f113 0 2024-05-25T06:11:48.972 INFO:tasks.workunit.client.0.smithi160.stdout:4/861: dwrite d4/d6/d5b/d66/d78/da1/dab/f10f [0,4194304] 0 2024-05-25T06:11:48.975 INFO:tasks.workunit.client.0.smithi160.stdout:1/808: mknod de/d21/d49/d1c/d41/d45/dac/c114 0 2024-05-25T06:11:48.979 INFO:tasks.workunit.client.0.smithi160.stdout:4/862: mknod d4/d6/d35/d3d/d9e/dea/df5/c118 0 2024-05-25T06:11:48.982 INFO:tasks.workunit.client.0.smithi160.stdout:5/958: dread db/f53 [0,4194304] 0 2024-05-25T06:11:48.985 INFO:tasks.workunit.client.0.smithi160.stdout:5/959: write db/dc/d13/d1d/fc9 [234973,66756] 0 2024-05-25T06:11:48.986 INFO:tasks.workunit.client.0.smithi160.stdout:1/809: dwrite de/d21/d49/d28/fad [0,4194304] 0 2024-05-25T06:11:48.993 INFO:tasks.workunit.client.0.smithi160.stdout:1/810: link de/d21/d49/d1c/d52/dc7/fcc de/d21/f115 0 2024-05-25T06:11:48.997 INFO:tasks.workunit.client.0.smithi160.stdout:1/811: symlink de/d21/d49/d1c/d41/d45/d7d/db3/l116 0 2024-05-25T06:11:49.000 INFO:tasks.workunit.client.0.smithi160.stdout:3/886: dread d19/d2f/d38/d43/fdb [0,4194304] 0 2024-05-25T06:11:49.003 INFO:tasks.workunit.client.0.smithi160.stdout:1/812: dwrite de/d21/d49/d1c/d52/fbe [0,4194304] 0 2024-05-25T06:11:49.004 INFO:tasks.workunit.client.0.smithi160.stdout:3/887: creat d19/d2f/d38/d43/d5b/d65/d7f/d127/f129 x:0 0 0 2024-05-25T06:11:49.010 INFO:tasks.workunit.client.0.smithi160.stdout:1/813: dwrite de/d21/d49/d28/f5b [0,4194304] 0 2024-05-25T06:11:49.014 INFO:tasks.workunit.client.0.smithi160.stdout:1/814: symlink de/d21/d49/d35/l117 0 2024-05-25T06:11:49.017 INFO:tasks.workunit.client.0.smithi160.stdout:5/960: dread db/dc/d13/d1d/d23/f91 [0,4194304] 0 2024-05-25T06:11:49.019 INFO:tasks.workunit.client.0.smithi160.stdout:5/961: write db/dc/f15 [1057972,68970] 0 2024-05-25T06:11:49.022 INFO:tasks.workunit.client.0.smithi160.stdout:5/962: creat db/d108/f120 x:0 0 0 2024-05-25T06:11:49.024 INFO:tasks.workunit.client.0.smithi160.stdout:3/888: dwrite d19/d2f/d38/d43/d5b/d65/d7f/d127/f4d [4194304,4194304] 0 2024-05-25T06:11:49.026 INFO:tasks.workunit.client.0.smithi160.stdout:5/963: truncate db/dc/d13/d1d/d23/fed 561138 0 2024-05-25T06:11:49.031 INFO:tasks.workunit.client.0.smithi160.stdout:3/889: rename d19/d2f/d38/d43/d5b/d65/d7f to d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a 0 2024-05-25T06:11:49.033 INFO:tasks.workunit.client.0.smithi160.stdout:3/890: chown d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127 114191533 1 2024-05-25T06:11:49.036 INFO:tasks.workunit.client.0.smithi160.stdout:3/891: mknod d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/c12b 0 2024-05-25T06:11:49.037 INFO:tasks.workunit.client.0.smithi160.stdout:3/892: stat d19/d2f 0 2024-05-25T06:11:49.061 INFO:tasks.workunit.client.0.smithi160.stdout:8/795: sync 2024-05-25T06:11:49.065 INFO:tasks.workunit.client.0.smithi160.stdout:8/796: dread d5/d7/dcb/d75/f76 [0,4194304] 0 2024-05-25T06:11:49.066 INFO:tasks.workunit.client.0.smithi160.stdout:3/893: unlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/ca0 0 2024-05-25T06:11:49.071 INFO:tasks.workunit.client.0.smithi160.stdout:8/797: getdents d5/d44/d1f/da3/d28/de6/dc2 0 2024-05-25T06:11:49.071 INFO:tasks.workunit.client.0.smithi160.stdout:5/964: dwrite db/dc/d13/d1d/d3f/d72/d98/fa0 [0,4194304] 0 2024-05-25T06:11:49.072 INFO:tasks.workunit.client.0.smithi160.stdout:8/798: readlink d5/d7/lff 0 2024-05-25T06:11:49.073 INFO:tasks.workunit.client.0.smithi160.stdout:5/965: write db/dc/d13/d1d/d23/dbd/fef [400282,66542] 0 2024-05-25T06:11:49.077 INFO:tasks.workunit.client.0.smithi160.stdout:8/799: write d5/d44/d1f/da3/d28/d49/fc3 [4931970,122407] 0 2024-05-25T06:11:49.078 INFO:tasks.workunit.client.0.smithi160.stdout:5/966: truncate db/dc/d13/d1d/d3f/d72/f10b 688487 0 2024-05-25T06:11:49.081 INFO:tasks.workunit.client.0.smithi160.stdout:5/967: creat db/d47/d81/f121 x:0 0 0 2024-05-25T06:11:49.083 INFO:tasks.workunit.client.0.smithi160.stdout:5/968: symlink db/dc/d13/d1d/d23/l122 0 2024-05-25T06:11:49.085 INFO:tasks.workunit.client.0.smithi160.stdout:8/800: read d5/d44/d1f/da3/d28/d4a/d4e/fa1 [3173471,129280] 0 2024-05-25T06:11:49.085 INFO:tasks.workunit.client.0.smithi160.stdout:5/969: mknod db/dc/d13/d1d/d23/dbd/c123 0 2024-05-25T06:11:49.091 INFO:tasks.workunit.client.0.smithi160.stdout:8/801: write d5/d44/d1f/da3/d28/f3f [759742,48657] 0 2024-05-25T06:11:49.093 INFO:tasks.workunit.client.0.smithi160.stdout:8/802: creat d5/d44/d1f/da3/d28/d4a/d4e/d86/f108 x:0 0 0 2024-05-25T06:11:49.097 INFO:tasks.workunit.client.0.smithi160.stdout:8/803: write d5/d7/dcb/d75/feb [172871,64959] 0 2024-05-25T06:11:49.099 INFO:tasks.workunit.client.0.smithi160.stdout:8/804: symlink d5/l109 0 2024-05-25T06:11:49.105 INFO:tasks.workunit.client.0.smithi160.stdout:8/805: dwrite d5/d9/d72/daf/db9/ff8 [0,4194304] 0 2024-05-25T06:11:49.107 INFO:tasks.workunit.client.0.smithi160.stdout:8/806: write d5/d7/dcb/d7a/f92 [5082832,70689] 0 2024-05-25T06:11:49.112 INFO:tasks.workunit.client.0.smithi160.stdout:8/807: getdents d5/d9/d72/daf/db9 0 2024-05-25T06:11:49.208 INFO:tasks.workunit.client.0.smithi160.stdout:9/750: sync 2024-05-25T06:11:49.208 INFO:tasks.workunit.client.0.smithi160.stdout:9/751: chown d4/d11/d14/f3a 16323 1 2024-05-25T06:11:49.212 INFO:tasks.workunit.client.0.smithi160.stdout:9/752: write d4/d11/d1e/d24/f82 [186811,13447] 0 2024-05-25T06:11:49.214 INFO:tasks.workunit.client.0.smithi160.stdout:9/753: chown d4/d11/c4a 58269640 1 2024-05-25T06:11:49.216 INFO:tasks.workunit.client.0.smithi160.stdout:9/754: unlink d4/d11/d1e/d3e/d2f/d79/fe0 0 2024-05-25T06:11:49.221 INFO:tasks.workunit.client.0.smithi160.stdout:9/755: rename d4/d11/d12/d2e/d45/f9e to d4/d11/d1e/d3e/d25/d27/d95/fef 0 2024-05-25T06:11:49.223 INFO:tasks.workunit.client.0.smithi160.stdout:9/756: mknod d4/d11/d14/d8e/d9c/cf0 0 2024-05-25T06:11:49.232 INFO:tasks.workunit.client.0.smithi160.stdout:9/757: dwrite d4/d11/d12/d2e/f32 [0,4194304] 0 2024-05-25T06:11:49.234 INFO:tasks.workunit.client.0.smithi160.stdout:9/758: mkdir d4/d11/d1e/dbc/df1 0 2024-05-25T06:11:49.242 INFO:tasks.workunit.client.0.smithi160.stdout:9/759: dwrite d4/d11/d1e/d24/d66/f94 [0,4194304] 0 2024-05-25T06:11:49.256 INFO:tasks.workunit.client.0.smithi160.stdout:9/760: dwrite d4/d11/d12/f35 [0,4194304] 0 2024-05-25T06:11:49.259 INFO:tasks.workunit.client.0.smithi160.stdout:9/761: mknod d4/d11/d1e/d3e/d2f/d79/cf2 0 2024-05-25T06:11:49.266 INFO:tasks.workunit.client.0.smithi160.stdout:9/762: dwrite d4/d11/f26 [0,4194304] 0 2024-05-25T06:11:49.271 INFO:tasks.workunit.client.0.smithi160.stdout:9/763: creat d4/d11/d1e/d3e/d2f/d79/ff3 x:0 0 0 2024-05-25T06:11:49.276 INFO:tasks.workunit.client.0.smithi160.stdout:9/764: dread d4/d11/d1e/d24/d66/f94 [0,4194304] 0 2024-05-25T06:11:49.278 INFO:tasks.workunit.client.0.smithi160.stdout:9/765: write d4/d11/d14/fb6 [1165674,122954] 0 2024-05-25T06:11:49.279 INFO:tasks.workunit.client.0.smithi160.stdout:9/766: stat d4/d11/d1e/d24/fd1 0 2024-05-25T06:11:49.284 INFO:tasks.workunit.client.0.smithi160.stdout:9/767: rename d4/d11/d1e/d3e/d2f/d79/f9b to d4/d11/d1e/d3e/ff4 0 2024-05-25T06:11:49.286 INFO:tasks.workunit.client.0.smithi160.stdout:9/768: rmdir d4 39 2024-05-25T06:11:49.295 INFO:tasks.workunit.client.0.smithi160.stdout:9/769: dwrite d4/d11/d1e/d3e/d25/d5e/fda [0,4194304] 0 2024-05-25T06:11:49.304 INFO:tasks.workunit.client.0.smithi160.stdout:9/770: dwrite d4/d11/d1e/d3e/d25/f8b [0,4194304] 0 2024-05-25T06:11:49.322 INFO:tasks.workunit.client.0.smithi160.stdout:9/771: dwrite d4/d11/d12/d58/fbb [0,4194304] 0 2024-05-25T06:11:49.325 INFO:tasks.workunit.client.0.smithi160.stdout:9/772: write d4/d11/d1e/d3e/d25/f50 [4180269,56353] 0 2024-05-25T06:11:49.326 INFO:tasks.workunit.client.0.smithi160.stdout:9/773: chown d4/d11/d14/f3a 2 1 2024-05-25T06:11:49.327 INFO:tasks.workunit.client.0.smithi160.stdout:9/774: write d4/d11/d1e/fd3 [1053778,104526] 0 2024-05-25T06:11:49.335 INFO:tasks.workunit.client.0.smithi160.stdout:9/775: creat d4/d11/d1e/ff5 x:0 0 0 2024-05-25T06:11:49.336 INFO:tasks.workunit.client.0.smithi160.stdout:9/776: stat d4/d11/d1e/d3e/d25/d27/dc5 0 2024-05-25T06:11:49.340 INFO:tasks.workunit.client.0.smithi160.stdout:9/777: rename d4/d11/d1e/d24/c5f to d4/d11/d12/d2e/d45/d83/cf6 0 2024-05-25T06:11:49.341 INFO:tasks.workunit.client.0.smithi160.stdout:9/778: chown d4/d11/d1e/d24/d66/fbf 154173 1 2024-05-25T06:11:49.342 INFO:tasks.workunit.client.0.smithi160.stdout:9/779: chown d4/d11/d7a/feb 18 1 2024-05-25T06:11:49.348 INFO:tasks.workunit.client.0.smithi160.stdout:9/780: truncate d4/d11/d1e/d3e/d25/d27/f77 4250473 0 2024-05-25T06:11:49.353 INFO:tasks.workunit.client.0.smithi160.stdout:9/781: rename d4/d11/d14/d8e to d4/d11/d1e/d24/df7 0 2024-05-25T06:11:49.360 INFO:tasks.workunit.client.0.smithi160.stdout:9/782: dwrite d4/fa [0,4194304] 0 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:1/815: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:4/863: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:3/894: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:2/955: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:5/970: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:8/808: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:6/884: sync 2024-05-25T06:11:49.379 INFO:tasks.workunit.client.0.smithi160.stdout:0/968: sync 2024-05-25T06:11:49.380 INFO:tasks.workunit.client.0.smithi160.stdout:7/930: sync 2024-05-25T06:11:49.386 INFO:tasks.workunit.client.0.smithi160.stdout:5/971: symlink db/d47/d5e/d6e/l124 0 2024-05-25T06:11:49.388 INFO:tasks.workunit.client.0.smithi160.stdout:4/864: dread d4/d6/d28/f9d [0,4194304] 0 2024-05-25T06:11:49.391 INFO:tasks.workunit.client.0.smithi160.stdout:0/969: read d2b/d31/d35/d36/fd3 [2350416,94925] 0 2024-05-25T06:11:49.393 INFO:tasks.workunit.client.0.smithi160.stdout:1/816: fsync de/d21/d49/d1c/d41/d96/fa8 0 2024-05-25T06:11:49.395 INFO:tasks.workunit.client.0.smithi160.stdout:3/895: creat d19/d2f/d38/d43/d5b/d65/d83/d120/f12c x:0 0 0 2024-05-25T06:11:49.397 INFO:tasks.workunit.client.0.smithi160.stdout:1/817: chown de/d21/c61 1700 1 2024-05-25T06:11:49.398 INFO:tasks.workunit.client.0.smithi160.stdout:6/885: mknod d5/d39/d8f/c131 0 2024-05-25T06:11:49.399 INFO:tasks.workunit.client.0.smithi160.stdout:0/970: creat d2b/d46/d63/da7/da6/f137 x:0 0 0 2024-05-25T06:11:49.405 INFO:tasks.workunit.client.0.smithi160.stdout:9/783: dread d4/d11/d14/f53 [0,4194304] 0 2024-05-25T06:11:49.406 INFO:tasks.workunit.client.0.smithi160.stdout:7/931: mkdir d6/d32/d5b/d10e/d128/d12b 0 2024-05-25T06:11:49.408 INFO:tasks.workunit.client.0.smithi160.stdout:4/865: dwrite d4/d6/d28/fc7 [0,4194304] 0 2024-05-25T06:11:49.410 INFO:tasks.workunit.client.0.smithi160.stdout:8/809: write d5/d44/d1f/da3/d28/d4a/f34 [91558,124265] 0 2024-05-25T06:11:49.412 INFO:tasks.workunit.client.0.smithi160.stdout:1/818: rename de/d21/d49/dc6/ce1 to de/d21/d49/d1c/d59/d7f/c118 0 2024-05-25T06:11:49.416 INFO:tasks.workunit.client.0.smithi160.stdout:7/932: truncate d6/dc/d21/f97 9196059 0 2024-05-25T06:11:49.417 INFO:tasks.workunit.client.0.smithi160.stdout:3/896: creat d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/f12d x:0 0 0 2024-05-25T06:11:49.420 INFO:tasks.workunit.client.0.smithi160.stdout:9/784: chown d4/d11/d1e/d24/fd1 6 1 2024-05-25T06:11:49.420 INFO:tasks.workunit.client.0.smithi160.stdout:1/819: fsync de/d21/d49/d35/f6e 0 2024-05-25T06:11:49.422 INFO:tasks.workunit.client.0.smithi160.stdout:5/972: dwrite db/d108/fb2 [0,4194304] 0 2024-05-25T06:11:49.424 INFO:tasks.workunit.client.0.smithi160.stdout:1/820: write f9 [1596287,68846] 0 2024-05-25T06:11:49.425 INFO:tasks.workunit.client.0.smithi160.stdout:7/933: creat d6/dc/d21/f12c x:0 0 0 2024-05-25T06:11:49.431 INFO:tasks.workunit.client.0.smithi160.stdout:4/866: mkdir d4/d63/d64/d88/d119 0 2024-05-25T06:11:49.432 INFO:tasks.workunit.client.0.smithi160.stdout:6/886: truncate d5/d39/d8f/d2b/f88 3256323 0 2024-05-25T06:11:49.432 INFO:tasks.workunit.client.0.smithi160.stdout:9/785: fdatasync d4/d11/d1e/d24/df7/fb8 0 2024-05-25T06:11:49.433 INFO:tasks.workunit.client.0.smithi160.stdout:4/867: stat d4/d1a/f50 0 2024-05-25T06:11:49.434 INFO:tasks.workunit.client.0.smithi160.stdout:8/810: rename d5/d9/l20 to d5/d44/d1f/da3/d28/dc6/de8/l10a 0 2024-05-25T06:11:49.436 INFO:tasks.workunit.client.0.smithi160.stdout:1/821: symlink de/d21/d49/d1c/d41/d45/d7d/db3/dba/l119 0 2024-05-25T06:11:49.437 INFO:tasks.workunit.client.0.smithi160.stdout:7/934: readlink d6/d3e/d6f/d73/l87 0 2024-05-25T06:11:49.437 INFO:tasks.workunit.client.0.smithi160.stdout:0/971: link d2b/d31/d35/d43/fc4 d2b/d46/d63/da7/da6/da9/f138 0 2024-05-25T06:11:49.439 INFO:tasks.workunit.client.0.smithi160.stdout:4/868: fsync d4/d1a/f59 0 2024-05-25T06:11:49.443 INFO:tasks.workunit.client.0.smithi160.stdout:3/897: symlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/da8/l12e 0 2024-05-25T06:11:49.444 INFO:tasks.workunit.client.0.smithi160.stdout:2/956: dread d12/d1c/d37/fcc [0,4194304] 0 2024-05-25T06:11:49.445 INFO:tasks.workunit.client.0.smithi160.stdout:4/869: dread - d4/d6/dbd/fd4 zero size 2024-05-25T06:11:49.447 INFO:tasks.workunit.client.0.smithi160.stdout:5/973: dwrite db/dc/d13/d1d/d3f/d72/dc5/fdc [0,4194304] 0 2024-05-25T06:11:49.449 INFO:tasks.workunit.client.0.smithi160.stdout:6/887: truncate d5/de/ff 885147 0 2024-05-25T06:11:49.450 INFO:tasks.workunit.client.0.smithi160.stdout:0/972: mknod d2b/d46/d63/da7/db5/c139 0 2024-05-25T06:11:49.450 INFO:tasks.workunit.client.0.smithi160.stdout:4/870: stat d4/d63/d64/l73 0 2024-05-25T06:11:49.451 INFO:tasks.workunit.client.0.smithi160.stdout:7/935: write d6/d32/d7e/de0/fe5 [2328618,12642] 0 2024-05-25T06:11:49.451 INFO:tasks.workunit.client.0.smithi160.stdout:3/898: chown c17 5067 1 2024-05-25T06:11:49.452 INFO:tasks.workunit.client.0.smithi160.stdout:2/957: chown d12/d3e/d5a/d8d/df9 9 1 2024-05-25T06:11:49.452 INFO:tasks.workunit.client.0.smithi160.stdout:0/973: chown d2b/d31/d35/d43/d98 2874 1 2024-05-25T06:11:49.454 INFO:tasks.workunit.client.0.smithi160.stdout:2/958: fdatasync d12/d3e/d5a/d8d/db4/ffb 0 2024-05-25T06:11:49.455 INFO:tasks.workunit.client.0.smithi160.stdout:6/888: dread - d5/d30/d36/d73/d6e/f12b zero size 2024-05-25T06:11:49.459 INFO:tasks.workunit.client.0.smithi160.stdout:3/899: chown d19/l21 7 1 2024-05-25T06:11:49.459 INFO:tasks.workunit.client.0.smithi160.stdout:9/786: getdents d4/d11/d1e/d3e/d2f 0 2024-05-25T06:11:49.459 INFO:tasks.workunit.client.0.smithi160.stdout:2/959: fsync d12/d1c/d2a/f3a 0 2024-05-25T06:11:49.461 INFO:tasks.workunit.client.0.smithi160.stdout:7/936: creat d6/dc/d64/d5a/dc4/f12d x:0 0 0 2024-05-25T06:11:49.463 INFO:tasks.workunit.client.0.smithi160.stdout:5/974: creat db/dc/d13/d1d/d3f/d72/dc5/f125 x:0 0 0 2024-05-25T06:11:49.467 INFO:tasks.workunit.client.0.smithi160.stdout:3/900: creat d19/d2f/d38/d43/d5b/d65/d83/ddf/d117/f12f x:0 0 0 2024-05-25T06:11:49.468 INFO:tasks.workunit.client.0.smithi160.stdout:1/822: rename de/d21/d49/d35/c4b to de/d21/d49/d1c/c11a 0 2024-05-25T06:11:49.468 INFO:tasks.workunit.client.0.smithi160.stdout:0/974: read d2b/d46/d63/da7/d67/db8/fd0 [352119,70094] 0 2024-05-25T06:11:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:11:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:11:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:49 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.470 INFO:tasks.workunit.client.0.smithi160.stdout:2/960: write d12/d5c/d85/dc1/dea/f12b [836192,50705] 0 2024-05-25T06:11:49.472 INFO:tasks.workunit.client.0.smithi160.stdout:0/975: chown d2b/d46/d63/db3/dd6/f10f 59000683 1 2024-05-25T06:11:49.477 INFO:tasks.workunit.client.0.smithi160.stdout:0/976: unlink d2b/d46/d63/da7/da6/dcd/c100 0 2024-05-25T06:11:49.477 INFO:tasks.workunit.client.0.smithi160.stdout:3/901: fdatasync d19/d22/f39 0 2024-05-25T06:11:49.480 INFO:tasks.workunit.client.0.smithi160.stdout:4/871: dread d4/d6/d5b/d66/d78/da1/dab/f10f [0,4194304] 0 2024-05-25T06:11:49.482 INFO:tasks.workunit.client.0.smithi160.stdout:6/889: getdents d5/d30/d36/d73/d6e/df7 0 2024-05-25T06:11:49.482 INFO:tasks.workunit.client.0.smithi160.stdout:7/937: dwrite d6/d3e/d6f/f10f [0,4194304] 0 2024-05-25T06:11:49.485 INFO:tasks.workunit.client.0.smithi160.stdout:1/823: mknod de/d39/c11b 0 2024-05-25T06:11:49.491 INFO:tasks.workunit.client.0.smithi160.stdout:4/872: chown d4/d1c/d54/dbc/dda/ce6 3 1 2024-05-25T06:11:49.492 INFO:tasks.workunit.client.0.smithi160.stdout:9/787: truncate d4/d11/d1e/d24/f82 234274 0 2024-05-25T06:11:49.493 INFO:tasks.workunit.client.0.smithi160.stdout:6/890: write d5/d39/d8f/d2b/f106 [955384,30779] 0 2024-05-25T06:11:49.498 INFO:tasks.workunit.client.0.smithi160.stdout:7/938: truncate d6/d2b/f5c 137102 0 2024-05-25T06:11:49.499 INFO:tasks.workunit.client.0.smithi160.stdout:1/824: mknod de/d21/d49/d1c/d59/d7f/da1/dbd/c11c 0 2024-05-25T06:11:49.501 INFO:tasks.workunit.client.0.smithi160.stdout:5/975: dwrite db/f53 [0,4194304] 0 2024-05-25T06:11:49.503 INFO:tasks.workunit.client.0.smithi160.stdout:8/811: dread d5/d44/d1f/da3/d28/f38 [0,4194304] 0 2024-05-25T06:11:49.503 INFO:tasks.workunit.client.0.smithi160.stdout:0/977: rename d2b/d31/d35/d43/fc4 to d2b/d46/d63/da7/d67/d6a/f13a 0 2024-05-25T06:11:49.504 INFO:tasks.workunit.client.0.smithi160.stdout:6/891: unlink d5/d39/dc1/d113/l123 0 2024-05-25T06:11:49.504 INFO:tasks.workunit.client.0.smithi160.stdout:4/873: read d4/d6/d21/d42/fa4 [301806,56613] 0 2024-05-25T06:11:49.507 INFO:tasks.workunit.client.0.smithi160.stdout:8/812: stat d5/d44/dba/ddb/cf3 0 2024-05-25T06:11:49.508 INFO:tasks.workunit.client.0.smithi160.stdout:3/902: mknod d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d111/c130 0 2024-05-25T06:11:49.509 INFO:tasks.workunit.client.0.smithi160.stdout:2/961: dread d12/d75/dae/d135/d113/f99 [0,4194304] 0 2024-05-25T06:11:49.511 INFO:tasks.workunit.client.0.smithi160.stdout:0/978: creat d2b/d31/d35/d36/d86/dd5/f13b x:0 0 0 2024-05-25T06:11:49.511 INFO:tasks.workunit.client.0.smithi160.stdout:7/939: chown c3 34 1 2024-05-25T06:11:49.513 INFO:tasks.workunit.client.0.smithi160.stdout:4/874: truncate d4/d6/dbd/df0/ffc 435420 0 2024-05-25T06:11:49.514 INFO:tasks.workunit.client.0.smithi160.stdout:3/903: read - d19/d2f/d38/d60/f11f zero size 2024-05-25T06:11:49.517 INFO:tasks.workunit.client.0.smithi160.stdout:4/875: fsync d4/d6/d21/f10e 0 2024-05-25T06:11:49.518 INFO:tasks.workunit.client.0.smithi160.stdout:6/892: creat d5/de/d32/d9f/dd5/f132 x:0 0 0 2024-05-25T06:11:49.521 INFO:tasks.workunit.client.0.smithi160.stdout:0/979: dread - d2b/d46/d63/da7/d67/fad zero size 2024-05-25T06:11:49.529 INFO:tasks.workunit.client.0.smithi160.stdout:9/788: write d4/d11/d12/f2b [1771260,115444] 0 2024-05-25T06:11:49.530 INFO:tasks.workunit.client.0.smithi160.stdout:5/976: dwrite db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/d113/db3/fba [0,4194304] 0 2024-05-25T06:11:49.534 INFO:tasks.workunit.client.0.smithi160.stdout:9/789: write d4/d11/fe9 [859799,8415] 0 2024-05-25T06:11:49.537 INFO:tasks.workunit.client.0.smithi160.stdout:6/893: dwrite d5/de/d32/d74/f116 [0,4194304] 0 2024-05-25T06:11:49.540 INFO:tasks.workunit.client.0.smithi160.stdout:2/962: link d12/lbc d12/d3e/d5a/d8d/db4/dcb/ded/l13e 0 2024-05-25T06:11:49.541 INFO:tasks.workunit.client.0.smithi160.stdout:8/813: dread d5/d7/d9a/db5/fe5 [0,4194304] 0 2024-05-25T06:11:49.541 INFO:tasks.workunit.client.0.smithi160.stdout:3/904: write d19/d2f/d38/d47/f57 [1656359,104923] 0 2024-05-25T06:11:49.544 INFO:tasks.workunit.client.0.smithi160.stdout:1/825: dread de/d21/d49/d28/f3c [0,4194304] 0 2024-05-25T06:11:49.546 INFO:tasks.workunit.client.0.smithi160.stdout:9/790: rename d4/d11/d1e/f1f to d4/d11/d12/ff8 0 2024-05-25T06:11:49.546 INFO:tasks.workunit.client.0.smithi160.stdout:2/963: dread - d12/d3e/d5a/d8d/db4/f123 zero size 2024-05-25T06:11:49.548 INFO:tasks.workunit.client.0.smithi160.stdout:1/826: chown de/d21/d49/d28 2846476 1 2024-05-25T06:11:49.549 INFO:tasks.workunit.client.0.smithi160.stdout:4/876: creat d4/d1c/d54/dbc/dda/f11a x:0 0 0 2024-05-25T06:11:49.549 INFO:tasks.workunit.client.0.smithi160.stdout:6/894: symlink d5/de/d7d/l133 0 2024-05-25T06:11:49.550 INFO:tasks.workunit.client.0.smithi160.stdout:2/964: readlink d12/d75/l92 0 2024-05-25T06:11:49.555 INFO:tasks.workunit.client.0.smithi160.stdout:8/814: truncate d5/d7/dcb/d7a/f99 1002643 0 2024-05-25T06:11:49.555 INFO:tasks.workunit.client.0.smithi160.stdout:2/965: readlink d12/d3e/d5a/l67 0 2024-05-25T06:11:49.555 INFO:tasks.workunit.client.0.smithi160.stdout:4/877: chown d4/d6/d5b/d66/d82/fa8 679781 1 2024-05-25T06:11:49.556 INFO:tasks.workunit.client.0.smithi160.stdout:3/905: mknod d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/da8/df8/d110/c131 0 2024-05-25T06:11:49.558 INFO:tasks.workunit.client.0.smithi160.stdout:9/791: read d4/d11/d1e/d3e/d2f/f98 [81460,33612] 0 2024-05-25T06:11:49.561 INFO:tasks.workunit.client.0.smithi160.stdout:0/980: dwrite d2b/d46/d63/da7/f85 [0,4194304] 0 2024-05-25T06:11:49.561 INFO:tasks.workunit.client.0.smithi160.stdout:1/827: readlink de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/l10b 0 2024-05-25T06:11:49.562 INFO:tasks.workunit.client.0.smithi160.stdout:8/815: chown d5/d44/d1f/d58/cfa 14573 1 2024-05-25T06:11:49.565 INFO:tasks.workunit.client.0.smithi160.stdout:4/878: symlink d4/d6/dbd/l11b 0 2024-05-25T06:11:49.566 INFO:tasks.workunit.client.0.smithi160.stdout:7/940: dread d6/d2b/da0/f101 [0,4194304] 0 2024-05-25T06:11:49.569 INFO:tasks.workunit.client.0.smithi160.stdout:3/906: creat d19/d2f/d38/d43/d5b/d65/d83/ddf/f132 x:0 0 0 2024-05-25T06:11:49.572 INFO:tasks.workunit.client.0.smithi160.stdout:0/981: chown d2b/da8/db2/f120 1572 1 2024-05-25T06:11:49.572 INFO:tasks.workunit.client.0.smithi160.stdout:4/879: mkdir d4/d6/dbd/d11c 0 2024-05-25T06:11:49.573 INFO:tasks.workunit.client.0.smithi160.stdout:6/895: dwrite d5/d30/d36/d73/d6e/f11c [4194304,4194304] 0 2024-05-25T06:11:49.574 INFO:tasks.workunit.client.0.smithi160.stdout:9/792: rename d4/d11/d7a/f72 to d4/d11/d1e/d24/df7/d9c/ff9 0 2024-05-25T06:11:49.575 INFO:tasks.workunit.client.0.smithi160.stdout:7/941: mkdir d6/d32/d7e/de0/d12e 0 2024-05-25T06:11:49.577 INFO:tasks.workunit.client.0.smithi160.stdout:2/966: dwrite f11 [0,4194304] 0 2024-05-25T06:11:49.582 INFO:tasks.workunit.client.0.smithi160.stdout:1/828: creat de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/f11d x:0 0 0 2024-05-25T06:11:49.582 INFO:tasks.workunit.client.0.smithi160.stdout:6/896: write d5/de/d7d/f9b [698160,110218] 0 2024-05-25T06:11:49.584 INFO:tasks.workunit.client.0.smithi160.stdout:8/816: creat d5/d44/d1f/da3/d28/de6/dc2/dd6/f10b x:0 0 0 2024-05-25T06:11:49.585 INFO:tasks.workunit.client.0.smithi160.stdout:5/977: dwrite db/d47/f68 [0,4194304] 0 2024-05-25T06:11:49.587 INFO:tasks.workunit.client.0.smithi160.stdout:2/967: dread - d12/d3e/d5a/d8d/df9/f10d zero size 2024-05-25T06:11:49.587 INFO:tasks.workunit.client.0.smithi160.stdout:3/907: symlink d19/d2f/d38/d43/d5b/l133 0 2024-05-25T06:11:49.588 INFO:tasks.workunit.client.0.smithi160.stdout:7/942: fdatasync d6/d3e/d6f/f76 0 2024-05-25T06:11:49.589 INFO:tasks.workunit.client.0.smithi160.stdout:4/880: symlink d4/d1c/d54/l11d 0 2024-05-25T06:11:49.591 INFO:tasks.workunit.client.0.smithi160.stdout:1/829: symlink de/d21/d49/d1c/d41/d96/l11e 0 2024-05-25T06:11:49.597 INFO:tasks.workunit.client.0.smithi160.stdout:9/793: link d4/d11/d1e/d3e/d25/d5e/da2/faa d4/d11/d12/d2e/ffa 0 2024-05-25T06:11:49.599 INFO:tasks.workunit.client.0.smithi160.stdout:3/908: mkdir d19/d2f/d38/d43/d5b/d65/d83/d120/d134 0 2024-05-25T06:11:49.599 INFO:tasks.workunit.client.0.smithi160.stdout:5/978: chown db/dc/d13/d1d/d3f/dc0/lee 9320 1 2024-05-25T06:11:49.600 INFO:tasks.workunit.client.0.smithi160.stdout:0/982: dwrite d2b/da8/fb0 [0,4194304] 0 2024-05-25T06:11:49.602 INFO:tasks.workunit.client.0.smithi160.stdout:9/794: write d4/d11/d7a/feb [3994409,21553] 0 2024-05-25T06:11:49.602 INFO:tasks.workunit.client.0.smithi160.stdout:5/979: chown db/d108/fb7 24 1 2024-05-25T06:11:49.606 INFO:tasks.workunit.client.0.smithi160.stdout:1/830: rename de/d21/d49/d1c/d52/dc7/ddb/lf3 to de/d21/d49/d1c/d59/d7f/da1/l11f 0 2024-05-25T06:11:49.609 INFO:tasks.workunit.client.0.smithi160.stdout:2/968: link d12/d1c/d120/f132 d12/d75/dae/d135/d74/d7a/de3/f13f 0 2024-05-25T06:11:49.614 INFO:tasks.workunit.client.0.smithi160.stdout:4/881: dwrite d4/d6/d35/f53 [0,4194304] 0 2024-05-25T06:11:49.614 INFO:tasks.workunit.client.0.smithi160.stdout:1/831: chown de/d21/d49/d1c/d41/d45/d7d/db3/l116 34242 1 2024-05-25T06:11:49.614 INFO:tasks.workunit.client.0.smithi160.stdout:0/983: creat d2b/d31/d135/f13c x:0 0 0 2024-05-25T06:11:49.615 INFO:tasks.workunit.client.0.smithi160.stdout:2/969: read - d12/d75/dae/d135/d74/d7a/d104/f129 zero size 2024-05-25T06:11:49.615 INFO:tasks.workunit.client.0.smithi160.stdout:5/980: mkdir db/dc/d13/d1d/d23/dbd/d126 0 2024-05-25T06:11:49.616 INFO:tasks.workunit.client.0.smithi160.stdout:3/909: dwrite d19/d2f/d38/f121 [0,4194304] 0 2024-05-25T06:11:49.621 INFO:tasks.workunit.client.0.smithi160.stdout:9/795: creat d4/d11/d1e/d3e/d25/d27/ffb x:0 0 0 2024-05-25T06:11:49.622 INFO:tasks.workunit.client.0.smithi160.stdout:2/970: symlink d12/d3c/df1/l140 0 2024-05-25T06:11:49.623 INFO:tasks.workunit.client.0.smithi160.stdout:4/882: creat d4/d6/dbd/f11e x:0 0 0 2024-05-25T06:11:49.623 INFO:tasks.workunit.client.0.smithi160.stdout:9/796: chown d4/l10 20649699 1 2024-05-25T06:11:49.624 INFO:tasks.workunit.client.0.smithi160.stdout:8/817: dread d5/d44/d1f/da3/d28/d49/fc3 [0,4194304] 0 2024-05-25T06:11:49.624 INFO:tasks.workunit.client.0.smithi160.stdout:1/832: symlink de/d21/d49/d1c/d59/d71/l120 0 2024-05-25T06:11:49.625 INFO:tasks.workunit.client.0.smithi160.stdout:4/883: write d4/d1c/d54/dbc/ff2 [403605,100004] 0 2024-05-25T06:11:49.627 INFO:tasks.workunit.client.0.smithi160.stdout:5/981: unlink db/dc/d13/d1d/d23/cb0 0 2024-05-25T06:11:49.631 INFO:tasks.workunit.client.0.smithi160.stdout:7/943: dread d6/dc/d64/f78 [0,4194304] 0 2024-05-25T06:11:49.633 INFO:tasks.workunit.client.0.smithi160.stdout:0/984: rename f12 to d2b/d31/d35/d43/d134/f13d 0 2024-05-25T06:11:49.633 INFO:tasks.workunit.client.0.smithi160.stdout:4/884: dwrite d4/d1c/d54/d9b/f10d [0,4194304] 0 2024-05-25T06:11:49.636 INFO:tasks.workunit.client.0.smithi160.stdout:3/910: read d19/d2f/d38/d43/d5b/d65/d7e/fd4 [3209576,100486] 0 2024-05-25T06:11:49.636 INFO:tasks.workunit.client.0.smithi160.stdout:9/797: symlink d4/d11/d1e/d3e/d25/d5e/da2/lfc 0 2024-05-25T06:11:49.637 INFO:tasks.workunit.client.0.smithi160.stdout:3/911: chown d19/d2f/d38 7615 1 2024-05-25T06:11:49.638 INFO:tasks.workunit.client.0.smithi160.stdout:1/833: mkdir de/d21/d49/d1c/d59/d7c/d93/d121 0 2024-05-25T06:11:49.642 INFO:tasks.workunit.client.0.smithi160.stdout:2/971: fdatasync d12/d1c/f6c 0 2024-05-25T06:11:49.643 INFO:tasks.workunit.client.0.smithi160.stdout:5/982: creat db/dc/d13/d1d/d23/dbd/d126/f127 x:0 0 0 2024-05-25T06:11:49.644 INFO:tasks.workunit.client.0.smithi160.stdout:3/912: mknod d19/d2f/d38/d47/c135 0 2024-05-25T06:11:49.646 INFO:tasks.workunit.client.0.smithi160.stdout:4/885: truncate d4/d63/fd3 4541099 0 2024-05-25T06:11:49.648 INFO:tasks.workunit.client.0.smithi160.stdout:7/944: dwrite d6/d2b/da0/f101 [0,4194304] 0 2024-05-25T06:11:49.649 INFO:tasks.workunit.client.0.smithi160.stdout:5/983: symlink db/dc/d13/d9a/l128 0 2024-05-25T06:11:49.649 INFO:tasks.workunit.client.0.smithi160.stdout:2/972: chown d12/d1c/l23 74111367 1 2024-05-25T06:11:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:11:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:11:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:11:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:11:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:49 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:49.650 INFO:tasks.workunit.client.0.smithi160.stdout:8/818: link d5/d44/d1f/da3/d28/d49/d61/c62 d5/d44/d1f/da3/d28/d49/c10c 0 2024-05-25T06:11:49.651 INFO:tasks.workunit.client.0.smithi160.stdout:9/798: creat d4/d11/d1e/d3e/d25/d5e/ffd x:0 0 0 2024-05-25T06:11:49.653 INFO:tasks.workunit.client.0.smithi160.stdout:3/913: rmdir d19/db2 39 2024-05-25T06:11:49.653 INFO:tasks.workunit.client.0.smithi160.stdout:4/886: mkdir d4/d6/d5b/d11f 0 2024-05-25T06:11:49.656 INFO:tasks.workunit.client.0.smithi160.stdout:2/973: fdatasync d12/d3e/d5a/d8d/db4/dcb/d128/f116 0 2024-05-25T06:11:49.656 INFO:tasks.workunit.client.0.smithi160.stdout:9/799: fsync d4/d11/d1e/d3e/d25/fb3 0 2024-05-25T06:11:49.658 INFO:tasks.workunit.client.0.smithi160.stdout:5/984: fsync db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/f117 0 2024-05-25T06:11:49.659 INFO:tasks.workunit.client.0.smithi160.stdout:1/834: dwrite de/d21/d49/d90/feb [0,4194304] 0 2024-05-25T06:11:49.659 INFO:tasks.workunit.client.0.smithi160.stdout:2/974: chown d12/d75/dca/c28 338035954 1 2024-05-25T06:11:49.660 INFO:tasks.workunit.client.0.smithi160.stdout:4/887: dread - d4/d6/d5b/d66/d78/da1/dab/dfa/f115 zero size 2024-05-25T06:11:49.662 INFO:tasks.workunit.client.0.smithi160.stdout:4/888: truncate d4/d6/d21/f27 4724210 0 2024-05-25T06:11:49.667 INFO:tasks.workunit.client.0.smithi160.stdout:3/914: symlink d19/d2f/d38/d43/l136 0 2024-05-25T06:11:49.669 INFO:tasks.workunit.client.0.smithi160.stdout:7/945: dread d6/d32/f99 [0,4194304] 0 2024-05-25T06:11:49.670 INFO:tasks.workunit.client.0.smithi160.stdout:3/915: write d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/f4d [3574206,168] 0 2024-05-25T06:11:49.671 INFO:tasks.workunit.client.0.smithi160.stdout:7/946: chown d6/d3e/da1/de7 933409958 1 2024-05-25T06:11:49.671 INFO:tasks.workunit.client.0.smithi160.stdout:1/835: unlink de/d21/d49/d1c/d41/d96/lc4 0 2024-05-25T06:11:49.672 INFO:tasks.workunit.client.0.smithi160.stdout:7/947: dread - d6/dc/d64/d5a/d7c/f119 zero size 2024-05-25T06:11:49.674 INFO:tasks.workunit.client.0.smithi160.stdout:9/800: creat d4/db2/de5/ffe x:0 0 0 2024-05-25T06:11:49.675 INFO:tasks.workunit.client.0.smithi160.stdout:4/889: rmdir d4/d63/d64 39 2024-05-25T06:11:49.676 INFO:tasks.workunit.client.0.smithi160.stdout:8/819: getdents d5/d44/d1f/d58/d65 0 2024-05-25T06:11:49.679 INFO:tasks.workunit.client.0.smithi160.stdout:3/916: unlink d19/d2f/d38/d43/fdb 0 2024-05-25T06:11:49.680 INFO:tasks.workunit.client.0.smithi160.stdout:2/975: write d12/d1c/d37/f43 [8225407,6187] 0 2024-05-25T06:11:49.681 INFO:tasks.workunit.client.0.smithi160.stdout:5/985: mkdir db/dc/d13/d1d/d3f/d72/d98/da5/d129 0 2024-05-25T06:11:49.682 INFO:tasks.workunit.client.0.smithi160.stdout:1/836: creat de/d21/d49/d1c/d59/d7f/da1/f122 x:0 0 0 2024-05-25T06:11:49.687 INFO:tasks.workunit.client.0.smithi160.stdout:1/837: chown de/d21/d49/d35/c2f 1 1 2024-05-25T06:11:49.687 INFO:tasks.workunit.client.0.smithi160.stdout:8/820: creat d5/d44/d1f/da3/d28/dc6/de8/f10d x:0 0 0 2024-05-25T06:11:49.690 INFO:tasks.workunit.client.0.smithi160.stdout:1/838: write de/f3d [837741,48661] 0 2024-05-25T06:11:49.690 INFO:tasks.workunit.client.0.smithi160.stdout:5/986: readlink db/dc/d13/d1d/le4 0 2024-05-25T06:11:49.692 INFO:tasks.workunit.client.0.smithi160.stdout:3/917: dwrite d19/d2f/d38/d43/d5b/d65/d83/dee/f114 [0,4194304] 0 2024-05-25T06:11:49.692 INFO:tasks.workunit.client.0.smithi160.stdout:0/985: link d2b/d31/d35/d43/d134/f13d d2b/d46/f13e 0 2024-05-25T06:11:49.693 INFO:tasks.workunit.client.0.smithi160.stdout:5/987: stat db/dc/d13/f65 0 2024-05-25T06:11:49.698 INFO:tasks.workunit.client.0.smithi160.stdout:3/918: truncate d19/d2f/d38/ffb 203631 0 2024-05-25T06:11:49.700 INFO:tasks.workunit.client.0.smithi160.stdout:0/986: chown d2b/d31/d6d/fe3 5675614 1 2024-05-25T06:11:49.700 INFO:tasks.workunit.client.0.smithi160.stdout:4/890: rename d4/d1c/d54/l83 to d4/d63/d64/d88/l120 0 2024-05-25T06:11:49.701 INFO:tasks.workunit.client.0.smithi160.stdout:7/948: dread d6/d2b/d5d/d89/fa7 [0,4194304] 0 2024-05-25T06:11:49.701 INFO:tasks.workunit.client.0.smithi160.stdout:1/839: readlink de/d21/d49/d1c/d41/d45/l78 0 2024-05-25T06:11:49.702 INFO:tasks.workunit.client.0.smithi160.stdout:5/988: mkdir db/d47/d81/dc1/d12a 0 2024-05-25T06:11:49.703 INFO:tasks.workunit.client.0.smithi160.stdout:9/801: truncate d4/d11/d1e/d24/d66/f94 2352946 0 2024-05-25T06:11:49.705 INFO:tasks.workunit.client.0.smithi160.stdout:2/976: write d12/d3e/f62 [4634474,106039] 0 2024-05-25T06:11:49.708 INFO:tasks.workunit.client.0.smithi160.stdout:4/891: symlink d4/d1c/d54/dbc/dda/d94/l121 0 2024-05-25T06:11:49.708 INFO:tasks.workunit.client.0.smithi160.stdout:8/821: dwrite d5/d7/d9a/db5/fbd [0,4194304] 0 2024-05-25T06:11:49.709 INFO:tasks.workunit.client.0.smithi160.stdout:9/802: write d4/d11/d12/d58/fbb [4984277,123262] 0 2024-05-25T06:11:49.720 INFO:tasks.workunit.client.0.smithi160.stdout:1/840: unlink de/d21/d49/d1c/d41/d45/dac/ccd 0 2024-05-25T06:11:49.721 INFO:tasks.workunit.client.0.smithi160.stdout:3/919: fdatasync d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/f4d 0 2024-05-25T06:11:49.722 INFO:tasks.workunit.client.0.smithi160.stdout:5/989: symlink db/dc/d13/d1d/d3f/d72/d98/da5/db5/l12b 0 2024-05-25T06:11:49.724 INFO:tasks.workunit.client.0.smithi160.stdout:5/990: chown f9 1 1 2024-05-25T06:11:49.728 INFO:tasks.workunit.client.0.smithi160.stdout:2/977: dwrite d12/d1c/d2a/f3a [4194304,4194304] 0 2024-05-25T06:11:49.731 INFO:tasks.workunit.client.0.smithi160.stdout:7/949: dwrite d6/dc/d21/f63 [0,4194304] 0 2024-05-25T06:11:49.733 INFO:tasks.workunit.client.0.smithi160.stdout:7/950: chown c3 1 1 2024-05-25T06:11:49.735 INFO:tasks.workunit.client.0.smithi160.stdout:2/978: dread d12/d3c/f126 [0,4194304] 0 2024-05-25T06:11:49.742 INFO:tasks.workunit.client.0.smithi160.stdout:7/951: dwrite d6/d32/d7e/f105 [0,4194304] 0 2024-05-25T06:11:49.743 INFO:tasks.workunit.client.0.smithi160.stdout:0/987: dread d2b/d46/d63/da7/d67/db8/fd0 [0,4194304] 0 2024-05-25T06:11:49.748 INFO:tasks.workunit.client.0.smithi160.stdout:8/822: stat d5/d44/d1f/da3/d28/de6/fd3 0 2024-05-25T06:11:49.750 INFO:tasks.workunit.client.0.smithi160.stdout:8/823: chown d5/d9/d72/daf/dbf 506158049 1 2024-05-25T06:11:49.750 INFO:tasks.workunit.client.0.smithi160.stdout:4/892: creat d4/d6/d35/d3d/d9e/dea/f122 x:0 0 0 2024-05-25T06:11:49.751 INFO:tasks.workunit.client.0.smithi160.stdout:3/920: dread - d19/da6/f9c zero size 2024-05-25T06:11:49.753 INFO:tasks.workunit.client.0.smithi160.stdout:9/803: rename d4/d11/d12/d2e/d45/d90/cd8 to d4/d11/d1e/d3e/d25/cff 0 2024-05-25T06:11:49.756 INFO:tasks.workunit.client.0.smithi160.stdout:2/979: mknod d12/d3c/df1/c141 0 2024-05-25T06:11:49.758 INFO:tasks.workunit.client.0.smithi160.stdout:4/893: creat d4/d6/d5b/d66/d78/da1/f123 x:0 0 0 2024-05-25T06:11:49.759 INFO:tasks.workunit.client.0.smithi160.stdout:5/991: mknod db/dc/d13/d1d/d3f/c12c 0 2024-05-25T06:11:49.759 INFO:tasks.workunit.client.0.smithi160.stdout:1/841: truncate de/d21/d49/d28/f102 1311711 0 2024-05-25T06:11:49.761 INFO:tasks.workunit.client.0.smithi160.stdout:0/988: rename d2b/d46/d63/db3/dd6/c123 to d2b/d46/dcb/c13f 0 2024-05-25T06:11:49.764 INFO:tasks.workunit.client.0.smithi160.stdout:2/980: creat d12/d5c/f142 x:0 0 0 2024-05-25T06:11:49.764 INFO:tasks.workunit.client.0.smithi160.stdout:7/952: mkdir d6/dc/d12f 0 2024-05-25T06:11:49.765 INFO:tasks.workunit.client.0.smithi160.stdout:3/921: symlink d19/d2f/d38/d43/l137 0 2024-05-25T06:11:49.767 INFO:tasks.workunit.client.0.smithi160.stdout:4/894: dwrite d4/d6/d5b/d66/f110 [0,4194304] 0 2024-05-25T06:11:49.770 INFO:tasks.workunit.client.0.smithi160.stdout:7/953: dread d6/d2b/d5d/fc1 [0,4194304] 0 2024-05-25T06:11:49.772 INFO:tasks.workunit.client.0.smithi160.stdout:2/981: mknod d12/d3c/c143 0 2024-05-25T06:11:49.773 INFO:tasks.workunit.client.0.smithi160.stdout:4/895: mknod d4/d6/d5b/d66/d78/da1/c124 0 2024-05-25T06:11:49.773 INFO:tasks.workunit.client.0.smithi160.stdout:2/982: stat d12/d5c/f142 0 2024-05-25T06:11:49.774 INFO:tasks.workunit.client.0.smithi160.stdout:3/922: symlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/def/l138 0 2024-05-25T06:11:49.774 INFO:tasks.workunit.client.0.smithi160.stdout:2/983: readlink d12/d5c/d85/d9b/l127 0 2024-05-25T06:11:49.775 INFO:tasks.workunit.client.0.smithi160.stdout:1/842: link de/d39/lf4 de/d39/l123 0 2024-05-25T06:11:49.778 INFO:tasks.workunit.client.0.smithi160.stdout:4/896: creat d4/d6/d28/de4/f125 x:0 0 0 2024-05-25T06:11:49.778 INFO:tasks.workunit.client.0.smithi160.stdout:2/984: dread - d12/d3e/d5a/fc5 zero size 2024-05-25T06:11:49.779 INFO:tasks.workunit.client.0.smithi160.stdout:5/992: dread db/dc/d13/d1d/d23/f46 [0,4194304] 0 2024-05-25T06:11:49.782 INFO:tasks.workunit.client.0.smithi160.stdout:4/897: symlink d4/d6/d35/d3d/d9e/l126 0 2024-05-25T06:11:49.783 INFO:tasks.workunit.client.0.smithi160.stdout:0/989: dread d2b/d31/d35/d36/f4a [0,4194304] 0 2024-05-25T06:11:49.783 INFO:tasks.workunit.client.0.smithi160.stdout:2/985: read d12/d75/dae/d135/df3/f125 [1719408,55908] 0 2024-05-25T06:11:49.783 INFO:tasks.workunit.client.0.smithi160.stdout:7/954: rmdir d6/d32/d5b/d10e/d128/d12b 0 2024-05-25T06:11:49.785 INFO:tasks.workunit.client.0.smithi160.stdout:7/955: readlink d6/dc/d21/ddd/lef 0 2024-05-25T06:11:49.786 INFO:tasks.workunit.client.0.smithi160.stdout:7/956: write d6/dc/d21/f97 [2207482,25011] 0 2024-05-25T06:11:49.788 INFO:tasks.workunit.client.0.smithi160.stdout:2/986: symlink d12/d1c/d2a/l144 0 2024-05-25T06:11:49.790 INFO:tasks.workunit.client.0.smithi160.stdout:2/987: truncate d12/d75/dae/d135/d74/d7a/fac 948689 0 2024-05-25T06:11:49.791 INFO:tasks.workunit.client.0.smithi160.stdout:3/923: link d19/d2f/d38/d60/d10f/l116 d19/d2f/d38/l139 0 2024-05-25T06:11:49.794 INFO:tasks.workunit.client.0.smithi160.stdout:5/993: creat db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/d113/f12d x:0 0 0 2024-05-25T06:11:49.796 INFO:tasks.workunit.client.0.smithi160.stdout:7/957: link d6/dc/d21/f2a d6/d2b/d5d/d89/f130 0 2024-05-25T06:11:49.798 INFO:tasks.workunit.client.0.smithi160.stdout:5/994: dread - db/dc/d13/d1d/d3f/d72/fea zero size 2024-05-25T06:11:49.800 INFO:tasks.workunit.client.0.smithi160.stdout:3/924: dwrite d19/d2f/d38/d47/f54 [0,4194304] 0 2024-05-25T06:11:49.800 INFO:tasks.workunit.client.0.smithi160.stdout:2/988: dwrite d12/d3e/d5a/d8d/db4/ffb [0,4194304] 0 2024-05-25T06:11:49.802 INFO:tasks.workunit.client.0.smithi160.stdout:7/958: dread d6/dc/f82 [0,4194304] 0 2024-05-25T06:11:49.804 INFO:tasks.workunit.client.0.smithi160.stdout:3/925: mknod d19/d2f/d38/d60/c13a 0 2024-05-25T06:11:49.815 INFO:tasks.workunit.client.0.smithi160.stdout:5/995: rename db/d47 to db/dc/d13/d1d/d23/d12e 0 2024-05-25T06:11:49.819 INFO:tasks.workunit.client.0.smithi160.stdout:3/926: unlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/da8/lb5 0 2024-05-25T06:11:49.820 INFO:tasks.workunit.client.0.smithi160.stdout:7/959: fsync d6/dc/d64/dae/f106 0 2024-05-25T06:11:49.822 INFO:tasks.workunit.client.0.smithi160.stdout:3/927: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/d13b 0 2024-05-25T06:11:49.824 INFO:tasks.workunit.client.0.smithi160.stdout:3/928: fdatasync d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/fa1 0 2024-05-25T06:11:49.825 INFO:tasks.workunit.client.0.smithi160.stdout:7/960: creat d6/dc/d21/ddd/f131 x:0 0 0 2024-05-25T06:11:49.826 INFO:tasks.workunit.client.0.smithi160.stdout:7/961: readlink d6/dc/d21/l48 0 2024-05-25T06:11:49.829 INFO:tasks.workunit.client.0.smithi160.stdout:3/929: rename d19/d2f/c62 to d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/c13c 0 2024-05-25T06:11:49.830 INFO:tasks.workunit.client.0.smithi160.stdout:7/962: symlink d6/d32/d7e/de0/l132 0 2024-05-25T06:11:49.831 INFO:tasks.workunit.client.0.smithi160.stdout:9/804: truncate d4/d11/d12/d2e/f32 391623 0 2024-05-25T06:11:49.833 INFO:tasks.workunit.client.0.smithi160.stdout:3/930: write d19/db2/fea [876981,79355] 0 2024-05-25T06:11:49.833 INFO:tasks.workunit.client.0.smithi160.stdout:7/963: symlink d6/dc/d21/l133 0 2024-05-25T06:11:49.837 INFO:tasks.workunit.client.0.smithi160.stdout:1/843: write de/d21/d49/d1c/d59/d71/f74 [448031,96842] 0 2024-05-25T06:11:49.839 INFO:tasks.workunit.client.0.smithi160.stdout:9/805: dread d4/d11/d12/d2e/d45/d83/fa9 [0,4194304] 0 2024-05-25T06:11:49.840 INFO:tasks.workunit.client.0.smithi160.stdout:7/964: creat d6/f134 x:0 0 0 2024-05-25T06:11:49.840 INFO:tasks.workunit.client.0.smithi160.stdout:0/990: write d2b/d46/d63/da7/f60 [3945594,14925] 0 2024-05-25T06:11:49.843 INFO:tasks.workunit.client.0.smithi160.stdout:1/844: creat de/d21/d49/d1c/d111/f124 x:0 0 0 2024-05-25T06:11:49.844 INFO:tasks.workunit.client.0.smithi160.stdout:7/965: write d6/d32/f10d [7794611,100666] 0 2024-05-25T06:11:49.845 INFO:tasks.workunit.client.0.smithi160.stdout:3/931: dwrite d19/d2f/d38/d43/d5b/d65/d83/dee/f114 [0,4194304] 0 2024-05-25T06:11:49.846 INFO:tasks.workunit.client.0.smithi160.stdout:7/966: stat d6/d2b/d5d/d89 0 2024-05-25T06:11:49.846 INFO:tasks.workunit.client.0.smithi160.stdout:2/989: truncate d12/f1d 3207783 0 2024-05-25T06:11:49.850 INFO:tasks.workunit.client.0.smithi160.stdout:9/806: mkdir d4/d11/d1e/d3e/d25/d27/dc5/d100 0 2024-05-25T06:11:49.851 INFO:tasks.workunit.client.0.smithi160.stdout:1/845: truncate de/d21/d49/d35/f6e 712387 0 2024-05-25T06:11:49.851 INFO:tasks.workunit.client.0.smithi160.stdout:2/990: fsync d12/d75/dae/d135/d74/ffd 0 2024-05-25T06:11:49.853 INFO:tasks.workunit.client.0.smithi160.stdout:0/991: creat d2b/d46/f140 x:0 0 0 2024-05-25T06:11:49.854 INFO:tasks.workunit.client.0.smithi160.stdout:0/992: fsync d2b/d31/fe0 0 2024-05-25T06:11:49.855 INFO:tasks.workunit.client.0.smithi160.stdout:1/846: write de/d21/d49/d90/f105 [1331743,60752] 0 2024-05-25T06:11:49.858 INFO:tasks.workunit.client.0.smithi160.stdout:2/991: unlink d12/d3e/d5a/d8d/df9/d100/c106 0 2024-05-25T06:11:49.859 INFO:tasks.workunit.client.0.smithi160.stdout:0/993: chown d2b/d31/d35/d43/dab/dc6 421 1 2024-05-25T06:11:49.859 INFO:tasks.workunit.client.0.smithi160.stdout:7/967: symlink d6/d3e/d6f/dfa/l135 0 2024-05-25T06:11:49.860 INFO:tasks.workunit.client.0.smithi160.stdout:2/992: mkdir d12/d75/dae/d135/d145 0 2024-05-25T06:11:49.861 INFO:tasks.workunit.client.0.smithi160.stdout:2/993: chown d12/d5c/d85/dc1/dea 1 1 2024-05-25T06:11:49.862 INFO:tasks.workunit.client.0.smithi160.stdout:7/968: mkdir d6/d32/d7e/de0/d136 0 2024-05-25T06:11:49.863 INFO:tasks.workunit.client.0.smithi160.stdout:0/994: creat d2b/d46/d63/da7/d67/d6a/f141 x:0 0 0 2024-05-25T06:11:49.863 INFO:tasks.workunit.client.0.smithi160.stdout:1/847: truncate de/d21/d49/d1c/fa5 1329652 0 2024-05-25T06:11:49.864 INFO:tasks.workunit.client.0.smithi160.stdout:1/848: truncate de/d39/d8f/fc2 3837134 0 2024-05-25T06:11:49.865 INFO:tasks.workunit.client.0.smithi160.stdout:2/994: creat d12/d3e/d5a/d8d/df9/d100/f146 x:0 0 0 2024-05-25T06:11:49.865 INFO:tasks.workunit.client.0.smithi160.stdout:1/849: stat de/d21/d49/d1c/d52/dc7/df6/l10e 0 2024-05-25T06:11:49.866 INFO:tasks.workunit.client.0.smithi160.stdout:7/969: mkdir d6/dba/d129/d137 0 2024-05-25T06:11:49.867 INFO:tasks.workunit.client.0.smithi160.stdout:2/995: write f9 [8610984,86622] 0 2024-05-25T06:11:49.868 INFO:tasks.workunit.client.0.smithi160.stdout:1/850: symlink de/d21/d49/d1c/d59/d7f/da1/dbd/l125 0 2024-05-25T06:11:49.868 INFO:tasks.workunit.client.0.smithi160.stdout:0/995: symlink d2b/d46/d63/da7/da6/dcd/dd8/l142 0 2024-05-25T06:11:49.869 INFO:tasks.workunit.client.0.smithi160.stdout:3/932: dread d19/d2f/d38/d43/d5b/d65/f89 [0,4194304] 0 2024-05-25T06:11:49.869 INFO:tasks.workunit.client.0.smithi160.stdout:7/970: symlink d6/dc/d64/l138 0 2024-05-25T06:11:49.870 INFO:tasks.workunit.client.0.smithi160.stdout:2/996: mknod d12/d5c/d85/d9b/c147 0 2024-05-25T06:11:49.871 INFO:tasks.workunit.client.0.smithi160.stdout:0/996: chown d2b/d46/d63/da7/d67/d6a/l6b 23489203 1 2024-05-25T06:11:49.871 INFO:tasks.workunit.client.0.smithi160.stdout:1/851: mkdir de/d21/d49/d1c/d52/dc7/ddb/d126 0 2024-05-25T06:11:49.871 INFO:tasks.workunit.client.0.smithi160.stdout:2/997: dread - d12/d1c/d2a/fe1 zero size 2024-05-25T06:11:49.872 INFO:tasks.workunit.client.0.smithi160.stdout:3/933: creat d19/d22/f13d x:0 0 0 2024-05-25T06:11:49.873 INFO:tasks.workunit.client.0.smithi160.stdout:7/971: creat d6/dba/d129/f139 x:0 0 0 2024-05-25T06:11:49.874 INFO:tasks.workunit.client.0.smithi160.stdout:0/997: write d2b/d31/d35/d7b/ff3 [279331,62714] 0 2024-05-25T06:11:49.874 INFO:tasks.workunit.client.0.smithi160.stdout:1/852: creat de/d39/d8f/f127 x:0 0 0 2024-05-25T06:11:49.876 INFO:tasks.workunit.client.0.smithi160.stdout:1/853: stat de/d21/d49/d1c/d41/d45/dac/dd3 0 2024-05-25T06:11:49.878 INFO:tasks.workunit.client.0.smithi160.stdout:3/934: symlink d19/db2/d104/l13e 0 2024-05-25T06:11:49.880 INFO:tasks.workunit.client.0.smithi160.stdout:2/998: getdents d12/d1c/d59 0 2024-05-25T06:11:49.880 INFO:tasks.workunit.client.0.smithi160.stdout:1/854: dread - de/d21/d49/d1c/d59/d7c/d93/de8/f10f zero size 2024-05-25T06:11:49.881 INFO:tasks.workunit.client.0.smithi160.stdout:0/998: rename d2b/d31/l124 to d2b/d31/d35/d43/d98/l143 0 2024-05-25T06:11:49.883 INFO:tasks.workunit.client.0.smithi160.stdout:7/972: dwrite d6/d32/d92/fc9 [0,4194304] 0 2024-05-25T06:11:49.885 INFO:tasks.workunit.client.0.smithi160.stdout:2/999: dread - d12/d3e/d5a/d8d/dc9/f12a zero size 2024-05-25T06:11:49.890 INFO:tasks.workunit.client.0.smithi160.stdout:1/855: read de/d21/d49/d1c/d41/d45/dac/ff7 [711997,60269] 0 2024-05-25T06:11:49.890 INFO:tasks.workunit.client.0.smithi160.stdout:0/999: chown d2b/d46/d63/da7/ce9 1 1 2024-05-25T06:11:49.892 INFO:tasks.workunit.client.0.smithi160.stdout:7/973: creat d6/d2b/d8b/f13a x:0 0 0 2024-05-25T06:11:49.893 INFO:tasks.workunit.client.0.smithi160.stdout:1/856: fsync de/d21/d49/d28/f76 0 2024-05-25T06:11:49.893 INFO:tasks.workunit.client.0.smithi160.stdout:3/935: dwrite d19/d2f/d38/d43/d5b/d65/fbd [0,4194304] 0 2024-05-25T06:11:49.897 INFO:tasks.workunit.client.0.smithi160.stdout:3/936: unlink d19/d2f/d38/d43/d5b/d65/fff 0 2024-05-25T06:11:49.902 INFO:tasks.workunit.client.0.smithi160.stdout:3/937: rename d19/d2f/d38/d47 to d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/d13f 0 2024-05-25T06:11:49.903 INFO:tasks.workunit.client.0.smithi160.stdout:7/974: link d6/dc/f41 d6/dc/d64/d5a/d7c/f13b 0 2024-05-25T06:11:49.904 INFO:tasks.workunit.client.0.smithi160.stdout:3/938: write d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/f10e [781144,111098] 0 2024-05-25T06:11:49.907 INFO:tasks.workunit.client.0.smithi160.stdout:3/939: read - d19/d2f/d38/d43/d5b/d65/d83/ddf/f132 zero size 2024-05-25T06:11:49.911 INFO:tasks.workunit.client.0.smithi160.stdout:7/975: dwrite d6/d32/d92/ff2 [0,4194304] 0 2024-05-25T06:11:49.912 INFO:tasks.workunit.client.0.smithi160.stdout:7/976: chown d6/d3e/d59/c11a 15094 1 2024-05-25T06:11:49.915 INFO:tasks.workunit.client.0.smithi160.stdout:3/940: truncate d19/d2f/d38/d43/d5b/d65/d7e/f128 371118 0 2024-05-25T06:11:49.921 INFO:tasks.workunit.client.0.smithi160.stdout:7/977: getdents d6/dc/d64/dac 0 2024-05-25T06:11:49.924 INFO:tasks.workunit.client.0.smithi160.stdout:7/978: creat d6/d32/d7e/de0/d12e/f13c x:0 0 0 2024-05-25T06:11:49.927 INFO:tasks.workunit.client.0.smithi160.stdout:3/941: write d19/da6/f9c [462182,77806] 0 2024-05-25T06:11:49.932 INFO:tasks.workunit.client.0.smithi160.stdout:7/979: dwrite d6/d32/d7e/f109 [0,4194304] 0 2024-05-25T06:11:49.932 INFO:tasks.workunit.client.0.smithi160.stdout:3/942: write d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/d13f/f57 [2961192,423] 0 2024-05-25T06:11:49.937 INFO:tasks.workunit.client.0.smithi160.stdout:3/943: chown d19/l1f 6799505 1 2024-05-25T06:11:49.937 INFO:tasks.workunit.client.0.smithi160.stdout:7/980: link d6/ca d6/d2b/d5d/de2/c13d 0 2024-05-25T06:11:49.940 INFO:tasks.workunit.client.0.smithi160.stdout:7/981: symlink d6/d2b/da0/l13e 0 2024-05-25T06:11:49.940 INFO:tasks.workunit.client.0.smithi160.stdout:7/982: chown d6/d2b/d5d/d89/dd6/cf8 1584 1 2024-05-25T06:11:49.942 INFO:tasks.workunit.client.0.smithi160.stdout:3/944: read d19/d2f/fc6 [1654660,125983] 0 2024-05-25T06:11:49.942 INFO:tasks.workunit.client.0.smithi160.stdout:7/983: creat d6/d32/d92/f13f x:0 0 0 2024-05-25T06:11:49.947 INFO:tasks.workunit.client.0.smithi160.stdout:3/945: link d19/d2f/d38/d43/d5b/d65/d83/dee/df3/f112 d19/d2f/d38/d43/d5b/d65/f140 0 2024-05-25T06:11:49.949 INFO:tasks.workunit.client.0.smithi160.stdout:7/984: dwrite d6/dc/f82 [0,4194304] 0 2024-05-25T06:11:49.954 INFO:tasks.workunit.client.0.smithi160.stdout:3/946: rmdir d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d10d 0 2024-05-25T06:11:49.958 INFO:tasks.workunit.client.0.smithi160.stdout:7/985: dwrite d6/d2b/d8b/f13a [0,4194304] 0 2024-05-25T06:11:49.958 INFO:tasks.workunit.client.0.smithi160.stdout:3/947: chown d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/da8/df8/l109 264 1 2024-05-25T06:11:49.959 INFO:tasks.workunit.client.0.smithi160.stdout:7/986: stat d6/d32/l124 0 2024-05-25T06:11:49.961 INFO:tasks.workunit.client.0.smithi160.stdout:7/987: write d6/dc/d64/d5a/fa6 [1764311,74325] 0 2024-05-25T06:11:49.961 INFO:tasks.workunit.client.0.smithi160.stdout:3/948: creat d19/da7/f141 x:0 0 0 2024-05-25T06:11:49.966 INFO:tasks.workunit.client.0.smithi160.stdout:3/949: unlink d19/db2/lf7 0 2024-05-25T06:11:49.966 INFO:tasks.workunit.client.0.smithi160.stdout:7/988: mknod d6/d3e/da1/ddb/c140 0 2024-05-25T06:11:49.968 INFO:tasks.workunit.client.0.smithi160.stdout:3/950: fsync d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/f95 0 2024-05-25T06:11:49.970 INFO:tasks.workunit.client.0.smithi160.stdout:3/951: readlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/l113 0 2024-05-25T06:11:49.972 INFO:tasks.workunit.client.0.smithi160.stdout:3/952: write d19/da6/f76 [174458,62674] 0 2024-05-25T06:11:49.973 INFO:tasks.workunit.client.0.smithi160.stdout:7/989: getdents d6/dba 0 2024-05-25T06:11:49.974 INFO:tasks.workunit.client.0.smithi160.stdout:3/953: chown d19/d2f/d38/d43/d5b/lfa 1940436323 1 2024-05-25T06:11:49.978 INFO:tasks.workunit.client.0.smithi160.stdout:7/990: dread d6/d2b/d5d/d89/fa7 [0,4194304] 0 2024-05-25T06:11:49.982 INFO:tasks.workunit.client.0.smithi160.stdout:7/991: creat d6/f141 x:0 0 0 2024-05-25T06:11:49.983 INFO:tasks.workunit.client.0.smithi160.stdout:7/992: creat d6/dc/d21/ddd/f142 x:0 0 0 2024-05-25T06:11:49.985 INFO:tasks.workunit.client.0.smithi160.stdout:7/993: stat d6/d3e/da1 0 2024-05-25T06:11:49.987 INFO:tasks.workunit.client.0.smithi160.stdout:3/954: dwrite d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/d13f/fc7 [0,4194304] 0 2024-05-25T06:11:49.988 INFO:tasks.workunit.client.0.smithi160.stdout:7/994: creat d6/d3e/d59/d65/f143 x:0 0 0 2024-05-25T06:11:49.989 INFO:tasks.workunit.client.0.smithi160.stdout:7/995: rename d6 to d6/d32/d7e/de0/d144 22 2024-05-25T06:11:49.990 INFO:tasks.workunit.client.0.smithi160.stdout:3/955: fdatasync d19/da6/f61 0 2024-05-25T06:11:49.991 INFO:tasks.workunit.client.0.smithi160.stdout:7/996: rmdir d6/d2b/d5d/d89 39 2024-05-25T06:11:49.995 INFO:tasks.workunit.client.0.smithi160.stdout:3/956: mkdir d19/d2f/d38/d43/d5b/d65/d83/d120/d142 0 2024-05-25T06:11:49.997 INFO:tasks.workunit.client.0.smithi160.stdout:7/997: mkdir d6/dc/d64/d5a/dc4/d145 0 2024-05-25T06:11:49.997 INFO:tasks.workunit.client.0.smithi160.stdout:3/957: stat d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/da8/df8 0 2024-05-25T06:11:49.999 INFO:tasks.workunit.client.0.smithi160.stdout:3/958: creat d19/da7/f143 x:0 0 0 2024-05-25T06:11:50.000 INFO:tasks.workunit.client.0.smithi160.stdout:7/998: write d6/d2b/d5d/d89/fa7 [425572,79558] 0 2024-05-25T06:11:50.002 INFO:tasks.workunit.client.0.smithi160.stdout:7/999: mkdir d6/dc/d64/dae/d146 0 2024-05-25T06:11:50.005 INFO:tasks.workunit.client.0.smithi160.stdout:3/959: rename d19/d2f/d38/d43/d5b/d65/d83/ddf/d117 to d19/d2f/d38/d43/d5b/d65/d83/d120/d134/d144 0 2024-05-25T06:11:50.007 INFO:tasks.workunit.client.0.smithi160.stdout:3/960: mkdir d19/da6/d145 0 2024-05-25T06:11:50.010 INFO:tasks.workunit.client.0.smithi160.stdout:3/961: mknod d19/d2f/d38/d43/c146 0 2024-05-25T06:11:50.016 INFO:tasks.workunit.client.0.smithi160.stdout:3/962: dwrite d19/d2f/d38/fb0 [0,4194304] 0 2024-05-25T06:11:50.018 INFO:tasks.workunit.client.0.smithi160.stdout:3/963: chown d19/d2f/d38/d43/d5b/d65/d83/dee/df3/f112 3 1 2024-05-25T06:11:50.020 INFO:tasks.workunit.client.0.smithi160.stdout:3/964: mkdir d19/da6/d145/d147 0 2024-05-25T06:11:50.023 INFO:tasks.workunit.client.0.smithi160.stdout:3/965: symlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/da8/df8/d110/l148 0 2024-05-25T06:11:50.026 INFO:tasks.workunit.client.0.smithi160.stdout:3/966: symlink d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/def/l149 0 2024-05-25T06:11:50.028 INFO:tasks.workunit.client.0.smithi160.stdout:3/967: stat d19/d2f/d38/d43/d5b/d65/d83/fcb 0 2024-05-25T06:11:50.031 INFO:tasks.workunit.client.0.smithi160.stdout:3/968: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/d13b/d14a 0 2024-05-25T06:11:50.032 INFO:tasks.workunit.client.0.smithi160.stdout:3/969: stat d19/d2f/d38/f121 0 2024-05-25T06:11:50.035 INFO:tasks.workunit.client.0.smithi160.stdout:3/970: creat d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/d13b/d14a/f14b x:0 0 0 2024-05-25T06:11:50.036 INFO:tasks.workunit.client.0.smithi160.stdout:3/971: dread - d19/d2f/d38/d43/d5b/d65/d83/ddf/f132 zero size 2024-05-25T06:11:50.038 INFO:tasks.workunit.client.0.smithi160.stdout:3/972: unlink d19/l1f 0 2024-05-25T06:11:50.040 INFO:tasks.workunit.client.0.smithi160.stdout:3/973: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/d14c 0 2024-05-25T06:11:50.042 INFO:tasks.workunit.client.0.smithi160.stdout:3/974: truncate d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/d13b/d14a/f14b 925691 0 2024-05-25T06:11:50.044 INFO:tasks.workunit.client.0.smithi160.stdout:3/975: mkdir d19/d2f/d38/d43/d5b/d65/d83/d120/d134/d144/d14d 0 2024-05-25T06:11:50.045 INFO:tasks.workunit.client.0.smithi160.stdout:3/976: chown d19/db2/d104 1 1 2024-05-25T06:11:50.045 INFO:tasks.workunit.client.0.smithi160.stdout:3/977: chown d19/d2f/d38/d43/d5b/c84 6 1 2024-05-25T06:11:50.051 INFO:tasks.workunit.client.0.smithi160.stdout:3/978: dwrite d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/d13f/f4b [0,4194304] 0 2024-05-25T06:11:50.055 INFO:tasks.workunit.client.0.smithi160.stdout:3/979: creat d19/d2f/d38/d43/d5b/d65/d83/dee/d14c/f14e x:0 0 0 2024-05-25T06:11:50.060 INFO:tasks.workunit.client.0.smithi160.stdout:3/980: rename d19/d2f/fa5 to d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/d67/d13b/f14f 0 2024-05-25T06:11:50.061 INFO:tasks.workunit.client.0.smithi160.stdout:3/981: chown d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/d127/f95 3182332 1 2024-05-25T06:11:50.063 INFO:tasks.workunit.client.0.smithi160.stdout:3/982: dread d19/d2f/d38/ffb [0,4194304] 0 2024-05-25T06:11:50.064 INFO:tasks.workunit.client.0.smithi160.stdout:3/983: chown d19/d2f/d38/d43/d5b/f6d 6763145 1 2024-05-25T06:11:50.077 INFO:tasks.workunit.client.0.smithi160.stdout:3/984: truncate d19/d2f/d38/d43/d5b/d65/fbd 3269863 0 2024-05-25T06:11:50.079 INFO:tasks.workunit.client.0.smithi160.stdout:3/985: symlink d19/d2f/d38/d43/d5b/d65/d83/d120/l150 0 2024-05-25T06:11:50.082 INFO:tasks.workunit.client.0.smithi160.stdout:3/986: creat d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/d13f/f151 x:0 0 0 2024-05-25T06:11:50.083 INFO:tasks.workunit.client.0.smithi160.stdout:3/987: write d19/d2f/d38/d60/f11f [558276,84482] 0 2024-05-25T06:11:50.084 INFO:tasks.workunit.client.0.smithi160.stdout:3/988: stat d19/da7/f141 0 2024-05-25T06:11:50.086 INFO:tasks.workunit.client.0.smithi160.stdout:3/989: write d19/d2f/d38/d43/d5b/d65/d83/ddf/f132 [1004296,121684] 0 2024-05-25T06:11:50.087 INFO:tasks.workunit.client.0.smithi160.stdout:3/990: write d19/d2f/f94 [1182241,37340] 0 2024-05-25T06:11:50.093 INFO:tasks.workunit.client.0.smithi160.stdout:3/991: dwrite d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/dd9/ded/d13f/f4c [0,4194304] 0 2024-05-25T06:11:50.096 INFO:tasks.workunit.client.0.smithi160.stdout:3/992: unlink d19/d2f/dad/f126 0 2024-05-25T06:11:50.098 INFO:tasks.workunit.client.0.smithi160.stdout:3/993: fdatasync d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d12a/f10e 0 2024-05-25T06:11:50.101 INFO:tasks.workunit.client.0.smithi160.stdout:3/994: creat d19/d2f/d38/d43/d5b/d65/d83/dee/d102/f152 x:0 0 0 2024-05-25T06:11:50.104 INFO:tasks.workunit.client.0.smithi160.stdout:3/995: creat d19/d2f/d38/d43/f153 x:0 0 0 2024-05-25T06:11:50.106 INFO:tasks.workunit.client.0.smithi160.stdout:3/996: mkdir d19/d2f/d38/d43/d5b/d65/d83/d120/d154 0 2024-05-25T06:11:50.107 INFO:tasks.workunit.client.0.smithi160.stdout:3/997: stat d19/d2f/d38/d60/f82 0 2024-05-25T06:11:50.109 INFO:tasks.workunit.client.0.smithi160.stdout:3/998: mkdir d19/d2f/d38/d43/d5b/d65/d83/dee/d102/d155 0 2024-05-25T06:11:50.109 INFO:tasks.workunit.client.0.smithi160.stdout:3/999: stat d19/d22/l25 0 2024-05-25T06:11:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:50 smithi160 ceph-mon[29041]: Reconfiguring node-exporter.smithi160 (dependencies changed)... 2024-05-25T06:11:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:50 smithi160 ceph-mon[29041]: Reconfiguring daemon node-exporter.smithi160 on smithi160 2024-05-25T06:11:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:50 smithi160 ceph-mon[29041]: pgmap v25: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 106 MiB/s rd, 238 MiB/s wr, 532 op/s 2024-05-25T06:11:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:50 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:50.516 INFO:tasks.workunit.client.0.smithi160.stdout:6/897: sync 2024-05-25T06:11:50.516 INFO:tasks.workunit.client.0.smithi160.stdout:6/898: fsync d5/d39/d112/da4/f102 0 2024-05-25T06:11:50.518 INFO:tasks.workunit.client.0.smithi160.stdout:6/899: mknod d5/d45/d4a/dbc/dc4/c134 0 2024-05-25T06:11:50.520 INFO:tasks.workunit.client.0.smithi160.stdout:6/900: mknod d5/d45/c135 0 2024-05-25T06:11:50.521 INFO:tasks.workunit.client.0.smithi160.stdout:6/901: mknod d5/d39/d112/da4/ddf/c136 0 2024-05-25T06:11:50.524 INFO:tasks.workunit.client.0.smithi160.stdout:6/902: link d5/d30/f5a d5/d39/d8f/f137 0 2024-05-25T06:11:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:50 smithi203 ceph-mon[36030]: Reconfiguring node-exporter.smithi160 (dependencies changed)... 2024-05-25T06:11:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:50 smithi203 ceph-mon[36030]: Reconfiguring daemon node-exporter.smithi160 on smithi160 2024-05-25T06:11:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:50 smithi203 ceph-mon[36030]: pgmap v25: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 106 MiB/s rd, 238 MiB/s wr, 532 op/s 2024-05-25T06:11:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:50 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:11:50.714 INFO:tasks.workunit.client.0.smithi160.stdout:5/996: sync 2024-05-25T06:11:50.714 INFO:tasks.workunit.client.0.smithi160.stdout:1/857: sync 2024-05-25T06:11:50.714 INFO:tasks.workunit.client.0.smithi160.stdout:9/807: sync 2024-05-25T06:11:50.714 INFO:tasks.workunit.client.0.smithi160.stdout:8/824: sync 2024-05-25T06:11:50.715 INFO:tasks.workunit.client.0.smithi160.stdout:4/898: sync 2024-05-25T06:11:50.715 INFO:tasks.workunit.client.0.smithi160.stdout:8/825: readlink d5/d9/d72/l83 0 2024-05-25T06:11:50.716 INFO:tasks.workunit.client.0.smithi160.stdout:9/808: stat d4/d11/d14/f53 0 2024-05-25T06:11:50.717 INFO:tasks.workunit.client.0.smithi160.stdout:1/858: mknod de/c128 0 2024-05-25T06:11:50.717 INFO:tasks.workunit.client.0.smithi160.stdout:9/809: write d4/d11/d1e/d3e/f44 [1320170,89178] 0 2024-05-25T06:11:50.718 INFO:tasks.workunit.client.0.smithi160.stdout:4/899: mkdir d4/d6/d35/d3d/d127 0 2024-05-25T06:11:50.735 INFO:tasks.workunit.client.0.smithi160.stdout:5/997: dread db/d108/f32 [4194304,4194304] 0 2024-05-25T06:11:50.741 INFO:tasks.workunit.client.0.smithi160.stdout:8/826: write d5/d9/fc [1810948,85105] 0 2024-05-25T06:11:50.771 INFO:tasks.workunit.client.0.smithi160.stdout:1/859: rename de/d21/fb4 to de/d21/d49/d1c/d52/dc7/ddb/d126/f129 0 2024-05-25T06:11:50.772 INFO:tasks.workunit.client.0.smithi160.stdout:9/810: symlink d4/d11/d1e/d3e/d25/d5e/da2/l101 0 2024-05-25T06:11:50.778 INFO:tasks.workunit.client.0.smithi160.stdout:9/811: rename d4/d11/d1e/c36 to d4/d11/d1e/dbc/df1/c102 0 2024-05-25T06:11:50.781 INFO:tasks.workunit.client.0.smithi160.stdout:1/860: link de/d21/d49/d1c/d59/d7f/da1/l11f de/d21/d49/d1c/d52/l12a 0 2024-05-25T06:11:50.782 INFO:tasks.workunit.client.0.smithi160.stdout:1/861: write f9 [3997060,53113] 0 2024-05-25T06:11:50.784 INFO:tasks.workunit.client.0.smithi160.stdout:4/900: dread d4/d6/d28/de4/f56 [4194304,4194304] 0 2024-05-25T06:11:50.785 INFO:tasks.workunit.client.0.smithi160.stdout:1/862: fdatasync de/f15 0 2024-05-25T06:11:50.788 INFO:tasks.workunit.client.0.smithi160.stdout:1/863: rmdir de/d21/d49/d1c/d59/d7f 39 2024-05-25T06:11:50.795 INFO:tasks.workunit.client.0.smithi160.stdout:4/901: dwrite d4/d6/dbd/f11e [0,4194304] 0 2024-05-25T06:11:50.796 INFO:tasks.workunit.client.0.smithi160.stdout:1/864: truncate de/d21/d49/d1c/f53 2289279 0 2024-05-25T06:11:50.796 INFO:tasks.workunit.client.0.smithi160.stdout:9/812: dread d4/fae [0,4194304] 0 2024-05-25T06:11:50.797 INFO:tasks.workunit.client.0.smithi160.stdout:4/902: chown d4/d6/d28/fc7 114834491 1 2024-05-25T06:11:50.798 INFO:tasks.workunit.client.0.smithi160.stdout:1/865: stat de/d21/d49/d1c/d59/d71/fc0 0 2024-05-25T06:11:50.799 INFO:tasks.workunit.client.0.smithi160.stdout:4/903: creat d4/d6/dbd/f128 x:0 0 0 2024-05-25T06:11:50.802 INFO:tasks.workunit.client.0.smithi160.stdout:1/866: symlink de/d21/d49/dc6/l12b 0 2024-05-25T06:11:50.803 INFO:tasks.workunit.client.0.smithi160.stdout:1/867: write de/d39/f72 [3099850,117062] 0 2024-05-25T06:11:50.804 INFO:tasks.workunit.client.0.smithi160.stdout:4/904: mknod d4/c129 0 2024-05-25T06:11:50.807 INFO:tasks.workunit.client.0.smithi160.stdout:4/905: truncate d4/d6/d21/f48 4348524 0 2024-05-25T06:11:50.808 INFO:tasks.workunit.client.0.smithi160.stdout:4/906: stat d4/d1c/d54/dbc/dda/d94/dec 0 2024-05-25T06:11:50.809 INFO:tasks.workunit.client.0.smithi160.stdout:1/868: mknod de/d21/d49/d1c/d59/d7f/c12c 0 2024-05-25T06:11:50.810 INFO:tasks.workunit.client.0.smithi160.stdout:1/869: fdatasync de/f19 0 2024-05-25T06:11:50.810 INFO:tasks.workunit.client.0.smithi160.stdout:4/907: dread - d4/d63/fed zero size 2024-05-25T06:11:50.811 INFO:tasks.workunit.client.0.smithi160.stdout:4/908: stat d4/d6/d5b/d66/d78/fcb 0 2024-05-25T06:11:50.813 INFO:tasks.workunit.client.0.smithi160.stdout:1/870: creat de/d21/d49/d1c/d59/d7c/f12d x:0 0 0 2024-05-25T06:11:50.815 INFO:tasks.workunit.client.0.smithi160.stdout:4/909: mkdir d4/d12a 0 2024-05-25T06:11:50.816 INFO:tasks.workunit.client.0.smithi160.stdout:1/871: write de/d21/d49/d1c/d59/d71/f5a [1929774,60380] 0 2024-05-25T06:11:50.817 INFO:tasks.workunit.client.0.smithi160.stdout:8/827: dwrite d5/d44/f37 [0,4194304] 0 2024-05-25T06:11:50.823 INFO:tasks.workunit.client.0.smithi160.stdout:4/910: creat d4/d6/d5b/d66/d78/da1/f12b x:0 0 0 2024-05-25T06:11:50.824 INFO:tasks.workunit.client.0.smithi160.stdout:9/813: dwrite d4/d11/d1e/d24/fd1 [0,4194304] 0 2024-05-25T06:11:50.825 INFO:tasks.workunit.client.0.smithi160.stdout:4/911: write d4/d6/dbd/f11e [254823,73470] 0 2024-05-25T06:11:50.825 INFO:tasks.workunit.client.0.smithi160.stdout:8/828: creat d5/d7/ded/f10e x:0 0 0 2024-05-25T06:11:50.826 INFO:tasks.workunit.client.0.smithi160.stdout:4/912: dread - d4/f100 zero size 2024-05-25T06:11:50.831 INFO:tasks.workunit.client.0.smithi160.stdout:9/814: chown d4/d11/d1e/d24/cbe 693456 1 2024-05-25T06:11:50.836 INFO:tasks.workunit.client.0.smithi160.stdout:8/829: symlink d5/d9/d72/d104/l10f 0 2024-05-25T06:11:50.838 INFO:tasks.workunit.client.0.smithi160.stdout:4/913: creat d4/d63/d64/d88/d119/f12c x:0 0 0 2024-05-25T06:11:50.838 INFO:tasks.workunit.client.0.smithi160.stdout:4/914: dread - d4/d6/d5b/fef zero size 2024-05-25T06:11:50.839 INFO:tasks.workunit.client.0.smithi160.stdout:8/830: symlink d5/d44/d1f/da3/d28/d4a/db8/l110 0 2024-05-25T06:11:50.840 INFO:tasks.workunit.client.0.smithi160.stdout:4/915: symlink d4/d63/d64/l12d 0 2024-05-25T06:11:50.840 INFO:tasks.workunit.client.0.smithi160.stdout:1/872: write de/d21/d49/d1c/fc9 [751617,21398] 0 2024-05-25T06:11:50.841 INFO:tasks.workunit.client.0.smithi160.stdout:8/831: dread - d5/d44/d1f/da3/d28/d4a/d4e/fac zero size 2024-05-25T06:11:50.841 INFO:tasks.workunit.client.0.smithi160.stdout:4/916: truncate d4/d6/f107 110424 0 2024-05-25T06:11:50.842 INFO:tasks.workunit.client.0.smithi160.stdout:1/873: symlink de/d21/d49/d1c/d59/d67/l12e 0 2024-05-25T06:11:50.843 INFO:tasks.workunit.client.0.smithi160.stdout:4/917: dread d4/d1a/f8b [0,4194304] 0 2024-05-25T06:11:50.843 INFO:tasks.workunit.client.0.smithi160.stdout:8/832: symlink d5/d44/d1f/d58/d65/da2/l111 0 2024-05-25T06:11:50.847 INFO:tasks.workunit.client.0.smithi160.stdout:4/918: creat d4/d1c/d54/dbc/dda/f12e x:0 0 0 2024-05-25T06:11:50.848 INFO:tasks.workunit.client.0.smithi160.stdout:4/919: chown d4/d1c/d54/dbc/dda/f104 0 1 2024-05-25T06:11:50.851 INFO:tasks.workunit.client.0.smithi160.stdout:4/920: symlink d4/d1c/d54/l12f 0 2024-05-25T06:11:50.854 INFO:tasks.workunit.client.0.smithi160.stdout:1/874: dwrite de/d21/d49/d1c/d59/d7f/faa [0,4194304] 0 2024-05-25T06:11:50.859 INFO:tasks.workunit.client.0.smithi160.stdout:1/875: getdents de/d21 0 2024-05-25T06:11:50.861 INFO:tasks.workunit.client.0.smithi160.stdout:1/876: read - de/d39/d8f/fff zero size 2024-05-25T06:11:50.863 INFO:tasks.workunit.client.0.smithi160.stdout:4/921: dwrite d4/d63/d64/f80 [0,4194304] 0 2024-05-25T06:11:50.864 INFO:tasks.workunit.client.0.smithi160.stdout:4/922: stat d4/d1c/d54/d9b 0 2024-05-25T06:11:50.865 INFO:tasks.workunit.client.0.smithi160.stdout:4/923: stat d4/d6/d21/f77 0 2024-05-25T06:11:50.865 INFO:tasks.workunit.client.0.smithi160.stdout:1/877: creat de/d21/d49/d1c/d52/dc7/df6/f12f x:0 0 0 2024-05-25T06:11:50.871 INFO:tasks.workunit.client.0.smithi160.stdout:4/924: dread d4/d6/d21/fd0 [0,4194304] 0 2024-05-25T06:11:50.874 INFO:tasks.workunit.client.0.smithi160.stdout:4/925: creat d4/d6/d5b/d66/d78/f130 x:0 0 0 2024-05-25T06:11:50.877 INFO:tasks.workunit.client.0.smithi160.stdout:4/926: getdents d4/d6/d35/d3d/d9e 0 2024-05-25T06:11:50.881 INFO:tasks.workunit.client.0.smithi160.stdout:4/927: link d4/d1c/d54/l12f d4/d6/l131 0 2024-05-25T06:11:50.882 INFO:tasks.workunit.client.0.smithi160.stdout:4/928: symlink d4/d1a/l132 0 2024-05-25T06:11:50.888 INFO:tasks.workunit.client.0.smithi160.stdout:6/903: sync 2024-05-25T06:11:50.889 INFO:tasks.workunit.client.0.smithi160.stdout:6/904: symlink d5/d39/d112/da4/ddf/l138 0 2024-05-25T06:11:50.891 INFO:tasks.workunit.client.0.smithi160.stdout:6/905: mknod d5/d39/dc1/d113/d115/c139 0 2024-05-25T06:11:50.892 INFO:tasks.workunit.client.0.smithi160.stdout:6/906: write d5/d45/d4a/fa8 [285700,46882] 0 2024-05-25T06:11:50.894 INFO:tasks.workunit.client.0.smithi160.stdout:6/907: fsync d5/d45/f9a 0 2024-05-25T06:11:50.896 INFO:tasks.workunit.client.0.smithi160.stdout:6/908: mkdir d5/d39/dc1/d113/d13a 0 2024-05-25T06:11:50.899 INFO:tasks.workunit.client.0.smithi160.stdout:6/909: truncate d5/de/d32/d35/dab/ffb 1034702 0 2024-05-25T06:11:50.903 INFO:tasks.workunit.client.0.smithi160.stdout:6/910: write d5/de1/f7e [899435,126317] 0 2024-05-25T06:11:50.910 INFO:tasks.workunit.client.0.smithi160.stdout:6/911: dwrite d5/de/f22 [0,4194304] 0 2024-05-25T06:11:50.918 INFO:tasks.workunit.client.0.smithi160.stdout:6/912: dwrite d5/d39/da6/f12f [0,4194304] 0 2024-05-25T06:11:51.210 INFO:tasks.workunit.client.0.smithi160.stdout:5/998: sync 2024-05-25T06:11:51.210 INFO:tasks.workunit.client.0.smithi160.stdout:9/815: sync 2024-05-25T06:11:51.211 INFO:tasks.workunit.client.0.smithi160.stdout:9/816: fdatasync d4/d11/d14/f37 0 2024-05-25T06:11:51.212 INFO:tasks.workunit.client.0.smithi160.stdout:5/999: rmdir db/dc/d13/d1d/d3f/d72/d98/da5/db5/d101/d113/db3 39 2024-05-25T06:11:51.213 INFO:tasks.workunit.client.0.smithi160.stdout:9/817: chown d4/d11/d1e/d24/df7/caf 509 1 2024-05-25T06:11:51.215 INFO:tasks.workunit.client.0.smithi160.stdout:9/818: mknod d4/d11/d12/d2e/d45/d83/c103 0 2024-05-25T06:11:51.216 INFO:tasks.workunit.client.0.smithi160.stdout:9/819: chown d4/d11/d12/d2e/d45/d83/fcd 4069 1 2024-05-25T06:11:51.217 INFO:tasks.workunit.client.0.smithi160.stdout:9/820: rmdir d4/d11/d1e/d24/df7/d9c 39 2024-05-25T06:11:51.218 INFO:tasks.workunit.client.0.smithi160.stdout:9/821: write d4/fc2 [1237149,9779] 0 2024-05-25T06:11:51.221 INFO:tasks.workunit.client.0.smithi160.stdout:9/822: getdents d4/d11/d1e/d24/df7 0 2024-05-25T06:11:51.223 INFO:tasks.workunit.client.0.smithi160.stdout:9/823: creat d4/d11/d1e/dbc/df1/f104 x:0 0 0 2024-05-25T06:11:51.224 INFO:tasks.workunit.client.0.smithi160.stdout:9/824: fsync d4/d11/d1e/fac 0 2024-05-25T06:11:51.226 INFO:tasks.workunit.client.0.smithi160.stdout:9/825: unlink d4/d11/f80 0 2024-05-25T06:11:51.228 INFO:tasks.workunit.client.0.smithi160.stdout:9/826: creat d4/d11/d12/d2e/f105 x:0 0 0 2024-05-25T06:11:51.241 INFO:tasks.workunit.client.0.smithi160.stdout:9/827: dread d4/d11/d1e/d24/df7/d9c/ff9 [0,4194304] 0 2024-05-25T06:11:51.243 INFO:tasks.workunit.client.0.smithi160.stdout:9/828: chown d4/d11/d1e/d3e/c52 8037 1 2024-05-25T06:11:51.244 INFO:tasks.workunit.client.0.smithi160.stdout:9/829: creat d4/d11/d12/f106 x:0 0 0 2024-05-25T06:11:51.250 INFO:tasks.workunit.client.0.smithi160.stdout:9/830: dwrite d4/fde [0,4194304] 0 2024-05-25T06:11:51.257 INFO:tasks.workunit.client.0.smithi160.stdout:9/831: dwrite d4/fc2 [4194304,4194304] 0 2024-05-25T06:11:51.273 INFO:tasks.workunit.client.0.smithi160.stdout:9/832: dread d4/d11/d1e/d3e/d25/d5e/fda [0,4194304] 0 2024-05-25T06:11:51.274 INFO:tasks.workunit.client.0.smithi160.stdout:9/833: rmdir d4/d11/d7a 39 2024-05-25T06:11:51.397 INFO:tasks.workunit.client.0.smithi160.stdout:8/833: sync 2024-05-25T06:11:51.401 INFO:tasks.workunit.client.0.smithi160.stdout:8/834: write d5/d9c/ff9 [660457,59634] 0 2024-05-25T06:11:51.404 INFO:tasks.workunit.client.0.smithi160.stdout:8/835: symlink d5/d7/dcb/d6a/l112 0 2024-05-25T06:11:51.411 INFO:tasks.workunit.client.0.smithi160.stdout:8/836: dwrite d5/d9/fbb [0,4194304] 0 2024-05-25T06:11:51.413 INFO:tasks.workunit.client.0.smithi160.stdout:8/837: symlink d5/d44/d1f/da3/d28/d4a/d4e/dee/l113 0 2024-05-25T06:11:51.416 INFO:tasks.workunit.client.0.smithi160.stdout:8/838: read d5/d44/d1f/fb6 [4184967,40603] 0 2024-05-25T06:11:51.418 INFO:tasks.workunit.client.0.smithi160.stdout:8/839: fsync d5/d7/d9a/fca 0 2024-05-25T06:11:51.422 INFO:tasks.workunit.client.0.smithi160.stdout:8/840: dread d5/d9/fbb [0,4194304] 0 2024-05-25T06:11:51.424 INFO:tasks.workunit.client.0.smithi160.stdout:8/841: mknod d5/d44/dba/ddb/de3/c114 0 2024-05-25T06:11:51.426 INFO:tasks.workunit.client.0.smithi160.stdout:8/842: truncate d5/d7/dcb/d75/fa5 1168328 0 2024-05-25T06:11:51.427 INFO:tasks.workunit.client.0.smithi160.stdout:8/843: chown d5/d44/d1f/da3/d28/f3f 239684836 1 2024-05-25T06:11:51.427 INFO:tasks.workunit.client.0.smithi160.stdout:8/844: fsync d5/d7/ded/f10e 0 2024-05-25T06:11:51.433 INFO:tasks.workunit.client.0.smithi160.stdout:8/845: dwrite d5/d44/d1f/da3/d28/f3f [4194304,4194304] 0 2024-05-25T06:11:51.437 INFO:tasks.workunit.client.0.smithi160.stdout:8/846: write d5/d9/fbb [1543753,53423] 0 2024-05-25T06:11:51.439 INFO:tasks.workunit.client.0.smithi160.stdout:8/847: symlink d5/d7/dcb/d7a/l115 0 2024-05-25T06:11:51.442 INFO:tasks.workunit.client.0.smithi160.stdout:8/848: truncate d5/d7/d9a/fdd 413843 0 2024-05-25T06:11:51.446 INFO:tasks.workunit.client.0.smithi160.stdout:8/849: getdents d5/d44/d1f/da3/d28/dc6/de8 0 2024-05-25T06:11:51.448 INFO:tasks.workunit.client.0.smithi160.stdout:8/850: mknod d5/d44/dba/c116 0 2024-05-25T06:11:51.458 INFO:tasks.workunit.client.0.smithi160.stdout:8/851: dwrite d5/d7/dcb/fe1 [0,4194304] 0 2024-05-25T06:11:51.460 INFO:tasks.workunit.client.0.smithi160.stdout:8/852: fdatasync d5/d9/d72/daf/dbf/fea 0 2024-05-25T06:11:51.466 INFO:tasks.workunit.client.0.smithi160.stdout:8/853: write d5/d9/d72/daf/ffc [103894,83093] 0 2024-05-25T06:11:51.469 INFO:tasks.workunit.client.0.smithi160.stdout:6/913: sync 2024-05-25T06:11:51.469 INFO:tasks.workunit.client.0.smithi160.stdout:1/878: sync 2024-05-25T06:11:51.469 INFO:tasks.workunit.client.0.smithi160.stdout:4/929: sync 2024-05-25T06:11:51.470 INFO:tasks.workunit.client.0.smithi160.stdout:4/930: fdatasync d4/d6/dbd/df0/ffc 0 2024-05-25T06:11:51.470 INFO:tasks.workunit.client.0.smithi160.stdout:6/914: chown d5/d39/dc1/le0 0 1 2024-05-25T06:11:51.472 INFO:tasks.workunit.client.0.smithi160.stdout:6/915: write d5/de/d9e/dcd/f10d [5652498,5207] 0 2024-05-25T06:11:51.473 INFO:tasks.workunit.client.0.smithi160.stdout:8/854: rename d5/d44/d1f/da3/d28/d49/fc3 to d5/d44/d1f/da3/d28/de6/f117 0 2024-05-25T06:11:51.476 INFO:tasks.workunit.client.0.smithi160.stdout:6/916: rmdir d5/d30/d36/d73/d6e/df7 39 2024-05-25T06:11:51.480 INFO:tasks.workunit.client.0.smithi160.stdout:4/931: read d4/d1a/fa2 [3056742,37245] 0 2024-05-25T06:11:51.480 INFO:tasks.workunit.client.0.smithi160.stdout:1/879: dread de/d21/d49/ff2 [0,4194304] 0 2024-05-25T06:11:51.482 INFO:tasks.workunit.client.0.smithi160.stdout:8/855: dwrite d5/d9/d72/daf/db9/ff8 [0,4194304] 0 2024-05-25T06:11:51.482 INFO:tasks.workunit.client.0.smithi160.stdout:6/917: rename d5/d39/d112/d107/f11e to d5/d45/f13b 0 2024-05-25T06:11:51.485 INFO:tasks.workunit.client.0.smithi160.stdout:8/856: write d5/d7/dcb/fa0 [1068508,115248] 0 2024-05-25T06:11:51.488 INFO:tasks.workunit.client.0.smithi160.stdout:8/857: write d5/d44/d1f/da3/d28/d4a/f5c [7200842,3222] 0 2024-05-25T06:11:51.495 INFO:tasks.workunit.client.0.smithi160.stdout:8/858: rmdir d5/d44/d1f/da3/d28/d4a 39 2024-05-25T06:11:51.495 INFO:tasks.workunit.client.0.smithi160.stdout:6/918: dwrite d5/d39/da6/f10c [0,4194304] 0 2024-05-25T06:11:51.495 INFO:tasks.workunit.client.0.smithi160.stdout:4/932: link d4/d6/d7b/lc6 d4/d6/d28/l133 0 2024-05-25T06:11:51.498 INFO:tasks.workunit.client.0.smithi160.stdout:1/880: dwrite de/d21/d49/d90/feb [0,4194304] 0 2024-05-25T06:11:51.499 INFO:tasks.workunit.client.0.smithi160.stdout:4/933: write d4/f76 [1198900,38438] 0 2024-05-25T06:11:51.501 INFO:tasks.workunit.client.0.smithi160.stdout:1/881: chown de/d21/d49/d1c/d59/d7c/d93/de8/f10f 0 1 2024-05-25T06:11:51.503 INFO:tasks.workunit.client.0.smithi160.stdout:6/919: symlink d5/de/l13c 0 2024-05-25T06:11:51.503 INFO:tasks.workunit.client.0.smithi160.stdout:1/882: read de/d21/d49/f104 [269659,52448] 0 2024-05-25T06:11:51.506 INFO:tasks.workunit.client.0.smithi160.stdout:8/859: readlink d5/d44/d1f/da3/d28/d4a/d4e/d86/dd4/ldf 0 2024-05-25T06:11:51.510 INFO:tasks.workunit.client.0.smithi160.stdout:8/860: rmdir d5/d44/d1f/da3/d28/d4a/d4e/dee 39 2024-05-25T06:11:51.510 INFO:tasks.workunit.client.0.smithi160.stdout:8/861: readlink d5/d7/l51 0 2024-05-25T06:11:51.513 INFO:tasks.workunit.client.0.smithi160.stdout:8/862: chown d5/d44/d1f/da3/d28/d49/c4c 3237477 1 2024-05-25T06:11:51.515 INFO:tasks.workunit.client.0.smithi160.stdout:8/863: creat d5/d44/d1f/d58/d65/da2/f118 x:0 0 0 2024-05-25T06:11:51.516 INFO:tasks.workunit.client.0.smithi160.stdout:8/864: write d5/d7/ded/f10e [922292,123468] 0 2024-05-25T06:11:51.518 INFO:tasks.workunit.client.0.smithi160.stdout:8/865: creat d5/d44/dba/ddb/f119 x:0 0 0 2024-05-25T06:11:51.520 INFO:tasks.workunit.client.0.smithi160.stdout:8/866: chown d5/d44/d1f/da3/d28/dc6/de8/f10d 273 1 2024-05-25T06:11:51.527 INFO:tasks.workunit.client.0.smithi160.stdout:8/867: dwrite d5/d44/d1f/da3/d28/d4a/d93/fab [0,4194304] 0 2024-05-25T06:11:51.532 INFO:tasks.workunit.client.0.smithi160.stdout:1/883: dread de/d21/d49/d1c/d41/ffc [0,4194304] 0 2024-05-25T06:11:51.536 INFO:tasks.workunit.client.0.smithi160.stdout:1/884: fdatasync de/d21/d49/d28/fd9 0 2024-05-25T06:11:51.537 INFO:tasks.workunit.client.0.smithi160.stdout:6/920: dread d5/d45/f9a [0,4194304] 0 2024-05-25T06:11:51.537 INFO:tasks.workunit.client.0.smithi160.stdout:8/868: dread d5/d44/d1f/da3/d28/de6/f117 [4194304,4194304] 0 2024-05-25T06:11:51.541 INFO:tasks.workunit.client.0.smithi160.stdout:6/921: dread d5/d45/d4a/dbc/dc4/f100 [0,4194304] 0 2024-05-25T06:11:51.541 INFO:tasks.workunit.client.0.smithi160.stdout:8/869: write d5/d44/d1f/da3/d28/de6/f117 [8398716,48783] 0 2024-05-25T06:11:51.545 INFO:tasks.workunit.client.0.smithi160.stdout:1/885: dwrite de/d39/d8f/f127 [0,4194304] 0 2024-05-25T06:11:51.549 INFO:tasks.workunit.client.0.smithi160.stdout:1/886: creat de/d21/d49/d1c/d52/dc7/f130 x:0 0 0 2024-05-25T06:11:51.555 INFO:tasks.workunit.client.0.smithi160.stdout:1/887: dwrite de/d39/d8f/fbc [0,4194304] 0 2024-05-25T06:11:51.557 INFO:tasks.workunit.client.0.smithi160.stdout:1/888: chown de/d21/d49/d28/f102 31581964 1 2024-05-25T06:11:51.560 INFO:tasks.workunit.client.0.smithi160.stdout:1/889: creat de/d21/d49/d1c/d59/d71/f131 x:0 0 0 2024-05-25T06:11:51.563 INFO:tasks.workunit.client.0.smithi160.stdout:1/890: creat de/d21/d49/d1c/d41/d45/d7d/db3/dba/f132 x:0 0 0 2024-05-25T06:11:51.564 INFO:tasks.workunit.client.0.smithi160.stdout:1/891: write de/d39/d8f/fc2 [280935,36941] 0 2024-05-25T06:11:51.565 INFO:tasks.workunit.client.0.smithi160.stdout:1/892: stat de/f3d 0 2024-05-25T06:11:51.569 INFO:tasks.workunit.client.0.smithi160.stdout:1/893: truncate de/d21/d49/d1c/d52/dc7/ddb/ff8 615199 0 2024-05-25T06:11:51.569 INFO:tasks.workunit.client.0.smithi160.stdout:1/894: chown de/d21/d49/d1c/d59 281942817 1 2024-05-25T06:11:51.577 INFO:tasks.workunit.client.0.smithi160.stdout:6/922: dread d5/de/d32/d35/f82 [0,4194304] 0 2024-05-25T06:11:51.577 INFO:tasks.workunit.client.0.smithi160.stdout:1/895: dread de/d21/d49/d1c/d59/d67/f83 [0,4194304] 0 2024-05-25T06:11:51.579 INFO:tasks.workunit.client.0.smithi160.stdout:1/896: fsync de/d21/d49/d1c/d41/d45/ffb 0 2024-05-25T06:11:51.580 INFO:tasks.workunit.client.0.smithi160.stdout:8/870: dread d5/d7/d9a/db5/fe5 [0,4194304] 0 2024-05-25T06:11:51.580 INFO:tasks.workunit.client.0.smithi160.stdout:1/897: readlink de/d21/d49/dc6/l12b 0 2024-05-25T06:11:51.581 INFO:tasks.workunit.client.0.smithi160.stdout:6/923: truncate d5/d45/d4a/f76 7771478 0 2024-05-25T06:11:51.582 INFO:tasks.workunit.client.0.smithi160.stdout:8/871: symlink d5/d9/d72/l11a 0 2024-05-25T06:11:51.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/898: mknod de/d21/d49/d1c/d59/c133 0 2024-05-25T06:11:51.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/899: chown de/d39 2662990 1 2024-05-25T06:11:51.584 INFO:tasks.workunit.client.0.smithi160.stdout:6/924: truncate d5/de/d32/d35/f78 1649516 0 2024-05-25T06:11:51.586 INFO:tasks.workunit.client.0.smithi160.stdout:8/872: rename d5/d44/d1f/da3/d28/de6/f117 to d5/d7/dcb/d75/f11b 0 2024-05-25T06:11:51.588 INFO:tasks.workunit.client.0.smithi160.stdout:6/925: fdatasync d5/d45/d4a/f90 0 2024-05-25T06:11:51.591 INFO:tasks.workunit.client.0.smithi160.stdout:1/900: creat de/d21/d49/d1c/d41/d81/f134 x:0 0 0 2024-05-25T06:11:51.598 INFO:tasks.workunit.client.0.smithi160.stdout:8/873: write d5/d44/dba/ddb/fe9 [908130,95111] 0 2024-05-25T06:11:51.599 INFO:tasks.workunit.client.0.smithi160.stdout:6/926: dwrite d5/de/d32/d74/fbb [4194304,4194304] 0 2024-05-25T06:11:51.599 INFO:tasks.workunit.client.0.smithi160.stdout:1/901: write de/f15 [1080637,67428] 0 2024-05-25T06:11:51.602 INFO:tasks.workunit.client.0.smithi160.stdout:8/874: rmdir d5/d44/d1f/da3/d28/d4a/d4e 39 2024-05-25T06:11:51.607 INFO:tasks.workunit.client.0.smithi160.stdout:6/927: link d5/d45/d4a/dbc/dea/f122 d5/de/d58/f13d 0 2024-05-25T06:11:51.608 INFO:tasks.workunit.client.0.smithi160.stdout:8/875: truncate d5/d44/d1f/da3/d28/de6/fe0 91213 0 2024-05-25T06:11:51.610 INFO:tasks.workunit.client.0.smithi160.stdout:8/876: rmdir d5/d44/d1f/da3/d28/d4a 39 2024-05-25T06:11:51.614 INFO:tasks.workunit.client.0.smithi160.stdout:6/928: dwrite d5/d39/da6/f12f [0,4194304] 0 2024-05-25T06:11:51.616 INFO:tasks.workunit.client.0.smithi160.stdout:8/877: getdents d5/d44/d1f/da3/d28/d49 0 2024-05-25T06:11:51.619 INFO:tasks.workunit.client.0.smithi160.stdout:8/878: read d5/fc5 [797096,18052] 0 2024-05-25T06:11:51.622 INFO:tasks.workunit.client.0.smithi160.stdout:1/902: dwrite de/d21/d49/d1c/d41/fee [0,4194304] 0 2024-05-25T06:11:51.626 INFO:tasks.workunit.client.0.smithi160.stdout:8/879: chown d5/d44/d1f/da3/d28/d4a/d4e 2433032 1 2024-05-25T06:11:51.646 INFO:tasks.workunit.client.0.smithi160.stdout:1/903: write de/d21/d49/d35/f6e [1594060,97859] 0 2024-05-25T06:11:51.650 INFO:tasks.workunit.client.0.smithi160.stdout:8/880: dwrite d5/d44/d1f/da3/d28/d49/fb4 [0,4194304] 0 2024-05-25T06:11:51.654 INFO:tasks.workunit.client.0.smithi160.stdout:8/881: creat d5/d44/d1f/da3/d28/d49/d61/f11c x:0 0 0 2024-05-25T06:11:51.655 INFO:tasks.workunit.client.0.smithi160.stdout:1/904: dwrite de/d21/d49/d1c/d52/dc7/df6/f12f [0,4194304] 0 2024-05-25T06:11:51.660 INFO:tasks.workunit.client.0.smithi160.stdout:1/905: creat de/d21/d49/db1/d106/f135 x:0 0 0 2024-05-25T06:11:51.661 INFO:tasks.workunit.client.0.smithi160.stdout:1/906: chown de/d21/c9b 7530 1 2024-05-25T06:11:51.664 INFO:tasks.workunit.client.0.smithi160.stdout:1/907: symlink de/d21/d49/d1c/d41/d81/l136 0 2024-05-25T06:11:51.665 INFO:tasks.workunit.client.0.smithi160.stdout:1/908: read de/d39/d8f/f127 [3459684,108214] 0 2024-05-25T06:11:51.668 INFO:tasks.workunit.client.0.smithi160.stdout:9/834: sync 2024-05-25T06:11:51.669 INFO:tasks.workunit.client.0.smithi160.stdout:9/835: fdatasync d4/d11/d1e/d24/fd1 0 2024-05-25T06:11:51.670 INFO:tasks.workunit.client.0.smithi160.stdout:1/909: dwrite de/d39/d8f/f127 [0,4194304] 0 2024-05-25T06:11:51.671 INFO:tasks.workunit.client.0.smithi160.stdout:9/836: stat d4/d11/d1e/d3e/d25/d27/l8a 0 2024-05-25T06:11:51.678 INFO:tasks.workunit.client.0.smithi160.stdout:8/882: dread d5/fe [0,4194304] 0 2024-05-25T06:11:51.682 INFO:tasks.workunit.client.0.smithi160.stdout:8/883: getdents d5/d44/d1f/da3/d28/de6/dc2/dd6 0 2024-05-25T06:11:51.683 INFO:tasks.workunit.client.0.smithi160.stdout:1/910: truncate de/d21/d49/d28/fad 3129076 0 2024-05-25T06:11:51.685 INFO:tasks.workunit.client.0.smithi160.stdout:8/884: mkdir d5/d44/d1f/da3/d28/d49/d11d 0 2024-05-25T06:11:51.687 INFO:tasks.workunit.client.0.smithi160.stdout:8/885: symlink d5/d44/l11e 0 2024-05-25T06:11:51.689 INFO:tasks.workunit.client.0.smithi160.stdout:9/837: dwrite d4/d11/d1e/d3e/d25/d5e/fa6 [0,4194304] 0 2024-05-25T06:11:51.689 INFO:tasks.workunit.client.0.smithi160.stdout:8/886: fdatasync d5/d44/d1f/d58/d65/da2/fef 0 2024-05-25T06:11:51.693 INFO:tasks.workunit.client.0.smithi160.stdout:1/911: link de/d21/d49/d1c/d59/d7f/da1/dbd/l125 de/d21/d49/d1c/d59/d7c/d93/d121/l137 0 2024-05-25T06:11:51.695 INFO:tasks.workunit.client.0.smithi160.stdout:8/887: mkdir d5/d44/d1f/da3/d28/dc6/de8/d11f 0 2024-05-25T06:11:51.697 INFO:tasks.workunit.client.0.smithi160.stdout:9/838: rename d4/d11/d1e/d24/df7/d9c/fb5 to d4/db2/f107 0 2024-05-25T06:11:51.698 INFO:tasks.workunit.client.0.smithi160.stdout:1/912: truncate de/d21/d49/d28/f3c 298844 0 2024-05-25T06:11:51.700 INFO:tasks.workunit.client.0.smithi160.stdout:8/888: readlink d5/d44/d1f/da3/d28/l8d 0 2024-05-25T06:11:51.705 INFO:tasks.workunit.client.0.smithi160.stdout:1/913: creat de/d21/d49/d1c/d41/f138 x:0 0 0 2024-05-25T06:11:51.708 INFO:tasks.workunit.client.0.smithi160.stdout:8/889: dread d5/d44/d1f/da3/d28/d49/f70 [0,4194304] 0 2024-05-25T06:11:51.709 INFO:tasks.workunit.client.0.smithi160.stdout:8/890: write d5/d44/d1f/da3/d28/d4a/d93/fab [3761359,29073] 0 2024-05-25T06:11:51.709 INFO:tasks.workunit.client.0.smithi160.stdout:1/914: rename de/d39/c11b to de/d21/d49/d1c/d52/dc7/df6/c139 0 2024-05-25T06:11:51.710 INFO:tasks.workunit.client.0.smithi160.stdout:1/915: dread - de/d21/d49/d28/df0/f103 zero size 2024-05-25T06:11:51.712 INFO:tasks.workunit.client.0.smithi160.stdout:1/916: write de/f19 [4186845,35301] 0 2024-05-25T06:11:51.712 INFO:tasks.workunit.client.0.smithi160.stdout:8/891: dread - d5/d7/dcb/d6a/ff0 zero size 2024-05-25T06:11:51.714 INFO:tasks.workunit.client.0.smithi160.stdout:1/917: write de/d39/d8f/fbc [4402289,110618] 0 2024-05-25T06:11:51.715 INFO:tasks.workunit.client.0.smithi160.stdout:8/892: read d5/d9/d72/daf/ffc [24425,65688] 0 2024-05-25T06:11:51.716 INFO:tasks.workunit.client.0.smithi160.stdout:9/839: write d4/fb1 [1023097,13813] 0 2024-05-25T06:11:51.717 INFO:tasks.workunit.client.0.smithi160.stdout:1/918: stat de/d39/d8f/fbc 0 2024-05-25T06:11:51.721 INFO:tasks.workunit.client.0.smithi160.stdout:9/840: mknod d4/d11/d1e/d3e/d2f/da7/c108 0 2024-05-25T06:11:51.722 INFO:tasks.workunit.client.0.smithi160.stdout:1/919: mknod de/d21/d49/d28/dde/c13a 0 2024-05-25T06:11:51.724 INFO:tasks.workunit.client.0.smithi160.stdout:8/893: link d5/d44/d1f/da3/d28/d4a/d4e/d86/f108 d5/d44/d1f/da3/d28/d4a/d4e/d86/dd4/f120 0 2024-05-25T06:11:51.724 INFO:tasks.workunit.client.0.smithi160.stdout:9/841: read - d4/d11/d1e/d3e/d25/d5e/da2/fd0 zero size 2024-05-25T06:11:51.734 INFO:tasks.workunit.client.0.smithi160.stdout:1/920: dwrite de/f10c [0,4194304] 0 2024-05-25T06:11:51.771 INFO:tasks.workunit.client.0.smithi160.stdout:9/842: creat d4/db2/de5/f109 x:0 0 0 2024-05-25T06:11:51.771 INFO:tasks.workunit.client.0.smithi160.stdout:8/894: write d5/d44/d1f/da3/d28/d49/fc4 [1063507,16847] 0 2024-05-25T06:11:51.773 INFO:tasks.workunit.client.0.smithi160.stdout:9/843: chown d4/d11/d1e/d3e/d25/d27/fce 7 1 2024-05-25T06:11:51.775 INFO:tasks.workunit.client.0.smithi160.stdout:8/895: creat d5/d9/d72/d104/f121 x:0 0 0 2024-05-25T06:11:51.776 INFO:tasks.workunit.client.0.smithi160.stdout:9/844: creat d4/d11/d12/d2e/d45/d83/f10a x:0 0 0 2024-05-25T06:11:51.776 INFO:tasks.workunit.client.0.smithi160.stdout:8/896: dread - d5/d44/d1f/da3/d28/d4a/d4e/d86/f101 zero size 2024-05-25T06:11:51.779 INFO:tasks.workunit.client.0.smithi160.stdout:8/897: creat d5/d44/d1f/da3/d28/de6/f122 x:0 0 0 2024-05-25T06:11:51.782 INFO:tasks.workunit.client.0.smithi160.stdout:9/845: rename d4/d11/d12/d2e/d45/d90 to d4/d11/d12/d2e/d10b 0 2024-05-25T06:11:51.784 INFO:tasks.workunit.client.0.smithi160.stdout:8/898: creat d5/d9/d72/daf/dbf/f123 x:0 0 0 2024-05-25T06:11:51.785 INFO:tasks.workunit.client.0.smithi160.stdout:8/899: chown d5/d44/d1f/fb6 220975551 1 2024-05-25T06:11:51.785 INFO:tasks.workunit.client.0.smithi160.stdout:9/846: creat d4/db2/de5/f10c x:0 0 0 2024-05-25T06:11:51.786 INFO:tasks.workunit.client.0.smithi160.stdout:6/929: sync 2024-05-25T06:11:51.786 INFO:tasks.workunit.client.0.smithi160.stdout:4/934: sync 2024-05-25T06:11:51.790 INFO:tasks.workunit.client.0.smithi160.stdout:4/935: symlink d4/d63/d64/l134 0 2024-05-25T06:11:51.793 INFO:tasks.workunit.client.0.smithi160.stdout:4/936: mkdir d4/d6/d5b/d66/d82/d135 0 2024-05-25T06:11:51.795 INFO:tasks.workunit.client.0.smithi160.stdout:4/937: dread - d4/d1c/d54/d9b/f106 zero size 2024-05-25T06:11:51.796 INFO:tasks.workunit.client.0.smithi160.stdout:6/930: dwrite d5/d39/d112/ff6 [0,4194304] 0 2024-05-25T06:11:51.802 INFO:tasks.workunit.client.0.smithi160.stdout:6/931: symlink d5/de/l13e 0 2024-05-25T06:11:51.805 INFO:tasks.workunit.client.0.smithi160.stdout:4/938: dwrite d4/d6/d5b/d66/d78/da1/dab/dfa/f115 [0,4194304] 0 2024-05-25T06:11:51.809 INFO:tasks.workunit.client.0.smithi160.stdout:6/932: dread d5/d45/f9a [0,4194304] 0 2024-05-25T06:11:51.810 INFO:tasks.workunit.client.0.smithi160.stdout:9/847: dread d4/d11/d14/f31 [0,4194304] 0 2024-05-25T06:11:51.812 INFO:tasks.workunit.client.0.smithi160.stdout:6/933: creat d5/d45/d4a/dbc/dea/f13f x:0 0 0 2024-05-25T06:11:51.816 INFO:tasks.workunit.client.0.smithi160.stdout:6/934: symlink d5/de/d32/d74/d77/d89/l140 0 2024-05-25T06:11:51.819 INFO:tasks.workunit.client.0.smithi160.stdout:6/935: creat d5/d39/dc1/d113/f141 x:0 0 0 2024-05-25T06:11:51.819 INFO:tasks.workunit.client.0.smithi160.stdout:6/936: chown d5/d39/d112/da4/fb2 1337708 1 2024-05-25T06:11:51.820 INFO:tasks.workunit.client.0.smithi160.stdout:9/848: link d4/d11/d1e/d3e/d25/d27/ffb d4/d11/d1e/f10d 0 2024-05-25T06:11:51.824 INFO:tasks.workunit.client.0.smithi160.stdout:1/921: write de/d21/d49/d1c/d59/d71/f5c [735297,77360] 0 2024-05-25T06:11:51.825 INFO:tasks.workunit.client.0.smithi160.stdout:6/937: rename d5/d39/l66 to d5/d30/l142 0 2024-05-25T06:11:51.829 INFO:tasks.workunit.client.0.smithi160.stdout:9/849: dread d4/d11/d1e/fd3 [0,4194304] 0 2024-05-25T06:11:51.831 INFO:tasks.workunit.client.0.smithi160.stdout:4/939: write d4/d6/d5b/d66/d78/da1/dab/dfa/f115 [4540266,118148] 0 2024-05-25T06:11:51.831 INFO:tasks.workunit.client.0.smithi160.stdout:1/922: rename de/d21/d49/d1c/d59/d7c/d93/d121 to de/d21/d49/d1c/d59/d7c/d93/d13b 0 2024-05-25T06:11:51.836 INFO:tasks.workunit.client.0.smithi160.stdout:9/850: mknod d4/d11/d1e/d3e/d25/c10e 0 2024-05-25T06:11:51.837 INFO:tasks.workunit.client.0.smithi160.stdout:4/940: fsync d4/d6/d5b/d66/d78/da1/dab/f114 0 2024-05-25T06:11:51.838 INFO:tasks.workunit.client.0.smithi160.stdout:6/938: write d5/d45/d4a/dbc/dea/f122 [744048,100726] 0 2024-05-25T06:11:51.839 INFO:tasks.workunit.client.0.smithi160.stdout:8/900: dwrite d5/d44/d1f/da3/d28/de6/fe0 [0,4194304] 0 2024-05-25T06:11:51.847 INFO:tasks.workunit.client.0.smithi160.stdout:6/939: rename d5/de/d58/f13d to d5/d39/f143 0 2024-05-25T06:11:51.848 INFO:tasks.workunit.client.0.smithi160.stdout:4/941: read d4/d6/d28/f17 [2836957,128530] 0 2024-05-25T06:11:51.849 INFO:tasks.workunit.client.0.smithi160.stdout:8/901: truncate d5/d7/dcb/d7a/fcd 1630535 0 2024-05-25T06:11:51.852 INFO:tasks.workunit.client.0.smithi160.stdout:9/851: rename d4/d11/d1e/d3e/d2f to d4/d11/d12/d2e/d45/d10f 0 2024-05-25T06:11:51.852 INFO:tasks.workunit.client.0.smithi160.stdout:6/940: creat d5/d45/d4a/dbc/dc4/f144 x:0 0 0 2024-05-25T06:11:51.854 INFO:tasks.workunit.client.0.smithi160.stdout:8/902: unlink d5/d44/dba/ddb/cf3 0 2024-05-25T06:11:51.855 INFO:tasks.workunit.client.0.smithi160.stdout:4/942: mkdir d4/d6/d5b/d66/d78/d136 0 2024-05-25T06:11:51.856 INFO:tasks.workunit.client.0.smithi160.stdout:9/852: symlink d4/d11/d1e/d24/df7/l110 0 2024-05-25T06:11:51.859 INFO:tasks.workunit.client.0.smithi160.stdout:8/903: creat d5/d7/dcb/d7a/f124 x:0 0 0 2024-05-25T06:11:51.859 INFO:tasks.workunit.client.0.smithi160.stdout:1/923: dread f5 [4194304,4194304] 0 2024-05-25T06:11:51.860 INFO:tasks.workunit.client.0.smithi160.stdout:6/941: rename d5/d39/dc1/d113/d115 to d5/d30/d36/d73/d6e/df7/d145 0 2024-05-25T06:11:51.863 INFO:tasks.workunit.client.0.smithi160.stdout:4/943: creat d4/d12a/f137 x:0 0 0 2024-05-25T06:11:51.863 INFO:tasks.workunit.client.0.smithi160.stdout:9/853: truncate d4/d11/d14/f31 1408748 0 2024-05-25T06:11:51.864 INFO:tasks.workunit.client.0.smithi160.stdout:8/904: creat d5/d9/d72/d104/f125 x:0 0 0 2024-05-25T06:11:51.867 INFO:tasks.workunit.client.0.smithi160.stdout:6/942: rename d5/d39/d8f/f9d to d5/d39/d112/d119/f146 0 2024-05-25T06:11:51.872 INFO:tasks.workunit.client.0.smithi160.stdout:4/944: rename d4/d6/d28/de4/f56 to d4/d6/d21/d42/f138 0 2024-05-25T06:11:51.874 INFO:tasks.workunit.client.0.smithi160.stdout:1/924: dread de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/f89 [0,4194304] 0 2024-05-25T06:11:51.874 INFO:tasks.workunit.client.0.smithi160.stdout:6/943: fdatasync d5/d45/d4a/f5d 0 2024-05-25T06:11:51.877 INFO:tasks.workunit.client.0.smithi160.stdout:9/854: unlink d4/d11/d7a/f7b 0 2024-05-25T06:11:51.877 INFO:tasks.workunit.client.0.smithi160.stdout:8/905: dwrite d5/d44/d1f/da3/d28/f38 [0,4194304] 0 2024-05-25T06:11:51.880 INFO:tasks.workunit.client.0.smithi160.stdout:4/945: creat d4/d6/d35/d3d/d9e/dea/df5/f139 x:0 0 0 2024-05-25T06:11:51.882 INFO:tasks.workunit.client.0.smithi160.stdout:4/946: write d4/d6/d28/f25 [1501640,69062] 0 2024-05-25T06:11:51.883 INFO:tasks.workunit.client.0.smithi160.stdout:1/925: rename de/d21/d49/d1c/d59/d71/c80 to de/d21/d49/d1c/d59/d7c/c13c 0 2024-05-25T06:11:51.887 INFO:tasks.workunit.client.0.smithi160.stdout:8/906: dwrite d5/d9/d72/d104/f125 [0,4194304] 0 2024-05-25T06:11:51.889 INFO:tasks.workunit.client.0.smithi160.stdout:8/907: chown d5/d44 9 1 2024-05-25T06:11:51.898 INFO:tasks.workunit.client.0.smithi160.stdout:1/926: dwrite de/d39/d8f/fff [0,4194304] 0 2024-05-25T06:11:51.900 INFO:tasks.workunit.client.0.smithi160.stdout:8/908: dread d5/d9/d72/d104/f125 [0,4194304] 0 2024-05-25T06:11:51.905 INFO:tasks.workunit.client.0.smithi160.stdout:1/927: truncate de/d21/d49/d28/fad 899484 0 2024-05-25T06:11:51.905 INFO:tasks.workunit.client.0.smithi160.stdout:8/909: chown d5/d44/d1f/da3/d28/d4a/f5c 15563 1 2024-05-25T06:11:51.912 INFO:tasks.workunit.client.0.smithi160.stdout:9/855: dwrite d4/d11/d12/d2e/f32 [0,4194304] 0 2024-05-25T06:11:51.914 INFO:tasks.workunit.client.0.smithi160.stdout:1/928: chown de/d21/d49/d1c/d41/d45/d7d/lb7 39 1 2024-05-25T06:11:51.917 INFO:tasks.workunit.client.0.smithi160.stdout:4/947: dwrite d4/d6/d28/de4/f79 [0,4194304] 0 2024-05-25T06:11:51.919 INFO:tasks.workunit.client.0.smithi160.stdout:9/856: mkdir d4/d11/d1e/d3e/d25/d27/d95/d111 0 2024-05-25T06:11:51.920 INFO:tasks.workunit.client.0.smithi160.stdout:4/948: truncate d4/d6/fe 301237 0 2024-05-25T06:11:51.921 INFO:tasks.workunit.client.0.smithi160.stdout:8/910: link d5/d7/dcb/d6a/ff0 d5/d44/d1f/da3/f126 0 2024-05-25T06:11:51.924 INFO:tasks.workunit.client.0.smithi160.stdout:8/911: chown d5/d9/d72/daf/dbf/f123 3868 1 2024-05-25T06:11:51.925 INFO:tasks.workunit.client.0.smithi160.stdout:1/929: mkdir de/d21/d49/d1c/d52/dc7/d13d 0 2024-05-25T06:11:51.926 INFO:tasks.workunit.client.0.smithi160.stdout:9/857: truncate d4/d11/d12/d2e/d45/d10f/f9d 862635 0 2024-05-25T06:11:51.927 INFO:tasks.workunit.client.0.smithi160.stdout:1/930: write de/d21/d49/d1c/d59/d7f/da1/f122 [62569,28935] 0 2024-05-25T06:11:51.927 INFO:tasks.workunit.client.0.smithi160.stdout:4/949: unlink d4/d6/d5b/d66/d78/f130 0 2024-05-25T06:11:51.929 INFO:tasks.workunit.client.0.smithi160.stdout:8/912: fsync d5/d44/d1f/d58/d65/da2/fb7 0 2024-05-25T06:11:51.930 INFO:tasks.workunit.client.0.smithi160.stdout:1/931: write de/d21/d49/d1c/d111/f124 [767091,111595] 0 2024-05-25T06:11:51.932 INFO:tasks.workunit.client.0.smithi160.stdout:1/932: dread - de/d21/d49/db1/d106/f135 zero size 2024-05-25T06:11:51.932 INFO:tasks.workunit.client.0.smithi160.stdout:8/913: creat d5/d7/dcb/d6a/f127 x:0 0 0 2024-05-25T06:11:51.934 INFO:tasks.workunit.client.0.smithi160.stdout:4/950: getdents d4/d6/d5b 0 2024-05-25T06:11:51.937 INFO:tasks.workunit.client.0.smithi160.stdout:8/914: creat d5/d44/d1f/da3/d28/d4a/d4e/d86/dd4/f128 x:0 0 0 2024-05-25T06:11:51.938 INFO:tasks.workunit.client.0.smithi160.stdout:4/951: rename d4/dcd to d4/d6/d28/de4/d13a 0 2024-05-25T06:11:51.941 INFO:tasks.workunit.client.0.smithi160.stdout:8/915: mknod d5/d44/dba/c129 0 2024-05-25T06:11:51.944 INFO:tasks.workunit.client.0.smithi160.stdout:1/933: getdents de/d21/d49/d35 0 2024-05-25T06:11:51.944 INFO:tasks.workunit.client.0.smithi160.stdout:4/952: dread d4/d6/d5b/d66/fc4 [0,4194304] 0 2024-05-25T06:11:51.945 INFO:tasks.workunit.client.0.smithi160.stdout:8/916: creat d5/d7/dcb/d75/f12a x:0 0 0 2024-05-25T06:11:51.947 INFO:tasks.workunit.client.0.smithi160.stdout:4/953: unlink d4/d63/d64/d88/feb 0 2024-05-25T06:11:51.948 INFO:tasks.workunit.client.0.smithi160.stdout:9/858: write d4/d11/d12/d2e/d45/d10f/d79/fc3 [503663,107624] 0 2024-05-25T06:11:51.952 INFO:tasks.workunit.client.0.smithi160.stdout:4/954: unlink d4/d1c/d54/dbc/dda/d94/fe8 0 2024-05-25T06:11:51.957 INFO:tasks.workunit.client.0.smithi160.stdout:4/955: rename d4/d6/d35/d3d/d9e/l126 to d4/d6/d28/de4/l13b 0 2024-05-25T06:11:51.959 INFO:tasks.workunit.client.0.smithi160.stdout:4/956: readlink d4/d1c/d54/l5d 0 2024-05-25T06:11:51.960 INFO:tasks.workunit.client.0.smithi160.stdout:9/859: link d4/d11/d1e/d24/d66/cee d4/d11/d12/d2e/d45/d10f/c112 0 2024-05-25T06:11:51.962 INFO:tasks.workunit.client.0.smithi160.stdout:8/917: write d5/d44/d1f/da3/d28/d4a/d4e/d86/f101 [193843,79082] 0 2024-05-25T06:11:51.963 INFO:tasks.workunit.client.0.smithi160.stdout:9/860: creat d4/d11/d12/d58/f113 x:0 0 0 2024-05-25T06:11:51.964 INFO:tasks.workunit.client.0.smithi160.stdout:4/957: mknod d4/d6/d21/d117/c13c 0 2024-05-25T06:11:51.966 INFO:tasks.workunit.client.0.smithi160.stdout:1/934: write de/d21/d49/d1c/d59/d7f/da1/dbd/f7b [60055,107611] 0 2024-05-25T06:11:51.968 INFO:tasks.workunit.client.0.smithi160.stdout:4/958: write d4/d1c/d54/dbc/dda/fe3 [117806,99092] 0 2024-05-25T06:11:51.968 INFO:tasks.workunit.client.0.smithi160.stdout:9/861: mkdir d4/d11/d1e/d114 0 2024-05-25T06:11:51.970 INFO:tasks.workunit.client.0.smithi160.stdout:1/935: read - de/d21/d49/d1c/d59/d71/f131 zero size 2024-05-25T06:11:51.971 INFO:tasks.workunit.client.0.smithi160.stdout:9/862: unlink d4/cab 0 2024-05-25T06:11:51.975 INFO:tasks.workunit.client.0.smithi160.stdout:1/936: write de/d21/d49/d90/f105 [1965320,122983] 0 2024-05-25T06:11:51.975 INFO:tasks.workunit.client.0.smithi160.stdout:4/959: creat d4/d6/d35/d3d/d9e/dea/df5/f13d x:0 0 0 2024-05-25T06:11:51.975 INFO:tasks.workunit.client.0.smithi160.stdout:8/918: dwrite d5/d44/d1f/da3/d28/d49/d61/f11c [0,4194304] 0 2024-05-25T06:11:51.977 INFO:tasks.workunit.client.0.smithi160.stdout:4/960: write d4/d1c/d54/d9b/f10d [2906925,80540] 0 2024-05-25T06:11:51.983 INFO:tasks.workunit.client.0.smithi160.stdout:4/961: fsync d4/d6/d35/d3d/fb8 0 2024-05-25T06:11:51.984 INFO:tasks.workunit.client.0.smithi160.stdout:9/863: getdents d4 0 2024-05-25T06:11:51.986 INFO:tasks.workunit.client.0.smithi160.stdout:8/919: truncate d5/d7/dcb/d7a/fcd 1414323 0 2024-05-25T06:11:51.987 INFO:tasks.workunit.client.0.smithi160.stdout:1/937: link de/d21/d49/d1c/fc9 de/d21/d49/d28/f13e 0 2024-05-25T06:11:51.989 INFO:tasks.workunit.client.0.smithi160.stdout:4/962: creat d4/d6/d5b/d66/d78/da1/dab/f13e x:0 0 0 2024-05-25T06:11:51.989 INFO:tasks.workunit.client.0.smithi160.stdout:4/963: fdatasync d4/f76 0 2024-05-25T06:11:51.993 INFO:tasks.workunit.client.0.smithi160.stdout:1/938: rename de/d21/d49/l6b to de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/l13f 0 2024-05-25T06:11:51.997 INFO:tasks.workunit.client.0.smithi160.stdout:1/939: write de/d21/d49/d1c/d59/d71/f5c [1933963,103554] 0 2024-05-25T06:11:51.997 INFO:tasks.workunit.client.0.smithi160.stdout:4/964: dwrite d4/d1c/d54/f65 [0,4194304] 0 2024-05-25T06:11:51.997 INFO:tasks.workunit.client.0.smithi160.stdout:8/920: truncate d5/d7/dcb/d75/feb 61171 0 2024-05-25T06:11:52.002 INFO:tasks.workunit.client.0.smithi160.stdout:1/940: mknod de/d21/d49/c140 0 2024-05-25T06:11:52.002 INFO:tasks.workunit.client.0.smithi160.stdout:9/864: dread d4/d11/d1e/d3e/d25/f50 [0,4194304] 0 2024-05-25T06:11:52.003 INFO:tasks.workunit.client.0.smithi160.stdout:1/941: chown de/d21/d49/d1c/d52/dc7/ddb/d126 384688 1 2024-05-25T06:11:52.003 INFO:tasks.workunit.client.0.smithi160.stdout:9/865: chown d4/d11/d1e/d3e/d25/d5e/da2 3142944 1 2024-05-25T06:11:52.004 INFO:tasks.workunit.client.0.smithi160.stdout:9/866: chown d4/d11/fcf 151799495 1 2024-05-25T06:11:52.006 INFO:tasks.workunit.client.0.smithi160.stdout:4/965: rename d4/d63 to d4/d6/d5b/d66/d78/da1/d13f 0 2024-05-25T06:11:52.010 INFO:tasks.workunit.client.0.smithi160.stdout:1/942: mkdir de/d21/d49/d1c/d41/d45/d141 0 2024-05-25T06:11:52.012 INFO:tasks.workunit.client.0.smithi160.stdout:4/966: truncate d4/d6/f7 1605386 0 2024-05-25T06:11:52.013 INFO:tasks.workunit.client.0.smithi160.stdout:9/867: truncate d4/d11/d1e/d3e/f74 3078617 0 2024-05-25T06:11:52.016 INFO:tasks.workunit.client.0.smithi160.stdout:8/921: truncate d5/d44/d1f/da3/d28/f3f 3217863 0 2024-05-25T06:11:52.017 INFO:tasks.workunit.client.0.smithi160.stdout:9/868: symlink d4/db2/de5/l115 0 2024-05-25T06:11:52.019 INFO:tasks.workunit.client.0.smithi160.stdout:4/967: link d4/d6/f107 d4/d6/d21/d117/f140 0 2024-05-25T06:11:52.020 INFO:tasks.workunit.client.0.smithi160.stdout:4/968: fsync d4/d6/d21/f27 0 2024-05-25T06:11:52.020 INFO:tasks.workunit.client.0.smithi160.stdout:1/943: getdents de/d21/d49/d1c/d59/d7f/da1/dbd 0 2024-05-25T06:11:52.022 INFO:tasks.workunit.client.0.smithi160.stdout:1/944: readlink de/d21/d49/d1c/d52/dc7/le0 0 2024-05-25T06:11:52.025 INFO:tasks.workunit.client.0.smithi160.stdout:4/969: creat d4/d6/d5b/d66/d78/da1/dab/dfa/f141 x:0 0 0 2024-05-25T06:11:52.026 INFO:tasks.workunit.client.0.smithi160.stdout:1/945: symlink de/d21/d49/d1c/d41/d45/d7d/l142 0 2024-05-25T06:11:52.028 INFO:tasks.workunit.client.0.smithi160.stdout:4/970: mknod d4/d6/dbd/df0/c142 0 2024-05-25T06:11:52.029 INFO:tasks.workunit.client.0.smithi160.stdout:4/971: chown d4/d6/d21/f3f 506590 1 2024-05-25T06:11:52.029 INFO:tasks.workunit.client.0.smithi160.stdout:8/922: getdents d5/d44/d1f/da3/d28 0 2024-05-25T06:11:52.032 INFO:tasks.workunit.client.0.smithi160.stdout:9/869: dread d4/fc2 [0,4194304] 0 2024-05-25T06:11:52.035 INFO:tasks.workunit.client.0.smithi160.stdout:8/923: mkdir d5/d44/d1f/d58/d65/d12b 0 2024-05-25T06:11:52.037 INFO:tasks.workunit.client.0.smithi160.stdout:1/946: link de/d21/d49/d1c/d59/d67/c79 de/d21/d49/d90/c143 0 2024-05-25T06:11:52.038 INFO:tasks.workunit.client.0.smithi160.stdout:9/870: mknod d4/d11/d1e/d24/c116 0 2024-05-25T06:11:52.039 INFO:tasks.workunit.client.0.smithi160.stdout:4/972: rmdir d4/d6/d35/d3d/d127 0 2024-05-25T06:11:52.041 INFO:tasks.workunit.client.0.smithi160.stdout:1/947: symlink de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/l144 0 2024-05-25T06:11:52.041 INFO:tasks.workunit.client.0.smithi160.stdout:9/871: chown d4/d11/c19 1832 1 2024-05-25T06:11:52.042 INFO:tasks.workunit.client.0.smithi160.stdout:4/973: symlink d4/d6/d21/d42/dae/l143 0 2024-05-25T06:11:52.059 INFO:tasks.workunit.client.0.smithi160.stdout:9/872: symlink d4/d11/d12/d2e/d45/l117 0 2024-05-25T06:11:52.062 INFO:tasks.workunit.client.0.smithi160.stdout:1/948: rename de/d21/d49/ff2 to de/d21/d49/d1c/d41/f145 0 2024-05-25T06:11:52.062 INFO:tasks.workunit.client.0.smithi160.stdout:4/974: symlink d4/d6/d28/de4/l144 0 2024-05-25T06:11:52.062 INFO:tasks.workunit.client.0.smithi160.stdout:1/949: write de/d21/d49/d90/feb [3148472,48668] 0 2024-05-25T06:11:52.066 INFO:tasks.workunit.client.0.smithi160.stdout:4/975: rename d4/d1a/f59 to d4/d6/d35/d3d/d9e/dea/f145 0 2024-05-25T06:11:52.070 INFO:tasks.workunit.client.0.smithi160.stdout:1/950: rename de/d21/d49/d1c/d52/dc7/ddb/d126 to de/d21/d49/d1c/d41/d45/d141/d146 0 2024-05-25T06:11:52.071 INFO:tasks.workunit.client.0.smithi160.stdout:4/976: creat d4/d6/d28/de4/d13a/f146 x:0 0 0 2024-05-25T06:11:52.073 INFO:tasks.workunit.client.0.smithi160.stdout:1/951: rmdir de 39 2024-05-25T06:11:52.074 INFO:tasks.workunit.client.0.smithi160.stdout:4/977: symlink d4/d6/d5b/d66/d78/da1/dab/dfa/l147 0 2024-05-25T06:11:52.077 INFO:tasks.workunit.client.0.smithi160.stdout:4/978: chown d4/d1c/d54/dbc/dda/le1 239 1 2024-05-25T06:11:52.077 INFO:tasks.workunit.client.0.smithi160.stdout:4/979: write d4/d6/d35/d3d/d9e/fc2 [3575124,129847] 0 2024-05-25T06:11:52.080 INFO:tasks.workunit.client.0.smithi160.stdout:1/952: rename de/d21/d49/d1c/d41/d45/dac/dd3 to de/d21/d49/d1c/d41/d45/d7d/d147 0 2024-05-25T06:11:52.081 INFO:tasks.workunit.client.0.smithi160.stdout:9/873: write d4/d11/d1e/d3e/d25/d5e/da2/fd0 [341111,84306] 0 2024-05-25T06:11:52.082 INFO:tasks.workunit.client.0.smithi160.stdout:4/980: symlink d4/d6/d21/d42/dae/l148 0 2024-05-25T06:11:52.083 INFO:tasks.workunit.client.0.smithi160.stdout:1/953: creat de/d21/d49/d1c/d59/d7c/d93/f148 x:0 0 0 2024-05-25T06:11:52.083 INFO:tasks.workunit.client.0.smithi160.stdout:4/981: chown d4/d6/d35/d3d/fb8 31004097 1 2024-05-25T06:11:52.084 INFO:tasks.workunit.client.0.smithi160.stdout:9/874: getdents d4/d11/d1e/d3e/d25/d27/dc5/d100 0 2024-05-25T06:11:52.084 INFO:tasks.workunit.client.0.smithi160.stdout:1/954: symlink de/d21/d49/d1c/d52/l149 0 2024-05-25T06:11:52.085 INFO:tasks.workunit.client.0.smithi160.stdout:4/982: mknod d4/d6/d5b/d66/d78/da1/dab/dfa/c149 0 2024-05-25T06:11:52.086 INFO:tasks.workunit.client.0.smithi160.stdout:4/983: dread - d4/d1c/ff9 zero size 2024-05-25T06:11:52.087 INFO:tasks.workunit.client.0.smithi160.stdout:1/955: truncate de/d21/d49/d1c/d59/f62 3870557 0 2024-05-25T06:11:52.088 INFO:tasks.workunit.client.0.smithi160.stdout:1/956: write de/d21/d49/d1c/d52/fd4 [70629,65674] 0 2024-05-25T06:11:52.090 INFO:tasks.workunit.client.0.smithi160.stdout:1/957: write de/d21/d49/d1c/d41/f145 [739336,75763] 0 2024-05-25T06:11:52.105 INFO:tasks.workunit.client.0.smithi160.stdout:4/984: dread d4/d6/d21/d42/f138 [0,4194304] 0 2024-05-25T06:11:52.107 INFO:tasks.workunit.client.0.smithi160.stdout:4/985: symlink d4/d1a/df1/l14a 0 2024-05-25T06:11:52.109 INFO:tasks.workunit.client.0.smithi160.stdout:4/986: dread - d4/d6/d28/de4/f125 zero size 2024-05-25T06:11:52.109 INFO:tasks.workunit.client.0.smithi160.stdout:1/958: dread de/d21/d49/d1c/d59/d71/f74 [0,4194304] 0 2024-05-25T06:11:52.111 INFO:tasks.workunit.client.0.smithi160.stdout:4/987: creat d4/d6/d21/f14b x:0 0 0 2024-05-25T06:11:52.111 INFO:tasks.workunit.client.0.smithi160.stdout:1/959: mknod de/d21/d49/d1c/d59/d71/dfe/c14a 0 2024-05-25T06:11:52.113 INFO:tasks.workunit.client.0.smithi160.stdout:4/988: truncate d4/d6/d5b/d66/d78/da1/f123 677228 0 2024-05-25T06:11:52.115 INFO:tasks.workunit.client.0.smithi160.stdout:4/989: write d4/d1c/d54/f65 [3718609,90862] 0 2024-05-25T06:11:52.117 INFO:tasks.workunit.client.0.smithi160.stdout:1/960: link de/d21/d49/d1c/d41/d45/d7d/fdd de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/db5/f14b 0 2024-05-25T06:11:52.119 INFO:tasks.workunit.client.0.smithi160.stdout:4/990: link d4/d6/l16 d4/d6/d28/de4/d13a/l14c 0 2024-05-25T06:11:52.133 INFO:tasks.workunit.client.0.smithi160.stdout:1/961: dread de/d21/d49/d1c/d59/d67/fb9 [0,4194304] 0 2024-05-25T06:11:52.136 INFO:tasks.workunit.client.0.smithi160.stdout:4/991: dwrite d4/d6/d5b/d66/d78/da1/d13f/f81 [0,4194304] 0 2024-05-25T06:11:52.137 INFO:tasks.workunit.client.0.smithi160.stdout:4/992: chown d4/d1c/d54/d9b/fcc 16768043 1 2024-05-25T06:11:52.137 INFO:tasks.workunit.client.0.smithi160.stdout:4/993: chown d4/d6/d28/de4/fa9 142245 1 2024-05-25T06:11:52.137 INFO:tasks.workunit.client.0.smithi160.stdout:1/962: mkdir de/d21/d49/d1c/d41/d96/d14c 0 2024-05-25T06:11:52.138 INFO:tasks.workunit.client.0.smithi160.stdout:4/994: write d4/d6/dbd/f11e [969364,71243] 0 2024-05-25T06:11:52.139 INFO:tasks.workunit.client.0.smithi160.stdout:4/995: stat d4/d6/d28/de4/l96 0 2024-05-25T06:11:52.140 INFO:tasks.workunit.client.0.smithi160.stdout:1/963: mknod de/d21/d49/d1c/d59/d7c/d93/de8/c14d 0 2024-05-25T06:11:52.141 INFO:tasks.workunit.client.0.smithi160.stdout:4/996: write d4/d1a/df1/ffe [546162,111528] 0 2024-05-25T06:11:52.144 INFO:tasks.workunit.client.0.smithi160.stdout:4/997: dread d4/d1c/d54/dbc/dda/fe3 [0,4194304] 0 2024-05-25T06:11:52.147 INFO:tasks.workunit.client.0.smithi160.stdout:4/998: symlink d4/d1c/d54/dbc/dda/d94/dec/l14d 0 2024-05-25T06:11:52.150 INFO:tasks.workunit.client.0.smithi160.stdout:4/999: mknod d4/d6/d5b/d66/d78/da1/d13f/d64/d88/d119/c14e 0 2024-05-25T06:11:52.162 INFO:tasks.workunit.client.0.smithi160.stdout:1/964: truncate de/d21/fa6 8030087 0 2024-05-25T06:11:52.172 INFO:tasks.workunit.client.0.smithi160.stdout:1/965: dwrite de/d21/d49/d1c/f9d [0,4194304] 0 2024-05-25T06:11:52.176 INFO:tasks.workunit.client.0.smithi160.stdout:1/966: getdents de/d21/d49/d35 0 2024-05-25T06:11:52.177 INFO:tasks.workunit.client.0.smithi160.stdout:1/967: chown de/d21/d49/dc6/fcf 1972577 1 2024-05-25T06:11:52.178 INFO:tasks.workunit.client.0.smithi160.stdout:1/968: rmdir de/d21/d49/d35 39 2024-05-25T06:11:52.182 INFO:tasks.workunit.client.0.smithi160.stdout:1/969: mknod de/d21/d49/d1c/d59/d7f/da1/dbd/dd0/c14e 0 2024-05-25T06:11:52.184 INFO:tasks.workunit.client.0.smithi160.stdout:1/970: mkdir de/d21/d49/d1c/d41/d45/d141/d146/d14f 0 2024-05-25T06:11:52.187 INFO:tasks.workunit.client.0.smithi160.stdout:1/971: dread de/d21/d49/d1c/d41/d96/fa8 [0,4194304] 0 2024-05-25T06:11:52.190 INFO:tasks.workunit.client.0.smithi160.stdout:1/972: mknod de/d21/d49/d1c/d59/d7f/da1/c150 0 2024-05-25T06:11:52.191 INFO:tasks.workunit.client.0.smithi160.stdout:1/973: write de/d21/d49/d1c/d59/d67/f83 [758904,129444] 0 2024-05-25T06:11:52.195 INFO:tasks.workunit.client.0.smithi160.stdout:1/974: rename de/d21/d49/d1c/d59/d71/f74 to de/d21/d49/d1c/f151 0 2024-05-25T06:11:52.197 INFO:tasks.workunit.client.0.smithi160.stdout:1/975: truncate de/d21/d49/fa2 1256229 0 2024-05-25T06:11:52.198 INFO:tasks.workunit.client.0.smithi160.stdout:1/976: write de/d21/d49/f6f [588429,112324] 0 2024-05-25T06:11:52.200 INFO:tasks.workunit.client.0.smithi160.stdout:1/977: mknod de/d21/d49/dc6/c152 0 2024-05-25T06:11:52.202 INFO:tasks.workunit.client.0.smithi160.stdout:1/978: mknod de/d21/d49/d1c/d41/c153 0 2024-05-25T06:11:52.215 INFO:tasks.workunit.client.0.smithi160.stdout:1/979: dwrite de/d21/d49/d1c/d41/d45/d141/d146/f129 [0,4194304] 0 2024-05-25T06:11:52.218 INFO:tasks.workunit.client.0.smithi160.stdout:1/980: unlink de/d21/d49/d28/c2a 0 2024-05-25T06:11:52.225 INFO:tasks.workunit.client.0.smithi160.stdout:1/981: dwrite de/d21/f115 [4194304,4194304] 0 2024-05-25T06:11:52.239 INFO:tasks.workunit.client.0.smithi160.stdout:1/982: dwrite de/d21/d49/d1c/d59/d71/fc0 [0,4194304] 0 2024-05-25T06:11:52.244 INFO:tasks.workunit.client.0.smithi160.stdout:1/983: rename de/d21/d49/d1c/d59/cd1 to de/d21/d49/d1c/c154 0 2024-05-25T06:11:52.247 INFO:tasks.workunit.client.0.smithi160.stdout:1/984: fsync de/d21/f9a 0 2024-05-25T06:11:52.250 INFO:tasks.workunit.client.0.smithi160.stdout:1/985: fsync de/d21/d49/d28/f3c 0 2024-05-25T06:11:52.255 INFO:tasks.workunit.client.0.smithi160.stdout:6/944: sync 2024-05-25T06:11:52.256 INFO:tasks.workunit.client.0.smithi160.stdout:1/986: dwrite de/f15 [4194304,4194304] 0 2024-05-25T06:11:52.257 INFO:tasks.workunit.client.0.smithi160.stdout:6/945: mknod d5/d39/d112/da4/c147 0 2024-05-25T06:11:52.260 INFO:tasks.workunit.client.0.smithi160.stdout:1/987: unlink de/d21/d49/d1c/d41/d45/d141/d146/f129 0 2024-05-25T06:11:52.264 INFO:tasks.workunit.client.0.smithi160.stdout:1/988: symlink de/d21/d49/d1c/d59/d71/dfe/l155 0 2024-05-25T06:11:52.264 INFO:tasks.workunit.client.0.smithi160.stdout:6/946: dread d5/d30/d36/d73/f64 [0,4194304] 0 2024-05-25T06:11:52.267 INFO:tasks.workunit.client.0.smithi160.stdout:6/947: creat d5/de/d32/d35/d12e/d98/f148 x:0 0 0 2024-05-25T06:11:52.268 INFO:tasks.workunit.client.0.smithi160.stdout:6/948: stat d5/d30/d36/d73/d6e/df7/d145/lef 0 2024-05-25T06:11:52.270 INFO:tasks.workunit.client.0.smithi160.stdout:6/949: unlink d5/d45/l52 0 2024-05-25T06:11:52.273 INFO:tasks.workunit.client.0.smithi160.stdout:6/950: creat d5/de/d32/d35/dc3/f149 x:0 0 0 2024-05-25T06:11:52.275 INFO:tasks.workunit.client.0.smithi160.stdout:6/951: symlink d5/d45/d4a/dbc/dc4/l14a 0 2024-05-25T06:11:52.279 INFO:tasks.workunit.client.0.smithi160.stdout:6/952: rename d5/d45/d4a/dbc/dc4/l14a to d5/d126/l14b 0 2024-05-25T06:11:52.284 INFO:tasks.workunit.client.0.smithi160.stdout:1/989: dread de/d21/d49/d35/f6e [0,4194304] 0 2024-05-25T06:11:52.291 INFO:tasks.workunit.client.0.smithi160.stdout:6/953: dwrite d5/de/f17 [16777216,4194304] 0 2024-05-25T06:11:52.296 INFO:tasks.workunit.client.0.smithi160.stdout:6/954: creat d5/d39/da6/dac/f14c x:0 0 0 2024-05-25T06:11:52.300 INFO:tasks.workunit.client.0.smithi160.stdout:6/955: rename d5/d39/d8f/c131 to d5/de/d32/d35/dc3/c14d 0 2024-05-25T06:11:52.302 INFO:tasks.workunit.client.0.smithi160.stdout:6/956: write d5/d39/d8f/d2b/f121 [987914,104773] 0 2024-05-25T06:11:52.305 INFO:tasks.workunit.client.0.smithi160.stdout:6/957: rename d5/d39/dc1/d113/d13a to d5/d30/d36/d14e 0 2024-05-25T06:11:52.307 INFO:tasks.workunit.client.0.smithi160.stdout:6/958: mknod d5/d30/c14f 0 2024-05-25T06:11:52.310 INFO:tasks.workunit.client.0.smithi160.stdout:6/959: creat d5/de/d7d/f150 x:0 0 0 2024-05-25T06:11:52.312 INFO:tasks.workunit.client.0.smithi160.stdout:6/960: creat d5/de/d32/d9f/f151 x:0 0 0 2024-05-25T06:11:52.321 INFO:tasks.workunit.client.0.smithi160.stdout:6/961: rmdir d5/d30/d36 39 2024-05-25T06:11:52.325 INFO:tasks.workunit.client.0.smithi160.stdout:6/962: mkdir d5/d39/da6/dac/de6/d152 0 2024-05-25T06:11:52.328 INFO:tasks.workunit.client.0.smithi160.stdout:6/963: dread d5/d45/fe2 [0,4194304] 0 2024-05-25T06:11:52.336 INFO:tasks.workunit.client.0.smithi160.stdout:6/964: dwrite d5/d39/da6/dac/f14c [0,4194304] 0 2024-05-25T06:11:52.339 INFO:tasks.workunit.client.0.smithi160.stdout:6/965: creat d5/de/d32/d74/d77/f153 x:0 0 0 2024-05-25T06:11:52.341 INFO:tasks.workunit.client.0.smithi160.stdout:6/966: dread - d5/de/d32/d9f/dd5/f132 zero size 2024-05-25T06:11:52.344 INFO:tasks.workunit.client.0.smithi160.stdout:6/967: mkdir d5/d45/d4a/d154 0 2024-05-25T06:11:52.374 INFO:tasks.workunit.client.0.smithi160.stdout:6/968: dread d5/de/d32/fd7 [0,4194304] 0 2024-05-25T06:11:52.375 INFO:tasks.workunit.client.0.smithi160.stdout:6/969: chown d5/d45/fe2 0 1 2024-05-25T06:11:52.378 INFO:tasks.workunit.client.0.smithi160.stdout:6/970: symlink d5/de/d32/d74/d77/d89/l155 0 2024-05-25T06:11:52.385 INFO:tasks.workunit.client.0.smithi160.stdout:6/971: dwrite d5/d39/da6/f10c [0,4194304] 0 2024-05-25T06:11:52.387 INFO:tasks.workunit.client.0.smithi160.stdout:6/972: write d5/de/d32/fd7 [1522062,38662] 0 2024-05-25T06:11:52.390 INFO:tasks.workunit.client.0.smithi160.stdout:6/973: mknod d5/d39/da6/dac/de6/c156 0 2024-05-25T06:11:52.463 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:52 smithi160 ceph-mon[29041]: pgmap v26: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 107 MiB/s rd, 243 MiB/s wr, 552 op/s 2024-05-25T06:11:52.577 INFO:tasks.workunit.client.0.smithi160.stdout:8/924: sync 2024-05-25T06:11:52.578 INFO:tasks.workunit.client.0.smithi160.stdout:1/990: sync 2024-05-25T06:11:52.578 INFO:tasks.workunit.client.0.smithi160.stdout:9/875: sync 2024-05-25T06:11:52.580 INFO:tasks.workunit.client.0.smithi160.stdout:1/991: write de/d21/d49/d1c/d59/d7c/f12d [463080,44093] 0 2024-05-25T06:11:52.581 INFO:tasks.workunit.client.0.smithi160.stdout:1/992: readlink de/d21/d49/d1c/l6a 0 2024-05-25T06:11:52.581 INFO:tasks.workunit.client.0.smithi160.stdout:8/925: truncate d5/d44/d1f/fb6 4182619 0 2024-05-25T06:11:52.584 INFO:tasks.workunit.client.0.smithi160.stdout:9/876: rename d4/d11/d12/d2e/d45/d83/fdd to d4/d11/d1e/d24/df7/f118 0 2024-05-25T06:11:52.584 INFO:tasks.workunit.client.0.smithi160.stdout:1/993: truncate de/d21/d49/d1c/d59/f95 2848229 0 2024-05-25T06:11:52.585 INFO:tasks.workunit.client.0.smithi160.stdout:9/877: mkdir d4/db2/d119 0 2024-05-25T06:11:52.587 INFO:tasks.workunit.client.0.smithi160.stdout:8/926: link d5/d44/d1f/da3/d28/f38 d5/d44/d1f/da3/d28/d4a/d93/f12c 0 2024-05-25T06:11:52.590 INFO:tasks.workunit.client.0.smithi160.stdout:8/927: creat d5/d7/d9a/f12d x:0 0 0 2024-05-25T06:11:52.591 INFO:tasks.workunit.client.0.smithi160.stdout:8/928: mknod d5/d9/d72/c12e 0 2024-05-25T06:11:52.597 INFO:tasks.workunit.client.0.smithi160.stdout:8/929: rename d5/d44/d1f/da3/d28/d49/c4c to d5/d44/d1f/da3/d28/dc6/de8/c12f 0 2024-05-25T06:11:52.598 INFO:tasks.workunit.client.0.smithi160.stdout:8/930: chown d5/d9/d72/l83 0 1 2024-05-25T06:11:52.604 INFO:tasks.workunit.client.0.smithi160.stdout:1/994: dwrite de/d21/f9a [0,4194304] 0 2024-05-25T06:11:52.605 INFO:tasks.workunit.client.0.smithi160.stdout:9/878: dwrite d4/d11/d1e/fac [0,4194304] 0 2024-05-25T06:11:52.605 INFO:tasks.workunit.client.0.smithi160.stdout:8/931: write d5/d7/dcb/d75/f76 [483150,66729] 0 2024-05-25T06:11:52.611 INFO:tasks.workunit.client.0.smithi160.stdout:1/995: fdatasync de/d21/d49/d1c/d41/fe6 0 2024-05-25T06:11:52.616 INFO:tasks.workunit.client.0.smithi160.stdout:1/996: creat de/d21/d49/d1c/d59/d7f/da1/dbd/f156 x:0 0 0 2024-05-25T06:11:52.617 INFO:tasks.workunit.client.0.smithi160.stdout:9/879: truncate d4/d11/d12/d2e/d10b/fba 2645368 0 2024-05-25T06:11:52.619 INFO:tasks.workunit.client.0.smithi160.stdout:9/880: readlink d4/d11/l21 0 2024-05-25T06:11:52.630 INFO:tasks.workunit.client.0.smithi160.stdout:8/932: dwrite d5/d7/d9a/fdd [0,4194304] 0 2024-05-25T06:11:52.631 INFO:tasks.workunit.client.0.smithi160.stdout:8/933: readlink d5/d7/dcb/d6a/l112 0 2024-05-25T06:11:52.634 INFO:tasks.workunit.client.0.smithi160.stdout:8/934: chown d5/f82 225708 1 2024-05-25T06:11:52.637 INFO:tasks.workunit.client.0.smithi160.stdout:8/935: rmdir d5/d44/dba/ddb/de3 39 2024-05-25T06:11:52.637 INFO:tasks.workunit.client.0.smithi160.stdout:9/881: write d4/d11/d1e/d3e/f74 [4120978,81584] 0 2024-05-25T06:11:52.639 INFO:tasks.workunit.client.0.smithi160.stdout:8/936: chown d5/d44/d1f/da3/d28/d4a/d4e/c5a 790625 1 2024-05-25T06:11:52.642 INFO:tasks.workunit.client.0.smithi160.stdout:8/937: mkdir d5/d44/d1f/da3/d28/d130 0 2024-05-25T06:11:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:52 smithi203 ceph-mon[36030]: pgmap v26: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 107 MiB/s rd, 243 MiB/s wr, 552 op/s 2024-05-25T06:11:52.656 INFO:tasks.workunit.client.0.smithi160.stdout:9/882: dread d4/d11/d1e/d24/fc [0,4194304] 0 2024-05-25T06:11:52.657 INFO:tasks.workunit.client.0.smithi160.stdout:8/938: dwrite d5/d44/d1f/da3/d28/d4a/d93/fab [0,4194304] 0 2024-05-25T06:11:52.661 INFO:tasks.workunit.client.0.smithi160.stdout:8/939: read d5/d9/fbb [1767136,38692] 0 2024-05-25T06:11:52.668 INFO:tasks.workunit.client.0.smithi160.stdout:9/883: dwrite d4/d11/d1e/d3e/d25/d5e/da2/fb0 [0,4194304] 0 2024-05-25T06:11:52.671 INFO:tasks.workunit.client.0.smithi160.stdout:9/884: unlink d4/d11/d1e/c49 0 2024-05-25T06:11:52.674 INFO:tasks.workunit.client.0.smithi160.stdout:9/885: mknod d4/d11/d12/d2e/d10b/c11a 0 2024-05-25T06:11:52.676 INFO:tasks.workunit.client.0.smithi160.stdout:9/886: creat d4/d11/d7a/f11b x:0 0 0 2024-05-25T06:11:52.678 INFO:tasks.workunit.client.0.smithi160.stdout:9/887: creat d4/db2/de5/f11c x:0 0 0 2024-05-25T06:11:52.681 INFO:tasks.workunit.client.0.smithi160.stdout:9/888: creat d4/d11/d1e/d24/df7/f11d x:0 0 0 2024-05-25T06:11:52.683 INFO:tasks.workunit.client.0.smithi160.stdout:9/889: mkdir d4/d11/d12/d2e/d45/d10f/da7/d11e 0 2024-05-25T06:11:52.684 INFO:tasks.workunit.client.0.smithi160.stdout:9/890: fsync d4/d11/d1e/d3e/d25/d5e/fe4 0 2024-05-25T06:11:52.685 INFO:tasks.workunit.client.0.smithi160.stdout:9/891: rmdir d4/d11/d1e/d6c 39 2024-05-25T06:11:52.696 INFO:tasks.workunit.client.0.smithi160.stdout:9/892: dwrite d4/d11/d1e/d3e/d25/d27/f77 [0,4194304] 0 2024-05-25T06:11:52.700 INFO:tasks.workunit.client.0.smithi160.stdout:9/893: rename d4/d11/f38 to d4/d11/d12/d2e/d45/d10f/d79/f11f 0 2024-05-25T06:11:52.704 INFO:tasks.workunit.client.0.smithi160.stdout:9/894: creat d4/d11/d1e/d3e/d25/d27/dc5/d100/f120 x:0 0 0 2024-05-25T06:11:52.708 INFO:tasks.workunit.client.0.smithi160.stdout:9/895: rename d4/d11/d12/d2e/d45/d10f/d79/ff3 to d4/d11/d12/f121 0 2024-05-25T06:11:52.711 INFO:tasks.workunit.client.0.smithi160.stdout:9/896: mkdir d4/d11/d1e/d3e/d25/d5e/d122 0 2024-05-25T06:11:52.716 INFO:tasks.workunit.client.0.smithi160.stdout:9/897: dwrite d4/d11/d1e/d24/fd1 [0,4194304] 0 2024-05-25T06:11:52.719 INFO:tasks.workunit.client.0.smithi160.stdout:9/898: truncate d4/d11/d1e/d3e/d25/d5e/ffd 771482 0 2024-05-25T06:11:52.723 INFO:tasks.workunit.client.0.smithi160.stdout:9/899: creat d4/d11/d1e/d3e/d25/d5e/d122/f123 x:0 0 0 2024-05-25T06:11:52.723 INFO:tasks.workunit.client.0.smithi160.stdout:9/900: chown d4/c17 24220 1 2024-05-25T06:11:52.724 INFO:tasks.workunit.client.0.smithi160.stdout:9/901: write d4/d11/d1e/d24/d66/f94 [5014503,106458] 0 2024-05-25T06:11:52.727 INFO:tasks.workunit.client.0.smithi160.stdout:9/902: creat d4/d11/d1e/d114/f124 x:0 0 0 2024-05-25T06:11:52.735 INFO:tasks.workunit.client.0.smithi160.stdout:9/903: rename d4/db2/de5/f11c to d4/d11/d1e/d6c/f125 0 2024-05-25T06:11:52.764 INFO:tasks.workunit.client.0.smithi160.stdout:9/904: write d4/d11/d1e/d24/df7/fad [127071,59587] 0 2024-05-25T06:11:52.766 INFO:tasks.workunit.client.0.smithi160.stdout:9/905: mknod d4/db2/de5/c126 0 2024-05-25T06:11:52.768 INFO:tasks.workunit.client.0.smithi160.stdout:9/906: mknod d4/d11/d1e/d24/df7/c127 0 2024-05-25T06:11:52.769 INFO:tasks.workunit.client.0.smithi160.stdout:9/907: chown d4/d11/d1e/d24/df7/l110 241957 1 2024-05-25T06:11:52.784 INFO:tasks.workunit.client.0.smithi160.stdout:9/908: dwrite d4/f5c [0,4194304] 0 2024-05-25T06:11:52.791 INFO:tasks.workunit.client.0.smithi160.stdout:9/909: dwrite d4/d11/d1e/dbc/df1/f104 [0,4194304] 0 2024-05-25T06:11:52.794 INFO:tasks.workunit.client.0.smithi160.stdout:9/910: mknod d4/d11/d1e/d3e/d25/d5e/da2/c128 0 2024-05-25T06:11:52.794 INFO:tasks.workunit.client.0.smithi160.stdout:9/911: stat d4/d11/f26 0 2024-05-25T06:11:52.796 INFO:tasks.workunit.client.0.smithi160.stdout:9/912: readlink d4/d11/d1e/d3e/d25/lcb 0 2024-05-25T06:11:52.803 INFO:tasks.workunit.client.0.smithi160.stdout:9/913: write d4/d11/d12/d2e/d45/d10f/f9d [1225341,80677] 0 2024-05-25T06:11:52.816 INFO:tasks.workunit.client.0.smithi160.stdout:9/914: dwrite d4/d11/d12/d58/f64 [0,4194304] 0 2024-05-25T06:11:52.820 INFO:tasks.workunit.client.0.smithi160.stdout:9/915: link d4/d11/d14/f3a d4/d11/d12/d2e/d45/d10f/f129 0 2024-05-25T06:11:52.823 INFO:tasks.workunit.client.0.smithi160.stdout:9/916: mknod d4/d11/d1e/d3e/d25/d27/c12a 0 2024-05-25T06:11:52.825 INFO:tasks.workunit.client.0.smithi160.stdout:9/917: creat d4/d11/d1e/dbc/f12b x:0 0 0 2024-05-25T06:11:52.827 INFO:tasks.workunit.client.0.smithi160.stdout:9/918: creat d4/d11/d1e/d6c/f12c x:0 0 0 2024-05-25T06:11:52.828 INFO:tasks.workunit.client.0.smithi160.stdout:9/919: chown d4/d11/fe9 192 1 2024-05-25T06:11:52.831 INFO:tasks.workunit.client.0.smithi160.stdout:9/920: getdents d4/d11/d1e/d24 0 2024-05-25T06:11:52.832 INFO:tasks.workunit.client.0.smithi160.stdout:9/921: mknod d4/d11/d1e/d114/c12d 0 2024-05-25T06:11:52.834 INFO:tasks.workunit.client.0.smithi160.stdout:9/922: dread - d4/d11/d1e/d3e/ff4 zero size 2024-05-25T06:11:52.835 INFO:tasks.workunit.client.0.smithi160.stdout:9/923: mkdir d4/db2/d12e 0 2024-05-25T06:11:52.838 INFO:tasks.workunit.client.0.smithi160.stdout:9/924: read d4/f69 [615013,33961] 0 2024-05-25T06:11:52.840 INFO:tasks.workunit.client.0.smithi160.stdout:9/925: symlink d4/d11/d1e/d3e/d25/d5e/da2/l12f 0 2024-05-25T06:11:52.842 INFO:tasks.workunit.client.0.smithi160.stdout:9/926: symlink d4/d11/d1e/dbc/df1/l130 0 2024-05-25T06:11:52.843 INFO:tasks.workunit.client.0.smithi160.stdout:6/974: sync 2024-05-25T06:11:52.844 INFO:tasks.workunit.client.0.smithi160.stdout:9/927: mkdir d4/d11/d12/d2e/d45/d83/d131 0 2024-05-25T06:11:52.846 INFO:tasks.workunit.client.0.smithi160.stdout:9/928: creat d4/d11/d1e/d3e/d25/d5e/d122/f132 x:0 0 0 2024-05-25T06:11:52.847 INFO:tasks.workunit.client.0.smithi160.stdout:6/975: dread d5/d45/f8e [4194304,4194304] 0 2024-05-25T06:11:52.847 INFO:tasks.workunit.client.0.smithi160.stdout:9/929: fsync d4/d11/d12/d2e/d45/fe6 0 2024-05-25T06:11:52.849 INFO:tasks.workunit.client.0.smithi160.stdout:6/976: fdatasync d5/de/d32/d35/f82 0 2024-05-25T06:11:52.853 INFO:tasks.workunit.client.0.smithi160.stdout:9/930: rmdir d4/d11/d1e/d3e/d25/d27/d95/d111 0 2024-05-25T06:11:52.854 INFO:tasks.workunit.client.0.smithi160.stdout:9/931: dread - d4/d11/d1e/ff5 zero size 2024-05-25T06:11:52.855 INFO:tasks.workunit.client.0.smithi160.stdout:9/932: dread - d4/d11/d1e/d114/f124 zero size 2024-05-25T06:11:52.862 INFO:tasks.workunit.client.0.smithi160.stdout:1/997: sync 2024-05-25T06:11:52.862 INFO:tasks.workunit.client.0.smithi160.stdout:9/933: dwrite d4/d11/d12/d2e/f105 [0,4194304] 0 2024-05-25T06:11:52.863 INFO:tasks.workunit.client.0.smithi160.stdout:6/977: dread d5/de/d32/d35/dab/faf [4194304,4194304] 0 2024-05-25T06:11:52.866 INFO:tasks.workunit.client.0.smithi160.stdout:1/998: rename de/c75 to de/d21/d49/d35/c157 0 2024-05-25T06:11:52.876 INFO:tasks.workunit.client.0.smithi160.stdout:6/978: dread d5/d39/f6c [0,4194304] 0 2024-05-25T06:11:52.878 INFO:tasks.workunit.client.0.smithi160.stdout:6/979: creat d5/de/d32/d9f/dd5/f157 x:0 0 0 2024-05-25T06:11:52.882 INFO:tasks.workunit.client.0.smithi160.stdout:1/999: dwrite de/d21/f31 [0,4194304] 0 2024-05-25T06:11:52.884 INFO:tasks.workunit.client.0.smithi160.stdout:6/980: truncate d5/de/d32/d74/f92 1901421 0 2024-05-25T06:11:52.887 INFO:tasks.workunit.client.0.smithi160.stdout:9/934: dread d4/d11/d1e/d3e/d25/d5e/fda [0,4194304] 0 2024-05-25T06:11:52.894 INFO:tasks.workunit.client.0.smithi160.stdout:6/981: dread d5/d45/d4a/f5d [0,4194304] 0 2024-05-25T06:11:52.894 INFO:tasks.workunit.client.0.smithi160.stdout:9/935: dwrite d4/d11/d12/d2e/f105 [0,4194304] 0 2024-05-25T06:11:52.896 INFO:tasks.workunit.client.0.smithi160.stdout:9/936: stat d4/d11/d1e/d3e/d25/d27/ffb 0 2024-05-25T06:11:52.903 INFO:tasks.workunit.client.0.smithi160.stdout:9/937: rename d4/d11/d12/d2e/d45/d10f/d79/f11f to d4/d11/f133 0 2024-05-25T06:11:52.905 INFO:tasks.workunit.client.0.smithi160.stdout:9/938: dread - d4/d11/d1e/d3e/d25/d5e/d122/f132 zero size 2024-05-25T06:11:52.908 INFO:tasks.workunit.client.0.smithi160.stdout:6/982: dread d5/de1/f7e [0,4194304] 0 2024-05-25T06:11:52.910 INFO:tasks.workunit.client.0.smithi160.stdout:6/983: mkdir d5/d39/d112/da4/d158 0 2024-05-25T06:11:52.912 INFO:tasks.workunit.client.0.smithi160.stdout:6/984: mkdir d5/de/d32/d35/d12e/d98/d159 0 2024-05-25T06:11:52.917 INFO:tasks.workunit.client.0.smithi160.stdout:8/940: sync 2024-05-25T06:11:52.917 INFO:tasks.workunit.client.0.smithi160.stdout:9/939: dwrite d4/d11/d12/d2e/d45/d83/fcd [0,4194304] 0 2024-05-25T06:11:52.918 INFO:tasks.workunit.client.0.smithi160.stdout:6/985: rmdir d5/d39 39 2024-05-25T06:11:52.927 INFO:tasks.workunit.client.0.smithi160.stdout:9/940: creat d4/d11/d1e/d3e/d25/d27/f134 x:0 0 0 2024-05-25T06:11:52.936 INFO:tasks.workunit.client.0.smithi160.stdout:6/986: dwrite d5/d45/fe2 [0,4194304] 0 2024-05-25T06:11:52.937 INFO:tasks.workunit.client.0.smithi160.stdout:9/941: rename d4/d11/d12/d2e/d45/d10f/d79/cf2 to d4/d11/d12/d2e/c135 0 2024-05-25T06:11:52.938 INFO:tasks.workunit.client.0.smithi160.stdout:8/941: dwrite d5/d7/dcb/d7a/fcd [0,4194304] 0 2024-05-25T06:11:52.945 INFO:tasks.workunit.client.0.smithi160.stdout:8/942: rmdir d5/d7/dcb/d75 39 2024-05-25T06:11:52.946 INFO:tasks.workunit.client.0.smithi160.stdout:6/987: symlink d5/d45/d4a/d154/l15a 0 2024-05-25T06:11:52.946 INFO:tasks.workunit.client.0.smithi160.stdout:8/943: chown d5/d9/d72/d104/f121 1084012 1 2024-05-25T06:11:52.949 INFO:tasks.workunit.client.0.smithi160.stdout:9/942: creat d4/d11/d1e/d3e/d25/d27/f136 x:0 0 0 2024-05-25T06:11:52.953 INFO:tasks.workunit.client.0.smithi160.stdout:8/944: read d5/d44/d1f/da3/d28/d49/d61/fad [234349,49505] 0 2024-05-25T06:11:52.953 INFO:tasks.workunit.client.0.smithi160.stdout:9/943: creat d4/d11/d12/d2e/d45/d10f/da7/f137 x:0 0 0 2024-05-25T06:11:52.953 INFO:tasks.workunit.client.0.smithi160.stdout:8/945: chown d5/d7/lff 12809773 1 2024-05-25T06:11:52.956 INFO:tasks.workunit.client.0.smithi160.stdout:9/944: mknod d4/d11/d1e/d3e/d25/d27/dc5/d100/c138 0 2024-05-25T06:11:52.965 INFO:tasks.workunit.client.0.smithi160.stdout:6/988: write d5/d39/d8f/f118 [1892511,112541] 0 2024-05-25T06:11:52.967 INFO:tasks.workunit.client.0.smithi160.stdout:6/989: creat d5/d39/da6/dac/f15b x:0 0 0 2024-05-25T06:11:52.968 INFO:tasks.workunit.client.0.smithi160.stdout:6/990: truncate d5/d39/f143 1118272 0 2024-05-25T06:11:52.970 INFO:tasks.workunit.client.0.smithi160.stdout:6/991: mkdir d5/d39/dc1/d15c 0 2024-05-25T06:11:52.974 INFO:tasks.workunit.client.0.smithi160.stdout:6/992: unlink d5/de/ff 0 2024-05-25T06:11:52.983 INFO:tasks.workunit.client.0.smithi160.stdout:6/993: dwrite d5/d39/f41 [0,4194304] 0 2024-05-25T06:11:52.988 INFO:tasks.workunit.client.0.smithi160.stdout:6/994: getdents d5/de/d32/d35 0 2024-05-25T06:11:52.990 INFO:tasks.workunit.client.0.smithi160.stdout:6/995: readlink d5/d30/d36/d73/d6e/df7/d145/lef 0 2024-05-25T06:11:52.994 INFO:tasks.workunit.client.0.smithi160.stdout:6/996: rmdir d5/de/d32/d35/d12e/d98/d159 0 2024-05-25T06:11:52.996 INFO:tasks.workunit.client.0.smithi160.stdout:6/997: chown d5/de/d7d/l86 5483646 1 2024-05-25T06:11:52.996 INFO:tasks.workunit.client.0.smithi160.stdout:6/998: fdatasync d5/de/d32/d9f/dd5/f157 0 2024-05-25T06:11:52.998 INFO:tasks.workunit.client.0.smithi160.stdout:6/999: chown d5/d39/d112/d107/f117 33572 1 2024-05-25T06:11:53.348 INFO:tasks.workunit.client.0.smithi160.stdout:8/946: sync 2024-05-25T06:11:53.348 INFO:tasks.workunit.client.0.smithi160.stdout:9/945: sync 2024-05-25T06:11:53.349 INFO:tasks.workunit.client.0.smithi160.stdout:8/947: dread - d5/d44/d1f/da3/d28/d4a/d4e/d86/f108 zero size 2024-05-25T06:11:53.357 INFO:tasks.workunit.client.0.smithi160.stdout:8/948: getdents d5/d7/dcb/d7a/dc9 0 2024-05-25T06:11:53.360 INFO:tasks.workunit.client.0.smithi160.stdout:8/949: mknod d5/d7/dcb/d75/c131 0 2024-05-25T06:11:53.367 INFO:tasks.workunit.client.0.smithi160.stdout:9/946: dwrite d4/d11/d14/f61 [0,4194304] 0 2024-05-25T06:11:53.368 INFO:tasks.workunit.client.0.smithi160.stdout:8/950: write d5/d44/d1f/da3/d28/d4a/d4e/d86/f66 [74860,60094] 0 2024-05-25T06:11:53.378 INFO:tasks.workunit.client.0.smithi160.stdout:9/947: write d4/d11/d1e/d24/f82 [113052,35568] 0 2024-05-25T06:11:53.379 INFO:tasks.workunit.client.0.smithi160.stdout:8/951: link d5/d7/dcb/d6a/cd5 d5/d44/d1f/da3/d28/de6/dc2/dd6/c132 0 2024-05-25T06:11:53.382 INFO:tasks.workunit.client.0.smithi160.stdout:9/948: creat d4/d11/d1e/d3e/d25/d5e/f139 x:0 0 0 2024-05-25T06:11:53.384 INFO:tasks.workunit.client.0.smithi160.stdout:9/949: mknod d4/db2/de5/c13a 0 2024-05-25T06:11:53.404 INFO:tasks.workunit.client.0.smithi160.stdout:8/952: dread d5/d7/dcb/d7a/f92 [0,4194304] 0 2024-05-25T06:11:53.410 INFO:tasks.workunit.client.0.smithi160.stdout:8/953: rename d5/d9c to d5/d7/d133 0 2024-05-25T06:11:53.415 INFO:tasks.workunit.client.0.smithi160.stdout:8/954: write d5/d44/d1f/da3/d28/d4a/d4e/d86/fa4 [114030,28778] 0 2024-05-25T06:11:53.422 INFO:tasks.workunit.client.0.smithi160.stdout:8/955: dwrite d5/d7/f19 [0,4194304] 0 2024-05-25T06:11:53.423 INFO:tasks.workunit.client.0.smithi160.stdout:8/956: fsync d5/d7/d9a/db5/fbd 0 2024-05-25T06:11:53.426 INFO:tasks.workunit.client.0.smithi160.stdout:8/957: creat d5/d44/dba/f134 x:0 0 0 2024-05-25T06:11:53.429 INFO:tasks.workunit.client.0.smithi160.stdout:8/958: fdatasync d5/d44/d1f/da3/f126 0 2024-05-25T06:11:53.429 INFO:tasks.workunit.client.0.smithi160.stdout:8/959: chown d5 1327182 1 2024-05-25T06:11:53.432 INFO:tasks.workunit.client.0.smithi160.stdout:8/960: write d5/d7/dcb/d6a/ff0 [61139,77522] 0 2024-05-25T06:11:53.435 INFO:tasks.workunit.client.0.smithi160.stdout:8/961: mkdir d5/d44/d1f/da3/d28/d49/d135 0 2024-05-25T06:11:53.437 INFO:tasks.workunit.client.0.smithi160.stdout:8/962: write d5/d44/d1f/da3/d28/d4a/d4e/d86/dd4/f128 [457709,104701] 0 2024-05-25T06:11:53.439 INFO:tasks.workunit.client.0.smithi160.stdout:8/963: fdatasync d5/d44/d1f/fd8 0 2024-05-25T06:11:53.440 INFO:tasks.workunit.client.0.smithi160.stdout:8/964: mknod d5/d7/dcb/d7a/dc9/c136 0 2024-05-25T06:11:53.443 INFO:tasks.workunit.client.0.smithi160.stdout:8/965: creat d5/d44/d1f/d58/d65/da2/f137 x:0 0 0 2024-05-25T06:11:53.444 INFO:tasks.workunit.client.0.smithi160.stdout:8/966: fdatasync d5/d44/d1f/da3/d28/d49/d61/f11c 0 2024-05-25T06:11:53.445 INFO:tasks.workunit.client.0.smithi160.stdout:8/967: read d5/d7/dcb/d7a/f99 [939280,51806] 0 2024-05-25T06:11:53.447 INFO:tasks.workunit.client.0.smithi160.stdout:8/968: fdatasync d5/d44/d1f/f55 0 2024-05-25T06:11:53.450 INFO:tasks.workunit.client.0.smithi160.stdout:8/969: dread d5/d44/d1f/fd8 [0,4194304] 0 2024-05-25T06:11:53.452 INFO:tasks.workunit.client.0.smithi160.stdout:8/970: mkdir d5/d44/dba/ddb/de3/d138 0 2024-05-25T06:11:53.454 INFO:tasks.workunit.client.0.smithi160.stdout:8/971: creat d5/d44/dba/ddb/de3/d138/f139 x:0 0 0 2024-05-25T06:11:53.455 INFO:tasks.workunit.client.0.smithi160.stdout:8/972: readlink d5/d7/lff 0 2024-05-25T06:11:53.458 INFO:tasks.workunit.client.0.smithi160.stdout:8/973: fdatasync d5/d44/d1f/da3/d28/f3f 0 2024-05-25T06:11:53.460 INFO:tasks.workunit.client.0.smithi160.stdout:8/974: read d5/d44/d1f/da3/d28/d49/f70 [366656,92783] 0 2024-05-25T06:11:53.462 INFO:tasks.workunit.client.0.smithi160.stdout:8/975: mknod d5/d9/d72/c13a 0 2024-05-25T06:11:53.463 INFO:tasks.workunit.client.0.smithi160.stdout:8/976: mkdir d5/d7/dcb/d75/d13b 0 2024-05-25T06:11:53.469 INFO:tasks.workunit.client.0.smithi160.stdout:8/977: dwrite d5/d7/d133/ff9 [0,4194304] 0 2024-05-25T06:11:53.471 INFO:tasks.workunit.client.0.smithi160.stdout:8/978: mkdir d5/d13c 0 2024-05-25T06:11:53.474 INFO:tasks.workunit.client.0.smithi160.stdout:8/979: write d5/d44/d1f/da3/d28/d4a/d4e/d86/f66 [29480,7962] 0 2024-05-25T06:11:53.479 INFO:tasks.workunit.client.0.smithi160.stdout:8/980: rename d5/d44/dba/ddb to d5/d44/d1f/da3/d28/d13d 0 2024-05-25T06:11:53.688 INFO:tasks.workunit.client.0.smithi160.stdout:9/950: sync 2024-05-25T06:11:53.689 INFO:tasks.workunit.client.0.smithi160.stdout:9/951: dread - d4/d11/d12/d2e/fec zero size 2024-05-25T06:11:53.690 INFO:tasks.workunit.client.0.smithi160.stdout:9/952: chown d4/d11/d12/d2e/d45/d83/d131 1 1 2024-05-25T06:11:53.692 INFO:tasks.workunit.client.0.smithi160.stdout:9/953: rmdir d4/d11/d1e/d3e/d25/d27 39 2024-05-25T06:11:53.694 INFO:tasks.workunit.client.0.smithi160.stdout:9/954: dread - d4/d11/d1e/d3e/d25/d27/ffb zero size 2024-05-25T06:11:53.701 INFO:tasks.workunit.client.0.smithi160.stdout:9/955: mknod d4/d11/d1e/c13b 0 2024-05-25T06:11:53.707 INFO:tasks.workunit.client.0.smithi160.stdout:9/956: dwrite d4/d11/d12/d2e/f32 [4194304,4194304] 0 2024-05-25T06:11:53.713 INFO:tasks.workunit.client.0.smithi160.stdout:9/957: rename d4/d11/d1e/d3e/d25/d5e/fe4 to d4/d11/d1e/d114/f13c 0 2024-05-25T06:11:53.720 INFO:tasks.workunit.client.0.smithi160.stdout:9/958: dwrite d4/d11/d1e/d3e/d25/d5e/ffd [0,4194304] 0 2024-05-25T06:11:53.721 INFO:tasks.workunit.client.0.smithi160.stdout:9/959: fdatasync d4/d11/d1e/d24/f82 0 2024-05-25T06:11:53.722 INFO:tasks.workunit.client.0.smithi160.stdout:9/960: stat d4/d11/d12/d2e/d45/d10f/lc9 0 2024-05-25T06:11:53.730 INFO:tasks.workunit.client.0.smithi160.stdout:9/961: dwrite d4/d11/d12/f106 [0,4194304] 0 2024-05-25T06:11:53.731 INFO:tasks.workunit.client.0.smithi160.stdout:8/981: sync 2024-05-25T06:11:53.733 INFO:tasks.workunit.client.0.smithi160.stdout:9/962: mknod d4/d11/d14/c13d 0 2024-05-25T06:11:53.740 INFO:tasks.workunit.client.0.smithi160.stdout:9/963: unlink d4/d11/d1e/d3e/d25/d5e/da2/fd0 0 2024-05-25T06:11:53.742 INFO:tasks.workunit.client.0.smithi160.stdout:9/964: readlink d4/d11/d1e/d3e/d25/d27/l87 0 2024-05-25T06:11:53.822 INFO:tasks.workunit.client.0.smithi160.stdout:9/965: symlink d4/db2/d12e/l13e 0 2024-05-25T06:11:53.824 INFO:tasks.workunit.client.0.smithi160.stdout:9/966: symlink d4/db2/l13f 0 2024-05-25T06:11:53.827 INFO:tasks.workunit.client.0.smithi160.stdout:9/967: symlink d4/d11/d1e/d24/l140 0 2024-05-25T06:11:53.828 INFO:tasks.workunit.client.0.smithi160.stdout:8/982: write d5/d44/d1f/d58/d65/da2/fb7 [511229,119509] 0 2024-05-25T06:11:53.830 INFO:tasks.workunit.client.0.smithi160.stdout:9/968: creat d4/d11/d12/d2e/d45/d10f/d79/f141 x:0 0 0 2024-05-25T06:11:53.831 INFO:tasks.workunit.client.0.smithi160.stdout:8/983: creat d5/d44/f13e x:0 0 0 2024-05-25T06:11:53.834 INFO:tasks.workunit.client.0.smithi160.stdout:8/984: unlink d5/d44/d1f/da3/l56 0 2024-05-25T06:11:53.856 INFO:tasks.workunit.client.0.smithi160.stdout:9/969: write d4/d11/d12/d2e/fec [254868,27601] 0 2024-05-25T06:11:53.857 INFO:tasks.workunit.client.0.smithi160.stdout:8/985: write d5/d9/f40 [3900129,78592] 0 2024-05-25T06:11:53.861 INFO:tasks.workunit.client.0.smithi160.stdout:9/970: link d4/d11/d12/d58/f64 d4/d11/d12/d2e/d45/d10f/da7/f142 0 2024-05-25T06:11:53.862 INFO:tasks.workunit.client.0.smithi160.stdout:8/986: mkdir d5/d44/d1f/da3/d28/d49/d135/d13f 0 2024-05-25T06:11:53.862 INFO:tasks.workunit.client.0.smithi160.stdout:9/971: chown d4/d11/d1e/d3e/d25/d5e 944578651 1 2024-05-25T06:11:53.867 INFO:tasks.workunit.client.0.smithi160.stdout:8/987: write d5/d44/f68 [3910048,5101] 0 2024-05-25T06:11:53.867 INFO:tasks.workunit.client.0.smithi160.stdout:9/972: dread d4/d11/d12/d58/f64 [0,4194304] 0 2024-05-25T06:11:53.873 INFO:tasks.workunit.client.0.smithi160.stdout:8/988: mkdir d5/d44/d1f/da3/d28/d4a/d4e/d140 0 2024-05-25T06:11:53.874 INFO:tasks.workunit.client.0.smithi160.stdout:9/973: creat d4/d11/d1e/d24/df7/d9c/f143 x:0 0 0 2024-05-25T06:11:53.877 INFO:tasks.workunit.client.0.smithi160.stdout:8/989: getdents d5/d9 0 2024-05-25T06:11:53.877 INFO:tasks.workunit.client.0.smithi160.stdout:9/974: mknod d4/d11/d12/d58/c144 0 2024-05-25T06:11:53.878 INFO:tasks.workunit.client.0.smithi160.stdout:8/990: rmdir d5/d44/d1f/da3 39 2024-05-25T06:11:53.882 INFO:tasks.workunit.client.0.smithi160.stdout:9/975: rename d4/d11/d1e/dbc/df1/c102 to d4/d11/d1e/d24/c145 0 2024-05-25T06:11:53.883 INFO:tasks.workunit.client.0.smithi160.stdout:9/976: dread - d4/d11/d1e/d6c/f12c zero size 2024-05-25T06:11:53.886 INFO:tasks.workunit.client.0.smithi160.stdout:9/977: truncate d4/d11/d14/f31 167650 0 2024-05-25T06:11:53.888 INFO:tasks.workunit.client.0.smithi160.stdout:9/978: symlink d4/d11/d1e/l146 0 2024-05-25T06:11:53.890 INFO:tasks.workunit.client.0.smithi160.stdout:9/979: truncate d4/d11/d1e/d24/df7/fad 256697 0 2024-05-25T06:11:53.892 INFO:tasks.workunit.client.0.smithi160.stdout:9/980: mknod d4/d11/d1e/d3e/c147 0 2024-05-25T06:11:53.893 INFO:tasks.workunit.client.0.smithi160.stdout:9/981: chown d4/d11/d1e/dbc/df1/l130 13909470 1 2024-05-25T06:11:53.897 INFO:tasks.workunit.client.0.smithi160.stdout:9/982: link d4/d11/d1e/d3e/d25/d5e/da2/faa d4/d11/d1e/d3e/f148 0 2024-05-25T06:11:53.903 INFO:tasks.workunit.client.0.smithi160.stdout:9/983: dwrite d4/d11/d1e/d24/df7/d9c/f143 [0,4194304] 0 2024-05-25T06:11:53.907 INFO:tasks.workunit.client.0.smithi160.stdout:9/984: chown d4/d11/d12/d2e/d45/d83/cf6 38678413 1 2024-05-25T06:11:53.911 INFO:tasks.workunit.client.0.smithi160.stdout:9/985: mknod d4/d11/d12/d2e/d45/c149 0 2024-05-25T06:11:53.915 INFO:tasks.workunit.client.0.smithi160.stdout:9/986: rename d4/d11/d12/d2e/fec to d4/db2/d119/f14a 0 2024-05-25T06:11:53.917 INFO:tasks.workunit.client.0.smithi160.stdout:9/987: mknod d4/d11/d1e/d3e/d25/d27/dc5/d100/c14b 0 2024-05-25T06:11:53.918 INFO:tasks.workunit.client.0.smithi160.stdout:9/988: chown d4/d11/d1e/d3e/d25/d27/f134 418548096 1 2024-05-25T06:11:53.918 INFO:tasks.workunit.client.0.smithi160.stdout:9/989: readlink d4/l10 0 2024-05-25T06:11:53.920 INFO:tasks.workunit.client.0.smithi160.stdout:9/990: write d4/d11/d1e/d24/df7/f11d [303039,12632] 0 2024-05-25T06:11:53.927 INFO:tasks.workunit.client.0.smithi160.stdout:9/991: write d4/d11/d12/fcc [755911,91482] 0 2024-05-25T06:11:53.929 INFO:tasks.workunit.client.0.smithi160.stdout:9/992: symlink d4/d11/l14c 0 2024-05-25T06:11:53.930 INFO:tasks.workunit.client.0.smithi160.stdout:9/993: stat d4/d11/d12/d2e/d45/d83/cf6 0 2024-05-25T06:11:53.932 INFO:tasks.workunit.client.0.smithi160.stdout:9/994: mknod d4/db2/d119/c14d 0 2024-05-25T06:11:53.934 INFO:tasks.workunit.client.0.smithi160.stdout:9/995: mkdir d4/d11/d12/d2e/d14e 0 2024-05-25T06:11:53.938 INFO:tasks.workunit.client.0.smithi160.stdout:9/996: getdents d4/d11/d1e/d24/d66 0 2024-05-25T06:11:53.943 INFO:tasks.workunit.client.0.smithi160.stdout:9/997: truncate d4/d11/d14/f53 3955855 0 2024-05-25T06:11:53.945 INFO:tasks.workunit.client.0.smithi160.stdout:9/998: mkdir d4/d11/d12/d2e/d14e/d14f 0 2024-05-25T06:11:53.946 INFO:tasks.workunit.client.0.smithi160.stdout:9/999: read - d4/d11/d1e/dbc/f12b zero size 2024-05-25T06:11:54.109 INFO:tasks.workunit.client.0.smithi160.stdout:8/991: sync 2024-05-25T06:11:54.110 INFO:tasks.workunit.client.0.smithi160.stdout:8/992: write d5/d9/f40 [4350204,123058] 0 2024-05-25T06:11:54.115 INFO:tasks.workunit.client.0.smithi160.stdout:8/993: rename d5/d44/d1f/da3/d28/d4a/d4e/fac to d5/d44/d1f/da3/d28/f141 0 2024-05-25T06:11:54.121 INFO:tasks.workunit.client.0.smithi160.stdout:8/994: write d5/f48 [2496126,20777] 0 2024-05-25T06:11:54.123 INFO:tasks.workunit.client.0.smithi160.stdout:8/995: mknod d5/d44/d1f/da3/d28/d4a/db8/c142 0 2024-05-25T06:11:54.129 INFO:tasks.workunit.client.0.smithi160.stdout:8/996: dwrite d5/d44/d1f/da3/f126 [0,4194304] 0 2024-05-25T06:11:54.131 INFO:tasks.workunit.client.0.smithi160.stdout:8/997: chown d5/d44/d1f/da3/d28/l2b 382939998 1 2024-05-25T06:11:54.133 INFO:tasks.workunit.client.0.smithi160.stdout:8/998: write d5/d7/dcb/d75/f76 [1467289,28235] 0 2024-05-25T06:11:54.324 INFO:tasks.workunit.client.0.smithi160.stdout:8/999: sync 2024-05-25T06:11:54.329 INFO:tasks.workunit.client.0.smithi160.stderr:+ rm -rf -- ./tmp.bD4KoTPC4G 2024-05-25T06:11:54.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:54 smithi160 ceph-mon[29041]: pgmap v27: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 130 MiB/s rd, 297 MiB/s wr, 711 op/s 2024-05-25T06:11:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:54 smithi203 ceph-mon[36030]: pgmap v27: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 130 MiB/s rd, 297 MiB/s wr, 711 op/s 2024-05-25T06:11:56.464 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:56 smithi160 ceph-mon[29041]: pgmap v28: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 73 MiB/s rd, 171 MiB/s wr, 427 op/s 2024-05-25T06:11:56.465 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:56 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:56.465 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:56 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:56 smithi203 ceph-mon[36030]: pgmap v28: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 73 MiB/s rd, 171 MiB/s wr, 427 op/s 2024-05-25T06:11:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:56 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:56 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:11:57.611 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:57 smithi160 ceph-mon[29041]: Reconfiguring alertmanager.smithi160 (dependencies changed)... 2024-05-25T06:11:57.611 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:57 smithi160 ceph-mon[29041]: Reconfiguring daemon alertmanager.smithi160 on smithi160 2024-05-25T06:11:57.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:57 smithi203 ceph-mon[36030]: Reconfiguring alertmanager.smithi160 (dependencies changed)... 2024-05-25T06:11:57.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:57 smithi203 ceph-mon[36030]: Reconfiguring daemon alertmanager.smithi160 on smithi160 2024-05-25T06:11:58.609 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:11:58 smithi160 ceph-mon[29041]: pgmap v29: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 83 MiB/s rd, 195 MiB/s wr, 550 op/s 2024-05-25T06:11:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:11:58 smithi203 ceph-mon[36030]: pgmap v29: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 83 MiB/s rd, 195 MiB/s wr, 550 op/s 2024-05-25T06:11:59.309 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-25T06:11:59.310 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-25T06:12:00.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:00 smithi160 ceph-mon[29041]: pgmap v30: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 34 MiB/s rd, 84 MiB/s wr, 301 op/s 2024-05-25T06:12:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:00 smithi203 ceph-mon[36030]: pgmap v30: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 34 MiB/s rd, 84 MiB/s wr, 301 op/s 2024-05-25T06:12:02.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:02 smithi160 ceph-mon[29041]: pgmap v31: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 35 MiB/s rd, 84 MiB/s wr, 327 op/s 2024-05-25T06:12:02.422 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:02 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:02.422 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:02 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:02.422 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:02 smithi160 ceph-mon[29041]: Reconfiguring grafana.smithi160 (dependencies changed)... 2024-05-25T06:12:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:02 smithi203 ceph-mon[36030]: pgmap v31: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 35 MiB/s rd, 84 MiB/s wr, 327 op/s 2024-05-25T06:12:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:02 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:02 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:02 smithi203 ceph-mon[36030]: Reconfiguring grafana.smithi160 (dependencies changed)... 2024-05-25T06:12:03.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:03 smithi160 ceph-mon[29041]: Reconfiguring daemon grafana.smithi160 on smithi160 2024-05-25T06:12:03.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:03 smithi203 ceph-mon[36030]: Reconfiguring daemon grafana.smithi160 on smithi160 2024-05-25T06:12:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:04 smithi160 ceph-mon[29041]: pgmap v32: 65 pgs: 65 active+clean; 1.6 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 33 MiB/s rd, 80 MiB/s wr, 402 op/s 2024-05-25T06:12:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:04 smithi203 ceph-mon[36030]: pgmap v32: 65 pgs: 65 active+clean; 1.6 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 33 MiB/s rd, 80 MiB/s wr, 402 op/s 2024-05-25T06:12:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:05 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:05.662 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-25T06:12:05.662 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-25T06:12:05.709 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:05 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:06.549 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:06 smithi160 ceph-mon[29041]: pgmap v33: 65 pgs: 65 active+clean; 1.6 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 10 MiB/s rd, 25 MiB/s wr, 243 op/s 2024-05-25T06:12:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:06 smithi203 ceph-mon[36030]: pgmap v33: 65 pgs: 65 active+clean; 1.6 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 10 MiB/s rd, 25 MiB/s wr, 243 op/s 2024-05-25T06:12:07.962 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:07 smithi160 ceph-mon[29041]: pgmap v34: 65 pgs: 65 active+clean; 691 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 399 op/s 2024-05-25T06:12:07.962 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:07 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:07.963 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:07 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:07.963 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:07 smithi160 ceph-mon[29041]: Reconfiguring prometheus.smithi160 (dependencies changed)... 2024-05-25T06:12:07.963 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:07 smithi160 ceph-mon[29041]: Reconfiguring daemon prometheus.smithi160 on smithi160 2024-05-25T06:12:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:07 smithi203 ceph-mon[36030]: pgmap v34: 65 pgs: 65 active+clean; 691 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 399 op/s 2024-05-25T06:12:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:07 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:07 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:07 smithi203 ceph-mon[36030]: Reconfiguring prometheus.smithi160 (dependencies changed)... 2024-05-25T06:12:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:07 smithi203 ceph-mon[36030]: Reconfiguring daemon prometheus.smithi160 on smithi160 2024-05-25T06:12:09.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:09 smithi160 ceph-mon[29041]: pgmap v35: 65 pgs: 65 active+clean; 691 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-05-25T06:12:10.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:09 smithi203 ceph-mon[36030]: pgmap v35: 65 pgs: 65 active+clean; 691 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-05-25T06:12:11.963 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:11 smithi160 ceph-mon[29041]: pgmap v36: 65 pgs: 65 active+clean; 655 MiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-05-25T06:12:12.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:11 smithi203 ceph-mon[36030]: pgmap v36: 65 pgs: 65 active+clean; 655 MiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 -- 172.21.15.160:0/1026285693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80662c0 msgr2=0x7f31d80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1026285693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80662c0 0x7f31d80ff9a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f31c8004940 tx=0x7f31c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 -- 172.21.15.160:0/1026285693 shutdown_connections 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1026285693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80ffee0 0x7f31d8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1026285693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80662c0 0x7f31d80ff9a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 -- 172.21.15.160:0/1026285693 >> 172.21.15.160:0/1026285693 conn(0x7f31d80f7430 msgr2=0x7f31d80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:12.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 -- 172.21.15.160:0/1026285693 shutdown_connections 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 -- 172.21.15.160:0/1026285693 wait complete. 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.421+0000 7f31dde9b700 1 Processor -- start 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31dde9b700 1 -- start start 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31dde9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 0x7f31d80fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31dde9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80ffee0 0x7f31d80fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31dde9b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d81116c0 con 0x7f31d80ffee0 2024-05-25T06:12:12.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31dde9b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8111830 con 0x7f31d80662c0 2024-05-25T06:12:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31d77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 0x7f31d80fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80ffee0 0x7f31d80fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31d77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 0x7f31d80fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34550/0 (socket says 172.21.15.160:34550) 2024-05-25T06:12:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.422+0000 7f31d77fe700 1 -- 172.21.15.160:0/1418066089 learned_addr learned my addr 172.21.15.160:0/1418066089 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31d77fe700 1 -- 172.21.15.160:0/1418066089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80ffee0 msgr2=0x7f31d80fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31d77fe700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80ffee0 0x7f31d80fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31d77fe700 1 -- 172.21.15.160:0/1418066089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c8005040 con 0x7f31d80662c0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31d77fe700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 0x7f31d80fd3d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f31c8000c00 tx=0x7f31c8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c800e040 con 0x7f31d80662c0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d80fbea0 con 0x7f31d80662c0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31c800bbf0 con 0x7f31d80662c0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c801c630 con 0x7f31d80662c0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.423+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d80fc410 con 0x7f31d80662c0 2024-05-25T06:12:12.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.425+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d805f5c0 con 0x7f31d80662c0 2024-05-25T06:12:12.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.426+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f31c800bd60 con 0x7f31d80662c0 2024-05-25T06:12:12.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.426+0000 7f31cffff700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f31c406b790 0x7f31c406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:12.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.426+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f31c808b700 con 0x7f31d80662c0 2024-05-25T06:12:12.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.427+0000 7f31d6ffd700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f31c406b790 0x7f31c406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:12.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.427+0000 7f31d6ffd700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f31c406b790 0x7f31c406dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f31c00102e0 tx=0x7f31c000b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:12.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.432+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f31c8055240 con 0x7f31d80662c0 2024-05-25T06:12:12.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.828+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31d8105a10 con 0x7f31c406b790 2024-05-25T06:12:12.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.830+0000 7f31cffff700 1 -- 172.21.15.160:0/1418066089 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f31d8105a10 con 0x7f31c406b790 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f31c406b790 msgr2=0x7f31c406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f31c406b790 0x7f31c406dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f31c00102e0 tx=0x7f31c000b040 comp rx=0 tx=0).stop 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 msgr2=0x7f31d80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 0x7f31d80fd3d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f31c8000c00 tx=0x7f31c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 shutdown_connections 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d80662c0 0x7f31d80fd3d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.832+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f31c406b790 0x7f31c406dc40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.833+0000 7f31dde9b700 1 --2- 172.21.15.160:0/1418066089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d80ffee0 0x7f31d80fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.833+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 >> 172.21.15.160:0/1418066089 conn(0x7f31d80f7430 msgr2=0x7f31d80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.833+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 shutdown_connections 2024-05-25T06:12:12.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.833+0000 7f31dde9b700 1 -- 172.21.15.160:0/1418066089 wait complete. 2024-05-25T06:12:12.849 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:12:12.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 -- 172.21.15.160:0/1870737577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 msgr2=0x7fd7900faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 --2- 172.21.15.160:0/1870737577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 0x7fd7900faf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd780004940 tx=0x7fd780009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 -- 172.21.15.160:0/1870737577 shutdown_connections 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 --2- 172.21.15.160:0/1870737577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 0x7fd7900faf30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 --2- 172.21.15.160:0/1870737577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd7900fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 -- 172.21.15.160:0/1870737577 >> 172.21.15.160:0/1870737577 conn(0x7fd7900f6b80 msgr2=0x7fd7900f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 -- 172.21.15.160:0/1870737577 shutdown_connections 2024-05-25T06:12:12.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.981+0000 7fd797c47700 1 -- 172.21.15.160:0/1870737577 wait complete. 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd797c47700 1 Processor -- start 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd797c47700 1 -- start start 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd797c47700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd79010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd797c47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 0x7fd79010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd797c47700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79010e4c0 con 0x7fd7900fedf0 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd797c47700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79010ed10 con 0x7fd7900fe4a0 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd7959e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd79010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd7959e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd79010d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34582/0 (socket says 172.21.15.160:34582) 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.982+0000 7fd7959e3700 1 -- 172.21.15.160:0/167093229 learned_addr learned my addr 172.21.15.160:0/167093229 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7951e2700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 0x7fd79010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:12.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7959e3700 1 -- 172.21.15.160:0/167093229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 msgr2=0x7fd79010dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7959e3700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 0x7fd79010dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7959e3700 1 -- 172.21.15.160:0/167093229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780005040 con 0x7fd7900fe4a0 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7959e3700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd79010d980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00eb40 tx=0x7fd78c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd78c007cc0 con 0x7fd7900fe4a0 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79010ef90 con 0x7fd7900fe4a0 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79010f4d0 con 0x7fd7900fe4a0 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd78c00f040 con 0x7fd7900fe4a0 2024-05-25T06:12:12.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.983+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd78c00ca10 con 0x7fd7900fe4a0 2024-05-25T06:12:12.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.984+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7740052e0 con 0x7fd7900fe4a0 2024-05-25T06:12:12.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.985+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fd78c00cc80 con 0x7fd7900fe4a0 2024-05-25T06:12:12.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.985+0000 7fd7867fc700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fd77c06b790 0x7fd77c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:12.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.985+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd78c08a040 con 0x7fd7900fe4a0 2024-05-25T06:12:12.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.986+0000 7fd7951e2700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fd77c06b790 0x7fd77c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:12.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.986+0000 7fd7951e2700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fd77c06b790 0x7fd77c06dc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd780009b30 tx=0x7fd780009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:12.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:12.992+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fd78c050210 con 0x7fd7900fe4a0 2024-05-25T06:12:13.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.230+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd774000bf0 con 0x7fd77c06b790 2024-05-25T06:12:13.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.231+0000 7fd7867fc700 1 -- 172.21.15.160:0/167093229 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fd774000bf0 con 0x7fd77c06b790 2024-05-25T06:12:13.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fd77c06b790 msgr2=0x7fd77c06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fd77c06b790 0x7fd77c06dc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd780009b30 tx=0x7fd780009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 msgr2=0x7fd79010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd79010d980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00eb40 tx=0x7fd78c00b040 comp rx=0 tx=0).stop 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 shutdown_connections 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7900fe4a0 0x7fd79010d980 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fd77c06b790 0x7fd77c06dc40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 --2- 172.21.15.160:0/167093229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7900fedf0 0x7fd79010dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 >> 172.21.15.160:0/167093229 conn(0x7fd7900f6b80 msgr2=0x7fd7900f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 shutdown_connections 2024-05-25T06:12:13.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.234+0000 7fd797c47700 1 -- 172.21.15.160:0/167093229 wait complete. 2024-05-25T06:12:13.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 -- 172.21.15.160:0/752614198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fbcb0 msgr2=0x7fa3100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 --2- 172.21.15.160:0/752614198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fbcb0 0x7fa3100fc0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa300004880 tx=0x7fa300009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:13.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 -- 172.21.15.160:0/752614198 shutdown_connections 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 --2- 172.21.15.160:0/752614198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fcf60 0x7fa3100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 --2- 172.21.15.160:0/752614198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fbcb0 0x7fa3100fc0c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 -- 172.21.15.160:0/752614198 >> 172.21.15.160:0/752614198 conn(0x7fa3100f7430 msgr2=0x7fa3100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 -- 172.21.15.160:0/752614198 shutdown_connections 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 -- 172.21.15.160:0/752614198 wait complete. 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.364+0000 7fa314f10700 1 Processor -- start 2024-05-25T06:12:13.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa314f10700 1 -- start start 2024-05-25T06:12:13.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa314f10700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 0x7fa310112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:13.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa314f10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fcf60 0x7fa310115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa314f10700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310066220 con 0x7fa3100fcf60 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa314f10700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310066360 con 0x7fa3100fbcb0 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fcf60 0x7fa310115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30e59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 0x7fa310112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30e59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 0x7fa310112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34598/0 (socket says 172.21.15.160:34598) 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fcf60 0x7fa310115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50064/0 (socket says 172.21.15.160:50064) 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30e59c700 1 -- 172.21.15.160:0/3844682756 learned_addr learned my addr 172.21.15.160:0/3844682756 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30e59c700 1 -- 172.21.15.160:0/3844682756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fcf60 msgr2=0x7fa310115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30e59c700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fcf60 0x7fa310115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.365+0000 7fa30e59c700 1 -- 172.21.15.160:0/3844682756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304005040 con 0x7fa3100fbcb0 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.366+0000 7fa30e59c700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 0x7fa310112b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa300008fb0 tx=0x7fa300009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.366+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30000e030 con 0x7fa3100fbcb0 2024-05-25T06:12:13.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.366+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa300013e60 con 0x7fa3100fbcb0 2024-05-25T06:12:13.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.366+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3000123c0 con 0x7fa3100fbcb0 2024-05-25T06:12:13.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.366+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa300005040 con 0x7fa3100fbcb0 2024-05-25T06:12:13.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.366+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa310115840 con 0x7fa3100fbcb0 2024-05-25T06:12:13.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.367+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31005f5c0 con 0x7fa3100fbcb0 2024-05-25T06:12:13.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.368+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fa30001c770 con 0x7fa3100fbcb0 2024-05-25T06:12:13.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.368+0000 7fa2feffd700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fa2f806b790 0x7fa2f806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:13.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.368+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa30008c160 con 0x7fa3100fbcb0 2024-05-25T06:12:13.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.368+0000 7fa30dd9b700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fa2f806b790 0x7fa2f806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:13.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.369+0000 7fa30dd9b700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fa2f806b790 0x7fa2f806dc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa304009fd0 tx=0x7fa30400b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:13.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.374+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fa300055e00 con 0x7fa3100fbcb0 2024-05-25T06:12:13.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.655+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa31005f5c0 con 0x7fa2f806b790 2024-05-25T06:12:13.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.677+0000 7fa2feffd700 1 -- 172.21.15.160:0/3844682756 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fa31005f5c0 con 0x7fa2f806b790 2024-05-25T06:12:13.677 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:12:13.677 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 starting - - - - 2024-05-25T06:12:13.677 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (9m) 56s ago 9m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (8m) 44s ago 8m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 starting - - - - 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (3m) 56s ago 3m 57.5M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (3m) 56s ago 3m 99.2M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (3m) 44s ago 3m 66.4M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (3m) 44s ago 3m 102M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:9283 running (10m) 56s ago 10m 385M - 17.2.7-958-g7b01852c baf1547092e7 fbeba8913b6d 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (89s) 44s ago 8m 490M - 19.0.0-2843-gea0684c0 f9a9809cafd6 fbc025680ca5 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (10m) 56s ago 10m 49.6M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (8m) 44s ago 8m 39.3M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 starting - - - - 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (8m) 44s ago 8m 15.5M - 1.5.0 0da6a335fe13 aec5d8f8dcff 2024-05-25T06:12:13.678 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (7m) 56s ago 7m 112M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:12:13.679 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (7m) 56s ago 7m 125M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:12:13.679 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (6m) 56s ago 6m 104M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:12:13.679 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (6m) 44s ago 6m 132M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:12:13.679 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (5m) 44s ago 5m 121M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:12:13.679 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (5m) 44s ago 5m 109M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:12:13.679 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 starting - - - - 2024-05-25T06:12:13.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.680+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fa2f806b790 msgr2=0x7fa2f806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fa2f806b790 0x7fa2f806dc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa304009fd0 tx=0x7fa30400b040 comp rx=0 tx=0).stop 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 msgr2=0x7fa310112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 0x7fa310112b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa300008fb0 tx=0x7fa300009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 shutdown_connections 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3100fbcb0 0x7fa310112b20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fa2f806b790 0x7fa2f806dc40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 --2- 172.21.15.160:0/3844682756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3100fcf60 0x7fa310115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 >> 172.21.15.160:0/3844682756 conn(0x7fa3100f7430 msgr2=0x7fa3100f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.681+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 shutdown_connections 2024-05-25T06:12:13.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.682+0000 7fa314f10700 1 -- 172.21.15.160:0/3844682756 wait complete. 2024-05-25T06:12:13.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 -- 172.21.15.160:0/3396254890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fbcc0 msgr2=0x7f69900fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 --2- 172.21.15.160:0/3396254890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fbcc0 0x7f69900fc0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f698c004940 tx=0x7f698c009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 -- 172.21.15.160:0/3396254890 shutdown_connections 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 --2- 172.21.15.160:0/3396254890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fcf70 0x7f69900fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 --2- 172.21.15.160:0/3396254890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fbcc0 0x7f69900fc0d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 -- 172.21.15.160:0/3396254890 >> 172.21.15.160:0/3396254890 conn(0x7f69900f7420 msgr2=0x7f69900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 -- 172.21.15.160:0/3396254890 shutdown_connections 2024-05-25T06:12:13.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.843+0000 7f6998333700 1 -- 172.21.15.160:0/3396254890 wait complete. 2024-05-25T06:12:13.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.844+0000 7f6998333700 1 Processor -- start 2024-05-25T06:12:13.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f6998333700 1 -- start start 2024-05-25T06:12:13.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f6998333700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 0x7f699010c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f6998333700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fcf70 0x7f699010ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f6998333700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699010d460 con 0x7f69900fcf70 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f6998333700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699010dcb0 con 0x7f69900fbcc0 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f69960cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 0x7f699010c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f69960cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 0x7f699010c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:34624/0 (socket says 172.21.15.160:34624) 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.845+0000 7f69960cf700 1 -- 172.21.15.160:0/2321058952 learned_addr learned my addr 172.21.15.160:0/2321058952 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.846+0000 7f69958ce700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fcf70 0x7f699010ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:13.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.846+0000 7f69958ce700 1 -- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 msgr2=0x7f699010c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:13.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.846+0000 7f69958ce700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 0x7f699010c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:13.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.846+0000 7f69958ce700 1 -- 172.21.15.160:0/2321058952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c005040 con 0x7f69900fcf70 2024-05-25T06:12:13.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.846+0000 7f69960cf700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 0x7f699010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:12:13.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.846+0000 7f69958ce700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fcf70 0x7f699010ce60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6980002820 tx=0x7f6980004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:13.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.847+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6980007cc0 con 0x7f69900fcf70 2024-05-25T06:12:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.847+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699010df90 con 0x7f69900fcf70 2024-05-25T06:12:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.847+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f698000f040 con 0x7f69900fcf70 2024-05-25T06:12:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.847+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6980013920 con 0x7f69900fcf70 2024-05-25T06:12:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.847+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699010e4d0 con 0x7f69900fcf70 2024-05-25T06:12:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.848+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699005f5c0 con 0x7f69900fcf70 2024-05-25T06:12:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.849+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f698001c400 con 0x7f69900fcf70 2024-05-25T06:12:13.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.849+0000 7f6986ffd700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f697c06b790 0x7f697c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:13.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.849+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f698008a0b0 con 0x7f69900fcf70 2024-05-25T06:12:13.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.849+0000 7f69960cf700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f697c06b790 0x7f697c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:13.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.850+0000 7f69960cf700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f697c06b790 0x7f697c06dc40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b3b0 tx=0x7f698c009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:13.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:13.854+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f6980057720 con 0x7f69900fcf70 2024-05-25T06:12:13.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:13 smithi160 ceph-mon[29041]: pgmap v37: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 449 op/s 2024-05-25T06:12:13.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:13 smithi160 ceph-mon[29041]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:13.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:13 smithi160 ceph-mon[29041]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:13.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:13 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:13.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:13 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:13 smithi203 ceph-mon[36030]: pgmap v37: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 449 op/s 2024-05-25T06:12:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:13 smithi203 ceph-mon[36030]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:13 smithi203 ceph-mon[36030]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:13 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:14.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:13 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:14.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.177+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69901047b0 con 0x7f69900fcf70 2024-05-25T06:12:14.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.177+0000 7f6986ffd700 1 -- 172.21.15.160:0/2321058952 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+844 (secure 0 0 0) 0x7f69800572b0 con 0x7f69900fcf70 2024-05-25T06:12:14.177 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:12:14.177 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 1, 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:14.178 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:12:14.179 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 13, 2024-05-25T06:12:14.179 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:12:14.179 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:12:14.179 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f697c06b790 msgr2=0x7f697c06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f697c06b790 0x7f697c06dc40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b3b0 tx=0x7f698c009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fcf70 msgr2=0x7f699010ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fcf70 0x7f699010ce60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6980002820 tx=0x7f6980004570 comp rx=0 tx=0).stop 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 shutdown_connections 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69900fbcc0 0x7f699010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7f697c06b790 0x7f697c06dc40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 --2- 172.21.15.160:0/2321058952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69900fcf70 0x7f699010ce60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 >> 172.21.15.160:0/2321058952 conn(0x7f69900f7420 msgr2=0x7f69900f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:14.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 shutdown_connections 2024-05-25T06:12:14.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.180+0000 7f6998333700 1 -- 172.21.15.160:0/2321058952 wait complete. 2024-05-25T06:12:14.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 -- 172.21.15.160:0/1943227006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 msgr2=0x7fca4c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:14.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1943227006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c0fd3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fca34004940 tx=0x7fca34009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:14.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 -- 172.21.15.160:0/1943227006 shutdown_connections 2024-05-25T06:12:14.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1943227006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c0fd3d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1943227006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0fbcb0 0x7fca4c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 -- 172.21.15.160:0/1943227006 >> 172.21.15.160:0/1943227006 conn(0x7fca4c0f7430 msgr2=0x7fca4c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 -- 172.21.15.160:0/1943227006 shutdown_connections 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.290+0000 7fca51b0f700 1 -- 172.21.15.160:0/1943227006 wait complete. 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca51b0f700 1 Processor -- start 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca51b0f700 1 -- start start 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca51b0f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0fbcb0 0x7fca4c10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca51b0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca51b0f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c110700 con 0x7fca4c0fcf60 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca51b0f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c110f00 con 0x7fca4c0fbcb0 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca4affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca4affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50096/0 (socket says 172.21.15.160:50096) 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca4affd700 1 -- 172.21.15.160:0/1454882615 learned_addr learned my addr 172.21.15.160:0/1454882615 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:14.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.291+0000 7fca4b7fe700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0fbcb0 0x7fca4c10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:14.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca4affd700 1 -- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0fbcb0 msgr2=0x7fca4c10fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca4affd700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0fbcb0 0x7fca4c10fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca4affd700 1 -- 172.21.15.160:0/1454882615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34005040 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca4affd700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c110100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fca34013870 tx=0x7fca34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca34021070 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca34009430 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c111180 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3400e070 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.292+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c1116c0 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.293+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c1018f0 con 0x7fca4c0fcf60 2024-05-25T06:12:14.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.293+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fca34013af0 con 0x7fca4c0fcf60 2024-05-25T06:12:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.294+0000 7fca43fff700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fca3806b790 0x7fca3806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.294+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fca3408b510 con 0x7fca4c0fcf60 2024-05-25T06:12:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.294+0000 7fca4b7fe700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fca3806b790 0x7fca3806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.295+0000 7fca4b7fe700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fca3806b790 0x7fca3806dc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009b80 tx=0x7fca3c00b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.298+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fca34058d60 con 0x7fca4c0fcf60 2024-05-25T06:12:14.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.522+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 --> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca4c1048b0 con 0x7fca3806b790 2024-05-25T06:12:14.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.523+0000 7fca43fff700 1 -- 172.21.15.160:0/1454882615 <== mgr.24381 v2:172.21.15.203:6828/2357665398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fca4c1048b0 con 0x7fca3806b790 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "1/2 daemons upgraded", 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "", 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:12:14.524 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:12:14.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fca3806b790 msgr2=0x7fca3806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fca3806b790 0x7fca3806dc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009b80 tx=0x7fca3c00b040 comp rx=0 tx=0).stop 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 msgr2=0x7fca4c110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c110100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fca34013870 tx=0x7fca34009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 shutdown_connections 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0fbcb0 0x7fca4c10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.203:6828/2357665398,v1:172.21.15.203:6829/2357665398] conn(0x7fca3806b790 0x7fca3806dc40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 --2- 172.21.15.160:0/1454882615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c0fcf60 0x7fca4c110100 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.525+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 >> 172.21.15.160:0/1454882615 conn(0x7fca4c0f7430 msgr2=0x7fca4c065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.526+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 shutdown_connections 2024-05-25T06:12:14.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:14.526+0000 7fca51b0f700 1 -- 172.21.15.160:0/1454882615 wait complete. 2024-05-25T06:12:15.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:14 smithi203 ceph-mon[36030]: Reconfiguring node-exporter.smithi203 (dependencies changed)... 2024-05-25T06:12:15.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:14 smithi203 ceph-mon[36030]: Reconfiguring daemon node-exporter.smithi203 on smithi203 2024-05-25T06:12:15.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:14 smithi203 ceph-mon[36030]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:15.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:14 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2321058952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:12:15.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:14 smithi160 ceph-mon[29041]: Reconfiguring node-exporter.smithi203 (dependencies changed)... 2024-05-25T06:12:15.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:14 smithi160 ceph-mon[29041]: Reconfiguring daemon node-exporter.smithi203 on smithi203 2024-05-25T06:12:15.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:14 smithi160 ceph-mon[29041]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:15.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:14 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2321058952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:12:16.050 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:15 smithi203 ceph-mon[36030]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:16.051 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:15 smithi203 ceph-mon[36030]: pgmap v38: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 354 op/s 2024-05-25T06:12:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:15 smithi160 ceph-mon[29041]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:15 smithi160 ceph-mon[29041]: pgmap v38: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 354 op/s 2024-05-25T06:12:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:17 smithi203 ceph-mon[36030]: pgmap v39: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 593 op/s 2024-05-25T06:12:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:17 smithi160 ceph-mon[29041]: pgmap v39: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 593 op/s 2024-05-25T06:12:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:19 smithi203 ceph-mon[36030]: pgmap v40: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 437 op/s 2024-05-25T06:12:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:19 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:12:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:19 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:19 smithi160 ceph-mon[29041]: pgmap v40: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 437 op/s 2024-05-25T06:12:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:19 smithi160 ceph-mon[29041]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:12:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:19 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:21 smithi203 ceph-mon[36030]: pgmap v41: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 450 op/s 2024-05-25T06:12:22.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:21 smithi160 ceph-mon[29041]: pgmap v41: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 450 op/s 2024-05-25T06:12:24.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: pgmap v42: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 561 op/s 2024-05-25T06:12:24.122 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: Upgrade: Need to upgrade myself (mgr.smithi203.rotqex) 2024-05-25T06:12:24.123 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:24 smithi160 ceph-mon[29041]: Upgrade: Need to upgrade myself (mgr.smithi203.rotqex) 2024-05-25T06:12:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: pgmap v42: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 561 op/s 2024-05-25T06:12:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: Upgrade: Need to upgrade myself (mgr.smithi203.rotqex) 2024-05-25T06:12:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:24 smithi203 ceph-mon[36030]: Upgrade: Need to upgrade myself (mgr.smithi203.rotqex) 2024-05-25T06:12:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: pgmap v43: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 377 op/s 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: Upgrade: Updating mgr.smithi160.nalzlg 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:12:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:25 smithi160 ceph-mon[29041]: Deploying daemon mgr.smithi160.nalzlg on smithi160 2024-05-25T06:12:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: pgmap v43: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 377 op/s 2024-05-25T06:12:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: Upgrade: Updating mgr.smithi160.nalzlg 2024-05-25T06:12:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:12:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:12:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:12:26.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:12:26.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:25 smithi203 ceph-mon[36030]: Deploying daemon mgr.smithi160.nalzlg on smithi160 2024-05-25T06:12:27.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:27 smithi160 ceph-mon[29041]: pgmap v44: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 543 op/s 2024-05-25T06:12:28.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:27 smithi203 ceph-mon[36030]: pgmap v44: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 543 op/s 2024-05-25T06:12:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:29 smithi160 ceph-mon[29041]: pgmap v45: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-05-25T06:12:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:29 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:29 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:29 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:30.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:29 smithi203 ceph-mon[36030]: pgmap v45: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-05-25T06:12:30.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:29 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:30.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:29 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:30.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:29 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:31 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:31 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:31.552 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:31 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:31.552 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:31 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:32 smithi160 ceph-mon[29041]: pgmap v46: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 318 op/s 2024-05-25T06:12:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:32 smithi203 ceph-mon[36030]: pgmap v46: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 318 op/s 2024-05-25T06:12:34.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:33 smithi160 ceph-mon[29041]: pgmap v47: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 454 op/s 2024-05-25T06:12:34.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:33 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:34.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:33 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:33 smithi203 ceph-mon[36030]: pgmap v47: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 454 op/s 2024-05-25T06:12:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:33 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:33 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:35.129 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:34 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:34 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:36.342 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:35 smithi160 ceph-mon[29041]: pgmap v48: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 329 op/s 2024-05-25T06:12:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:35 smithi203 ceph-mon[36030]: pgmap v48: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 329 op/s 2024-05-25T06:12:37.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:36 smithi160 ceph-mon[29041]: mgrmap e26: smithi203.rotqex(active, since 91s), standbys: smithi160.nalzlg 2024-05-25T06:12:37.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:36 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:37.124 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:36 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:36 smithi203 ceph-mon[36030]: mgrmap e26: smithi203.rotqex(active, since 91s), standbys: smithi160.nalzlg 2024-05-25T06:12:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:36 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:36 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:37 smithi203 ceph-mon[36030]: pgmap v49: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 518 op/s 2024-05-25T06:12:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:37 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:37 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:38.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:37 smithi160 ceph-mon[29041]: pgmap v49: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 518 op/s 2024-05-25T06:12:38.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:37 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:38.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:37 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:39.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:38 smithi160 ceph-mon[29041]: Standby manager daemon smithi160.nalzlg restarted 2024-05-25T06:12:39.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:38 smithi160 ceph-mon[29041]: Standby manager daemon smithi160.nalzlg started 2024-05-25T06:12:39.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:38 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/crt"}]: dispatch 2024-05-25T06:12:39.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:38 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:12:39.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:38 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/key"}]: dispatch 2024-05-25T06:12:39.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:38 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:12:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:38 smithi203 ceph-mon[36030]: Standby manager daemon smithi160.nalzlg restarted 2024-05-25T06:12:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:38 smithi203 ceph-mon[36030]: Standby manager daemon smithi160.nalzlg started 2024-05-25T06:12:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:38 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/crt"}]: dispatch 2024-05-25T06:12:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:38 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:12:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:38 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi160.nalzlg/key"}]: dispatch 2024-05-25T06:12:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:38 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:12:40.152 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:40 smithi160 ceph-mon[29041]: pgmap v50: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 352 op/s 2024-05-25T06:12:40.152 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:40 smithi160 ceph-mon[29041]: mgrmap e27: smithi203.rotqex(active, since 94s), standbys: smithi160.nalzlg 2024-05-25T06:12:40.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:40 smithi203 ceph-mon[36030]: pgmap v50: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 352 op/s 2024-05-25T06:12:40.317 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:40 smithi203 ceph-mon[36030]: mgrmap e27: smithi203.rotqex(active, since 94s), standbys: smithi160.nalzlg 2024-05-25T06:12:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:42 smithi160 ceph-mon[29041]: pgmap v51: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 369 op/s 2024-05-25T06:12:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:42 smithi203 ceph-mon[36030]: pgmap v51: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 369 op/s 2024-05-25T06:12:43.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:43.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:43.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:12:43.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:12:43.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:43.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr fail", "who": "smithi203.rotqex"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr fail", "who": "smithi203.rotqex"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: osdmap e48: 6 total, 6 up, 6 in 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd='[{"prefix": "mgr fail", "who": "smithi203.rotqex"}]': finished 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: mgrmap e28: smithi160.nalzlg(active, starting, since 0.00693907s) 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:12:43.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:12:43.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:43.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:43.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:12:43.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:12:43.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:43 smithi160 ceph-mon[29041]: [25/May/2024:06:12:43] ENGINE Bus STARTING 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:43.771 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 172.21.15.203:0/1595062332' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr fail", "who": "smithi203.rotqex"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "mgr fail", "who": "smithi203.rotqex"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: osdmap e48: 6 total, 6 up, 6 in 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.24381 ' entity='mgr.smithi203.rotqex' cmd='[{"prefix": "mgr fail", "who": "smithi203.rotqex"}]': finished 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: mgrmap e28: smithi160.nalzlg(active, starting, since 0.00693907s) 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:12:43.772 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:12:43.773 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:43 smithi203 ceph-mon[36030]: [25/May/2024:06:12:43] ENGINE Bus STARTING 2024-05-25T06:12:44.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.675+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2404571506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fbe30 msgr2=0x7fd6180fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:44.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.675+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2404571506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fbe30 0x7fd6180fc240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6140049c0 tx=0x7fd614009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:44.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.675+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2404571506 shutdown_connections 2024-05-25T06:12:44.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.675+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2404571506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fcfa0 0x7fd6180fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:44.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.675+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2404571506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fbe30 0x7fd6180fc240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:44.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.676+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2404571506 >> 172.21.15.160:0/2404571506 conn(0x7fd6180f7440 msgr2=0x7fd6180f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:44.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.676+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2404571506 shutdown_connections 2024-05-25T06:12:44.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.676+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2404571506 wait complete. 2024-05-25T06:12:44.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.677+0000 7fd61f9c5700 1 Processor -- start 2024-05-25T06:12:44.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.677+0000 7fd61f9c5700 1 -- start start 2024-05-25T06:12:44.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.677+0000 7fd61f9c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fbe30 0x7fd61810b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:44.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.677+0000 7fd61f9c5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fcfa0 0x7fd61810bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:44.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.677+0000 7fd61f9c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61810c2e0 con 0x7fd6180fbe30 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.677+0000 7fd61f9c5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61810cb30 con 0x7fd6180fcfa0 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61cf60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fcfa0 0x7fd61810bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61cf60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fcfa0 0x7fd61810bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:43038/0 (socket says 172.21.15.160:43038) 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61cf60700 1 -- 172.21.15.160:0/2192399627 learned_addr learned my addr 172.21.15.160:0/2192399627 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61d761700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fbe30 0x7fd61810b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61d761700 1 -- 172.21.15.160:0/2192399627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fcfa0 msgr2=0x7fd61810bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61d761700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fcfa0 0x7fd61810bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.678+0000 7fd61d761700 1 -- 172.21.15.160:0/2192399627 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614005040 con 0x7fd6180fbe30 2024-05-25T06:12:44.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.679+0000 7fd61d761700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fbe30 0x7fd61810b750 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d5c0 tx=0x7fd614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:44.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.679+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd61401a3f0 con 0x7fd6180fbe30 2024-05-25T06:12:44.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.679+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd61401aa20 con 0x7fd6180fbe30 2024-05-25T06:12:44.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.679+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61810cdb0 con 0x7fd6180fbe30 2024-05-25T06:12:44.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.679+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6140199b0 con 0x7fd6180fbe30 2024-05-25T06:12:44.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.679+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61810d370 con 0x7fd6180fbe30 2024-05-25T06:12:44.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.682+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 29) v1 ==== 46241+0+0 (secure 0 0 0) 0x7fd61401ab90 con 0x7fd6180fbe30 2024-05-25T06:12:44.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.682+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61805f5c0 con 0x7fd6180fbe30 2024-05-25T06:12:44.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.682+0000 7fd60dffb700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd604038f10 0x7fd60403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:44.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.682+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd61404dc70 con 0x7fd6180fbe30 2024-05-25T06:12:44.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.683+0000 7fd61cf60700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd604038f10 0x7fd60403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:44.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.683+0000 7fd61cf60700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd604038f10 0x7fd60403b3c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd608008fd0 tx=0x7fd60800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:44.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:44.691+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fd614018e10 con 0x7fd6180fbe30 2024-05-25T06:12:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:44 smithi203 ceph-mon[36030]: [25/May/2024:06:12:43] ENGINE Serving on http://172.21.15.160:8765 2024-05-25T06:12:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:44 smithi203 ceph-mon[36030]: [25/May/2024:06:12:43] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:12:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:44 smithi203 ceph-mon[36030]: [25/May/2024:06:12:43] ENGINE Bus STARTED 2024-05-25T06:12:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:44 smithi203 ceph-mon[36030]: mgrmap e29: smithi160.nalzlg(active, since 1.01119s) 2024-05-25T06:12:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:44 smithi203 ceph-mon[36030]: pgmap v3: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:44 smithi160 ceph-mon[29041]: [25/May/2024:06:12:43] ENGINE Serving on http://172.21.15.160:8765 2024-05-25T06:12:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:44 smithi160 ceph-mon[29041]: [25/May/2024:06:12:43] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:12:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:44 smithi160 ceph-mon[29041]: [25/May/2024:06:12:43] ENGINE Bus STARTED 2024-05-25T06:12:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:44 smithi160 ceph-mon[29041]: mgrmap e29: smithi160.nalzlg(active, since 1.01119s) 2024-05-25T06:12:44.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:44 smithi160 ceph-mon[29041]: pgmap v3: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:45.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.033+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd61810d710 con 0x7fd604038f10 2024-05-25T06:12:45.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.036+0000 7fd60dffb700 1 -- 172.21.15.160:0/2192399627 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fd61810d710 con 0x7fd604038f10 2024-05-25T06:12:45.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd604038f10 msgr2=0x7fd60403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd604038f10 0x7fd60403b3c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd608008fd0 tx=0x7fd60800b040 comp rx=0 tx=0).stop 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fbe30 msgr2=0x7fd61810b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fbe30 0x7fd61810b750 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d5c0 tx=0x7fd614009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 shutdown_connections 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd604038f10 0x7fd60403b3c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6180fbe30 0x7fd61810b750 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 --2- 172.21.15.160:0/2192399627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6180fcfa0 0x7fd61810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 >> 172.21.15.160:0/2192399627 conn(0x7fd6180f7440 msgr2=0x7fd6180f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.040+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 shutdown_connections 2024-05-25T06:12:45.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.041+0000 7fd61f9c5700 1 -- 172.21.15.160:0/2192399627 wait complete. 2024-05-25T06:12:45.056 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:12:45.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 -- 172.21.15.160:0/2137590238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 msgr2=0x7f9fe40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 --2- 172.21.15.160:0/2137590238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe40fc0c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004940 tx=0x7f9fe0009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:45.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 -- 172.21.15.160:0/2137590238 shutdown_connections 2024-05-25T06:12:45.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 --2- 172.21.15.160:0/2137590238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe40fcf60 0x7f9fe40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 --2- 172.21.15.160:0/2137590238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe40fc0c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 -- 172.21.15.160:0/2137590238 >> 172.21.15.160:0/2137590238 conn(0x7f9fe40f7430 msgr2=0x7f9fe40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:45.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 -- 172.21.15.160:0/2137590238 shutdown_connections 2024-05-25T06:12:45.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.159+0000 7f9febd80700 1 -- 172.21.15.160:0/2137590238 wait complete. 2024-05-25T06:12:45.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9febd80700 1 Processor -- start 2024-05-25T06:12:45.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9febd80700 1 -- start start 2024-05-25T06:12:45.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9febd80700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe4110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:45.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9febd80700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe40fcf60 0x7f9fe4112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:45.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9febd80700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4066350 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9febd80700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4066490 con 0x7f9fe40fcf60 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe4110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe4110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:32796/0 (socket says 172.21.15.160:32796) 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 -- 172.21.15.160:0/2562569090 learned_addr learned my addr 172.21.15.160:0/2562569090 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe931b700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe40fcf60 0x7f9fe4112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 -- 172.21.15.160:0/2562569090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe40fcf60 msgr2=0x7f9fe4112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe40fcf60 0x7f9fe4112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 -- 172.21.15.160:0/2562569090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0005040 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.160+0000 7f9fe9b1c700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe4110700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b3b0 tx=0x7f9fe0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.161+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe000e040 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.161+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fe000bbf0 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.161+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe41131f0 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.161+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe001c630 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.161+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe4113780 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.162+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 29) v1 ==== 46241+0+0 (secure 0 0 0) 0x7f9fe000bd60 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.162+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe405f5c0 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.162+0000 7f9fd67fc700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f9fd0038e90 0x7f9fd003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:45.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.162+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9fe004d7c0 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.162+0000 7f9fe931b700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f9fd0038e90 0x7f9fd003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:45.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.163+0000 7f9fe931b700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f9fd0038e90 0x7f9fd003b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8004f40 tx=0x7f9fd800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:45.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.167+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f9fe00137a0 con 0x7f9fe40fbcb0 2024-05-25T06:12:45.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.463+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fe41012c0 con 0x7f9fd0038e90 2024-05-25T06:12:45.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.465+0000 7f9fd67fc700 1 -- 172.21.15.160:0/2562569090 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f9fe41012c0 con 0x7f9fd0038e90 2024-05-25T06:12:45.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f9fd0038e90 msgr2=0x7f9fd003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f9fd0038e90 0x7f9fd003b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8004f40 tx=0x7f9fd800b040 comp rx=0 tx=0).stop 2024-05-25T06:12:45.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 msgr2=0x7f9fe4110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe4110700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b3b0 tx=0x7f9fe0009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 shutdown_connections 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f9fd0038e90 0x7f9fd003b340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9fe40fbcb0 0x7f9fe4110700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 --2- 172.21.15.160:0/2562569090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe40fcf60 0x7f9fe4112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 >> 172.21.15.160:0/2562569090 conn(0x7f9fe40f7430 msgr2=0x7f9fe40f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 shutdown_connections 2024-05-25T06:12:45.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.467+0000 7f9febd80700 1 -- 172.21.15.160:0/2562569090 wait complete. 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.594+0000 7fdab9b6a700 1 -- 172.21.15.160:0/2163593329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 msgr2=0x7fdab40fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.594+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/2163593329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab40fd290 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004940 tx=0x7fdaa4009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.594+0000 7fdab9b6a700 1 -- 172.21.15.160:0/2163593329 shutdown_connections 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.594+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/2163593329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.594+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/2163593329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab40fd290 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.594+0000 7fdab9b6a700 1 -- 172.21.15.160:0/2163593329 >> 172.21.15.160:0/2163593329 conn(0x7fdab40f8540 msgr2=0x7fdab40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.595+0000 7fdab9b6a700 1 -- 172.21.15.160:0/2163593329 shutdown_connections 2024-05-25T06:12:45.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.595+0000 7fdab9b6a700 1 -- 172.21.15.160:0/2163593329 wait complete. 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.595+0000 7fdab9b6a700 1 Processor -- start 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.595+0000 7fdab9b6a700 1 -- start start 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab9b6a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab4071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab9b6a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab9b6a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4070860 con 0x7fdab40fce80 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab9b6a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40709d0 con 0x7fdab40fdff0 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab4071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab4071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45428/0 (socket says 172.21.15.160:45428) 2024-05-25T06:12:45.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:38458/0 (socket says 172.21.15.160:38458) 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 -- 172.21.15.160:0/3173369032 learned_addr learned my addr 172.21.15.160:0/3173369032 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 -- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 msgr2=0x7fdab4071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab4071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 -- 172.21.15.160:0/3173369032 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4005040 con 0x7fdab40fdff0 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.596+0000 7fdab2ffd700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40702b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800eb40 tx=0x7fdaa800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.597+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa8007cc0 con 0x7fdab40fdff0 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.597+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaa800f040 con 0x7fdab40fdff0 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.597+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab4070c50 con 0x7fdab40fdff0 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.597+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa800ca10 con 0x7fdab40fdff0 2024-05-25T06:12:45.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.597+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab41159e0 con 0x7fdab40fdff0 2024-05-25T06:12:45.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.598+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab405f5c0 con 0x7fdab40fdff0 2024-05-25T06:12:45.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.598+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 29) v1 ==== 46241+0+0 (secure 0 0 0) 0x7fdaa800cb70 con 0x7fdab40fdff0 2024-05-25T06:12:45.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.598+0000 7fda9bfff700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fda9c038e90 0x7fda9c03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:45.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.598+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdaa804c780 con 0x7fdab40fdff0 2024-05-25T06:12:45.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.598+0000 7fdab37fe700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fda9c038e90 0x7fda9c03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:45.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.599+0000 7fdab37fe700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fda9c038e90 0x7fda9c03b340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004890 tx=0x7fdaa4009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:45.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.605+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fdaa80095f0 con 0x7fdab40fdff0 2024-05-25T06:12:45.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.612+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 30) v1 ==== 46297+0+0 (secure 0 0 0) 0x7fdaa8022cd0 con 0x7fdab40fdff0 2024-05-25T06:12:45.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.879+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdab4102290 con 0x7fda9c038e90 2024-05-25T06:12:45.880 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:45 smithi160 ceph-mon[29041]: pgmap v4: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:45.880 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:45 smithi160 ceph-mon[29041]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.890+0000 7fda9bfff700 1 -- 172.21.15.160:0/3173369032 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdab4102290 con 0x7fda9c038e90 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (45s) 9s ago 9m 16.6M - 0.25.0 c8568f914cd2 2c52bb146e13 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (9m) 9s ago 9m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (8m) 12s ago 8m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (39s) 9s ago 9m 80.1M - 9.4.7 954c08fa6188 608a83ea61e4 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (4m) 9s ago 4m 70.9M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (4m) 9s ago 4m 137M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (4m) 12s ago 4m 85.4M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:12:45.890 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (4m) 12s ago 4m 151M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (16s) 9s ago 11m 114M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283 running (2m) 12s ago 8m 505M - 19.0.0-2843-gea0684c0 f9a9809cafd6 fbc025680ca5 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (11m) 9s ago 11m 52.1M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (8m) 12s ago 8m 37.2M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (51s) 9s ago 9m 16.4M - 1.5.0 0da6a335fe13 fbfdb6ff1f90 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (23s) 12s ago 8m 7982k - 1.5.0 0da6a335fe13 94608b0bb858 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (8m) 9s ago 7m 200M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (7m) 9s ago 7m 217M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (7m) 9s ago 7m 172M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (6m) 12s ago 6m 242M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (6m) 12s ago 6m 210M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (5m) 12s ago 5m 213M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:12:45.891 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (32s) 9s ago 9m 44.5M - 2.43.0 a07b618ecd1d 88bc63547022 2024-05-25T06:12:45.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fda9c038e90 msgr2=0x7fda9c03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fda9c038e90 0x7fda9c03b340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004890 tx=0x7fdaa4009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 msgr2=0x7fdab40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40702b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800eb40 tx=0x7fdaa800b040 comp rx=0 tx=0).stop 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 shutdown_connections 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fda9c038e90 0x7fda9c03b340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdab40fce80 0x7fdab4071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 --2- 172.21.15.160:0/3173369032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdab40fdff0 0x7fdab40702b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.892+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 >> 172.21.15.160:0/3173369032 conn(0x7fdab40f8540 msgr2=0x7fdab40f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.893+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 shutdown_connections 2024-05-25T06:12:45.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:45.893+0000 7fdab9b6a700 1 -- 172.21.15.160:0/3173369032 wait complete. 2024-05-25T06:12:45.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:45 smithi203 ceph-mon[36030]: pgmap v4: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:45.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:45 smithi203 ceph-mon[36030]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:46.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/2563130990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 msgr2=0x7fc9b40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/2563130990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40fc0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4004940 tx=0x7fc9a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/2563130990 shutdown_connections 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/2563130990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b40fcf90 0x7fc9b40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/2563130990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40fc0f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/2563130990 >> 172.21.15.160:0/2563130990 conn(0x7fc9b40f7440 msgr2=0x7fc9b40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/2563130990 shutdown_connections 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/2563130990 wait complete. 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.013+0000 7fc9b9dbb700 1 Processor -- start 2024-05-25T06:12:46.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b9dbb700 1 -- start start 2024-05-25T06:12:46.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b9dbb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:46.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b9dbb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b40fcf90 0x7fc9b40fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:46.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b9dbb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40fe860 con 0x7fc9b40fbce0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b9dbb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40fe9a0 con 0x7fc9b40fcf90 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45446/0 (socket says 172.21.15.160:45446) 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b37fe700 1 -- 172.21.15.160:0/462494849 learned_addr learned my addr 172.21.15.160:0/462494849 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b2ffd700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b40fcf90 0x7fc9b40fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b37fe700 1 -- 172.21.15.160:0/462494849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b40fcf90 msgr2=0x7fc9b40fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b37fe700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b40fcf90 0x7fc9b40fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.014+0000 7fc9b37fe700 1 -- 172.21.15.160:0/462494849 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a4005040 con 0x7fc9b40fbce0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.015+0000 7fc9b37fe700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40ffdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4013870 tx=0x7fc9a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.015+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a4021070 con 0x7fc9b40fbce0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.015+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9a4009430 con 0x7fc9b40fbce0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.015+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a400e070 con 0x7fc9b40fbce0 2024-05-25T06:12:46.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.015+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b40fec20 con 0x7fc9b40fbce0 2024-05-25T06:12:46.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.015+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b4115a00 con 0x7fc9b40fbce0 2024-05-25T06:12:46.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.016+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 30) v1 ==== 46297+0+0 (secure 0 0 0) 0x7fc9a4013af0 con 0x7fc9b40fbce0 2024-05-25T06:12:46.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.016+0000 7fc99bfff700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc99c038f10 0x7fc99c03b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:46.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.016+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b4101b20 con 0x7fc9b40fbce0 2024-05-25T06:12:46.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.016+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc9a404de00 con 0x7fc9b40fbce0 2024-05-25T06:12:46.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.017+0000 7fc9b2ffd700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc99c038f10 0x7fc99c03b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:46.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.017+0000 7fc9b2ffd700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc99c038f10 0x7fc99c03b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4100390 tx=0x7fc9a800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:46.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.022+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fc9a401c400 con 0x7fc9b40fbce0 2024-05-25T06:12:46.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.418+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc9b41048b0 con 0x7fc9b40fbce0 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.419+0000 7fc99bfff700 1 -- 172.21.15.160:0/462494849 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7fc9a4024310 con 0x7fc9b40fbce0 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:46.419 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 12, 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:12:46.420 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:12:46.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.421+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc99c038f10 msgr2=0x7fc99c03b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.421+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc99c038f10 0x7fc99c03b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4100390 tx=0x7fc9a800b040 comp rx=0 tx=0).stop 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.421+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 msgr2=0x7fc9b40ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.421+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40ffdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4013870 tx=0x7fc9a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 shutdown_connections 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc99c038f10 0x7fc99c03b3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9b40fbce0 0x7fc9b40ffdb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 --2- 172.21.15.160:0/462494849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b40fcf90 0x7fc9b40fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 >> 172.21.15.160:0/462494849 conn(0x7fc9b40f7440 msgr2=0x7fc9b40663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:46.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 shutdown_connections 2024-05-25T06:12:46.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.422+0000 7fc9b9dbb700 1 -- 172.21.15.160:0/462494849 wait complete. 2024-05-25T06:12:46.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.552+0000 7f64edffb700 1 -- 172.21.15.160:0/3199253078 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc013b70 con 0x7f64f00fcf70 2024-05-25T06:12:46.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 -- 172.21.15.160:0/3199253078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 msgr2=0x7f64f00fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 --2- 172.21.15.160:0/3199253078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f00fd3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64dc004940 tx=0x7f64dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 -- 172.21.15.160:0/3199253078 shutdown_connections 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 --2- 172.21.15.160:0/3199253078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f00fd3e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 --2- 172.21.15.160:0/3199253078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 0x7f64f00fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 -- 172.21.15.160:0/3199253078 >> 172.21.15.160:0/3199253078 conn(0x7f64f00f7420 msgr2=0x7f64f00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 -- 172.21.15.160:0/3199253078 shutdown_connections 2024-05-25T06:12:46.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.553+0000 7f64f5710700 1 -- 172.21.15.160:0/3199253078 wait complete. 2024-05-25T06:12:46.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64f5710700 1 Processor -- start 2024-05-25T06:12:46.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64f5710700 1 -- start start 2024-05-25T06:12:46.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64f5710700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 0x7f64f010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:46.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64f5710700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f01100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:46.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64f5710700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f01106b0 con 0x7f64f00fcf70 2024-05-25T06:12:46.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64f5710700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0110f00 con 0x7f64f00fbcc0 2024-05-25T06:12:46.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64ee7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f01100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:46.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64eeffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 0x7f64f010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:46.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64ee7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f01100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45472/0 (socket says 172.21.15.160:45472) 2024-05-25T06:12:46.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.554+0000 7f64ee7fc700 1 -- 172.21.15.160:0/3165354873 learned_addr learned my addr 172.21.15.160:0/3165354873 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:12:46.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64eeffd700 1 -- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 msgr2=0x7f64f01100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64eeffd700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f01100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64eeffd700 1 -- 172.21.15.160:0/3165354873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc005040 con 0x7f64f00fbcc0 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64eeffd700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 0x7f64f010fb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009fd0 tx=0x7f64e000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e0004e90 con 0x7f64f00fbcc0 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f0111180 con 0x7f64f00fbcc0 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64e000f040 con 0x7f64f00fbcc0 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e000ca50 con 0x7f64f00fbcc0 2024-05-25T06:12:46.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.555+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f01116c0 con 0x7f64f00fbcc0 2024-05-25T06:12:46.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.556+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 30) v1 ==== 46297+0+0 (secure 0 0 0) 0x7f64e000cc00 con 0x7f64f00fbcc0 2024-05-25T06:12:46.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.557+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f005f5c0 con 0x7f64f00fbcc0 2024-05-25T06:12:46.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.557+0000 7f64e77fe700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f64d4038f10 0x7f64d403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:12:46.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.557+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f64e004c810 con 0x7f64f00fbcc0 2024-05-25T06:12:46.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.557+0000 7f64ee7fc700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f64d4038f10 0x7f64d403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:12:46.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.557+0000 7f64ee7fc700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f64d4038f10 0x7f64d403b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009b30 tx=0x7f64dc009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:12:46.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.562+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f64e001b2c0 con 0x7f64f00fbcc0 2024-05-25T06:12:46.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.863+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64f0111a10 con 0x7f64d4038f10 2024-05-25T06:12:46.865 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:46 smithi160 ceph-mon[29041]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:46.865 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:46 smithi160 ceph-mon[29041]: mgrmap e30: smithi160.nalzlg(active, since 3s) 2024-05-25T06:12:46.865 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:46 smithi160 ceph-mon[29041]: from='client.24507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:46.865 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:46 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/462494849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:12:46.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.865+0000 7f64e77fe700 1 -- 172.21.15.160:0/3165354873 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f64f0111a10 con 0x7f64d4038f10 2024-05-25T06:12:46.865 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:12:46.865 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "2/2 daemons upgraded", 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "", 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:12:46.866 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.868+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f64d4038f10 msgr2=0x7f64d403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.868+0000 7f64f5710700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f64d4038f10 0x7f64d403b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009b30 tx=0x7f64dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.868+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 msgr2=0x7f64f010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.868+0000 7f64f5710700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 0x7f64f010fb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009fd0 tx=0x7f64e000b040 comp rx=0 tx=0).stop 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 shutdown_connections 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f00fbcc0 0x7f64f010fb70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f64d4038f10 0x7f64d403b3c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 --2- 172.21.15.160:0/3165354873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f00fcf70 0x7f64f01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 >> 172.21.15.160:0/3165354873 conn(0x7f64f00f7420 msgr2=0x7f64f00f7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 shutdown_connections 2024-05-25T06:12:46.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:12:46.869+0000 7f64f5710700 1 -- 172.21.15.160:0/3165354873 wait complete. 2024-05-25T06:12:46.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:46 smithi203 ceph-mon[36030]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:46.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:46 smithi203 ceph-mon[36030]: mgrmap e30: smithi160.nalzlg(active, since 3s) 2024-05-25T06:12:46.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:46 smithi203 ceph-mon[36030]: from='client.24507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:46.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:46 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/462494849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:12:47.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:47 smithi203 ceph-mon[36030]: pgmap v5: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:47.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:47 smithi203 ceph-mon[36030]: from='client.24515 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:47.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:47 smithi160 ceph-mon[29041]: pgmap v5: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:47.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:47 smithi160 ceph-mon[29041]: from='client.24515 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:12:49.840 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:49 smithi203 ceph-mon[36030]: pgmap v6: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:49.885 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:49 smithi160 ceph-mon[29041]: pgmap v6: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:12:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: pgmap v7: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 756 KiB/s rd, 847 KiB/s wr, 246 op/s 2024-05-25T06:12:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:12:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:12:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:12:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:12:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:51 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:12:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: pgmap v7: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 756 KiB/s rd, 847 KiB/s wr, 246 op/s 2024-05-25T06:12:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:12:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:12:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:12:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:12:51.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:51 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/2227890379' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:12:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:52 smithi203 ceph-mon[36030]: mgrmap e31: smithi160.nalzlg(active, since 8s), standbys: smithi203.rotqex 2024-05-25T06:12:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:52 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:12:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:52 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:52 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:52.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:52 smithi160 ceph-mon[29041]: mgrmap e31: smithi160.nalzlg(active, since 8s), standbys: smithi203.rotqex 2024-05-25T06:12:52.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:52 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:12:52.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:52 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:52.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:52 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:53.789 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:53 smithi160 ceph-mon[29041]: pgmap v8: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 588 KiB/s rd, 658 KiB/s wr, 191 op/s 2024-05-25T06:12:53.861 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:53 smithi203 ceph-mon[36030]: pgmap v8: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 588 KiB/s rd, 658 KiB/s wr, 191 op/s 2024-05-25T06:12:55.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:55 smithi160 ceph-mon[29041]: pgmap v9: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 305 op/s 2024-05-25T06:12:56.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:55 smithi203 ceph-mon[36030]: pgmap v9: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 305 op/s 2024-05-25T06:12:56.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:56 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:56.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:56 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:56 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:56 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:58 smithi203 ceph-mon[36030]: pgmap v10: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 932 KiB/s rd, 945 KiB/s wr, 279 op/s 2024-05-25T06:12:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:58 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:58 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:12:58 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:12:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:58 smithi160 ceph-mon[29041]: pgmap v10: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 932 KiB/s rd, 945 KiB/s wr, 279 op/s 2024-05-25T06:12:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:58 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:58 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:12:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:12:58 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:00 smithi203 ceph-mon[36030]: pgmap v11: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 932 KiB/s rd, 945 KiB/s wr, 279 op/s 2024-05-25T06:13:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:00 smithi160 ceph-mon[29041]: pgmap v11: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 932 KiB/s rd, 945 KiB/s wr, 279 op/s 2024-05-25T06:13:00.842 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-25T06:13:00.842 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-25T06:13:01.229 DEBUG:teuthology.parallel:result is None 2024-05-25T06:13:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:02 smithi203 ceph-mon[36030]: pgmap v12: 65 pgs: 65 active+clean; 270 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 416 op/s 2024-05-25T06:13:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:02 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:02 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:02 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:02 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:13:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:02 smithi160 ceph-mon[29041]: pgmap v12: 65 pgs: 65 active+clean; 270 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 416 op/s 2024-05-25T06:13:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:02 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:02.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:02 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:02.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:02 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:02.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:02 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:13:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:03 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:03.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:03 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:04 smithi160 ceph-mon[29041]: pgmap v13: 65 pgs: 65 active+clean; 270 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-25T06:13:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:04 smithi160 ceph-mon[29041]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:13:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:04 smithi160 ceph-mon[29041]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:13:04.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:04 smithi160 ceph-mon[29041]: Reconfiguring prometheus.smithi160 (dependencies changed)... 2024-05-25T06:13:04.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:04 smithi160 ceph-mon[29041]: Reconfiguring daemon prometheus.smithi160 on smithi160 2024-05-25T06:13:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:04 smithi203 ceph-mon[36030]: pgmap v13: 65 pgs: 65 active+clean; 270 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-25T06:13:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:04 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:13:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:04 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:13:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:04 smithi203 ceph-mon[36030]: Reconfiguring prometheus.smithi160 (dependencies changed)... 2024-05-25T06:13:04.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:04 smithi203 ceph-mon[36030]: Reconfiguring daemon prometheus.smithi160 on smithi160 2024-05-25T06:13:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:06 smithi203 ceph-mon[36030]: pgmap v14: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 458 op/s 2024-05-25T06:13:06.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:06 smithi160 ceph-mon[29041]: pgmap v14: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 458 op/s 2024-05-25T06:13:06.523 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-25T06:13:06.523 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-25T06:13:06.885 DEBUG:teuthology.parallel:result is None 2024-05-25T06:13:06.886 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-25T06:13:06.935 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-25T06:13:06.935 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-25T06:13:06.974 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-25T06:13:06.974 DEBUG:teuthology.parallel:result is None 2024-05-25T06:13:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:08 smithi203 ceph-mon[36030]: pgmap v15: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-25T06:13:08.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:08 smithi160 ceph-mon[29041]: pgmap v15: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-25T06:13:10.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:10 smithi160 ceph-mon[29041]: pgmap v16: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-25T06:13:10.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:10 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:10.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:10 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:10.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:10 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:13:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:10 smithi203 ceph-mon[36030]: pgmap v16: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-25T06:13:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:10 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:10 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:10 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:13:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:11 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:13:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:11 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:11 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:13:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:11 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:12 smithi203 ceph-mon[36030]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-05-25T06:13:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:12 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:13:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:12 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:13:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:12 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:12 smithi160 ceph-mon[29041]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-05-25T06:13:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:12 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:13:12.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:12 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:13:12.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:12 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:13 smithi203 ceph-mon[36030]: Upgrade: Updating mgr.smithi203.rotqex 2024-05-25T06:13:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:13 smithi203 ceph-mon[36030]: Deploying daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:13:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:13 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:13 smithi160 ceph-mon[29041]: Upgrade: Updating mgr.smithi203.rotqex 2024-05-25T06:13:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:13 smithi160 ceph-mon[29041]: Deploying daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:13:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:13 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:14.181 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:14 smithi203 ceph-mon[36030]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 958 KiB/s rd, 877 KiB/s wr, 289 op/s 2024-05-25T06:13:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:14 smithi160 ceph-mon[29041]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 958 KiB/s rd, 877 KiB/s wr, 289 op/s 2024-05-25T06:13:16.135 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:16 smithi203 ceph-mon[36030]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 974 KiB/s rd, 892 KiB/s wr, 292 op/s 2024-05-25T06:13:16.135 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:16 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:16.136 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:16 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:16.136 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:16 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:16 smithi160 ceph-mon[29041]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 974 KiB/s rd, 892 KiB/s wr, 292 op/s 2024-05-25T06:13:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:16 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:16.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:16 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:16.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:16 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:16.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.981+0000 7f2a2442c700 1 -- 172.21.15.160:0/727941559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 msgr2=0x7f2a1c0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.981+0000 7f2a2442c700 1 --2- 172.21.15.160:0/727941559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c0fe830 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009ce0 tx=0x7f2a0c004d10 comp rx=0 tx=0).stop 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 -- 172.21.15.160:0/727941559 shutdown_connections 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 --2- 172.21.15.160:0/727941559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a1c0fed70 0x7f2a1c0faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 --2- 172.21.15.160:0/727941559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c0fe830 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 -- 172.21.15.160:0/727941559 >> 172.21.15.160:0/727941559 conn(0x7f2a1c0f6be0 msgr2=0x7f2a1c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 -- 172.21.15.160:0/727941559 shutdown_connections 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 -- 172.21.15.160:0/727941559 wait complete. 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 Processor -- start 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.982+0000 7f2a2442c700 1 -- start start 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a2442c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a2442c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a1c0fed70 0x7f2a1c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a2442c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c10e540 con 0x7f2a1c0fe420 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a2442c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c10e6b0 con 0x7f2a1c0fed70 2024-05-25T06:13:16.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41816/0 (socket says 172.21.15.160:41816) 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 -- 172.21.15.160:0/3685380766 learned_addr learned my addr 172.21.15.160:0/3685380766 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a219c7700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a1c0fed70 0x7f2a1c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 -- 172.21.15.160:0/3685380766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a1c0fed70 msgr2=0x7f2a1c10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a1c0fed70 0x7f2a1c10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 -- 172.21.15.160:0/3685380766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c009940 con 0x7f2a1c0fe420 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a221c8700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c10da00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009380 tx=0x7f2a0c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c00e040 con 0x7f2a1c0fe420 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a0c0128e0 con 0x7f2a1c0fe420 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c1130b0 con 0x7f2a1c0fe420 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c01c930 con 0x7f2a1c0fe420 2024-05-25T06:13:16.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.983+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c113670 con 0x7f2a1c0fe420 2024-05-25T06:13:16.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.984+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c04c710 con 0x7f2a1c0fe420 2024-05-25T06:13:16.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.985+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 31) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f2a0c012a50 con 0x7f2a1c0fe420 2024-05-25T06:13:16.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.985+0000 7f2a1affd700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2a0406db70 0x7f2a04070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:16.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.986+0000 7f2a219c7700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2a0406db70 0x7f2a04070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:16.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.986+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2a0c08efd0 con 0x7f2a1c0fe420 2024-05-25T06:13:16.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.986+0000 7f2a219c7700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2a0406db70 0x7f2a04070020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009fd0 tx=0x7f2a1000b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:16.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:16.990+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f2a0c05c4b0 con 0x7f2a1c0fe420 2024-05-25T06:13:17.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.212+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a1c102470 con 0x7f2a0406db70 2024-05-25T06:13:17.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.214+0000 7f2a1affd700 1 -- 172.21.15.160:0/3685380766 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f2a1c102470 con 0x7f2a0406db70 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.217+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2a0406db70 msgr2=0x7f2a04070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.217+0000 7f2a2442c700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2a0406db70 0x7f2a04070020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009fd0 tx=0x7f2a1000b040 comp rx=0 tx=0).stop 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.217+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 msgr2=0x7f2a1c10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.217+0000 7f2a2442c700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c10da00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009380 tx=0x7f2a0c004d10 comp rx=0 tx=0).stop 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 shutdown_connections 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2a0406db70 0x7f2a04070020 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c0fe420 0x7f2a1c10da00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 --2- 172.21.15.160:0/3685380766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a1c0fed70 0x7f2a1c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 >> 172.21.15.160:0/3685380766 conn(0x7f2a1c0f6be0 msgr2=0x7f2a1c0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:17.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 shutdown_connections 2024-05-25T06:13:17.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.218+0000 7f2a2442c700 1 -- 172.21.15.160:0/3685380766 wait complete. 2024-05-25T06:13:17.231 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:13:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 -- 172.21.15.160:0/1816433708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fe420 msgr2=0x7f1af40fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/1816433708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fe420 0x7f1af40fe830 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4004940 tx=0x7f1ae4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 -- 172.21.15.160:0/1816433708 shutdown_connections 2024-05-25T06:13:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/1816433708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af40fed70 0x7f1af40faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/1816433708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fe420 0x7f1af40fe830 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 -- 172.21.15.160:0/1816433708 >> 172.21.15.160:0/1816433708 conn(0x7f1af40f6be0 msgr2=0x7f1af40f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:17.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 -- 172.21.15.160:0/1816433708 shutdown_connections 2024-05-25T06:13:17.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 -- 172.21.15.160:0/1816433708 wait complete. 2024-05-25T06:13:17.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 Processor -- start 2024-05-25T06:13:17.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.329+0000 7f1afa4b3700 1 -- start start 2024-05-25T06:13:17.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1afa4b3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af40fe420 0x7f1af4113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:17.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1afa4b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 0x7f1af4114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:17.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1afa4b3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4114a20 con 0x7f1af40fed70 2024-05-25T06:13:17.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1afa4b3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4114b90 con 0x7f1af40fe420 2024-05-25T06:13:17.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 0x7f1af4114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:17.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 0x7f1af4114420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41834/0 (socket says 172.21.15.160:41834) 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 -- 172.21.15.160:0/3962491897 learned_addr learned my addr 172.21.15.160:0/3962491897 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1af3fff700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af40fe420 0x7f1af4113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 -- 172.21.15.160:0/3962491897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af40fe420 msgr2=0x7f1af4113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af40fe420 0x7f1af4113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 -- 172.21.15.160:0/3962491897 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4005040 con 0x7f1af40fed70 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1aebfff700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 0x7f1af4114420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1adc002a50 tx=0x7f1adc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1adc007cc0 con 0x7f1af40fed70 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af41195f0 con 0x7f1af40fed70 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1adc00f040 con 0x7f1af40fed70 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1adc00da30 con 0x7f1af40fed70 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.330+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af4119bb0 con 0x7f1af40fed70 2024-05-25T06:13:17.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.331+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af4101b60 con 0x7f1af40fed70 2024-05-25T06:13:17.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.332+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 31) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f1adc00dbb0 con 0x7f1af40fed70 2024-05-25T06:13:17.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.332+0000 7f1af17fa700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1ae006db70 0x7f1ae0070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:17.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.332+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1adc08d1e0 con 0x7f1af40fed70 2024-05-25T06:13:17.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.332+0000 7f1af3fff700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1ae006db70 0x7f1ae0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:17.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.333+0000 7f1af3fff700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1ae006db70 0x7f1ae0070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b970 tx=0x7f1ae4009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:17.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.337+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f1adc05a6c0 con 0x7f1af40fed70 2024-05-25T06:13:17.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.557+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1af402b2f0 con 0x7f1ae006db70 2024-05-25T06:13:17.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.559+0000 7f1af17fa700 1 -- 172.21.15.160:0/3962491897 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f1af402b2f0 con 0x7f1ae006db70 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1ae006db70 msgr2=0x7f1ae0070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1ae006db70 0x7f1ae0070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b970 tx=0x7f1ae4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 msgr2=0x7f1af4114420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 0x7f1af4114420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1adc002a50 tx=0x7f1adc0046b0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 shutdown_connections 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af40fe420 0x7f1af4113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1ae006db70 0x7f1ae0070020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 --2- 172.21.15.160:0/3962491897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1af40fed70 0x7f1af4114420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.562+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 >> 172.21.15.160:0/3962491897 conn(0x7f1af40f6be0 msgr2=0x7f1af40f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.563+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 shutdown_connections 2024-05-25T06:13:17.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.563+0000 7f1afa4b3700 1 -- 172.21.15.160:0/3962491897 wait complete. 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 -- 172.21.15.160:0/4156479344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 msgr2=0x7f43e00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 --2- 172.21.15.160:0/4156479344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e00fe440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f43d4004940 tx=0x7f43d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 -- 172.21.15.160:0/4156479344 shutdown_connections 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 --2- 172.21.15.160:0/4156479344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e00fe440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 --2- 172.21.15.160:0/4156479344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43e00fce60 0x7f43e00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 -- 172.21.15.160:0/4156479344 >> 172.21.15.160:0/4156479344 conn(0x7f43e00f8540 msgr2=0x7f43e00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 -- 172.21.15.160:0/4156479344 shutdown_connections 2024-05-25T06:13:17.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.672+0000 7f43e527f700 1 -- 172.21.15.160:0/4156479344 wait complete. 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43e527f700 1 Processor -- start 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43e527f700 1 -- start start 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43e527f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43e00fce60 0x7f43e010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43e527f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43e527f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e010e460 con 0x7f43e00fdfd0 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43e527f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e010e5d0 con 0x7f43e00fce60 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43de7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.673+0000 7f43de7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41854/0 (socket says 172.21.15.160:41854) 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43de7fc700 1 -- 172.21.15.160:0/1783346004 learned_addr learned my addr 172.21.15.160:0/1783346004 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43deffd700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43e00fce60 0x7f43e010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:17.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43de7fc700 1 -- 172.21.15.160:0/1783346004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43e00fce60 msgr2=0x7f43e010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43de7fc700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43e00fce60 0x7f43e010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43de7fc700 1 -- 172.21.15.160:0/1783346004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d4005040 con 0x7f43e00fdfd0 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43de7fc700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e010de60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009150 tx=0x7f43d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d400e040 con 0x7f43e00fdfd0 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43d4004290 con 0x7f43e00fdfd0 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e0112fd0 con 0x7f43e00fdfd0 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d401c630 con 0x7f43e00fdfd0 2024-05-25T06:13:17.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.674+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e0113560 con 0x7f43e00fdfd0 2024-05-25T06:13:17.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.675+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e004c710 con 0x7f43e00fdfd0 2024-05-25T06:13:17.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.676+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 31) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f43d400bbf0 con 0x7f43e00fdfd0 2024-05-25T06:13:17.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.676+0000 7f43c77fe700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f43c806db70 0x7f43c8070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:17.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.676+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f43d408dee0 con 0x7f43e00fdfd0 2024-05-25T06:13:17.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.676+0000 7f43deffd700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f43c806db70 0x7f43c8070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:17.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.677+0000 7f43deffd700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f43c806db70 0x7f43c8070020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004f40 tx=0x7f43d000b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:17.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.681+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f43d405b4e0 con 0x7f43e00fdfd0 2024-05-25T06:13:17.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.889+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f43e01023b0 con 0x7f43c806db70 2024-05-25T06:13:17.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.901+0000 7f43c77fe700 1 -- 172.21.15.160:0/1783346004 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f43e01023b0 con 0x7f43c806db70 2024-05-25T06:13:17.901 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (77s) 27s ago 10m 17.8M - 0.25.0 c8568f914cd2 2c52bb146e13 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (10m) 27s ago 10m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (9m) 22s ago 9m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (71s) 27s ago 9m 82.3M - 9.4.7 954c08fa6188 608a83ea61e4 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (4m) 27s ago 4m 71.8M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (4m) 27s ago 4m 138M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (4m) 22s ago 4m 69.8M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (4m) 22s ago 4m 130M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (48s) 27s ago 11m 479M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 starting - - - - 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (11m) 27s ago 11m 55.5M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (9m) 22s ago 9m 40.6M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (83s) 27s ago 10m 15.7M - 1.5.0 0da6a335fe13 fbfdb6ff1f90 2024-05-25T06:13:17.902 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (55s) 22s ago 9m 13.3M - 1.5.0 0da6a335fe13 94608b0bb858 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (8m) 27s ago 8m 205M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (8m) 27s ago 8m 217M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (7m) 27s ago 7m 182M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (7m) 22s ago 7m 256M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (6m) 22s ago 6m 219M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (6m) 22s ago 6m 219M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:13:17.903 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 starting - - - - 2024-05-25T06:13:17.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f43c806db70 msgr2=0x7f43c8070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f43c806db70 0x7f43c8070020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004f40 tx=0x7f43d000b040 comp rx=0 tx=0).stop 2024-05-25T06:13:17.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 msgr2=0x7f43e010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e010de60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009150 tx=0x7f43d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 shutdown_connections 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43e00fce60 0x7f43e010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f43c806db70 0x7f43c8070020 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 --2- 172.21.15.160:0/1783346004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43e00fdfd0 0x7f43e010de60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 >> 172.21.15.160:0/1783346004 conn(0x7f43e00f8540 msgr2=0x7f43e00f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 shutdown_connections 2024-05-25T06:13:17.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:17.904+0000 7f43e527f700 1 -- 172.21.15.160:0/1783346004 wait complete. 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 -- 172.21.15.160:0/2762486742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fcf70 msgr2=0x7fae6c0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 --2- 172.21.15.160:0/2762486742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fd3e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae5c004940 tx=0x7fae5c009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 -- 172.21.15.160:0/2762486742 shutdown_connections 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 --2- 172.21.15.160:0/2762486742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fd3e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 --2- 172.21.15.160:0/2762486742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 -- 172.21.15.160:0/2762486742 >> 172.21.15.160:0/2762486742 conn(0x7fae6c0f7420 msgr2=0x7fae6c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 -- 172.21.15.160:0/2762486742 shutdown_connections 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.026+0000 7fae7361d700 1 -- 172.21.15.160:0/2762486742 wait complete. 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae7361d700 1 Processor -- start 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae7361d700 1 -- start start 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae7361d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:18.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae7361d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae7361d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0fe840 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae7361d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0fe980 con 0x7fae6c0fcf70 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae713b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae713b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41872/0 (socket says 172.21.15.160:41872) 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae713b9700 1 -- 172.21.15.160:0/3079195858 learned_addr learned my addr 172.21.15.160:0/3079195858 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae70bb8700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae713b9700 1 -- 172.21.15.160:0/3079195858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fcf70 msgr2=0x7fae6c0fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.027+0000 7fae713b9700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae713b9700 1 -- 172.21.15.160:0/3079195858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c005040 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae70bb8700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae713b9700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0ffd90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae6800eb40 tx=0x7fae6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:18.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae68007cc0 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c0fec00 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae6800f040 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6800ca10 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.028+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c0ff110 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.029+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c101b00 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.030+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 31) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fae6800cc80 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.030+0000 7fae61ffb700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fae5806db60 0x7fae58070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.031+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fae6808d010 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.031+0000 7fae70bb8700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fae5806db60 0x7fae58070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:18.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.031+0000 7fae70bb8700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fae5806db60 0x7fae58070010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b0d0 tx=0x7fae5c009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:18.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.035+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fae68056a70 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:18 smithi160 ceph-mon[29041]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 349 KiB/s rd, 312 KiB/s wr, 106 op/s 2024-05-25T06:13:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:18 smithi160 ceph-mon[29041]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:18.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.321+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae6c104930 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.322+0000 7fae61ffb700 1 -- 172.21.15.160:0/3079195858 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7fae6805a130 con 0x7fae6c0fbcc0 2024-05-25T06:13:18.322 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:13:18.322 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:13:18.322 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:13:18.322 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:18.322 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 12, 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:13:18.323 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fae5806db60 msgr2=0x7fae58070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fae5806db60 0x7fae58070010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b0d0 tx=0x7fae5c009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 msgr2=0x7fae6c0ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0ffd90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae6800eb40 tx=0x7fae6800b040 comp rx=0 tx=0).stop 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 shutdown_connections 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fae5806db60 0x7fae58070010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae6c0fbcc0 0x7fae6c0ffd90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 --2- 172.21.15.160:0/3079195858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae6c0fcf70 0x7fae6c0fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.324+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 >> 172.21.15.160:0/3079195858 conn(0x7fae6c0f7420 msgr2=0x7fae6c066350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.325+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 shutdown_connections 2024-05-25T06:13:18.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.325+0000 7fae7361d700 1 -- 172.21.15.160:0/3079195858 wait complete. 2024-05-25T06:13:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:18 smithi203 ceph-mon[36030]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 349 KiB/s rd, 312 KiB/s wr, 106 op/s 2024-05-25T06:13:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:18 smithi203 ceph-mon[36030]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 -- 172.21.15.160:0/4175351613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 msgr2=0x7f2abc0fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/4175351613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc0fd2a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4004940 tx=0x7f2ab4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 -- 172.21.15.160:0/4175351613 shutdown_connections 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/4175351613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2abc0fe090 0x7f2abc0fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/4175351613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc0fd2a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 -- 172.21.15.160:0/4175351613 >> 172.21.15.160:0/4175351613 conn(0x7f2abc0f85c0 msgr2=0x7f2abc0fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 -- 172.21.15.160:0/4175351613 shutdown_connections 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.430+0000 7f2ac43f7700 1 -- 172.21.15.160:0/4175351613 wait complete. 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac43f7700 1 Processor -- start 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac43f7700 1 -- start start 2024-05-25T06:13:18.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac43f7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac43f7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2abc0fe090 0x7f2abc10dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac43f7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc10e5b0 con 0x7f2abc0fce90 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac43f7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc10ee00 con 0x7f2abc0fe090 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac2193700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac2193700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc10da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41898/0 (socket says 172.21.15.160:41898) 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac2193700 1 -- 172.21.15.160:0/1756134111 learned_addr learned my addr 172.21.15.160:0/1756134111 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac1992700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2abc0fe090 0x7f2abc10dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac2193700 1 -- 172.21.15.160:0/1756134111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2abc0fe090 msgr2=0x7f2abc10dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac2193700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2abc0fe090 0x7f2abc10dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.431+0000 7f2ac2193700 1 -- 172.21.15.160:0/1756134111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4005040 con 0x7f2abc0fce90 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.432+0000 7f2ac2193700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc10da70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b970 tx=0x7f2ab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.432+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ab400e040 con 0x7f2abc0fce90 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.432+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ab400bbf0 con 0x7f2abc0fce90 2024-05-25T06:13:18.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.432+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc10f080 con 0x7f2abc0fce90 2024-05-25T06:13:18.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.432+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ab401c3f0 con 0x7f2abc0fce90 2024-05-25T06:13:18.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.432+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc10f5c0 con 0x7f2abc0fce90 2024-05-25T06:13:18.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.433+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc05f5c0 con 0x7f2abc0fce90 2024-05-25T06:13:18.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.433+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 31) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f2ab400bd60 con 0x7f2abc0fce90 2024-05-25T06:13:18.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.434+0000 7f2abaffd700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2aa406db60 0x7f2aa4070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:18.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.434+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2ab408e680 con 0x7f2abc0fce90 2024-05-25T06:13:18.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.434+0000 7f2ac1992700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2aa406db60 0x7f2aa4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:18.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.434+0000 7f2ac1992700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2aa406db60 0x7f2aa4070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0004f40 tx=0x7f2ab000b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:18.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.438+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f2ab405bc80 con 0x7f2abc0fce90 2024-05-25T06:13:18.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.665+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2abc10fa00 con 0x7f2aa406db60 2024-05-25T06:13:18.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.666+0000 7f2abaffd700 1 -- 172.21.15.160:0/1756134111 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f2abc10fa00 con 0x7f2aa406db60 2024-05-25T06:13:18.666 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:13:18.666 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:13:18.666 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:13:18.666 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:13:18.666 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:13:18.667 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "", 2024-05-25T06:13:18.667 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mgr daemons", 2024-05-25T06:13:18.667 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:13:18.667 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2aa406db60 msgr2=0x7f2aa4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2aa406db60 0x7f2aa4070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0004f40 tx=0x7f2ab000b040 comp rx=0 tx=0).stop 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 msgr2=0x7f2abc10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc10da70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b970 tx=0x7f2ab4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 shutdown_connections 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2aa406db60 0x7f2aa4070010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2abc0fce90 0x7f2abc10da70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 --2- 172.21.15.160:0/1756134111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2abc0fe090 0x7f2abc10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 >> 172.21.15.160:0/1756134111 conn(0x7f2abc0f85c0 msgr2=0x7f2abc0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 shutdown_connections 2024-05-25T06:13:18.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:18.668+0000 7f2ac43f7700 1 -- 172.21.15.160:0/1756134111 wait complete. 2024-05-25T06:13:19.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:19 smithi203 ceph-mon[36030]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:19.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:19 smithi203 ceph-mon[36030]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:19.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:19 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/3079195858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:19.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:19 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:19.295 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:19 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:19 smithi160 ceph-mon[29041]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:19 smithi160 ceph-mon[29041]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:19 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/3079195858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:19.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:19 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:19.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:19 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:20 smithi203 ceph-mon[36030]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 349 KiB/s rd, 312 KiB/s wr, 106 op/s 2024-05-25T06:13:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:20 smithi203 ceph-mon[36030]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:20 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:20 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:20.412 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:20 smithi160 ceph-mon[29041]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 349 KiB/s rd, 312 KiB/s wr, 106 op/s 2024-05-25T06:13:20.412 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:20 smithi160 ceph-mon[29041]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:20.412 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:20 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:20.412 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:20 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:22.196 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:22 smithi203 ceph-mon[36030]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 351 KiB/s rd, 313 KiB/s wr, 109 op/s 2024-05-25T06:13:22.196 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:22 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:22.196 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:22 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:22 smithi160 ceph-mon[29041]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 351 KiB/s rd, 313 KiB/s wr, 109 op/s 2024-05-25T06:13:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:22 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:22 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:23 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex restarted 2024-05-25T06:13:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:23 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:13:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:23 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:13:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:23 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:13:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:23 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:13:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:23 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:13:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:23 smithi160 ceph-mon[29041]: Standby manager daemon smithi203.rotqex restarted 2024-05-25T06:13:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:23 smithi160 ceph-mon[29041]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:13:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:23 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:13:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:23 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:13:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:23 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:13:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:23 smithi160 ceph-mon[29041]: from='mgr.? 172.21.15.203:0/1359971819' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:13:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 16 KiB/s wr, 5 op/s 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: mgrmap e32: smithi160.nalzlg(active, since 40s), standbys: smithi203.rotqex 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi160.nalzlg"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi160.nalzlg"}]': finished 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi203.rotqex"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi203.rotqex"}]': finished 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 16 KiB/s wr, 5 op/s 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: mgrmap e32: smithi160.nalzlg(active, since 40s), standbys: smithi203.rotqex 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi160.nalzlg"}]: dispatch 2024-05-25T06:13:24.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi160.nalzlg"}]': finished 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi203.rotqex"}]: dispatch 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi203.rotqex"}]': finished 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:24.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:24.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all mgr 2024-05-25T06:13:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all rgw 2024-05-25T06:13:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all rbd-mirror 2024-05-25T06:13:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-25T06:13:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all ceph-exporter 2024-05-25T06:13:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all iscsi 2024-05-25T06:13:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all nfs 2024-05-25T06:13:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:25 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all nvmeof 2024-05-25T06:13:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all mgr 2024-05-25T06:13:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all rgw 2024-05-25T06:13:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all rbd-mirror 2024-05-25T06:13:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-25T06:13:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all ceph-exporter 2024-05-25T06:13:25.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all iscsi 2024-05-25T06:13:25.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all nfs 2024-05-25T06:13:25.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:25 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all nvmeof 2024-05-25T06:13:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:26 smithi203 ceph-mon[36030]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-05-25T06:13:26.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:26 smithi160 ceph-mon[29041]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-05-25T06:13:27.301 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:27 smithi160 ceph-mon[29041]: Upgrade: Updating node-exporter.smithi160 (1/2) 2024-05-25T06:13:27.301 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:27 smithi160 ceph-mon[29041]: Deploying daemon node-exporter.smithi160 on smithi160 2024-05-25T06:13:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:27 smithi203 ceph-mon[36030]: Upgrade: Updating node-exporter.smithi160 (1/2) 2024-05-25T06:13:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:27 smithi203 ceph-mon[36030]: Deploying daemon node-exporter.smithi160 on smithi160 2024-05-25T06:13:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:28 smithi160 ceph-mon[29041]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:13:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:28 smithi203 ceph-mon[36030]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-25T06:13:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:29 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:29 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:29.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:29 smithi203 ceph-mon[36030]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-25T06:13:29.939 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:29 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:29.939 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:29 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:29.939 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:29 smithi160 ceph-mon[29041]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-25T06:13:30.881 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:30 smithi160 ceph-mon[29041]: Upgrade: Updating node-exporter.smithi203 (2/2) 2024-05-25T06:13:30.882 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:30 smithi160 ceph-mon[29041]: Deploying daemon node-exporter.smithi203 on smithi203 2024-05-25T06:13:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:30 smithi203 ceph-mon[36030]: Upgrade: Updating node-exporter.smithi203 (2/2) 2024-05-25T06:13:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:30 smithi203 ceph-mon[36030]: Deploying daemon node-exporter.smithi203 on smithi203 2024-05-25T06:13:31.896 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:31 smithi203 ceph-mon[36030]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-25T06:13:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:31 smithi160 ceph-mon[29041]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-25T06:13:34.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:33 smithi203 ceph-mon[36030]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-25T06:13:34.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:33 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:34.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:33 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:34.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:33 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:34.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:33 smithi160 ceph-mon[29041]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-25T06:13:34.223 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:33 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:34.223 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:33 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:34.223 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:33 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:36.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:35 smithi203 ceph-mon[36030]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-25T06:13:36.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:35 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:36.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:35 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:36.212 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:35 smithi160 ceph-mon[29041]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-25T06:13:36.212 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:35 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:36.212 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:35 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:37 smithi203 ceph-mon[36030]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-05-25T06:13:38.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:37 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:38.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:37 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:37 smithi160 ceph-mon[29041]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-05-25T06:13:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:37 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:37 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:39.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:39 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:39.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:39 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:39 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:39 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:40.377 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:40 smithi203 ceph-mon[36030]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-25T06:13:40.377 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:40 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:40.377 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:40 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:40.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:40 smithi160 ceph-mon[29041]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-25T06:13:40.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:40 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:40.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:40 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:41 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:41 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:41 smithi160 ceph-mon[29041]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-25T06:13:41.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:41 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:41.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:41 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:41.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:41 smithi203 ceph-mon[36030]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-25T06:13:43.858 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-25T06:13:43.858 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:43.859 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:43 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-25T06:13:44.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:44.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:44.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:43 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:45.940 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:45 smithi160 ceph-mon[29041]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-25T06:13:45.940 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:45 smithi160 ceph-mon[29041]: Upgrade: Updating prometheus.smithi160 2024-05-25T06:13:45.940 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:45 smithi160 ceph-mon[29041]: Deploying daemon prometheus.smithi160 on smithi160 2024-05-25T06:13:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:45 smithi203 ceph-mon[36030]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-25T06:13:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:45 smithi203 ceph-mon[36030]: Upgrade: Updating prometheus.smithi160 2024-05-25T06:13:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:45 smithi203 ceph-mon[36030]: Deploying daemon prometheus.smithi160 on smithi160 2024-05-25T06:13:47.935 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:47 smithi160 ceph-mon[29041]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:47.950 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:47 smithi203 ceph-mon[36030]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 -- 172.21.15.160:0/295926007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fce60 msgr2=0x7fa8fc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 --2- 172.21.15.160:0/295926007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc0fd270 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004940 tx=0x7fa8ec009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 -- 172.21.15.160:0/295926007 shutdown_connections 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 --2- 172.21.15.160:0/295926007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 --2- 172.21.15.160:0/295926007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc0fd270 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 -- 172.21.15.160:0/295926007 >> 172.21.15.160:0/295926007 conn(0x7fa8fc0f8540 msgr2=0x7fa8fc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 -- 172.21.15.160:0/295926007 shutdown_connections 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.777+0000 7fa901b91700 1 -- 172.21.15.160:0/295926007 wait complete. 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa901b91700 1 Processor -- start 2024-05-25T06:13:48.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa901b91700 1 -- start start 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa901b91700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa901b91700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa901b91700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc10e4a0 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa901b91700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc10e610 con 0x7fa8fc0fce60 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa8faffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa8faffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60148/0 (socket says 172.21.15.160:60148) 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.778+0000 7fa8faffd700 1 -- 172.21.15.160:0/138892282 learned_addr learned my addr 172.21.15.160:0/138892282 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8fb7fe700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8faffd700 1 -- 172.21.15.160:0/138892282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fce60 msgr2=0x7fa8fc10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8faffd700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:48.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8faffd700 1 -- 172.21.15.160:0/138892282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec005040 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8fb7fe700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8faffd700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc10dea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000c810 tx=0x7fa8f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8f0012810 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc113070 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8f0012e40 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc113630 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.779+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8f001b410 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.780+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc04c710 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.781+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 32) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fa8f0011810 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.781+0000 7fa8e3fff700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fa8e406db70 0x7fa8e4070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:48.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.781+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa8f008d720 con 0x7fa8fc0fdfd0 2024-05-25T06:13:48.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.781+0000 7fa8fb7fe700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fa8e406db70 0x7fa8e4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:48.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.782+0000 7fa8fb7fe700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fa8e406db70 0x7fa8e4070020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec000c00 tx=0x7fa8ec009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:48.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:48.786+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fa8f005ad10 con 0x7fa8fc0fdfd0 2024-05-25T06:13:49.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.004+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8fc102330 con 0x7fa8e406db70 2024-05-25T06:13:49.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.005+0000 7fa8e3fff700 1 -- 172.21.15.160:0/138892282 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fa8fc102330 con 0x7fa8e406db70 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fa8e406db70 msgr2=0x7fa8e4070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fa8e406db70 0x7fa8e4070020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec000c00 tx=0x7fa8ec009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 msgr2=0x7fa8fc10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc10dea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000c810 tx=0x7fa8f0004470 comp rx=0 tx=0).stop 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 shutdown_connections 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8fc0fce60 0x7fa8fc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fa8e406db70 0x7fa8e4070020 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 --2- 172.21.15.160:0/138892282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8fc0fdfd0 0x7fa8fc10dea0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 >> 172.21.15.160:0/138892282 conn(0x7fa8fc0f8540 msgr2=0x7fa8fc0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:49.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 shutdown_connections 2024-05-25T06:13:49.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.008+0000 7fa901b91700 1 -- 172.21.15.160:0/138892282 wait complete. 2024-05-25T06:13:49.019 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 -- 172.21.15.160:0/2815649765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fdfd0 msgr2=0x7f10c00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 --2- 172.21.15.160:0/2815649765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fdfd0 0x7f10c00fe440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10b4004940 tx=0x7f10b4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 -- 172.21.15.160:0/2815649765 shutdown_connections 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 --2- 172.21.15.160:0/2815649765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fdfd0 0x7f10c00fe440 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 --2- 172.21.15.160:0/2815649765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00fce60 0x7f10c00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 -- 172.21.15.160:0/2815649765 >> 172.21.15.160:0/2815649765 conn(0x7f10c00f8540 msgr2=0x7f10c00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 -- 172.21.15.160:0/2815649765 shutdown_connections 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.112+0000 7f10c63ea700 1 -- 172.21.15.160:0/2815649765 wait complete. 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10c63ea700 1 Processor -- start 2024-05-25T06:13:49.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10c63ea700 1 -- start start 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10c63ea700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 0x7f10c0071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10c63ea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00fdfd0 0x7f10c00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10c63ea700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c0072200 con 0x7f10c00fce60 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10c63ea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00707f0 con 0x7f10c00fdfd0 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 0x7f10c0071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 0x7f10c0071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60174/0 (socket says 172.21.15.160:60174) 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10bffff700 1 -- 172.21.15.160:0/724910288 learned_addr learned my addr 172.21.15.160:0/724910288 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.113+0000 7f10bf7fe700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00fdfd0 0x7f10c00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bffff700 1 -- 172.21.15.160:0/724910288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00fdfd0 msgr2=0x7f10c00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bffff700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00fdfd0 0x7f10c00702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bffff700 1 -- 172.21.15.160:0/724910288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b4005040 con 0x7f10c00fce60 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bffff700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 0x7f10c0071c00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10b0002a80 tx=0x7f10b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b0007cc0 con 0x7f10c00fce60 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c0070aa0 con 0x7f10c00fce60 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10b000f040 con 0x7f10c00fce60 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b000dac0 con 0x7f10c00fce60 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.114+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c0071060 con 0x7f10c00fce60 2024-05-25T06:13:49.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.115+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c004c710 con 0x7f10c00fce60 2024-05-25T06:13:49.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.115+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 32) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f10b000dc20 con 0x7f10c00fce60 2024-05-25T06:13:49.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.116+0000 7f10bcff9700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f10a806db70 0x7f10a8070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.116+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f10b008d0a0 con 0x7f10c00fce60 2024-05-25T06:13:49.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.116+0000 7f10bf7fe700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f10a806db70 0x7f10a8070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.116+0000 7f10bf7fe700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f10a806db70 0x7f10a8070020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f10b400b3b0 tx=0x7f10b4009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:49.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.120+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f10b005a580 con 0x7f10c00fce60 2024-05-25T06:13:49.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.351+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10c01022a0 con 0x7f10a806db70 2024-05-25T06:13:49.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.352+0000 7f10bcff9700 1 -- 172.21.15.160:0/724910288 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f10c01022a0 con 0x7f10a806db70 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f10a806db70 msgr2=0x7f10a8070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f10a806db70 0x7f10a8070020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f10b400b3b0 tx=0x7f10b4009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 msgr2=0x7f10c0071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 0x7f10c0071c00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10b0002a80 tx=0x7f10b00046b0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 shutdown_connections 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f10a806db70 0x7f10a8070020 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c00fce60 0x7f10c0071c00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 --2- 172.21.15.160:0/724910288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00fdfd0 0x7f10c00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 >> 172.21.15.160:0/724910288 conn(0x7f10c00f8540 msgr2=0x7f10c00f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 shutdown_connections 2024-05-25T06:13:49.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.355+0000 7f10c63ea700 1 -- 172.21.15.160:0/724910288 wait complete. 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- 172.21.15.160:0/414441841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 msgr2=0x7f8acc0faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/414441841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc0faf60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8abc004940 tx=0x7f8abc009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- 172.21.15.160:0/414441841 shutdown_connections 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/414441841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc0faf60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/414441841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8acc0fe4d0 0x7f8acc0fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- 172.21.15.160:0/414441841 >> 172.21.15.160:0/414441841 conn(0x7f8acc0f6bd0 msgr2=0x7f8acc0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- 172.21.15.160:0/414441841 shutdown_connections 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- 172.21.15.160:0/414441841 wait complete. 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 Processor -- start 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- start start 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8acc0fe4d0 0x7f8acc10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc10e420 con 0x7f8acc0fee20 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.461+0000 7f8ad44bd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc10e590 con 0x7f8acc0fe4d0 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60188/0 (socket says 172.21.15.160:60188) 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 -- 172.21.15.160:0/475524223 learned_addr learned my addr 172.21.15.160:0/475524223 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad2259700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8acc0fe4d0 0x7f8acc10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 -- 172.21.15.160:0/475524223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8acc0fe4d0 msgr2=0x7f8acc10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8acc0fe4d0 0x7f8acc10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 -- 172.21.15.160:0/475524223 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc005040 con 0x7f8acc0fee20 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad1a58700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc10de20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009150 tx=0x7f8abc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8abc00e040 con 0x7f8acc0fee20 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8abc004290 con 0x7f8acc0fee20 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc112f90 con 0x7f8acc0fee20 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8abc01c630 con 0x7f8acc0fee20 2024-05-25T06:13:49.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.462+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc113520 con 0x7f8acc0fee20 2024-05-25T06:13:49.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.463+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc04c710 con 0x7f8acc0fee20 2024-05-25T06:13:49.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.464+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 32) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f8abc00bbf0 con 0x7f8acc0fee20 2024-05-25T06:13:49.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.464+0000 7f8ac2ffd700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f8ab806db70 0x7f8ab8070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.464+0000 7f8ad2259700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f8ab806db70 0x7f8ab8070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.464+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8abc08dfe0 con 0x7f8acc0fee20 2024-05-25T06:13:49.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.465+0000 7f8ad2259700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f8ab806db70 0x7f8ab8070020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8004f40 tx=0x7f8ac800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:49.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.469+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f8abc05b540 con 0x7f8acc0fee20 2024-05-25T06:13:49.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.694+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8acc102560 con 0x7f8ab806db70 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.705+0000 7f8ac2ffd700 1 -- 172.21.15.160:0/475524223 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8acc102560 con 0x7f8ab806db70 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (109s) 11s ago 10m 25.7M - 0.25.0 c8568f914cd2 2c52bb146e13 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (11m) 11s ago 10m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (10m) 13s ago 10m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (103s) 11s ago 10m 81.4M - 9.4.7 954c08fa6188 608a83ea61e4 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (5m) 11s ago 5m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (5m) 11s ago 5m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (5m) 13s ago 5m 61.2M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:13:49.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (5m) 13s ago 5m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (80s) 11s ago 12m 497M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (34s) 13s ago 9m 452M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (12m) 11s ago 12m 56.8M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (9m) 13s ago 9m 41.7M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (21s) 11s ago 10m 7247k - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (17s) 13s ago 9m 3644k - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (9m) 11s ago 9m 216M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (8m) 11s ago 8m 224M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (8m) 11s ago 8m 189M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (7m) 13s ago 7m 267M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (7m) 13s ago 7m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (6m) 13s ago 6m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:13:49.707 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (40s) 11s ago 10m 48.7M - 2.43.0 a07b618ecd1d ddbf6badba8f 2024-05-25T06:13:49.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f8ab806db70 msgr2=0x7f8ab8070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f8ab806db70 0x7f8ab8070020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8004f40 tx=0x7f8ac800b040 comp rx=0 tx=0).stop 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 msgr2=0x7f8acc10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc10de20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009150 tx=0x7f8abc009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 shutdown_connections 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8acc0fe4d0 0x7f8acc10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f8ab806db70 0x7f8ab8070020 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 --2- 172.21.15.160:0/475524223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8acc0fee20 0x7f8acc10de20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 >> 172.21.15.160:0/475524223 conn(0x7f8acc0f6bd0 msgr2=0x7f8acc0f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 shutdown_connections 2024-05-25T06:13:49.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.707+0000 7f8ad44bd700 1 -- 172.21.15.160:0/475524223 wait complete. 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.818+0000 7f28b4b1b700 1 -- 172.21.15.160:0/3417299227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fbcb0 msgr2=0x7f28b00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.818+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/3417299227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fbcb0 0x7f28b00fc0c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28a0004940 tx=0x7f28a0009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.818+0000 7f28b4b1b700 1 -- 172.21.15.160:0/3417299227 shutdown_connections 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.818+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/3417299227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b00fcf60 0x7f28b00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.818+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/3417299227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fbcb0 0x7f28b00fc0c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.818+0000 7f28b4b1b700 1 -- 172.21.15.160:0/3417299227 >> 172.21.15.160:0/3417299227 conn(0x7f28b00f7430 msgr2=0x7f28b00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 -- 172.21.15.160:0/3417299227 shutdown_connections 2024-05-25T06:13:49.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 -- 172.21.15.160:0/3417299227 wait complete. 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 Processor -- start 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 -- start start 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b00fbcb0 0x7f28b010c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 0x7f28b010cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b010d3c0 con 0x7f28b00fcf60 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.819+0000 7f28b4b1b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b010dc10 con 0x7f28b00fbcb0 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 0x7f28b010cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 0x7f28b010cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60212/0 (socket says 172.21.15.160:60212) 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 -- 172.21.15.160:0/2453531101 learned_addr learned my addr 172.21.15.160:0/2453531101 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28ae59c700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b00fbcb0 0x7f28b010c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 -- 172.21.15.160:0/2453531101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b00fbcb0 msgr2=0x7f28b010c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b00fbcb0 0x7f28b010c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 -- 172.21.15.160:0/2453531101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0005040 con 0x7f28b00fcf60 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28add9b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 0x7f28b010cdc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28a4010d50 tx=0x7f28a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a400ed60 con 0x7f28b00fcf60 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b010de90 con 0x7f28b00fcf60 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28a40090d0 con 0x7f28b00fcf60 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a4012670 con 0x7f28b00fcf60 2024-05-25T06:13:49.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.820+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b010e3d0 con 0x7f28b00fcf60 2024-05-25T06:13:49.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.821+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b005f5c0 con 0x7f28b00fcf60 2024-05-25T06:13:49.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.822+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 32) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f28a4018030 con 0x7f28b00fcf60 2024-05-25T06:13:49.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.822+0000 7f289effd700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f289806db60 0x7f2898070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:49.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.823+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f28a4016070 con 0x7f28b00fcf60 2024-05-25T06:13:49.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.823+0000 7f28ae59c700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f289806db60 0x7f2898070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:49.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.823+0000 7f28ae59c700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f289806db60 0x7f2898070010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009b30 tx=0x7f28a0009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:49.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:49.827+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f28a405d1d0 con 0x7f28b00fcf60 2024-05-25T06:13:49.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:49 smithi160 ceph-mon[29041]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:13:49.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:49 smithi160 ceph-mon[29041]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:49.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:49 smithi160 ceph-mon[29041]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:50.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.125+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f28b004bed0 con 0x7f28b00fcf60 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.126+0000 7f289effd700 1 -- 172.21.15.160:0/2453531101 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7f28a405cd60 con 0x7f28b00fcf60 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:13:50.127 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 12, 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:13:50.128 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f289806db60 msgr2=0x7f2898070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f289806db60 0x7f2898070010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009b30 tx=0x7f28a0009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 msgr2=0x7f28b010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 0x7f28b010cdc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28a4010d50 tx=0x7f28a400b040 comp rx=0 tx=0).stop 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 shutdown_connections 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b00fbcb0 0x7f28b010c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f289806db60 0x7f2898070010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 --2- 172.21.15.160:0/2453531101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b00fcf60 0x7f28b010cdc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 >> 172.21.15.160:0/2453531101 conn(0x7f28b00f7430 msgr2=0x7f28b00f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:50.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 shutdown_connections 2024-05-25T06:13:50.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.128+0000 7f28b4b1b700 1 -- 172.21.15.160:0/2453531101 wait complete. 2024-05-25T06:13:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:49 smithi203 ceph-mon[36030]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:13:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:49 smithi203 ceph-mon[36030]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:49 smithi203 ceph-mon[36030]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 -- 172.21.15.160:0/110651011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fce80 msgr2=0x7fd0980fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/110651011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fce80 0x7fd0980fd290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd088004940 tx=0x7fd088009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 -- 172.21.15.160:0/110651011 shutdown_connections 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/110651011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0980fdff0 0x7fd0980fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/110651011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fce80 0x7fd0980fd290 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 -- 172.21.15.160:0/110651011 >> 172.21.15.160:0/110651011 conn(0x7fd0980f8540 msgr2=0x7fd0980fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 -- 172.21.15.160:0/110651011 shutdown_connections 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 -- 172.21.15.160:0/110651011 wait complete. 2024-05-25T06:13:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 Processor -- start 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.239+0000 7fd09d9e8700 1 -- start start 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd09d9e8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0980fce80 0x7fd09810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd09d9e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 0x7fd09810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd09d9e8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09810e5d0 con 0x7fd0980fdff0 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd09d9e8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09810edc0 con 0x7fd0980fce80 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 0x7fd09810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 0x7fd09810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60236/0 (socket says 172.21.15.160:60236) 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd096ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0980fce80 0x7fd09810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 -- 172.21.15.160:0/2806010629 learned_addr learned my addr 172.21.15.160:0/2806010629 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 -- 172.21.15.160:0/2806010629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0980fce80 msgr2=0x7fd09810da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0980fce80 0x7fd09810da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 -- 172.21.15.160:0/2806010629 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088005040 con 0x7fd0980fdff0 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd0967fc700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 0x7fd09810df40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b790 tx=0x7fd08c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08c00f6f0 con 0x7fd0980fdff0 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09810f0a0 con 0x7fd0980fdff0 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd08c00fd20 con 0x7fd0980fdff0 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08c0103a0 con 0x7fd0980fdff0 2024-05-25T06:13:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.240+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09810f660 con 0x7fd0980fdff0 2024-05-25T06:13:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.242+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09805f5c0 con 0x7fd0980fdff0 2024-05-25T06:13:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.242+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 32) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fd08c011400 con 0x7fd0980fdff0 2024-05-25T06:13:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.242+0000 7fd07f7fe700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd08006db60 0x7fd080070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:13:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.243+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd08c08d020 con 0x7fd0980fdff0 2024-05-25T06:13:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.243+0000 7fd096ffd700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd08006db60 0x7fd080070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:13:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.243+0000 7fd096ffd700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd08006db60 0x7fd080070010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd088009fd0 tx=0x7fd088009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:13:50.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.247+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fd08c05a730 con 0x7fd0980fdff0 2024-05-25T06:13:50.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.484+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd09810f9a0 con 0x7fd08006db60 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.486+0000 7fd07f7fe700 1 -- 172.21.15.160:0/2806010629 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fd09810f9a0 con 0x7fd08006db60 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:13:50.487 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "2/2 daemons upgraded", 2024-05-25T06:13:50.488 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading prometheus daemons", 2024-05-25T06:13:50.488 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:13:50.488 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:13:50.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd08006db60 msgr2=0x7fd080070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:50.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd08006db60 0x7fd080070010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd088009fd0 tx=0x7fd088009e30 comp rx=0 tx=0).stop 2024-05-25T06:13:50.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 msgr2=0x7fd09810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:13:50.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 0x7fd09810df40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b790 tx=0x7fd08c009250 comp rx=0 tx=0).stop 2024-05-25T06:13:50.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 shutdown_connections 2024-05-25T06:13:50.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0980fce80 0x7fd09810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fd08006db60 0x7fd080070010 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 --2- 172.21.15.160:0/2806010629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0980fdff0 0x7fd09810df40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:13:50.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 >> 172.21.15.160:0/2806010629 conn(0x7fd0980f8540 msgr2=0x7fd0980f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:13:50.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 shutdown_connections 2024-05-25T06:13:50.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:13:50.489+0000 7fd09d9e8700 1 -- 172.21.15.160:0/2806010629 wait complete. 2024-05-25T06:13:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:50 smithi160 ceph-mon[29041]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:50 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2453531101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:50 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:50.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:50 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:50.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:50 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:51.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:50 smithi203 ceph-mon[36030]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:51.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:50 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2453531101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:13:51.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:50 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:51.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:50 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:51.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:50 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:13:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:52 smithi160 ceph-mon[29041]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:52.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:52 smithi160 ceph-mon[29041]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:52.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:52 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:52.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:52 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:52.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:52 smithi203 ceph-mon[36030]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:13:52.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:52 smithi203 ceph-mon[36030]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:52.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:52 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:52.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:52 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:54.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:54 smithi160 ceph-mon[29041]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:54 smithi203 ceph-mon[36030]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:56 smithi160 ceph-mon[29041]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:13:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:56 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:56 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:56 smithi203 ceph-mon[36030]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:13:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:56 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:56 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:57.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:57 smithi160 ceph-mon[29041]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:57.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:57 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:57.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:57 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:57.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:57 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:57 smithi203 ceph-mon[36030]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:13:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:57 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:57 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:13:58.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:57 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:13:59.792 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:13:59 smithi160 ceph-mon[29041]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:00.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:13:59 smithi203 ceph-mon[36030]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:01.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:01 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:01.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:01.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:01 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:02.790 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:02 smithi160 ceph-mon[29041]: Upgrade: Updating alertmanager.smithi160 2024-05-25T06:14:02.790 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:02 smithi160 ceph-mon[29041]: Deploying daemon alertmanager.smithi160 on smithi160 2024-05-25T06:14:02.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:02 smithi203 ceph-mon[36030]: Upgrade: Updating alertmanager.smithi160 2024-05-25T06:14:02.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:02 smithi203 ceph-mon[36030]: Deploying daemon alertmanager.smithi160 on smithi160 2024-05-25T06:14:03.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:03 smithi203 ceph-mon[36030]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:03.908 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:03 smithi160 ceph-mon[29041]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:05.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:05 smithi160 ceph-mon[29041]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:05 smithi203 ceph-mon[36030]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:07.796 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:07 smithi160 ceph-mon[29041]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:07 smithi203 ceph-mon[36030]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:09.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:08 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:09.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:08 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:09.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:08 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:08 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:08 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:08 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:10.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:09 smithi160 ceph-mon[29041]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:09 smithi203 ceph-mon[36030]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:11 smithi160 ceph-mon[29041]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:11 smithi203 ceph-mon[36030]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:12 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:12 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:14 smithi203 ceph-mon[36030]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:14 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:14 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:14 smithi203 ceph-mon[36030]: mgrmap e33: smithi160.nalzlg(active, since 91s), standbys: smithi203.rotqex 2024-05-25T06:14:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:14 smithi160 ceph-mon[29041]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:14 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:14 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:14 smithi160 ceph-mon[29041]: mgrmap e33: smithi160.nalzlg(active, since 91s), standbys: smithi203.rotqex 2024-05-25T06:14:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:15 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:15.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:15 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:15 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:15 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:16 smithi160 ceph-mon[29041]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-25T06:14:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:16 smithi203 ceph-mon[36030]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-25T06:14:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:18 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:14:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:18.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:18 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:19.514 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:19 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:14:19.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:19 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-25T06:14:20.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 -- 172.21.15.160:0/1636692414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 msgr2=0x7f2f88102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:20.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/1636692414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88102310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c004940 tx=0x7f2f7c009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:20.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 -- 172.21.15.160:0/1636692414 shutdown_connections 2024-05-25T06:14:20.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/1636692414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88102310 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/1636692414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88066f70 0x7f2f880ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 -- 172.21.15.160:0/1636692414 >> 172.21.15.160:0/1636692414 conn(0x7f2f880f7420 msgr2=0x7f2f880f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:20.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 -- 172.21.15.160:0/1636692414 shutdown_connections 2024-05-25T06:14:20.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 -- 172.21.15.160:0/1636692414 wait complete. 2024-05-25T06:14:20.603 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:20 smithi160 ceph-mon[29041]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-25T06:14:20.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 Processor -- start 2024-05-25T06:14:20.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.602+0000 7f2f8ea44700 1 -- start start 2024-05-25T06:14:20.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f8ea44700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88066f70 0x7f2f88113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:20.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f8ea44700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88114510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:20.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f8ea44700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88114ba0 con 0x7f2f880ffef0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f8ea44700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881194f0 con 0x7f2f88066f70 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88114510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88114510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44396/0 (socket says 172.21.15.160:44396) 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 -- 172.21.15.160:0/3939988000 learned_addr learned my addr 172.21.15.160:0/3939988000 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f87fff700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88066f70 0x7f2f88113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 -- 172.21.15.160:0/3939988000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88066f70 msgr2=0x7f2f88113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88066f70 0x7f2f88113fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 -- 172.21.15.160:0/3939988000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c005040 con 0x7f2f880ffef0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f877fe700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88114510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009fd0 tx=0x7f2f7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f7c00e040 con 0x7f2f880ffef0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f881196f0 con 0x7f2f880ffef0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f7c00bcc0 con 0x7f2f880ffef0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.603+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f7c01c630 con 0x7f2f880ffef0 2024-05-25T06:14:20.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.604+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f88119c80 con 0x7f2f880ffef0 2024-05-25T06:14:20.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.604+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8804c710 con 0x7f2f880ffef0 2024-05-25T06:14:20.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.605+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f2f7c013980 con 0x7f2f880ffef0 2024-05-25T06:14:20.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.606+0000 7f2f84ff9700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2f7006da40 0x7f2f7006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:20.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.606+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2f7c08e3f0 con 0x7f2f880ffef0 2024-05-25T06:14:20.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.606+0000 7f2f87fff700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2f7006da40 0x7f2f7006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:20.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.606+0000 7f2f87fff700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2f7006da40 0x7f2f7006fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f88115460 tx=0x7f2f780091a0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:20.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.610+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f2f7c05bb70 con 0x7f2f880ffef0 2024-05-25T06:14:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:20 smithi203 ceph-mon[36030]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-25T06:14:20.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.833+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f880fd170 con 0x7f2f7006da40 2024-05-25T06:14:20.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.835+0000 7f2f84ff9700 1 -- 172.21.15.160:0/3939988000 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f2f880fd170 con 0x7f2f7006da40 2024-05-25T06:14:20.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2f7006da40 msgr2=0x7f2f7006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:20.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2f7006da40 0x7f2f7006fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f88115460 tx=0x7f2f780091a0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 msgr2=0x7f2f88114510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:20.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88114510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009fd0 tx=0x7f2f7c009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:20.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 shutdown_connections 2024-05-25T06:14:20.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88066f70 0x7f2f88113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f2f7006da40 0x7f2f7006fef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 --2- 172.21.15.160:0/3939988000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f880ffef0 0x7f2f88114510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 >> 172.21.15.160:0/3939988000 conn(0x7f2f880f7420 msgr2=0x7f2f880fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:20.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 shutdown_connections 2024-05-25T06:14:20.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.837+0000 7f2f8ea44700 1 -- 172.21.15.160:0/3939988000 wait complete. 2024-05-25T06:14:20.849 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:14:20.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 -- 172.21.15.160:0/706486966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fbcc0 msgr2=0x7fc1380fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:20.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 --2- 172.21.15.160:0/706486966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fbcc0 0x7fc1380fc0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc128004940 tx=0x7fc128009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:20.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 -- 172.21.15.160:0/706486966 shutdown_connections 2024-05-25T06:14:20.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 --2- 172.21.15.160:0/706486966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1380fcf70 0x7fc1380fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 --2- 172.21.15.160:0/706486966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fbcc0 0x7fc1380fc0d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 -- 172.21.15.160:0/706486966 >> 172.21.15.160:0/706486966 conn(0x7fc1380f7440 msgr2=0x7fc1380f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:20.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 -- 172.21.15.160:0/706486966 shutdown_connections 2024-05-25T06:14:20.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 -- 172.21.15.160:0/706486966 wait complete. 2024-05-25T06:14:20.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 Processor -- start 2024-05-25T06:14:20.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.942+0000 7fc13dae7700 1 -- start start 2024-05-25T06:14:20.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc13dae7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1380fbcc0 0x7fc13810fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:20.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc13dae7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 0x7fc138110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:20.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc13dae7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc138110790 con 0x7fc1380fcf70 2024-05-25T06:14:20.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc13dae7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1381150e0 con 0x7fc1380fbcc0 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 0x7fc138110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 0x7fc138110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44406/0 (socket says 172.21.15.160:44406) 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 -- 172.21.15.160:0/2800760029 learned_addr learned my addr 172.21.15.160:0/2800760029 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc1377fe700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1380fbcc0 0x7fc13810fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 -- 172.21.15.160:0/2800760029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1380fbcc0 msgr2=0x7fc13810fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1380fbcc0 0x7fc13810fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 -- 172.21.15.160:0/2800760029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128005040 con 0x7fc1380fcf70 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc12effd700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 0x7fc138110100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc12000b790 tx=0x7fc120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc12000f770 con 0x7fc1380fcf70 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc12000fda0 con 0x7fc1380fcf70 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.943+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc138115340 con 0x7fc1380fcf70 2024-05-25T06:14:20.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.944+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc12000d310 con 0x7fc1380fcf70 2024-05-25T06:14:20.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.944+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1381158b0 con 0x7fc1380fcf70 2024-05-25T06:14:20.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.945+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc138101900 con 0x7fc1380fcf70 2024-05-25T06:14:20.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.945+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fc12000f8d0 con 0x7fc1380fcf70 2024-05-25T06:14:20.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.945+0000 7fc134ff9700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc12406da40 0x7fc12406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:20.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.946+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc12008d200 con 0x7fc1380fcf70 2024-05-25T06:14:20.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.946+0000 7fc1377fe700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc12406da40 0x7fc12406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:20.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.946+0000 7fc1377fe700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc12406da40 0x7fc12406fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc12800b3b0 tx=0x7fc128009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:20.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:20.950+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fc12005a980 con 0x7fc1380fcf70 2024-05-25T06:14:21.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.172+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc13802b340 con 0x7fc12406da40 2024-05-25T06:14:21.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.174+0000 7fc134ff9700 1 -- 172.21.15.160:0/2800760029 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fc13802b340 con 0x7fc12406da40 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc12406da40 msgr2=0x7fc12406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc12406da40 0x7fc12406fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc12800b3b0 tx=0x7fc128009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 msgr2=0x7fc138110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 0x7fc138110100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc12000b790 tx=0x7fc120009250 comp rx=0 tx=0).stop 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 shutdown_connections 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1380fbcc0 0x7fc13810fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc12406da40 0x7fc12406fef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 --2- 172.21.15.160:0/2800760029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1380fcf70 0x7fc138110100 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 >> 172.21.15.160:0/2800760029 conn(0x7fc1380f7440 msgr2=0x7fc1380667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:21.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 shutdown_connections 2024-05-25T06:14:21.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.176+0000 7fc13dae7700 1 -- 172.21.15.160:0/2800760029 wait complete. 2024-05-25T06:14:21.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 -- 172.21.15.160:0/3898528751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fcc10 msgr2=0x7f62040fd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 --2- 172.21.15.160:0/3898528751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fcc10 0x7f62040fd080 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f61f4004940 tx=0x7f61f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:21.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 -- 172.21.15.160:0/3898528751 shutdown_connections 2024-05-25T06:14:21.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 --2- 172.21.15.160:0/3898528751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fcc10 0x7f62040fd080 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 --2- 172.21.15.160:0/3898528751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fc2c0 0x7f62040fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 -- 172.21.15.160:0/3898528751 >> 172.21.15.160:0/3898528751 conn(0x7f62040f6bd0 msgr2=0x7f62040f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.282+0000 7f620bfe8700 1 -- 172.21.15.160:0/3898528751 shutdown_connections 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 -- 172.21.15.160:0/3898528751 wait complete. 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 Processor -- start 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 -- start start 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 0x7f620410d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fcc10 0x7f620410df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620410e530 con 0x7f62040fc2c0 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f620bfe8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620410e6a0 con 0x7f62040fcc10 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f6209d84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 0x7f620410d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f6209d84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 0x7f620410d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44430/0 (socket says 172.21.15.160:44430) 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f6209d84700 1 -- 172.21.15.160:0/1924725844 learned_addr learned my addr 172.21.15.160:0/1924725844 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:21.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.283+0000 7f6209583700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fcc10 0x7f620410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f6209d84700 1 -- 172.21.15.160:0/1924725844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fcc10 msgr2=0x7f620410df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f6209d84700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fcc10 0x7f620410df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f6209d84700 1 -- 172.21.15.160:0/1924725844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4005040 con 0x7f62040fc2c0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f6209583700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fcc10 0x7f620410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f6209d84700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 0x7f620410d9f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6200002a80 tx=0x7f6200004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6200007cc0 con 0x7f62040fc2c0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f620000f040 con 0x7f62040fc2c0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62041130b0 con 0x7f62040fc2c0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f620000dad0 con 0x7f62040fc2c0 2024-05-25T06:14:21.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.284+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6204113670 con 0x7f62040fc2c0 2024-05-25T06:14:21.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.285+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204101c10 con 0x7f62040fc2c0 2024-05-25T06:14:21.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.286+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f620000dc30 con 0x7f62040fc2c0 2024-05-25T06:14:21.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.286+0000 7f61fa7fc700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f61f006da40 0x7f61f006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:21.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.286+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f620008ce80 con 0x7f62040fc2c0 2024-05-25T06:14:21.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.286+0000 7f6209583700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f61f006da40 0x7f61f006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:21.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.287+0000 7f6209583700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f61f006da40 0x7f61f006fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f61f4004f90 tx=0x7f61f4009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:21.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.291+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f620005a600 con 0x7f62040fc2c0 2024-05-25T06:14:21.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.523+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f620402b3b0 con 0x7f61f006da40 2024-05-25T06:14:21.523 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:21 smithi160 ceph-mon[29041]: Upgrade: Updating grafana.smithi160 2024-05-25T06:14:21.523 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:21 smithi160 ceph-mon[29041]: Deploying daemon grafana.smithi160 on smithi160 2024-05-25T06:14:21.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.534+0000 7f61fa7fc700 1 -- 172.21.15.160:0/1924725844 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f620402b3b0 con 0x7f61f006da40 2024-05-25T06:14:21.534 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:14:21.534 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (14s) 8s ago 11m 17.7M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:14:21.534 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (11m) 8s ago 11m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:14:21.534 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (10m) 45s ago 10m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:14:21.534 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (2m) 8s ago 11m 80.0M - 9.4.7 954c08fa6188 608a83ea61e4 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (5m) 8s ago 5m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (5m) 8s ago 5m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (5m) 45s ago 5m 61.2M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (5m) 45s ago 5m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (112s) 8s ago 12m 500M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (66s) 45s ago 10m 452M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (12m) 8s ago 12m 53.2M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (10m) 45s ago 10m 41.7M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (53s) 8s ago 11m 7591k - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (49s) 45s ago 10m 3644k - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:14:21.535 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (9m) 8s ago 9m 217M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:14:21.536 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (9m) 8s ago 9m 225M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:14:21.536 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (8m) 8s ago 8m 189M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:14:21.536 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (8m) 45s ago 8m 267M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:14:21.536 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (7m) 45s ago 7m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:14:21.536 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (7m) 45s ago 7m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:14:21.536 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (31s) 8s ago 10m 51.3M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:14:21.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f61f006da40 msgr2=0x7f61f006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f61f006da40 0x7f61f006fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f61f4004f90 tx=0x7f61f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:21.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 msgr2=0x7f620410d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 0x7f620410d9f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6200002a80 tx=0x7f6200004600 comp rx=0 tx=0).stop 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 shutdown_connections 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f61f006da40 0x7f61f006fef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62040fc2c0 0x7f620410d9f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 --2- 172.21.15.160:0/1924725844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62040fcc10 0x7f620410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 >> 172.21.15.160:0/1924725844 conn(0x7f62040f6bd0 msgr2=0x7f62040fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 shutdown_connections 2024-05-25T06:14:21.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.537+0000 7f620bfe8700 1 -- 172.21.15.160:0/1924725844 wait complete. 2024-05-25T06:14:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:21 smithi203 ceph-mon[36030]: Upgrade: Updating grafana.smithi160 2024-05-25T06:14:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:21 smithi203 ceph-mon[36030]: Deploying daemon grafana.smithi160 on smithi160 2024-05-25T06:14:21.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 -- 172.21.15.160:0/4101739154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 msgr2=0x7fc8a80ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 --2- 172.21.15.160:0/4101739154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80ff9b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc898004940 tx=0x7fc898009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:21.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 -- 172.21.15.160:0/4101739154 shutdown_connections 2024-05-25T06:14:21.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 --2- 172.21.15.160:0/4101739154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8a80ffef0 0x7fc8a8102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 --2- 172.21.15.160:0/4101739154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80ff9b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 -- 172.21.15.160:0/4101739154 >> 172.21.15.160:0/4101739154 conn(0x7fc8a80f7420 msgr2=0x7fc8a80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:21.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 -- 172.21.15.160:0/4101739154 shutdown_connections 2024-05-25T06:14:21.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.651+0000 7fc8ada18700 1 -- 172.21.15.160:0/4101739154 wait complete. 2024-05-25T06:14:21.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8ada18700 1 Processor -- start 2024-05-25T06:14:21.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8ada18700 1 -- start start 2024-05-25T06:14:21.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8ada18700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8ada18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8a80ffef0 0x7fc8a80fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8ada18700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80fbe40 con 0x7fc8a8066b50 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8ada18700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80fbf80 con 0x7fc8a80ffef0 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44454/0 (socket says 172.21.15.160:44454) 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a6ffd700 1 -- 172.21.15.160:0/1469035977 learned_addr learned my addr 172.21.15.160:0/1469035977 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a67fc700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8a80ffef0 0x7fc8a80fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a6ffd700 1 -- 172.21.15.160:0/1469035977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8a80ffef0 msgr2=0x7fc8a80fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a6ffd700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8a80ffef0 0x7fc8a80fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.652+0000 7fc8a6ffd700 1 -- 172.21.15.160:0/1469035977 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898005040 con 0x7fc8a8066b50 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.653+0000 7fc8a6ffd700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80fd390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc898000c00 tx=0x7fc898009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.653+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc89800e030 con 0x7fc8a8066b50 2024-05-25T06:14:21.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.653+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80fc200 con 0x7fc8a8066b50 2024-05-25T06:14:21.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.653+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc898013de0 con 0x7fc8a8066b50 2024-05-25T06:14:21.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.653+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc89801ce50 con 0x7fc8a8066b50 2024-05-25T06:14:21.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.653+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a8115a20 con 0x7fc8a8066b50 2024-05-25T06:14:21.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.654+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a805f5c0 con 0x7fc8a8066b50 2024-05-25T06:14:21.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.655+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fc89801c400 con 0x7fc8a8066b50 2024-05-25T06:14:21.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.655+0000 7fc88f7fe700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc89006d9d0 0x7fc89006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:21.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.655+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc89808eb20 con 0x7fc8a8066b50 2024-05-25T06:14:21.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.655+0000 7fc8a67fc700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc89006d9d0 0x7fc89006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:21.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.656+0000 7fc8a67fc700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc89006d9d0 0x7fc89006fe80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80fd970 tx=0x7fc89c00b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:21.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:21.660+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fc8980586f0 con 0x7fc8a8066b50 2024-05-25T06:14:22.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.020+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc8a8115bb0 con 0x7fc8a8066b50 2024-05-25T06:14:22.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.021+0000 7fc88f7fe700 1 -- 172.21.15.160:0/1469035977 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7fc89805bdb0 con 0x7fc8a8066b50 2024-05-25T06:14:22.021 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:14:22.022 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 12, 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:14:22.023 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc89006d9d0 msgr2=0x7fc89006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc89006d9d0 0x7fc89006fe80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80fd970 tx=0x7fc89c00b040 comp rx=0 tx=0).stop 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 msgr2=0x7fc8a80fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80fd390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc898000c00 tx=0x7fc898009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 shutdown_connections 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fc89006d9d0 0x7fc89006fe80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8a8066b50 0x7fc8a80fd390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 --2- 172.21.15.160:0/1469035977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8a80ffef0 0x7fc8a80fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 >> 172.21.15.160:0/1469035977 conn(0x7fc8a80f7420 msgr2=0x7fc8a80f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:22.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 shutdown_connections 2024-05-25T06:14:22.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.023+0000 7fc8ada18700 1 -- 172.21.15.160:0/1469035977 wait complete. 2024-05-25T06:14:22.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 -- 172.21.15.160:0/386397235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fe080 msgr2=0x7f12000fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:22.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 --2- 172.21.15.160:0/386397235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fe080 0x7f12000fe4f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f11f4004940 tx=0x7f11f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:22.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 -- 172.21.15.160:0/386397235 shutdown_connections 2024-05-25T06:14:22.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 --2- 172.21.15.160:0/386397235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fe080 0x7f12000fe4f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 --2- 172.21.15.160:0/386397235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12000fce80 0x7f12000fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 -- 172.21.15.160:0/386397235 >> 172.21.15.160:0/386397235 conn(0x7f12000f8590 msgr2=0x7f12000fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 -- 172.21.15.160:0/386397235 shutdown_connections 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.132+0000 7f1204f5c700 1 -- 172.21.15.160:0/386397235 wait complete. 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f1204f5c700 1 Processor -- start 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f1204f5c700 1 -- start start 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f1204f5c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 0x7f1200071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f1204f5c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12000fe080 0x7f12000702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f1204f5c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200070860 con 0x7f12000fce80 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f1204f5c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000709d0 con 0x7f12000fe080 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f11fe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 0x7f1200071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f11fe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 0x7f1200071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44474/0 (socket says 172.21.15.160:44474) 2024-05-25T06:14:22.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f11fe59c700 1 -- 172.21.15.160:0/3155551906 learned_addr learned my addr 172.21.15.160:0/3155551906 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f11fdd9b700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12000fe080 0x7f12000702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f11fe59c700 1 -- 172.21.15.160:0/3155551906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12000fe080 msgr2=0x7f12000702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.133+0000 7f11fe59c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12000fe080 0x7f12000702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.134+0000 7f11fe59c700 1 -- 172.21.15.160:0/3155551906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4005040 con 0x7f12000fce80 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.134+0000 7f11fe59c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 0x7f1200071de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f11f0002a80 tx=0x7f11f0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.134+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f0007cc0 con 0x7f12000fce80 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.134+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1200070cb0 con 0x7f12000fce80 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.134+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11f000f040 con 0x7f12000fce80 2024-05-25T06:14:22.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.134+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200115a50 con 0x7f12000fce80 2024-05-25T06:14:22.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.135+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f000da30 con 0x7f12000fce80 2024-05-25T06:14:22.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.135+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e00052e0 con 0x7f12000fce80 2024-05-25T06:14:22.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.136+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f11f000db90 con 0x7f12000fce80 2024-05-25T06:14:22.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.136+0000 7f11eeffd700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f11e806da40 0x7f11e806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:22.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.136+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f11f008ce40 con 0x7f12000fce80 2024-05-25T06:14:22.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.136+0000 7f11fdd9b700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f11e806da40 0x7f11e806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:22.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.137+0000 7f11fdd9b700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f11e806da40 0x7f11e806fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11f400b970 tx=0x7f11f4009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:22.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.141+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f11f0056a90 con 0x7f12000fce80 2024-05-25T06:14:22.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.405+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e0000bf0 con 0x7f11e806da40 2024-05-25T06:14:22.406 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:22 smithi160 ceph-mon[29041]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:14:22.406 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:22 smithi160 ceph-mon[29041]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:22.406 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:22 smithi160 ceph-mon[29041]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:22.406 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:22 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1469035977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:22.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.406+0000 7f11eeffd700 1 -- 172.21.15.160:0/3155551906 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f11e0000bf0 con 0x7f11e806da40 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "2/2 daemons upgraded", 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading grafana daemons", 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:14:22.407 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:14:22.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f11e806da40 msgr2=0x7f11e806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:22.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f11e806da40 0x7f11e806fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11f400b970 tx=0x7f11f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:22.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 msgr2=0x7f1200071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:22.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 0x7f1200071de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f11f0002a80 tx=0x7f11f0004600 comp rx=0 tx=0).stop 2024-05-25T06:14:22.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 shutdown_connections 2024-05-25T06:14:22.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f11e806da40 0x7f11e806fef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12000fce80 0x7f1200071de0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 --2- 172.21.15.160:0/3155551906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12000fe080 0x7f12000702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:22.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 >> 172.21.15.160:0/3155551906 conn(0x7f12000f8590 msgr2=0x7f12000f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:22.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 shutdown_connections 2024-05-25T06:14:22.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:22.409+0000 7f1204f5c700 1 -- 172.21.15.160:0/3155551906 wait complete. 2024-05-25T06:14:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:22 smithi203 ceph-mon[36030]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:14:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:22 smithi203 ceph-mon[36030]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:22 smithi203 ceph-mon[36030]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:22.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:22 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1469035977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:23.505 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:23 smithi160 ceph-mon[29041]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:23 smithi203 ceph-mon[36030]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:24 smithi203 ceph-mon[36030]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:24 smithi203 ceph-mon[36030]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:14:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:24 smithi160 ceph-mon[29041]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:24 smithi160 ceph-mon[29041]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-25T06:14:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:26 smithi203 ceph-mon[36030]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-25T06:14:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:26 smithi160 ceph-mon[29041]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-25T06:14:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:28 smithi203 ceph-mon[36030]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:28.658 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:28 smithi160 ceph-mon[29041]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:28.658 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:30.302 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:30 smithi160 ceph-mon[29041]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:30 smithi203 ceph-mon[36030]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:32.646 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:32 smithi160 ceph-mon[29041]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:32.646 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:32 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:32.646 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:32 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:32.646 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:32 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:32 smithi203 ceph-mon[36030]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:32 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:32 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:32 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:34 smithi203 ceph-mon[36030]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:34 smithi160 ceph-mon[29041]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:36 smithi160 ceph-mon[29041]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:36 smithi203 ceph-mon[36030]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:38.548 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:38 smithi160 ceph-mon[29041]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:38 smithi203 ceph-mon[36030]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:40 smithi203 ceph-mon[36030]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:40.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:40 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:40.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:40 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:40 smithi160 ceph-mon[29041]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:40 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:40 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:41 smithi160 ceph-mon[29041]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:41 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:41 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:41 smithi203 ceph-mon[36030]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:41 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:41 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:42 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:42 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:43 smithi160 ceph-mon[29041]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:43 smithi203 ceph-mon[36030]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:14:45.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-25T06:14:45.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:45.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:45 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-25T06:14:45.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:45.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:14:45.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:14:45.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:45 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:14:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:46 smithi203 ceph-mon[36030]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:14:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:46 smithi203 ceph-mon[36030]: Upgrade: Finalizing container_image settings 2024-05-25T06:14:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:46 smithi203 ceph-mon[36030]: Upgrade: Complete! 2024-05-25T06:14:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:46 smithi203 ceph-mon[36030]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:46 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:46.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:46 smithi160 ceph-mon[29041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-25T06:14:46.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:46 smithi160 ceph-mon[29041]: Upgrade: Finalizing container_image settings 2024-05-25T06:14:46.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:46 smithi160 ceph-mon[29041]: Upgrade: Complete! 2024-05-25T06:14:46.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:46 smithi160 ceph-mon[29041]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:46.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:46 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:48 smithi203 ceph-mon[36030]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:48 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:48.658 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:48 smithi160 ceph-mon[29041]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:48.658 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:48 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:14:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:50 smithi203 ceph-mon[36030]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:50 smithi160 ceph-mon[29041]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:52.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.528+0000 7f6ffc337700 1 -- 172.21.15.160:0/2370200150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 msgr2=0x7f6ff40fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:52.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.528+0000 7f6ffc337700 1 --2- 172.21.15.160:0/2370200150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff40fd3e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004940 tx=0x7f6ff0009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:52.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- 172.21.15.160:0/2370200150 shutdown_connections 2024-05-25T06:14:52.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 --2- 172.21.15.160:0/2370200150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff40fd3e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 --2- 172.21.15.160:0/2370200150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ff40fbcc0 0x7f6ff40fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- 172.21.15.160:0/2370200150 >> 172.21.15.160:0/2370200150 conn(0x7f6ff40f7440 msgr2=0x7f6ff40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:52.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- 172.21.15.160:0/2370200150 shutdown_connections 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- 172.21.15.160:0/2370200150 wait complete. 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 Processor -- start 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- start start 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ff40fbcc0 0x7f6ff410fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff4110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4110720 con 0x7f6ff40fcf70 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.529+0000 7f6ffc337700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4110890 con 0x7f6ff40fbcc0 2024-05-25T06:14:52.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff4110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff4110120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57914/0 (socket says 172.21.15.160:57914) 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 -- 172.21.15.160:0/3068736042 learned_addr learned my addr 172.21.15.160:0/3068736042 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ffa0d3700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ff40fbcc0 0x7f6ff410fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 -- 172.21.15.160:0/3068736042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ff40fbcc0 msgr2=0x7f6ff410fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ff40fbcc0 0x7f6ff410fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 -- 172.21.15.160:0/3068736042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0005040 con 0x7f6ff40fcf70 2024-05-25T06:14:52.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ff98d2700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff4110120 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009380 tx=0x7f6ff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:52.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff000e040 con 0x7f6ff40fcf70 2024-05-25T06:14:52.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff000bbf0 con 0x7f6ff40fcf70 2024-05-25T06:14:52.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff4115290 con 0x7f6ff40fcf70 2024-05-25T06:14:52.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff001c3f0 con 0x7f6ff40fcf70 2024-05-25T06:14:52.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.530+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff4115850 con 0x7f6ff40fcf70 2024-05-25T06:14:52.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.531+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff4101900 con 0x7f6ff40fcf70 2024-05-25T06:14:52.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.532+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f6ff000bd60 con 0x7f6ff40fcf70 2024-05-25T06:14:52.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.532+0000 7f6feaffd700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f6fe006da40 0x7f6fe006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:52.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.532+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6ff008e540 con 0x7f6ff40fcf70 2024-05-25T06:14:52.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.532+0000 7f6ffa0d3700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f6fe006da40 0x7f6fe006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:52.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.533+0000 7f6ffa0d3700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f6fe006da40 0x7f6fe006fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4007fd0 tx=0x7f6fe40043c0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:52.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.537+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f6ff005bc40 con 0x7f6ff40fcf70 2024-05-25T06:14:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:52 smithi203 ceph-mon[36030]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:52.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:52 smithi160 ceph-mon[29041]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:52.756 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.756+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ff402b2f0 con 0x7f6fe006da40 2024-05-25T06:14:52.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.757+0000 7f6feaffd700 1 -- 172.21.15.160:0/3068736042 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f6ff402b2f0 con 0x7f6fe006da40 2024-05-25T06:14:52.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f6fe006da40 msgr2=0x7f6fe006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f6fe006da40 0x7f6fe006fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4007fd0 tx=0x7f6fe40043c0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 msgr2=0x7f6ff4110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff4110120 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009380 tx=0x7f6ff0009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 shutdown_connections 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ff40fbcc0 0x7f6ff410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f6fe006da40 0x7f6fe006fef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 --2- 172.21.15.160:0/3068736042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff40fcf70 0x7f6ff4110120 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 >> 172.21.15.160:0/3068736042 conn(0x7f6ff40f7440 msgr2=0x7f6ff40668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 shutdown_connections 2024-05-25T06:14:52.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:52.760+0000 7f6ffc337700 1 -- 172.21.15.160:0/3068736042 wait complete. 2024-05-25T06:14:53.420 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-05-25T06:14:53.680 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:14:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:54 smithi203 ceph-mon[36030]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:54 smithi203 ceph-mon[36030]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:54 smithi160 ceph-mon[29041]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:54 smithi160 ceph-mon[29041]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 -- 172.21.15.160:0/1446622984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8102de0 msgr2=0x7f45b81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 --2- 172.21.15.160:0/1446622984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8102de0 0x7f45b81031b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004940 tx=0x7f45ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 -- 172.21.15.160:0/1446622984 shutdown_connections 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 --2- 172.21.15.160:0/1446622984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8103780 0x7f45b80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 --2- 172.21.15.160:0/1446622984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8102de0 0x7f45b81031b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 -- 172.21.15.160:0/1446622984 >> 172.21.15.160:0/1446622984 conn(0x7f45b80f8990 msgr2=0x7f45b80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 -- 172.21.15.160:0/1446622984 shutdown_connections 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.405+0000 7f45beade700 1 -- 172.21.15.160:0/1446622984 wait complete. 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45beade700 1 Processor -- start 2024-05-25T06:14:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45beade700 1 -- start start 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45beade700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8102de0 0x7f45b8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45beade700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 0x7f45b8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45beade700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8114500 con 0x7f45b8103780 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45beade700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b810db50 con 0x7f45b8102de0 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 0x7f45b8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 0x7f45b8113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57928/0 (socket says 172.21.15.160:57928) 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45b7fff700 1 -- 172.21.15.160:0/71421055 learned_addr learned my addr 172.21.15.160:0/71421055 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45bc87a700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8102de0 0x7f45b8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:55.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45b7fff700 1 -- 172.21.15.160:0/71421055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8102de0 msgr2=0x7f45b8113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45b7fff700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8102de0 0x7f45b8113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45b7fff700 1 -- 172.21.15.160:0/71421055 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac005040 con 0x7f45b8103780 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.406+0000 7f45bc87a700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8102de0 0x7f45b8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.407+0000 7f45b7fff700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 0x7f45b8113eb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c800 tx=0x7f45a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.407+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a400a760 con 0x7f45b8103780 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.407+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45a400ad90 con 0x7f45b8103780 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.407+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b810de30 con 0x7f45b8103780 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.407+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a40113a0 con 0x7f45b8103780 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.407+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b810e3f0 con 0x7f45b8103780 2024-05-25T06:14:55.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.408+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b81000f0 con 0x7f45b8103780 2024-05-25T06:14:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.408+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f45a4012400 con 0x7f45b8103780 2024-05-25T06:14:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.409+0000 7f45b57fa700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f45a806da40 0x7f45a806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.409+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f45a408d0d0 con 0x7f45b8103780 2024-05-25T06:14:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.409+0000 7f45bc87a700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f45a806da40 0x7f45a806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.410+0000 7f45bc87a700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f45a806da40 0x7f45a806fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004560 tx=0x7f45ac009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:55.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.414+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f45a405a8f0 con 0x7f45b8103780 2024-05-25T06:14:55.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.693+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f45b8109ee0 con 0x7f45b8103780 2024-05-25T06:14:55.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.693+0000 7f45b57fa700 1 -- 172.21.15.160:0/71421055 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7f45a405a480 con 0x7f45b8103780 2024-05-25T06:14:55.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f45a806da40 msgr2=0x7f45a806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:55.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f45a806da40 0x7f45a806fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004560 tx=0x7f45ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:55.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 msgr2=0x7f45b8113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:55.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 0x7f45b8113eb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c800 tx=0x7f45a4004470 comp rx=0 tx=0).stop 2024-05-25T06:14:55.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 shutdown_connections 2024-05-25T06:14:55.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8102de0 0x7f45b8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:55.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f45a806da40 0x7f45a806fef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:55.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 --2- 172.21.15.160:0/71421055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45b8103780 0x7f45b8113eb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:55.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 >> 172.21.15.160:0/71421055 conn(0x7f45b80f8990 msgr2=0x7f45b8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:55.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 shutdown_connections 2024-05-25T06:14:55.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:55.696+0000 7f45beade700 1 -- 172.21.15.160:0/71421055 wait complete. 2024-05-25T06:14:55.707 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:14:56.268 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-05-25T06:14:56.524 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:14:56.647 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:56 smithi160 ceph-mon[29041]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:56.647 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:56 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/71421055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:56 smithi203 ceph-mon[36030]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:14:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:56 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/71421055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:58.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 -- 172.21.15.160:0/3478169607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054101cf0 msgr2=0x7f10541020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:58.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 --2- 172.21.15.160:0/3478169607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054101cf0 0x7f10541020c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f103c004940 tx=0x7f103c009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:58.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 -- 172.21.15.160:0/3478169607 shutdown_connections 2024-05-25T06:14:58.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 --2- 172.21.15.160:0/3478169607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054102690 0x7f10540fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 --2- 172.21.15.160:0/3478169607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054101cf0 0x7f10541020c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 -- 172.21.15.160:0/3478169607 >> 172.21.15.160:0/3478169607 conn(0x7f10540f7860 msgr2=0x7f10540f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 -- 172.21.15.160:0/3478169607 shutdown_connections 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.092+0000 7f1058a76700 1 -- 172.21.15.160:0/3478169607 wait complete. 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1058a76700 1 Processor -- start 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1058a76700 1 -- start start 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1058a76700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054101cf0 0x7f105410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1058a76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 0x7f105410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1058a76700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105410f5c0 con 0x7f1054102690 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1058a76700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105410f700 con 0x7f1054101cf0 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1051d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 0x7f105410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1051d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 0x7f105410f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45846/0 (socket says 172.21.15.160:45846) 2024-05-25T06:14:58.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.093+0000 7f1051d9b700 1 -- 172.21.15.160:0/1095694308 learned_addr learned my addr 172.21.15.160:0/1095694308 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f105259c700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054101cf0 0x7f105410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f1051d9b700 1 -- 172.21.15.160:0/1095694308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054101cf0 msgr2=0x7f105410eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f1051d9b700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054101cf0 0x7f105410eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f1051d9b700 1 -- 172.21.15.160:0/1095694308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c005040 con 0x7f1054102690 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f105259c700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054101cf0 0x7f105410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f1051d9b700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 0x7f105410f080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f104400b790 tx=0x7f1044009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f104400f6f0 con 0x7f1054102690 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f104400fd20 con 0x7f1054102690 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1054117bb0 con 0x7f1054102690 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10440103a0 con 0x7f1054102690 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.094+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1054117fd0 con 0x7f1054102690 2024-05-25T06:14:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.095+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10540fd440 con 0x7f1054102690 2024-05-25T06:14:58.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.096+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f1044010500 con 0x7f1054102690 2024-05-25T06:14:58.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.096+0000 7f104affd700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f104006da40 0x7f104006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:14:58.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.096+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f104408d000 con 0x7f1054102690 2024-05-25T06:14:58.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.096+0000 7f105259c700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f104006da40 0x7f104006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:14:58.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.097+0000 7f105259c700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f104006da40 0x7f104006fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f103c004560 tx=0x7f103c009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:14:58.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.101+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f1044056bd0 con 0x7f1054102690 2024-05-25T06:14:58.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.380+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1054108e60 con 0x7f1054102690 2024-05-25T06:14:58.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.380+0000 7f104affd700 1 -- 172.21.15.160:0/1095694308 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7f104405a290 con 0x7f1054102690 2024-05-25T06:14:58.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f104006da40 msgr2=0x7f104006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:58.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f104006da40 0x7f104006fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f103c004560 tx=0x7f103c009e30 comp rx=0 tx=0).stop 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 msgr2=0x7f105410f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 0x7f105410f080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f104400b790 tx=0x7f1044009220 comp rx=0 tx=0).stop 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 shutdown_connections 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1054101cf0 0x7f105410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f104006da40 0x7f104006fef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 --2- 172.21.15.160:0/1095694308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1054102690 0x7f105410f080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.382+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 >> 172.21.15.160:0/1095694308 conn(0x7f10540f7860 msgr2=0x7f10540f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.383+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 shutdown_connections 2024-05-25T06:14:58.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:14:58.383+0000 7f1058a76700 1 -- 172.21.15.160:0/1095694308 wait complete. 2024-05-25T06:14:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)" 2024-05-25T06:14:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:58 smithi203 ceph-mon[36030]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:58 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:58.658 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:58 smithi160 ceph-mon[29041]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:14:58.658 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:58 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:14:59.101 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-05-25T06:14:59.359 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:14:59.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:14:59 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1095694308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:14:59.702 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:14:59 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1095694308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:00 smithi160 ceph-mon[29041]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:00 smithi203 ceph-mon[36030]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:01.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4210348504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 msgr2=0x7ff8e8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:01.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4210348504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e8101a30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0004940 tx=0x7ff8d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:01.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4210348504 shutdown_connections 2024-05-25T06:15:01.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4210348504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e8101a30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4210348504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e80fb5a0 0x7ff8e80fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4210348504 >> 172.21.15.160:0/4210348504 conn(0x7ff8e8073350 msgr2=0x7ff8e8073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4210348504 shutdown_connections 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4210348504 wait complete. 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.166+0000 7ff8ed3cf700 1 Processor -- start 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8ed3cf700 1 -- start start 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8ed3cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e80fb5a0 0x7ff8e80fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8ed3cf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e80fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8ed3cf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80fe510 con 0x7ff8e80fbf40 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8ed3cf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80fcfb0 con 0x7ff8e80fb5a0 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e80fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e80fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45858/0 (socket says 172.21.15.160:45858) 2024-05-25T06:15:01.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 -- 172.21.15.160:0/4023210909 learned_addr learned my addr 172.21.15.160:0/4023210909 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e6ffd700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e80fb5a0 0x7ff8e80fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 -- 172.21.15.160:0/4023210909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e80fb5a0 msgr2=0x7ff8e80fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e80fb5a0 0x7ff8e80fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 -- 172.21.15.160:0/4023210909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0005040 con 0x7ff8e80fbf40 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8e67fc700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e80fca70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b3b0 tx=0x7ff8d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.167+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d000e040 con 0x7ff8e80fbf40 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.168+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8d0013420 con 0x7ff8e80fbf40 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.168+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e80fd230 con 0x7ff8e80fbf40 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.168+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d001c630 con 0x7ff8e80fbf40 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.168+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e811a1d0 con 0x7ff8e80fbf40 2024-05-25T06:15:01.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.168+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e80fe670 con 0x7ff8e80fbf40 2024-05-25T06:15:01.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.169+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7ff8d0013590 con 0x7ff8e80fbf40 2024-05-25T06:15:01.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.169+0000 7ff8df7fe700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7ff8d406da40 0x7ff8d406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:01.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.170+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff8d008e3b0 con 0x7ff8e80fbf40 2024-05-25T06:15:01.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.170+0000 7ff8e6ffd700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7ff8d406da40 0x7ff8d406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:01.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.170+0000 7ff8e6ffd700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7ff8d406da40 0x7ff8d406fef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009bf0 tx=0x7ff8d80091a0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:01.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.174+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7ff8d00580b0 con 0x7ff8e80fbf40 2024-05-25T06:15:01.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.462+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8e8109f70 con 0x7ff8e80fbf40 2024-05-25T06:15:01.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.463+0000 7ff8df7fe700 1 -- 172.21.15.160:0/4023210909 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7ff8d0013840 con 0x7ff8e80fbf40 2024-05-25T06:15:01.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7ff8d406da40 msgr2=0x7ff8d406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7ff8d406da40 0x7ff8d406fef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009bf0 tx=0x7ff8d80091a0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 msgr2=0x7ff8e80fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e80fca70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b3b0 tx=0x7ff8d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 shutdown_connections 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e80fb5a0 0x7ff8e80fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7ff8d406da40 0x7ff8d406fef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 --2- 172.21.15.160:0/4023210909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e80fbf40 0x7ff8e80fca70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 >> 172.21.15.160:0/4023210909 conn(0x7ff8e8073350 msgr2=0x7ff8e80f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 shutdown_connections 2024-05-25T06:15:01.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:01.465+0000 7ff8ed3cf700 1 -- 172.21.15.160:0/4023210909 wait complete. 2024-05-25T06:15:01.477 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:15:02.148 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-05-25T06:15:02.413 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:02 smithi203 ceph-mon[36030]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:02 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/4023210909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:02.652 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:02 smithi160 ceph-mon[29041]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:02.652 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:02 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/4023210909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 -- 172.21.15.160:0/1267330661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 msgr2=0x7f76a00fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/1267330661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fcb80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7688004940 tx=0x7f7688009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 -- 172.21.15.160:0/1267330661 shutdown_connections 2024-05-25T06:15:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/1267330661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fcb80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/1267330661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a0101e20 0x7f76a01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 -- 172.21.15.160:0/1267330661 >> 172.21.15.160:0/1267330661 conn(0x7f76a0071460 msgr2=0x7f76a0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 -- 172.21.15.160:0/1267330661 shutdown_connections 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.126+0000 7f76a5ed6700 1 -- 172.21.15.160:0/1267330661 wait complete. 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.127+0000 7f76a5ed6700 1 Processor -- start 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.127+0000 7f76a5ed6700 1 -- start start 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.127+0000 7f76a5ed6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.127+0000 7f76a5ed6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a0101e20 0x7f76a00fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.127+0000 7f76a5ed6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00fe440 con 0x7f76a00fc710 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.127+0000 7f76a5ed6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00fe580 con 0x7f76a0101e20 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45882/0 (socket says 172.21.15.160:45882) 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 -- 172.21.15.160:0/2503393567 learned_addr learned my addr 172.21.15.160:0/2503393567 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f7696ffd700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a0101e20 0x7f76a00fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 -- 172.21.15.160:0/2503393567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a0101e20 msgr2=0x7f76a00fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a0101e20 0x7f76a00fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 -- 172.21.15.160:0/2503393567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7688005040 con 0x7f76a00fc710 2024-05-25T06:15:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769f7fe700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fd970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f769000cb10 tx=0x7f7690007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7690004200 con 0x7f76a00fc710 2024-05-25T06:15:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f769000ac70 con 0x7f76a00fc710 2024-05-25T06:15:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a0119e40 con 0x7f76a00fc710 2024-05-25T06:15:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7690003d30 con 0x7f76a00fc710 2024-05-25T06:15:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.128+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a011a2c0 con 0x7f76a00fc710 2024-05-25T06:15:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.129+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a00652f0 con 0x7f76a00fc710 2024-05-25T06:15:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.130+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f769000a7a0 con 0x7f76a00fc710 2024-05-25T06:15:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.130+0000 7f769cff9700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f768c06da40 0x7f768c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.130+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f769008d660 con 0x7f76a00fc710 2024-05-25T06:15:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.130+0000 7f7696ffd700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f768c06da40 0x7f768c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.131+0000 7f7696ffd700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f768c06da40 0x7f768c06fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ff2b0 tx=0x7f7688009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:04.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.135+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f7690057230 con 0x7f76a00fc710 2024-05-25T06:15:04.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:04.350+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "target": ["mon-mgr", ""]}) v1 -- 0x7f76a00ff060 con 0x7f768c06da40 2024-05-25T06:15:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:04 smithi203 ceph-mon[36030]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:04.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:04 smithi160 ceph-mon[29041]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:05 smithi203 ceph-mon[36030]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:05.685 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:05 smithi160 ceph-mon[29041]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:06.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.571+0000 7f769cff9700 1 -- 172.21.15.160:0/2503393567 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5091 (secure 0 0 0) 0x7f76a00ff060 con 0x7f768c06da40 2024-05-25T06:15:06.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f768c06da40 msgr2=0x7f768c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:06.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f768c06da40 0x7f768c06fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ff2b0 tx=0x7f7688009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 msgr2=0x7f76a00fd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fd970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f769000cb10 tx=0x7f7690007f90 comp rx=0 tx=0).stop 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 shutdown_connections 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f768c06da40 0x7f768c06fef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76a00fc710 0x7f76a00fd970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 --2- 172.21.15.160:0/2503393567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a0101e20 0x7f76a00fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 >> 172.21.15.160:0/2503393567 conn(0x7f76a0071460 msgr2=0x7f76a00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.573+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 shutdown_connections 2024-05-25T06:15:06.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:06.574+0000 7f76a5ed6700 1 -- 172.21.15.160:0/2503393567 wait complete. 2024-05-25T06:15:06.584 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:15:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:06 smithi160 ceph-mon[29041]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:06.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:06 smithi203 ceph-mon[36030]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:07.251 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch ps' 2024-05-25T06:15:07.513 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:08.659 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:08 smithi160 ceph-mon[29041]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:08.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:08 smithi203 ceph-mon[36030]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/660957171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc00fbd40 msgr2=0x7fbdc00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 --2- 172.21.15.160:0/660957171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc00fbd40 0x7fbdc00fc1b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4004940 tx=0x7fbdb4009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/660957171 shutdown_connections 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 --2- 172.21.15.160:0/660957171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc00fbd40 0x7fbdc00fc1b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 --2- 172.21.15.160:0/660957171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc0101d90 0x7fbdc0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/660957171 >> 172.21.15.160:0/660957171 conn(0x7fbdc0071d00 msgr2=0x7fbdc0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/660957171 shutdown_connections 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.393+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/660957171 wait complete. 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc7b1c700 1 Processor -- start 2024-05-25T06:15:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc7b1c700 1 -- start start 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc7b1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc00fbd40 0x7fbdc010ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc7b1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 0x7fbdc010f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc7b1c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc010f790 con 0x7fbdc0101d90 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc7b1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc010f8d0 con 0x7fbdc00fbd40 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 0x7fbdc010f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 0x7fbdc010f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57722/0 (socket says 172.21.15.160:57722) 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 -- 172.21.15.160:0/1373019513 learned_addr learned my addr 172.21.15.160:0/1373019513 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 -- 172.21.15.160:0/1373019513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc00fbd40 msgr2=0x7fbdc010ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc00fbd40 0x7fbdc010ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 -- 172.21.15.160:0/1373019513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4005040 con 0x7fbdc0101d90 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.394+0000 7fbdc50b7700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 0x7fbdc010f1c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009150 tx=0x7fbdb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:09.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.395+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdb400e040 con 0x7fbdc0101d90 2024-05-25T06:15:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.395+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/1373019513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc0117b90 con 0x7fbdc0101d90 2024-05-25T06:15:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.395+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdb400bbc0 con 0x7fbdc0101d90 2024-05-25T06:15:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.395+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdb401c630 con 0x7fbdc0101d90 2024-05-25T06:15:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.395+0000 7fbdc7b1c700 1 -- 172.21.15.160:0/1373019513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc0118120 con 0x7fbdc0101d90 2024-05-25T06:15:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.396+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc00fd430 con 0x7fbdc0101d90 2024-05-25T06:15:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.396+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fbdb400bd30 con 0x7fbdc0101d90 2024-05-25T06:15:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.397+0000 7fbdbe7fc700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fbda806da50 0x7fbda806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.397+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbdb408e4e0 con 0x7fbdc0101d90 2024-05-25T06:15:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.397+0000 7fbdc58b8700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fbda806da50 0x7fbda806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.397+0000 7fbdc58b8700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fbda806da50 0x7fbda806ff00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8006fd0 tx=0x7fbdb8006db0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:09.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.401+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fbdb4058090 con 0x7fbdc0101d90 2024-05-25T06:15:09.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.607+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdc005f5c0 con 0x7fbda806da50 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.617+0000 7fbdbe7fc700 1 -- 172.21.15.160:0/1373019513 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbdc005f5c0 con 0x7fbda806da50 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (62s) 29s ago 12m 17.7M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (12m) 29s ago 12m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (11m) 93s ago 11m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (38s) 29s ago 11m 52.7M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (6m) 29s ago 6m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (6m) 29s ago 6m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (6m) 93s ago 6m 61.2M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:15:09.617 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (6m) 93s ago 6m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (2m) 29s ago 13m 508M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (114s) 93s ago 11m 452M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (13m) 29s ago 13m 54.5M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (11m) 93s ago 11m 41.7M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (101s) 29s ago 12m 8300k - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (97s) 93s ago 11m 3644k - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (10m) 29s ago 10m 217M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (9m) 29s ago 9m 225M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (9m) 29s ago 9m 189M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (9m) 93s ago 9m 267M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (8m) 93s ago 8m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (8m) 93s ago 8m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:15:09.618 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (79s) 29s ago 11m 53.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fbda806da50 msgr2=0x7fbda806ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fbda806da50 0x7fbda806ff00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8006fd0 tx=0x7fbdb8006db0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 msgr2=0x7fbdc010f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 0x7fbdc010f1c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009150 tx=0x7fbdb4009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 shutdown_connections 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc00fbd40 0x7fbdc010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fbda806da50 0x7fbda806ff00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 --2- 172.21.15.160:0/1373019513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc0101d90 0x7fbdc010f1c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:09.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 >> 172.21.15.160:0/1373019513 conn(0x7fbdc0071d00 msgr2=0x7fbdc00fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:09.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 shutdown_connections 2024-05-25T06:15:09.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:09.619+0000 7fbda7fff700 1 -- 172.21.15.160:0/1373019513 wait complete. 2024-05-25T06:15:10.280 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-25T06:15:10.281 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:15:10.281 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-05-25T06:15:10.542 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:10.664 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:10 smithi160 ceph-mon[29041]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:10.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:10 smithi203 ceph-mon[36030]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:11.686 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:11 smithi160 ceph-mon[29041]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:11.686 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:11 smithi160 ceph-mon[29041]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:11.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:11 smithi203 ceph-mon[36030]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:11.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:11 smithi203 ceph-mon[36030]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:12.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.318+0000 7f48069a3700 1 -- 172.21.15.160:0/755066137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 msgr2=0x7f48001020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:12.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.318+0000 7f48069a3700 1 --2- 172.21.15.160:0/755066137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f48001020b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f47f0004940 tx=0x7f47f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:12.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- 172.21.15.160:0/755066137 shutdown_connections 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 --2- 172.21.15.160:0/755066137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4800102680 0x7f48000fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 --2- 172.21.15.160:0/755066137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f48001020b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- 172.21.15.160:0/755066137 >> 172.21.15.160:0/755066137 conn(0x7f48000f7850 msgr2=0x7f48000f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- 172.21.15.160:0/755066137 shutdown_connections 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- 172.21.15.160:0/755066137 wait complete. 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 Processor -- start 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- start start 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f4800113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4800102680 0x7f4800114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48001146c0 con 0x7f4800101ce0 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f48069a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480010dab0 con 0x7f4800102680 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f47fffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f4800113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:12.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f47fffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f4800113b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57750/0 (socket says 172.21.15.160:57750) 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.319+0000 7f47fffff700 1 -- 172.21.15.160:0/2084860401 learned_addr learned my addr 172.21.15.160:0/2084860401 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47f7fff700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4800102680 0x7f4800114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fffff700 1 -- 172.21.15.160:0/2084860401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4800102680 msgr2=0x7f4800114070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fffff700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4800102680 0x7f4800114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fffff700 1 -- 172.21.15.160:0/2084860401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0005040 con 0x7f4800101ce0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fffff700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f4800113b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47f000b3b0 tx=0x7f47f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f000e040 con 0x7f4800101ce0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47f000bbc0 con 0x7f4800101ce0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480010dd30 con 0x7f4800101ce0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f001c3f0 con 0x7f4800101ce0 2024-05-25T06:15:12.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.320+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480010e2f0 con 0x7f4800101ce0 2024-05-25T06:15:12.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.321+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48000fd430 con 0x7f4800101ce0 2024-05-25T06:15:12.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.322+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f47f000bd30 con 0x7f4800101ce0 2024-05-25T06:15:12.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.322+0000 7f47fd7fa700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f47ec06da40 0x7f47ec06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:12.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.322+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f47f008e380 con 0x7f4800101ce0 2024-05-25T06:15:12.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.322+0000 7f47f7fff700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f47ec06da40 0x7f47ec06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:12.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.322+0000 7f47f7fff700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f47ec06da40 0x7f47ec06fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f480010ec70 tx=0x7f47e800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:12.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.326+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f47f005bb00 con 0x7f4800101ce0 2024-05-25T06:15:12.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.524+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f480010dec0 con 0x7f4800101ce0 2024-05-25T06:15:12.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.528+0000 7f47fd7fa700 1 -- 172.21.15.160:0/2084860401 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7f480010dec0 con 0x7f4800101ce0 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f47ec06da40 msgr2=0x7f47ec06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f47ec06da40 0x7f47ec06fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f480010ec70 tx=0x7f47e800b040 comp rx=0 tx=0).stop 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 msgr2=0x7f4800113b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f4800113b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47f000b3b0 tx=0x7f47f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 shutdown_connections 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f47ec06da40 0x7f47ec06fef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4800101ce0 0x7f4800113b30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:12.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 --2- 172.21.15.160:0/2084860401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4800102680 0x7f4800114070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:12.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 >> 172.21.15.160:0/2084860401 conn(0x7f48000f7850 msgr2=0x7f48000f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:12.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 shutdown_connections 2024-05-25T06:15:12.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:12.531+0000 7f48069a3700 1 -- 172.21.15.160:0/2084860401 wait complete. 2024-05-25T06:15:13.194 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-25T06:15:13.194 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:15:13.195 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-25T06:15:13.450 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2084860401' entity='client.admin' 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:13.824 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:13 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:15:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2084860401' entity='client.admin' 2024-05-25T06:15:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:15:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:13 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:15:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 -- 172.21.15.160:0/4048156060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c1026b0 msgr2=0x7f589c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 --2- 172.21.15.160:0/4048156060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c1026b0 0x7f589c0fbd20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5884004940 tx=0x7f5884009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 -- 172.21.15.160:0/4048156060 shutdown_connections 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 --2- 172.21.15.160:0/4048156060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c1026b0 0x7f589c0fbd20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 --2- 172.21.15.160:0/4048156060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589c101d10 0x7f589c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 -- 172.21.15.160:0/4048156060 >> 172.21.15.160:0/4048156060 conn(0x7f589c0f7860 msgr2=0x7f589c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 -- 172.21.15.160:0/4048156060 shutdown_connections 2024-05-25T06:15:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.118+0000 7f58a2564700 1 -- 172.21.15.160:0/4048156060 wait complete. 2024-05-25T06:15:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f58a2564700 1 Processor -- start 2024-05-25T06:15:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f58a2564700 1 -- start start 2024-05-25T06:15:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f58a2564700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 0x7f589c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f58a2564700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589c1026b0 0x7f589c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f58a2564700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c114510 con 0x7f589c101d10 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f58a2564700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c10db30 con 0x7f589c1026b0 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 0x7f589c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 0x7f589c113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57768/0 (socket says 172.21.15.160:57768) 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 -- 172.21.15.160:0/4224602167 learned_addr learned my addr 172.21.15.160:0/4224602167 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f5893fff700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589c1026b0 0x7f589c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 -- 172.21.15.160:0/4224602167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589c1026b0 msgr2=0x7f589c113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589c1026b0 0x7f589c113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 -- 172.21.15.160:0/4224602167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884005040 con 0x7f589c101d10 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.119+0000 7f589bfff700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 0x7f589c113980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c830 tx=0x7f588c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.120+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f588c00a760 con 0x7f589c101d10 2024-05-25T06:15:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.120+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f588c00ad90 con 0x7f589c101d10 2024-05-25T06:15:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.120+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589c10de10 con 0x7f589c101d10 2024-05-25T06:15:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.120+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f588c0113a0 con 0x7f589c101d10 2024-05-25T06:15:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.120+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589c10e3d0 con 0x7f589c101d10 2024-05-25T06:15:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.121+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589c0fd460 con 0x7f589c101d10 2024-05-25T06:15:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.121+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f588c012400 con 0x7f589c101d10 2024-05-25T06:15:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.122+0000 7f58997fa700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f588806da40 0x7f588806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:15.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.122+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f588c08d170 con 0x7f589c101d10 2024-05-25T06:15:15.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.122+0000 7f5893fff700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f588806da40 0x7f588806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:15.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.122+0000 7f5893fff700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f588806da40 0x7f588806fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f588400b3b0 tx=0x7f5884009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:15.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.126+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f588c056e60 con 0x7f589c101d10 2024-05-25T06:15:15.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.322+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f589c04c710 con 0x7f589c101d10 2024-05-25T06:15:15.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.323+0000 7f58997fa700 1 -- 172.21.15.160:0/4224602167 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7f588c05a520 con 0x7f589c101d10 2024-05-25T06:15:15.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f588806da40 msgr2=0x7f588806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:15.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f588806da40 0x7f588806fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f588400b3b0 tx=0x7f5884009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:15.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 msgr2=0x7f589c113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 0x7f589c113980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c830 tx=0x7f588c004470 comp rx=0 tx=0).stop 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 shutdown_connections 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f588806da40 0x7f588806fef0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589c101d10 0x7f589c113980 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 --2- 172.21.15.160:0/4224602167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589c1026b0 0x7f589c113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 >> 172.21.15.160:0/4224602167 conn(0x7f589c0f7860 msgr2=0x7f589c0f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 shutdown_connections 2024-05-25T06:15:15.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:15.325+0000 7f58a2564700 1 -- 172.21.15.160:0/4224602167 wait complete. 2024-05-25T06:15:15.950 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:15 smithi160 ceph-mon[29041]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:15.977 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-25T06:15:16.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:15 smithi203 ceph-mon[36030]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:16.235 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:17.908 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:17 smithi160 ceph-mon[29041]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:17.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 -- 172.21.15.160:0/704084363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 msgr2=0x7f4fcc0fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:17.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/704084363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc0fe1d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc004940 tx=0x7f4fbc009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 -- 172.21.15.160:0/704084363 shutdown_connections 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/704084363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fcc0fe7a0 0x7f4fcc106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/704084363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc0fe1d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 -- 172.21.15.160:0/704084363 >> 172.21.15.160:0/704084363 conn(0x7f4fcc0f7870 msgr2=0x7f4fcc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 -- 172.21.15.160:0/704084363 shutdown_connections 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 -- 172.21.15.160:0/704084363 wait complete. 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 Processor -- start 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.908+0000 7f4fd1ced700 1 -- start start 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fd1ced700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fd1ced700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fcc0fe7a0 0x7f4fcc113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fd1ced700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc114630 con 0x7f4fcc0fde00 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fd1ced700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10dba0 con 0x7f4fcc0fe7a0 2024-05-25T06:15:17.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59232/0 (socket says 172.21.15.160:59232) 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 -- 172.21.15.160:0/1434223757 learned_addr learned my addr 172.21.15.160:0/1434223757 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcaffd700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fcc0fe7a0 0x7f4fcc113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 -- 172.21.15.160:0/1434223757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fcc0fe7a0 msgr2=0x7f4fcc113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fcc0fe7a0 0x7f4fcc113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 -- 172.21.15.160:0/1434223757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc005040 con 0x7f4fcc0fde00 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fcb7fe700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc113a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009380 tx=0x7f4fbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fbc00e040 con 0x7f4fcc0fde00 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fbc009af0 con 0x7f4fcc0fde00 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc10de20 con 0x7f4fcc0fde00 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fbc01c630 con 0x7f4fcc0fde00 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.909+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc10e360 con 0x7f4fcc0fde00 2024-05-25T06:15:17.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.910+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc1017f0 con 0x7f4fcc0fde00 2024-05-25T06:15:17.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.911+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f4fbc00bbc0 con 0x7f4fcc0fde00 2024-05-25T06:15:17.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.911+0000 7f4fb3fff700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f4fb406da40 0x7f4fb406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:17.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.911+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4fbc08e500 con 0x7f4fcc0fde00 2024-05-25T06:15:17.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.911+0000 7f4fcaffd700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f4fb406da40 0x7f4fb406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:17.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.912+0000 7f4fcaffd700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f4fb406da40 0x7f4fb406fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006fd0 tx=0x7f4fc0008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:17.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:17.916+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f4fbc05bc80 con 0x7f4fcc0fde00 2024-05-25T06:15:18.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.118+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f4fcc109f10 con 0x7f4fcc0fde00 2024-05-25T06:15:18.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.118+0000 7f4fb3fff700 1 -- 172.21.15.160:0/1434223757 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7f4fbc05b810 con 0x7f4fcc0fde00 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f4fb406da40 msgr2=0x7f4fb406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f4fb406da40 0x7f4fb406fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006fd0 tx=0x7f4fc0008040 comp rx=0 tx=0).stop 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 msgr2=0x7f4fcc113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc113a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009380 tx=0x7f4fbc009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 shutdown_connections 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f4fb406da40 0x7f4fb406fef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fcc0fde00 0x7f4fcc113a10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 --2- 172.21.15.160:0/1434223757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fcc0fe7a0 0x7f4fcc113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.120+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 >> 172.21.15.160:0/1434223757 conn(0x7f4fcc0f7870 msgr2=0x7f4fcc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.121+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 shutdown_connections 2024-05-25T06:15:18.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:18.121+0000 7f4fd1ced700 1 -- 172.21.15.160:0/1434223757 wait complete. 2024-05-25T06:15:18.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:17 smithi203 ceph-mon[36030]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:18.786 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-25T06:15:19.043 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:19.863 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:19 smithi160 ceph-mon[29041]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:19.952 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:19 smithi203 ceph-mon[36030]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 -- 172.21.15.160:0/839991562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43340fbd40 msgr2=0x7f43340fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 --2- 172.21.15.160:0/839991562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43340fbd40 0x7f43340fc1b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4324004940 tx=0x7f4324009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 -- 172.21.15.160:0/839991562 shutdown_connections 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 --2- 172.21.15.160:0/839991562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43340fbd40 0x7f43340fc1b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 --2- 172.21.15.160:0/839991562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4334101d90 0x7f4334102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 -- 172.21.15.160:0/839991562 >> 172.21.15.160:0/839991562 conn(0x7f4334071d00 msgr2=0x7f4334072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:20.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 -- 172.21.15.160:0/839991562 shutdown_connections 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 -- 172.21.15.160:0/839991562 wait complete. 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 Processor -- start 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.721+0000 7f433bfed700 1 -- start start 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f433bfed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43340fbd40 0x7f4334113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f433bfed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 0x7f43341140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f433bfed700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4334114790 con 0x7f4334101d90 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f433bfed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433410daf0 con 0x7f43340fbd40 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 0x7f43341140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 0x7f43341140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59250/0 (socket says 172.21.15.160:59250) 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 -- 172.21.15.160:0/3458849085 learned_addr learned my addr 172.21.15.160:0/3458849085 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339d89700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43340fbd40 0x7f4334113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:20.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 -- 172.21.15.160:0/3458849085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43340fbd40 msgr2=0x7f4334113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43340fbd40 0x7f4334113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 -- 172.21.15.160:0/3458849085 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324005040 con 0x7f4334101d90 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339d89700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43340fbd40 0x7f4334113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f4339588700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 0x7f43341140b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f432400b3b0 tx=0x7f4324009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f432400e040 con 0x7f4334101d90 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f432400bbc0 con 0x7f4334101d90 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433410dd70 con 0x7f4334101d90 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.722+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f432401c3f0 con 0x7f4334101d90 2024-05-25T06:15:20.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.723+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433410e330 con 0x7f4334101d90 2024-05-25T06:15:20.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.723+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43341063c0 con 0x7f4334101d90 2024-05-25T06:15:20.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.724+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f432400bd30 con 0x7f4334101d90 2024-05-25T06:15:20.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.724+0000 7f432a7fc700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f432006da40 0x7f432006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:20.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.725+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f432408e4a0 con 0x7f4334101d90 2024-05-25T06:15:20.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.725+0000 7f4339d89700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f432006da40 0x7f432006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:20.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.725+0000 7f4339d89700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f432006da40 0x7f432006fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:20.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.729+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f432405bba0 con 0x7f4334101d90 2024-05-25T06:15:20.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.929+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f433410f1c0 con 0x7f4334101d90 2024-05-25T06:15:20.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.929+0000 7f432a7fc700 1 -- 172.21.15.160:0/3458849085 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f432405b730 con 0x7f4334101d90 2024-05-25T06:15:20.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f432006da40 msgr2=0x7f432006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:20.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f432006da40 0x7f432006fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330008040 comp rx=0 tx=0).stop 2024-05-25T06:15:20.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 msgr2=0x7f43341140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:20.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 0x7f43341140b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f432400b3b0 tx=0x7f4324009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 shutdown_connections 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43340fbd40 0x7f4334113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f432006da40 0x7f432006fef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 --2- 172.21.15.160:0/3458849085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4334101d90 0x7f43341140b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 >> 172.21.15.160:0/3458849085 conn(0x7f4334071d00 msgr2=0x7f4334065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 shutdown_connections 2024-05-25T06:15:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:20.932+0000 7f433bfed700 1 -- 172.21.15.160:0/3458849085 wait complete. 2024-05-25T06:15:21.617 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-25T06:15:21.879 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:21.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:21 smithi160 ceph-mon[29041]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:21 smithi203 ceph-mon[36030]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 -- 172.21.15.160:0/3443803075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd08101d90 msgr2=0x7fdd08102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/3443803075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd08101d90 0x7fdd08102160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8004940 tx=0x7fdcf8009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 -- 172.21.15.160:0/3443803075 shutdown_connections 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/3443803075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080fbd40 0x7fdd080fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/3443803075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd08101d90 0x7fdd08102160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 -- 172.21.15.160:0/3443803075 >> 172.21.15.160:0/3443803075 conn(0x7fdd08071d00 msgr2=0x7fdd08072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:23.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 -- 172.21.15.160:0/3443803075 shutdown_connections 2024-05-25T06:15:23.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.615+0000 7fdd0d39d700 1 -- 172.21.15.160:0/3443803075 wait complete. 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.616+0000 7fdd0d39d700 1 Processor -- start 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.616+0000 7fdd0d39d700 1 -- start start 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.616+0000 7fdd0d39d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 0x7fdd0810eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd0d39d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd08101d90 0x7fdd0810efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd0d39d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0810f630 con 0x7fdd080fbd40 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd0d39d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0810f770 con 0x7fdd08101d90 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 0x7fdd0810eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 0x7fdd0810eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59266/0 (socket says 172.21.15.160:59266) 2024-05-25T06:15:23.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 -- 172.21.15.160:0/1562383147 learned_addr learned my addr 172.21.15.160:0/1562383147 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd067fc700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd08101d90 0x7fdd0810efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 -- 172.21.15.160:0/1562383147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd08101d90 msgr2=0x7fdd0810efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd08101d90 0x7fdd0810efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 -- 172.21.15.160:0/1562383147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8005040 con 0x7fdd080fbd40 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd06ffd700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 0x7fdd0810eaa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800b8b0 tx=0x7fdcf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf800e040 con 0x7fdd080fbd40 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd08117b10 con 0x7fdd080fbd40 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcf8004290 con 0x7fdd080fbd40 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.617+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf801c3f0 con 0x7fdd080fbd40 2024-05-25T06:15:23.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.618+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd08118020 con 0x7fdd080fbd40 2024-05-25T06:15:23.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.619+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd080fd430 con 0x7fdd080fbd40 2024-05-25T06:15:23.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.619+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fdcf8013820 con 0x7fdd080fbd40 2024-05-25T06:15:23.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.620+0000 7fdcff7fe700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fdcf406da40 0x7fdcf406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:23.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.620+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdcf808e320 con 0x7fdd080fbd40 2024-05-25T06:15:23.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.620+0000 7fdd067fc700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fdcf406da40 0x7fdcf406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:23.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.620+0000 7fdd067fc700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fdcf406da40 0x7fdcf406fef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd08065fe0 tx=0x7fdcf000b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:23.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.625+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fdcf8058010 con 0x7fdd080fbd40 2024-05-25T06:15:23.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.844+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd08065d90 con 0x7fdcf406da40 2024-05-25T06:15:23.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.849+0000 7fdcff7fe700 1 -- 172.21.15.160:0/1562383147 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fdd08065d90 con 0x7fdcf406da40 2024-05-25T06:15:23.850 INFO:teuthology.orchestra.run.smithi160.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:15:23.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fdcf406da40 msgr2=0x7fdcf406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:23.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fdcf406da40 0x7fdcf406fef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd08065fe0 tx=0x7fdcf000b040 comp rx=0 tx=0).stop 2024-05-25T06:15:23.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 msgr2=0x7fdd0810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:23.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 0x7fdd0810eaa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800b8b0 tx=0x7fdcf8009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 shutdown_connections 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fdcf406da40 0x7fdcf406fef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd080fbd40 0x7fdd0810eaa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 --2- 172.21.15.160:0/1562383147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd08101d90 0x7fdd0810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 >> 172.21.15.160:0/1562383147 conn(0x7fdd08071d00 msgr2=0x7fdd080fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 shutdown_connections 2024-05-25T06:15:23.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:23.852+0000 7fdd0d39d700 1 -- 172.21.15.160:0/1562383147 wait complete. 2024-05-25T06:15:23.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:23 smithi160 ceph-mon[29041]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:23 smithi203 ceph-mon[36030]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:24.539 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-25T06:15:24.539 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:15:24.539 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-25T06:15:24.820 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:15:25.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:25.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:15:25.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:25.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:25.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:25.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:15:25.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:25.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:24 smithi203 ceph-mon[36030]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:15:25.186 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:25.186 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:15:25.186 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:25.186 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:25.186 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:25.186 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:15:25.187 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:25.187 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:24 smithi160 ceph-mon[29041]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:ea0684c08cbcf83f4af9930de2267b3563ac5f78 2024-05-25T06:15:26.112 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:25 smithi160 ceph-mon[29041]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:26.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:25 smithi203 ceph-mon[36030]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:27.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 -- 172.21.15.160:0/2531641501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fde00 msgr2=0x7fe1f80fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:27.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 --2- 172.21.15.160:0/2531641501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fde00 0x7fe1f80fe1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8004940 tx=0x7fe1e8009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 -- 172.21.15.160:0/2531641501 shutdown_connections 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 --2- 172.21.15.160:0/2531641501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 --2- 172.21.15.160:0/2531641501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fde00 0x7fe1f80fe1d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 -- 172.21.15.160:0/2531641501 >> 172.21.15.160:0/2531641501 conn(0x7fe1f80f78b0 msgr2=0x7fe1f80f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 -- 172.21.15.160:0/2531641501 shutdown_connections 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.763+0000 7fe1fd647700 1 -- 172.21.15.160:0/2531641501 wait complete. 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1fd647700 1 Processor -- start 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1fd647700 1 -- start start 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1fd647700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f80fde00 0x7fe1f8115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1fd647700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1fd647700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8116680 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1fd647700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f810fc60 con 0x7fe1f80fde00 2024-05-25T06:15:27.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1effff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1effff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44430/0 (socket says 172.21.15.160:44430) 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.764+0000 7fe1effff700 1 -- 172.21.15.160:0/3494736550 learned_addr learned my addr 172.21.15.160:0/3494736550 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1effff700 1 -- 172.21.15.160:0/3494736550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f80fde00 msgr2=0x7fe1f8115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1effff700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f80fde00 0x7fe1f8115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1effff700 1 -- 172.21.15.160:0/3494736550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8005040 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1effff700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8116030 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c830 tx=0x7fe1e0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e000a760 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1e000ad90 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f810ff40 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e00113a0 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.765+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f8110480 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.766+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f8066b50 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.767+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fe1e0012400 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.767+0000 7fe1ef7fe700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe1e406da40 0x7fe1e406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:27.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.767+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe1e008d200 con 0x7fe1f80fe7a0 2024-05-25T06:15:27.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.767+0000 7fe1f6ffd700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe1e406da40 0x7fe1e406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:27.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.768+0000 7fe1f6ffd700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe1e406da40 0x7fe1e406fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8004f90 tx=0x7fe1e8009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:27.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:27.772+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fe1e005a980 con 0x7fe1f80fe7a0 2024-05-25T06:15:28.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.001+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1f80fce80 con 0x7fe1e406da40 2024-05-25T06:15:28.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.002+0000 7fe1ef7fe700 1 -- 172.21.15.160:0/3494736550 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe1f80fce80 con 0x7fe1e406da40 2024-05-25T06:15:28.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe1e406da40 msgr2=0x7fe1e406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe1e406da40 0x7fe1e406fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8004f90 tx=0x7fe1e8009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 msgr2=0x7fe1f8116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8116030 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c830 tx=0x7fe1e0004470 comp rx=0 tx=0).stop 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 shutdown_connections 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f80fde00 0x7fe1f8115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe1e406da40 0x7fe1e406fef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 --2- 172.21.15.160:0/3494736550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80fe7a0 0x7fe1f8116030 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 >> 172.21.15.160:0/3494736550 conn(0x7fe1f80f78b0 msgr2=0x7fe1f80f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 shutdown_connections 2024-05-25T06:15:28.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.004+0000 7fe1fd647700 1 -- 172.21.15.160:0/3494736550 wait complete. 2024-05-25T06:15:28.015 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 -- 172.21.15.160:0/623315167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 msgr2=0x7fe4f80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 --2- 172.21.15.160:0/623315167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f80fbcf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0004940 tx=0x7fe4e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 -- 172.21.15.160:0/623315167 shutdown_connections 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 --2- 172.21.15.160:0/623315167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f80fbcf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 --2- 172.21.15.160:0/623315167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 0x7fe4f81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 -- 172.21.15.160:0/623315167 >> 172.21.15.160:0/623315167 conn(0x7fe4f80f7850 msgr2=0x7fe4f80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:28.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 -- 172.21.15.160:0/623315167 shutdown_connections 2024-05-25T06:15:28.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.111+0000 7fe4fea16700 1 -- 172.21.15.160:0/623315167 wait complete. 2024-05-25T06:15:28.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4fea16700 1 Processor -- start 2024-05-25T06:15:28.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4fea16700 1 -- start start 2024-05-25T06:15:28.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4fea16700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 0x7fe4f8112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4fea16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f8112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4fea16700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8113630 con 0x7fe4f8102680 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4fea16700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f810ca20 con 0x7fe4f8101ce0 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4effff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f8112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4effff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f8112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44448/0 (socket says 172.21.15.160:44448) 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.112+0000 7fe4effff700 1 -- 172.21.15.160:0/3795069084 learned_addr learned my addr 172.21.15.160:0/3795069084 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4effff700 1 -- 172.21.15.160:0/3795069084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 msgr2=0x7fe4f8112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4f7fff700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 0x7fe4f8112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4effff700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 0x7fe4f8112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4effff700 1 -- 172.21.15.160:0/3795069084 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e0005040 con 0x7fe4f8102680 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4f7fff700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 0x7fe4f8112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4effff700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f8112fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009100 tx=0x7fe4e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e000e040 con 0x7fe4f8102680 2024-05-25T06:15:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4e0013420 con 0x7fe4f8102680 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f810cca0 con 0x7fe4f8102680 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e001c610 con 0x7fe4f8102680 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.113+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f810d260 con 0x7fe4f8102680 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.114+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f80fd430 con 0x7fe4f8102680 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.115+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fe4e0013590 con 0x7fe4f8102680 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.115+0000 7fe4f57fa700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe4e406da40 0x7fe4e406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.115+0000 7fe4f7fff700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe4e406da40 0x7fe4e406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.115+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe4e008e310 con 0x7fe4f8102680 2024-05-25T06:15:28.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.116+0000 7fe4f7fff700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe4e406da40 0x7fe4e406fef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8004d80 tx=0x7fe4e800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:28.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.120+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fe4e005bb30 con 0x7fe4f8102680 2024-05-25T06:15:28.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.345+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4f810e160 con 0x7fe4e406da40 2024-05-25T06:15:28.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.347+0000 7fe4f57fa700 1 -- 172.21.15.160:0/3795069084 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe4f810e160 con 0x7fe4e406da40 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe4e406da40 msgr2=0x7fe4e406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe4e406da40 0x7fe4e406fef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8004d80 tx=0x7fe4e800b040 comp rx=0 tx=0).stop 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 msgr2=0x7fe4f8112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f8112fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009100 tx=0x7fe4e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 shutdown_connections 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4f8101ce0 0x7fe4f8112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fe4e406da40 0x7fe4e406fef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 --2- 172.21.15.160:0/3795069084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4f8102680 0x7fe4f8112fe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 >> 172.21.15.160:0/3795069084 conn(0x7fe4f80f7850 msgr2=0x7fe4f80f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:28.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 shutdown_connections 2024-05-25T06:15:28.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.349+0000 7fe4fea16700 1 -- 172.21.15.160:0/3795069084 wait complete. 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 -- 172.21.15.160:0/1632219971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 msgr2=0x7f7b9c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1632219971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c102160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c004940 tx=0x7f7b8c009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 -- 172.21.15.160:0/1632219971 shutdown_connections 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1632219971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b9c0fbd40 0x7f7b9c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1632219971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c102160 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 -- 172.21.15.160:0/1632219971 >> 172.21.15.160:0/1632219971 conn(0x7f7b9c071d00 msgr2=0x7f7b9c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 -- 172.21.15.160:0/1632219971 shutdown_connections 2024-05-25T06:15:28.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.453+0000 7f7ba2502700 1 -- 172.21.15.160:0/1632219971 wait complete. 2024-05-25T06:15:28.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7ba2502700 1 Processor -- start 2024-05-25T06:15:28.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7ba2502700 1 -- start start 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7ba2502700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b9c0fbd40 0x7f7b9c10ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7ba2502700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c10f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7ba2502700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c10f760 con 0x7f7b9c101d90 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7ba2502700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c10f8a0 con 0x7f7b9c0fbd40 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c10f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c10f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44458/0 (socket says 172.21.15.160:44458) 2024-05-25T06:15:28.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.454+0000 7f7b93fff700 1 -- 172.21.15.160:0/1011397339 learned_addr learned my addr 172.21.15.160:0/1011397339 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b9bfff700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b9c0fbd40 0x7f7b9c10ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b93fff700 1 -- 172.21.15.160:0/1011397339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b9c0fbd40 msgr2=0x7f7b9c10ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b93fff700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b9c0fbd40 0x7f7b9c10ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b93fff700 1 -- 172.21.15.160:0/1011397339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c005040 con 0x7f7b9c101d90 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b93fff700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c10f220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c830 tx=0x7f7b84004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8400a760 con 0x7f7b9c101d90 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b8400ad90 con 0x7f7b9c101d90 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b840113a0 con 0x7f7b9c101d90 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c117b90 con 0x7f7b9c101d90 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.455+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c118120 con 0x7f7b9c101d90 2024-05-25T06:15:28.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.456+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c0fd430 con 0x7f7b9c101d90 2024-05-25T06:15:28.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.457+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f7b84012400 con 0x7f7b9c101d90 2024-05-25T06:15:28.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.457+0000 7f7b997fa700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f7b8806da50 0x7f7b8806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.457+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7b8408d280 con 0x7f7b9c101d90 2024-05-25T06:15:28.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.457+0000 7f7b9bfff700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f7b8806da50 0x7f7b8806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.458+0000 7f7b9bfff700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f7b8806da50 0x7f7b8806ff00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b3b0 tx=0x7f7b8c009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:28.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.462+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f7b84056ed0 con 0x7f7b9c101d90 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: Upgrade: Target is version 19.0.0-2843-gea0684c0 (squid) 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee'] 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: Upgrade: Setting container_image for all mgr 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:28.642 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-25T06:15:28.643 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:28 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:15:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: Upgrade: Target is version 19.0.0-2843-gea0684c0 (squid) 2024-05-25T06:15:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee'] 2024-05-25T06:15:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:28.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:28.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: Upgrade: Setting container_image for all mgr 2024-05-25T06:15:28.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:28.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-25T06:15:28.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:28 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:15:28.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.679+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b9c04bed0 con 0x7f7b8806da50 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.689+0000 7f7b997fa700 1 -- 172.21.15.160:0/1011397339 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7b9c04bed0 con 0x7f7b8806da50 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (81s) 48s ago 12m 17.7M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (12m) 48s ago 12m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (11m) 112s ago 11m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (57s) 48s ago 12m 52.7M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (7m) 48s ago 7m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (6m) 48s ago 6m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:15:28.689 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (6m) 112s ago 6m 61.2M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (7m) 112s ago 7m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (2m) 48s ago 13m 508M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (2m) 112s ago 11m 452M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (13m) 48s ago 14m 54.5M 2048M 17.2.7-958-g7b01852c baf1547092e7 6e785f1f742d 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (11m) 112s ago 11m 41.7M 2048M 17.2.7-958-g7b01852c baf1547092e7 6f4175efcd45 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (2m) 48s ago 12m 8300k - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (116s) 112s ago 11m 3644k - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (10m) 48s ago 10m 217M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (10m) 48s ago 10m 225M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (9m) 48s ago 9m 189M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (9m) 112s ago 9m 267M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (8m) 112s ago 8m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:15:28.690 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (8m) 112s ago 8m 229M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:15:28.691 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (98s) 48s ago 11m 53.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:15:28.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f7b8806da50 msgr2=0x7f7b8806ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f7b8806da50 0x7f7b8806ff00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b3b0 tx=0x7f7b8c009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 msgr2=0x7f7b9c10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c10f220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c830 tx=0x7f7b84004470 comp rx=0 tx=0).stop 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 shutdown_connections 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b9c0fbd40 0x7f7b9c10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f7b8806da50 0x7f7b8806ff00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 --2- 172.21.15.160:0/1011397339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b9c101d90 0x7f7b9c10f220 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 >> 172.21.15.160:0/1011397339 conn(0x7f7b9c071d00 msgr2=0x7f7b9c0fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 shutdown_connections 2024-05-25T06:15:28.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.691+0000 7f7ba2502700 1 -- 172.21.15.160:0/1011397339 wait complete. 2024-05-25T06:15:28.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 -- 172.21.15.160:0/2699832758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580702a0 msgr2=0x7f6258070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 --2- 172.21.15.160:0/2699832758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580702a0 0x7f6258070670 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6248004970 tx=0x7f6248009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 -- 172.21.15.160:0/2699832758 shutdown_connections 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 --2- 172.21.15.160:0/2699832758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6258070bb0 0x7f6258109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 --2- 172.21.15.160:0/2699832758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580702a0 0x7f6258070670 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 -- 172.21.15.160:0/2699832758 >> 172.21.15.160:0/2699832758 conn(0x7f62580f8990 msgr2=0x7f62580fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 -- 172.21.15.160:0/2699832758 shutdown_connections 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.793+0000 7f6260388700 1 -- 172.21.15.160:0/2699832758 wait complete. 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f6260388700 1 Processor -- start 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f6260388700 1 -- start start 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f6260388700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62580702a0 0x7f6258115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f6260388700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 0x7f62581160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f6260388700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62581167b0 con 0x7f6258070bb0 2024-05-25T06:15:28.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f6260388700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625810fd00 con 0x7f62580702a0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f625d923700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 0x7f62581160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f625d923700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 0x7f62581160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44484/0 (socket says 172.21.15.160:44484) 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.794+0000 7f625d923700 1 -- 172.21.15.160:0/2068089136 learned_addr learned my addr 172.21.15.160:0/2068089136 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f625e124700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62580702a0 0x7f6258115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f625d923700 1 -- 172.21.15.160:0/2068089136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62580702a0 msgr2=0x7f6258115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f625d923700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62580702a0 0x7f6258115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f625d923700 1 -- 172.21.15.160:0/2068089136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248005040 con 0x7f6258070bb0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f625e124700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62580702a0 0x7f6258115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f625d923700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 0x7f62581160d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f625400b790 tx=0x7f6254009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625400f770 con 0x7f6258070bb0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f625400fda0 con 0x7f6258070bb0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625810ff80 con 0x7f6258070bb0 2024-05-25T06:15:28.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625400d310 con 0x7f6258070bb0 2024-05-25T06:15:28.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.795+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62581104c0 con 0x7f6258070bb0 2024-05-25T06:15:28.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.796+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62580719e0 con 0x7f6258070bb0 2024-05-25T06:15:28.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.796+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f6254010420 con 0x7f6258070bb0 2024-05-25T06:15:28.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.797+0000 7f624effd700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f624406da40 0x7f624406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:28.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.797+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f625408d1e0 con 0x7f6258070bb0 2024-05-25T06:15:28.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.797+0000 7f625e124700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f624406da40 0x7f624406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:28.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.797+0000 7f625e124700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f624406da40 0x7f624406fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6248004560 tx=0x7f6248009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:28.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:28.801+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f625405a960 con 0x7f6258070bb0 2024-05-25T06:15:29.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.089+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f625804c710 con 0x7f6258070bb0 2024-05-25T06:15:29.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.090+0000 7f624effd700 1 -- 172.21.15.160:0/2068089136 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+737 (secure 0 0 0) 0x7f625405a4f0 con 0x7f6258070bb0 2024-05-25T06:15:29.090 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:15:29.090 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:15:29.091 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 12, 2024-05-25T06:15:29.092 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:15:29.092 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:15:29.092 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f624406da40 msgr2=0x7f624406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f624406da40 0x7f624406fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6248004560 tx=0x7f6248009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 msgr2=0x7f62581160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 0x7f62581160d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f625400b790 tx=0x7f6254009250 comp rx=0 tx=0).stop 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 shutdown_connections 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62580702a0 0x7f6258115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f624406da40 0x7f624406fef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 --2- 172.21.15.160:0/2068089136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6258070bb0 0x7f62581160d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 >> 172.21.15.160:0/2068089136 conn(0x7f62580f8990 msgr2=0x7f62580f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:29.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 shutdown_connections 2024-05-25T06:15:29.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.093+0000 7f6260388700 1 -- 172.21.15.160:0/2068089136 wait complete. 2024-05-25T06:15:29.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 -- 172.21.15.160:0/2556012654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 msgr2=0x7f2750103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 --2- 172.21.15.160:0/2556012654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f2750103220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2744004940 tx=0x7f2744009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:29.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 -- 172.21.15.160:0/2556012654 shutdown_connections 2024-05-25T06:15:29.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 --2- 172.21.15.160:0/2556012654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 0x7f2750065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 --2- 172.21.15.160:0/2556012654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f2750103220 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 -- 172.21.15.160:0/2556012654 >> 172.21.15.160:0/2556012654 conn(0x7f2750071d00 msgr2=0x7f2750072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:29.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 -- 172.21.15.160:0/2556012654 shutdown_connections 2024-05-25T06:15:29.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.200+0000 7f2755031700 1 -- 172.21.15.160:0/2556012654 wait complete. 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f2755031700 1 Processor -- start 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f2755031700 1 -- start start 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f2755031700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 0x7f2750113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f2755031700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f27501140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f2755031700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2750114710 con 0x7f2750102e50 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f2755031700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275010db00 con 0x7f2750065640 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f274e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f27501140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f274e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f27501140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44494/0 (socket says 172.21.15.160:44494) 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.201+0000 7f274e59c700 1 -- 172.21.15.160:0/1440993168 learned_addr learned my addr 172.21.15.160:0/1440993168 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f274ed9d700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 0x7f2750113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f274e59c700 1 -- 172.21.15.160:0/1440993168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 msgr2=0x7f2750113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f274e59c700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 0x7f2750113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f274e59c700 1 -- 172.21.15.160:0/1440993168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744005040 con 0x7f2750102e50 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f274ed9d700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 0x7f2750113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f274e59c700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f27501140c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f274000c8c0 tx=0x7f27400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274000a980 con 0x7f2750102e50 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f274000ce60 con 0x7f2750102e50 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27400115e0 con 0x7f2750102e50 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275010dde0 con 0x7f2750102e50 2024-05-25T06:15:29.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.202+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275010e3a0 con 0x7f2750102e50 2024-05-25T06:15:29.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.204+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2750100100 con 0x7f2750102e50 2024-05-25T06:15:29.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.204+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f2740011740 con 0x7f2750102e50 2024-05-25T06:15:29.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.204+0000 7f27377fe700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f273806da40 0x7f273806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.205+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f274008d450 con 0x7f2750102e50 2024-05-25T06:15:29.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.205+0000 7f274ed9d700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f273806da40 0x7f273806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.205+0000 7f274ed9d700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f273806da40 0x7f273806fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2744004560 tx=0x7f2744009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:29.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.209+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f274005ba80 con 0x7f2750102e50 2024-05-25T06:15:29.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.459+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f275010f1d0 con 0x7f2750102e50 2024-05-25T06:15:29.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.460+0000 7f27377fe700 1 -- 172.21.15.160:0/1440993168 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f274005b610 con 0x7f2750102e50 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:e24 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 24 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:15:29.460 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:12:18.728742+0000 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:15:29.461 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:15:29.462 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f273806da40 msgr2=0x7f273806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f273806da40 0x7f273806fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2744004560 tx=0x7f2744009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 msgr2=0x7f27501140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f27501140c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f274000c8c0 tx=0x7f27400044c0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 shutdown_connections 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2750065640 0x7f2750113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f273806da40 0x7f273806fef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 --2- 172.21.15.160:0/1440993168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2750102e50 0x7f27501140c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 >> 172.21.15.160:0/1440993168 conn(0x7f2750071d00 msgr2=0x7f27500fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:29.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 shutdown_connections 2024-05-25T06:15:29.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.462+0000 7f2755031700 1 -- 172.21.15.160:0/1440993168 wait complete. 2024-05-25T06:15:29.464 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 24 2024-05-25T06:15:29.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.570+0000 7fb578fb5700 1 -- 172.21.15.160:0/1050380575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 msgr2=0x7fb574109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.570+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1050380575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb574109150 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb564004970 tx=0x7fb564009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.570+0000 7fb578fb5700 1 -- 172.21.15.160:0/1050380575 shutdown_connections 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.570+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1050380575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb574109150 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.570+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1050380575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5740702a0 0x7fb574070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.570+0000 7fb578fb5700 1 -- 172.21.15.160:0/1050380575 >> 172.21.15.160:0/1050380575 conn(0x7fb5740f8950 msgr2=0x7fb5740fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 -- 172.21.15.160:0/1050380575 shutdown_connections 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 -- 172.21.15.160:0/1050380575 wait complete. 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 Processor -- start 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 -- start start 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5740702a0 0x7fb574115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb5741162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5741168f0 con 0x7fb574070bb0 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.571+0000 7fb578fb5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57410fcc0 con 0x7fb5740702a0 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb5741162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb5741162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44500/0 (socket says 172.21.15.160:44500) 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 -- 172.21.15.160:0/1377371499 learned_addr learned my addr 172.21.15.160:0/1377371499 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb57259c700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5740702a0 0x7fb574115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 -- 172.21.15.160:0/1377371499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5740702a0 msgr2=0x7fb574115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5740702a0 0x7fb574115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 -- 172.21.15.160:0/1377371499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564005040 con 0x7fb574070bb0 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb571d9b700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb5741162a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:29.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56400e040 con 0x7fb574070bb0 2024-05-25T06:15:29.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb56400bc10 con 0x7fb574070bb0 2024-05-25T06:15:29.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57410ff40 con 0x7fb574070bb0 2024-05-25T06:15:29.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56401c3f0 con 0x7fb574070bb0 2024-05-25T06:15:29.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.572+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb574110480 con 0x7fb574070bb0 2024-05-25T06:15:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.573+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740719e0 con 0x7fb574070bb0 2024-05-25T06:15:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.574+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7fb564013420 con 0x7fb574070bb0 2024-05-25T06:15:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.575+0000 7fb562ffd700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fb55c06da40 0x7fb55c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.575+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb56408ee50 con 0x7fb574070bb0 2024-05-25T06:15:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.575+0000 7fb57259c700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fb55c06da40 0x7fb55c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.575+0000 7fb57259c700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fb55c06da40 0x7fb55c06fef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb568004f40 tx=0x7fb56800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:29.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.579+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fb56405c720 con 0x7fb574070bb0 2024-05-25T06:15:29.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:29.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:29.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/2068089136' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:29.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:29.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:15:29.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:15:29.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:29 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/2068089136' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:15:29.706 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:29 smithi160 ceph-mon[29041]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:29.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.809+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 --> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57410c150 con 0x7fb55c06da40 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.814+0000 7fb562ffd700 1 -- 172.21.15.160:0/1377371499 <== mgr.14688 v2:172.21.15.160:6800/1875516529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb57410c150 con 0x7fb55c06da40 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "2/21 daemons upgraded", 2024-05-25T06:15:29.814 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mon daemons", 2024-05-25T06:15:29.815 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:15:29.815 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:15:29.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fb55c06da40 msgr2=0x7fb55c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fb55c06da40 0x7fb55c06fef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb568004f40 tx=0x7fb56800b040 comp rx=0 tx=0).stop 2024-05-25T06:15:29.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 msgr2=0x7fb5741162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb5741162a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:29.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 shutdown_connections 2024-05-25T06:15:29.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5740702a0 0x7fb574115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7fb55c06da40 0x7fb55c06fef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 --2- 172.21.15.160:0/1377371499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb574070bb0 0x7fb5741162a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 >> 172.21.15.160:0/1377371499 conn(0x7fb5740f8950 msgr2=0x7fb5740f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:29.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 shutdown_connections 2024-05-25T06:15:29.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.816+0000 7fb578fb5700 1 -- 172.21.15.160:0/1377371499 wait complete. 2024-05-25T06:15:29.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 -- 172.21.15.160:0/338458259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14102df0 msgr2=0x7f1d141031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 --2- 172.21.15.160:0/338458259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14102df0 0x7f1d141031c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d08004940 tx=0x7f1d08009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:29.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 -- 172.21.15.160:0/338458259 shutdown_connections 2024-05-25T06:15:29.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 --2- 172.21.15.160:0/338458259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d14103790 0x7f1d140fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 --2- 172.21.15.160:0/338458259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14102df0 0x7f1d141031c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 -- 172.21.15.160:0/338458259 >> 172.21.15.160:0/338458259 conn(0x7f1d140f8980 msgr2=0x7f1d140fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:29.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 -- 172.21.15.160:0/338458259 shutdown_connections 2024-05-25T06:15:29.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 -- 172.21.15.160:0/338458259 wait complete. 2024-05-25T06:15:29.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 Processor -- start 2024-05-25T06:15:29.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.921+0000 7f1d1b009700 1 -- start start 2024-05-25T06:15:29.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d1b009700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d14102df0 0x7f1d14113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d1b009700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 0x7f1d141140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d1b009700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14114730 con 0x7f1d14103790 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d1b009700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410db40 con 0x7f1d14102df0 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 0x7f1d141140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 0x7f1d141140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44526/0 (socket says 172.21.15.160:44526) 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 -- 172.21.15.160:0/564338294 learned_addr learned my addr 172.21.15.160:0/564338294 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d18da5700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d14102df0 0x7f1d14113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 -- 172.21.15.160:0/564338294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d14102df0 msgr2=0x7f1d14113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d14102df0 0x7f1d14113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 -- 172.21.15.160:0/564338294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08005040 con 0x7f1d14103790 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d13fff700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 0x7f1d141140e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1d00002a50 tx=0x7f1d00004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d00007cc0 con 0x7f1d14103790 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d0000f040 con 0x7f1d14103790 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1410de20 con 0x7f1d14103790 2024-05-25T06:15:29.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0000d9c0 con 0x7f1d14103790 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.922+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1410e3e0 con 0x7f1d14103790 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.923+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d14100100 con 0x7f1d14103790 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.924+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 33) v1 ==== 91927+0+0 (secure 0 0 0) 0x7f1d00009730 con 0x7f1d14103790 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.925+0000 7f1d117fa700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1d0406da40 0x7f1d0406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.925+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1d0008d4b0 con 0x7f1d14103790 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.925+0000 7f1d18da5700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1d0406da40 0x7f1d0406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:15:29.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.925+0000 7f1d18da5700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1d0406da40 0x7f1d0406fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d08004f90 tx=0x7f1d08009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:15:29.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:29.929+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f1d000571a0 con 0x7f1d14103790 2024-05-25T06:15:30.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.219+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1d1410f1f0 con 0x7f1d14103790 2024-05-25T06:15:30.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.219+0000 7f1d117fa700 1 -- 172.21.15.160:0/564338294 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1d0005a860 con 0x7f1d14103790 2024-05-25T06:15:30.220 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:15:30.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1d0406da40 msgr2=0x7f1d0406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:30.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1d0406da40 0x7f1d0406fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d08004f90 tx=0x7f1d08009e30 comp rx=0 tx=0).stop 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 msgr2=0x7f1d141140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 0x7f1d141140e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1d00002a50 tx=0x7f1d00004600 comp rx=0 tx=0).stop 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 shutdown_connections 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d14102df0 0x7f1d14113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:6800/1875516529,v1:172.21.15.160:6801/1875516529] conn(0x7f1d0406da40 0x7f1d0406fef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 --2- 172.21.15.160:0/564338294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d14103790 0x7f1d141140e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 >> 172.21.15.160:0/564338294 conn(0x7f1d140f8980 msgr2=0x7f1d141073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 shutdown_connections 2024-05-25T06:15:30.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:15:30.221+0000 7f1d1b009700 1 -- 172.21.15.160:0/564338294 wait complete. 2024-05-25T06:15:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:30 smithi203 ceph-mon[36030]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:30 smithi203 ceph-mon[36030]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:30 smithi203 ceph-mon[36030]: Upgrade: Updating mon.smithi160 2024-05-25T06:15:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:30 smithi203 ceph-mon[36030]: Deploying daemon mon.smithi160 on smithi160 2024-05-25T06:15:30.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:30 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/1440993168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:15:30.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:30 smithi203 ceph-mon[36030]: from='client.? 172.21.15.160:0/564338294' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:15:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:30 smithi160 ceph-mon[29041]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:30 smithi160 ceph-mon[29041]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:15:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:30 smithi160 ceph-mon[29041]: Upgrade: Updating mon.smithi160 2024-05-25T06:15:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:30 smithi160 ceph-mon[29041]: Deploying daemon mon.smithi160 on smithi160 2024-05-25T06:15:30.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:30 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/1440993168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:15:30.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:30 smithi160 ceph-mon[29041]: from='client.? 172.21.15.160:0/564338294' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:15:31.296 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:31 smithi160 systemd[1]: Stopping Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:15:31.687 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:31 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[29037]: 2024-05-25T06:15:31.295+0000 7f0ca6c87700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi160 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:15:31.687 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:31 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[29037]: 2024-05-25T06:15:31.295+0000 7f0ca6c87700 -1 mon.smithi160@0(leader) e2 *** Got Signal Terminated *** 2024-05-25T06:15:31.688 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:31 smithi160 podman[111926]: 2024-05-25 06:15:31.418921307 +0000 UTC m=+0.229998422 container died 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-7b01852, org.label-schema.vendor=CentOS) 2024-05-25T06:15:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:31 smithi160 podman[111926]: 2024-05-25 06:15:31.787368252 +0000 UTC m=+0.598445364 container cleanup 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=quincy-7b01852, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-25T06:15:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:31 smithi160 bash[111926]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160 2024-05-25T06:15:32.390 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 podman[111951]: 2024-05-25 06:15:32.064032565 +0000 UTC m=+0.644997975 container remove 6e785f1f742db6246720227a19cd3637de8d050578a2234f67920981f5f07ec6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, RELEASE=quincy-7b01852, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-25T06:15:32.390 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service: Deactivated successfully. 2024-05-25T06:15:32.390 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 systemd[1]: Stopped Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:15:32.391 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service: Consumed 12.891s CPU time. 2024-05-25T06:15:32.704 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 systemd[1]: Starting Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:15:32.704 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 podman[112074]: 2024-05-25 06:15:32.508556015 +0000 UTC m=+0.017541660 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:15:32.704 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 podman[112074]: 2024-05-25 06:15:32.615011699 +0000 UTC m=+0.123997376 container create f6abc627de68ea95abdb82f6adbc13d0d578c05ef00174eca8685fc42e114ef9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c) 2024-05-25T06:15:32.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 podman[112074]: 2024-05-25 06:15:32.809479246 +0000 UTC m=+0.318464896 container init f6abc627de68ea95abdb82f6adbc13d0d578c05ef00174eca8685fc42e114ef9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-25T06:15:32.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 podman[112074]: 2024-05-25 06:15:32.813187764 +0000 UTC m=+0.322173405 container start f6abc627de68ea95abdb82f6adbc13d0d578c05ef00174eca8685fc42e114ef9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: set uid:gid to 167:167 (ceph:ceph) 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc), process ceph-mon, pid 2 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: pidfile_write: ignore empty --pid-file 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: load: jerasure load: lrc 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: RocksDB version: 7.9.2 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Git sha 0 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Compile date 2024-05-23 20:42:41 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: DB SUMMARY 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: DB Session ID: TI4YTZLDF9OH0L9WWHN7 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: CURRENT file: CURRENT 2024-05-25T06:15:32.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: IDENTITY file: IDENTITY 2024-05-25T06:15:32.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1166 Bytes 2024-05-25T06:15:32.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi160/store.db dir, Total Num: 1, files: 000033.sst 2024-05-25T06:15:32.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi160/store.db: 000031.log size: 1963023 ; 2024-05-25T06:15:32.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.error_if_exists: 0 2024-05-25T06:15:32.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.create_if_missing: 0 2024-05-25T06:15:32.972 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.paranoid_checks: 1 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.env: 0x55a6ceb21c60 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.fs: PosixFileSystem 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.info_log: 0x55a6d01eb9e0 2024-05-25T06:15:32.973 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_file_opening_threads: 16 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.statistics: (nil) 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.use_fsync: 0 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_log_file_size: 0 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.keep_log_file_num: 1000 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.recycle_log_file_num: 0 2024-05-25T06:15:32.974 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.allow_fallocate: 1 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.allow_mmap_reads: 0 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.allow_mmap_writes: 0 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.use_direct_reads: 0 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.create_missing_column_families: 0 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.db_log_dir: 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.wal_dir: 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-25T06:15:32.975 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.advise_random_on_open: 1 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.db_write_buffer_size: 0 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.write_buffer_manager: 0x55a6d01bc1e0 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.rate_limiter: (nil) 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-25T06:15:32.976 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.wal_recovery_mode: 2 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.enable_thread_tracking: 0 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.enable_pipelined_write: 0 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.unordered_write: 0 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.row_cache: None 2024-05-25T06:15:32.977 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.wal_filter: None 2024-05-25T06:15:32.978 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.allow_ingest_behind: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.two_write_queues: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.manual_wal_flush: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.wal_compression: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.atomic_flush: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.log_readahead_size: 0 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-25T06:15:32.979 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.best_efforts_recovery: 0 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.allow_data_in_errors: 0 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.db_host_id: __hostname__ 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_background_jobs: 2 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_background_compactions: -1 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_subcompactions: 1 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_total_wal_size: 0 2024-05-25T06:15:32.980 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_open_files: -1 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bytes_per_sync: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_readahead_size: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_background_flushes: -1 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Compression algorithms supported: 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kZSTD supported: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kXpressCompression supported: 0 2024-05-25T06:15:32.981 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kLZ4HCCompression supported: 1 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kZlibCompression supported: 1 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kSnappyCompression supported: 1 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kLZ4Compression supported: 1 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: kBZip2Compression supported: 0 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi160/store.db/MANIFEST-000015 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.merge_operator: 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_filter: None 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_filter_factory: None 2024-05-25T06:15:32.982 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.sst_partitioner_factory: None 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a6d01ebae0) 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: cache_index_and_filter_blocks: 1 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: pin_top_level_index_and_filter: 1 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: index_type: 0 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: data_block_index_type: 0 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: index_shortening: 1 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: checksum: 4 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: no_block_cache: 0 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache: 0x55a6d01ed090 2024-05-25T06:15:32.983 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache_name: BinnedLRUCache 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache_options: 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: capacity : 536870912 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: num_shard_bits : 4 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: strict_capacity_limit : 0 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: high_pri_pool_ratio: 0.000 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_cache_compressed: (nil) 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: persistent_cache: (nil) 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_size: 4096 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_size_deviation: 10 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_restart_interval: 16 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: index_block_restart_interval: 1 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: metadata_block_size: 4096 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: partition_filters: 0 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: use_delta_encoding: 1 2024-05-25T06:15:32.984 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: filter_policy: bloomfilter 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: whole_key_filtering: 1 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: verify_compression: 0 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: read_amp_bytes_per_bit: 0 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: format_version: 5 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: enable_index_compression: 1 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: block_align: 0 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: max_auto_readahead_size: 262144 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: prepopulate_block_cache: 0 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: initial_auto_readahead_size: 8192 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: num_file_reads_for_auto_readahead: 2 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.write_buffer_size: 33554432 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_write_buffer_number: 2 2024-05-25T06:15:32.985 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression: NoCompression 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression: Disabled 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.prefix_extractor: nullptr 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.num_levels: 7 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-25T06:15:32.986 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.level: 32767 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.strategy: 0 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.enabled: false 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-25T06:15:32.987 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.target_file_size_base: 67108864 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-25T06:15:32.988 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.arena_block_size: 1048576 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.disable_auto_compactions: 0 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-25T06:15:32.989 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.table_properties_collectors: 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.inplace_update_support: 0 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-25T06:15:32.990 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.bloom_locality: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.max_successive_merges: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.paranoid_file_checks: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.force_consistency_checks: 1 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.report_bg_io_stats: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.ttl: 2592000 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-25T06:15:32.991 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.enable_blob_files: false 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.min_blob_size: 0 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.blob_file_size: 268435456 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.blob_file_starting_level: 0 2024-05-25T06:15:32.992 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi160/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12003, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 55c71099-df43-4919-9c1a-658159af5fb0 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617732842776, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617732849216, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1741069, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12004, "largest_seqno": 13534, "table_properties": {"data_size": 1733944, "index_size": 4382, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1733, "raw_key_size": 16277, "raw_average_key_size": 23, "raw_value_size": 1719363, "raw_average_value_size": 2499, "num_data_blocks": 200, "num_entries": 688, "num_filter_entries": 688, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716617732, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "55c71099-df43-4919-9c1a-658159af5fb0", "db_session_id": "TI4YTZLDF9OH0L9WWHN7", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617732849316, "job": 1, "event": "recovery_finished"} 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi160/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a6d02dc000 2024-05-25T06:15:32.993 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: DB pointer 0x55a6d02c6000 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** DB Stats ** 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-25T06:15:32.994 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: L0 1/0 1.66 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 514.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: L6 1/0 6.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Sum 2/0 8.56 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 514.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 514.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 514.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-25T06:15:32.995 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-25T06:15:32.996 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-25T06:15:32.997 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-25T06:15:32.997 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Cumulative compaction: 0.00 GB write, 184.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:15:32.997 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Interval compaction: 0.00 GB write, 184.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:15:32.997 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-25T06:15:32.997 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Block cache BinnedLRUCache@0x55a6d01ed090#2 capacity: 512.00 MB usage: 6.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-25T06:15:32.997 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.58 KB,0.000873208%) Misc(1,0.00 KB,0%) 2024-05-25T06:15:32.998 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:32.998 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-25T06:15:32.998 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: starting mon.smithi160 rank 0 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi160 fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:15:32.998 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???) e2 preinit fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:15:32.998 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).mds e24 new map 2024-05-25T06:15:32.998 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).mds e24 print_map 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: e24 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: legacy client fscid: 1 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: Filesystem 'cephfs' (1) 2024-05-25T06:15:32.999 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: fs_name cephfs 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: epoch 24 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: modified 2024-05-25T06:12:18.728742+0000 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: tableserver 0 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: root 0 2024-05-25T06:15:33.000 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: session_timeout 60 2024-05-25T06:15:33.001 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: session_autoclose 300 2024-05-25T06:15:33.001 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: max_file_size 1099511627776 2024-05-25T06:15:33.001 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: max_xattr_size 65536 2024-05-25T06:15:33.001 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: required_client_features {} 2024-05-25T06:15:33.001 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: last_failure 0 2024-05-25T06:15:33.001 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: last_failure_osd_epoch 46 2024-05-25T06:15:33.002 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:15:33.002 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: max_mds 2 2024-05-25T06:15:33.002 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: in 0,1 2024-05-25T06:15:33.002 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: up {0=14480,1=24271} 2024-05-25T06:15:33.002 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: failed 2024-05-25T06:15:33.002 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: damaged 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: stopped 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: data_pools [3] 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: metadata_pool 2 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: inline_data disabled 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: balancer 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: bal_rank_mask -1 2024-05-25T06:15:33.003 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: standby_count_wanted 1 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: qdb_cluster leader: 0 members: 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: [mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: [mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: [mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: [mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:33.004 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout: 2024-05-25T06:15:33.005 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-25T06:15:33.005 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-25T06:15:33.005 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-25T06:15:33.005 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-25T06:15:33.005 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 ceph-mon[112094]: mon.smithi160@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-05-25T06:15:33.005 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 bash[112074]: f6abc627de68ea95abdb82f6adbc13d0d578c05ef00174eca8685fc42e114ef9 2024-05-25T06:15:33.006 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:32 smithi160 systemd[1]: Started Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:15:36.178 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:15:36.178 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:15:36.178 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:36.178 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: mon.smithi160 calling monitor election 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: mon.smithi160 is new leader, mons smithi160,smithi203 in quorum (ranks 0,1) 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: monmap epoch 2 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: last_changed 2024-05-25T06:04:02.551194+0000 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: created 2024-05-25T06:01:25.425099+0000 2024-05-25T06:15:36.179 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: min_mon_release 17 (quincy) 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: election_strategy: 1 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: 0: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.smithi160 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.smithi203 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: osdmap e48: 6 total, 6 up, 6 in 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: mgrmap e33: smithi160.nalzlg(active, since 2m), standbys: smithi203.rotqex 2024-05-25T06:15:36.180 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: overall HEALTH_OK 2024-05-25T06:15:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: from='mgr.14688 ' entity='' 2024-05-25T06:15:36.181 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:35 smithi160 ceph-mon[112094]: mgrmap e34: smithi160.nalzlg(active, since 2m), standbys: smithi203.rotqex 2024-05-25T06:15:36.249 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:15:36.249 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: from='mgr.14688 172.21.15.160:0/622054027' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:15:36.249 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:15:36.249 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:15:36.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: mon.smithi160 calling monitor election 2024-05-25T06:15:36.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: mon.smithi160 is new leader, mons smithi160,smithi203 in quorum (ranks 0,1) 2024-05-25T06:15:36.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: monmap epoch 2 2024-05-25T06:15:36.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:15:36.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: last_changed 2024-05-25T06:04:02.551194+0000 2024-05-25T06:15:36.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: created 2024-05-25T06:01:25.425099+0000 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: min_mon_release 17 (quincy) 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: election_strategy: 1 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: 0: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.smithi160 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.smithi203 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: osdmap e48: 6 total, 6 up, 6 in 2024-05-25T06:15:36.251 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: mgrmap e33: smithi160.nalzlg(active, since 2m), standbys: smithi203.rotqex 2024-05-25T06:15:36.252 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: overall HEALTH_OK 2024-05-25T06:15:36.252 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: from='mgr.14688 ' entity='' 2024-05-25T06:15:36.252 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:35 smithi203 ceph-mon[36030]: mgrmap e34: smithi160.nalzlg(active, since 2m), standbys: smithi203.rotqex 2024-05-25T06:15:42.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:42 smithi160 ceph-mon[112094]: Standby manager daemon smithi203.rotqex restarted 2024-05-25T06:15:42.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:42 smithi160 ceph-mon[112094]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:15:42.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:42 smithi160 ceph-mon[112094]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:15:42.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:42 smithi160 ceph-mon[112094]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:15:42.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:42 smithi160 ceph-mon[112094]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:15:42.906 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:42 smithi160 ceph-mon[112094]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:15:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:42 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex restarted 2024-05-25T06:15:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:42 smithi203 ceph-mon[36030]: Standby manager daemon smithi203.rotqex started 2024-05-25T06:15:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:42 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/crt"}]: dispatch 2024-05-25T06:15:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:42 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-25T06:15:43.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:42 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi203.rotqex/key"}]: dispatch 2024-05-25T06:15:43.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:42 smithi203 ceph-mon[36030]: from='mgr.? 172.21.15.203:0/1412692429' entity='mgr.smithi203.rotqex' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-25T06:15:44.071 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: mgrmap e35: smithi160.nalzlg(active, since 3m), standbys: smithi203.rotqex 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: Active manager daemon smithi160.nalzlg restarted 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: osdmap e49: 6 total, 6 up, 6 in 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: mgrmap e36: smithi160.nalzlg(active, starting, since 0.00662517s), standbys: smithi203.rotqex 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:15:44.072 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:15:44.073 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:43 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:15:44.103 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: mgrmap e35: smithi160.nalzlg(active, since 3m), standbys: smithi203.rotqex 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: Active manager daemon smithi160.nalzlg restarted 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: Activating manager daemon smithi160.nalzlg 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: osdmap e49: 6 total, 6 up, 6 in 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: mgrmap e36: smithi160.nalzlg(active, starting, since 0.00662517s), standbys: smithi203.rotqex 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi160.nalzlg", "id": "smithi160.nalzlg"}]: dispatch 2024-05-25T06:15:44.104 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr metadata", "who": "smithi203.rotqex", "id": "smithi203.rotqex"}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: Manager daemon smithi160.nalzlg is now available 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:15:44.105 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/mirror_snapshot_schedule"}]: dispatch 2024-05-25T06:15:44.106 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi160.nalzlg/trash_purge_schedule"}]: dispatch 2024-05-25T06:15:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: [25/May/2024:06:15:43] ENGINE Bus STARTING 2024-05-25T06:15:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: [25/May/2024:06:15:43] ENGINE Serving on http://172.21.15.160:8765 2024-05-25T06:15:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: [25/May/2024:06:15:43] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:15:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: [25/May/2024:06:15:43] ENGINE Bus STARTED 2024-05-25T06:15:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: mgrmap e37: smithi160.nalzlg(active, since 1.01122s), standbys: smithi203.rotqex 2024-05-25T06:15:44.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:44.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:44 smithi203 ceph-mon[36030]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:15:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: [25/May/2024:06:15:43] ENGINE Bus STARTING 2024-05-25T06:15:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: [25/May/2024:06:15:43] ENGINE Serving on http://172.21.15.160:8765 2024-05-25T06:15:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: [25/May/2024:06:15:43] ENGINE Serving on https://172.21.15.160:7150 2024-05-25T06:15:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: [25/May/2024:06:15:43] ENGINE Bus STARTED 2024-05-25T06:15:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: mgrmap e37: smithi160.nalzlg(active, since 1.01122s), standbys: smithi203.rotqex 2024-05-25T06:15:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:45.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:44 smithi160 ceph-mon[112094]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:15:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:45 smithi203 ceph-mon[36030]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:45 smithi203 ceph-mon[36030]: mgrmap e38: smithi160.nalzlg(active, since 2s), standbys: smithi203.rotqex 2024-05-25T06:15:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:45 smithi160 ceph-mon[112094]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:45 smithi160 ceph-mon[112094]: mgrmap e38: smithi160.nalzlg(active, since 2s), standbys: smithi203.rotqex 2024-05-25T06:15:47.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:46 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:47.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:46 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:47.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:47.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:47 smithi160 ceph-mon[112094]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:47 smithi160 ceph-mon[112094]: mgrmap e39: smithi160.nalzlg(active, since 4s), standbys: smithi203.rotqex 2024-05-25T06:15:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:47 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:48.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:47 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:47 smithi203 ceph-mon[36030]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:47 smithi203 ceph-mon[36030]: mgrmap e39: smithi160.nalzlg(active, since 4s), standbys: smithi203.rotqex 2024-05-25T06:15:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:47 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:48.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:47 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:49.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:49.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:49.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:49 smithi160 ceph-mon[112094]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:49.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:49 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:49.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:49 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:49.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:49 smithi203 ceph-mon[36030]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:15:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:50 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:50 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:50.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:50 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:50.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:50 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:51.841 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:51 smithi160 ceph-mon[112094]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-05-25T06:15:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:51 smithi203 ceph-mon[36030]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-05-25T06:15:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:53 smithi160 ceph-mon[112094]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-25T06:15:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:53 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:53 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:53 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:53 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:15:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:53 smithi203 ceph-mon[36030]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-25T06:15:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:53 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:53 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:54.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:53 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:54.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:53 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:15:55.174 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Detected new or changed devices on smithi160 2024-05-25T06:15:55.174 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:55.175 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:54 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-25T06:15:55.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Detected new or changed devices on smithi160 2024-05-25T06:15:55.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-25T06:15:55.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:15:55.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-25T06:15:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:55 smithi160 ceph-mon[112094]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:15:56.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:55 smithi160 ceph-mon[112094]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:15:56.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:55 smithi160 ceph-mon[112094]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-25T06:15:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:55 smithi203 ceph-mon[36030]: Updating smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:15:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:55 smithi203 ceph-mon[36030]: Updating smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-25T06:15:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:55 smithi203 ceph-mon[36030]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: Upgrade: Updating mon.smithi203 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: Deploying daemon mon.smithi203 on smithi203 2024-05-25T06:15:57.269 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 ceph-mon[36030]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-25T06:15:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: Upgrade: Updating mon.smithi203 2024-05-25T06:15:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:15:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:15:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:15:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:15:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: Deploying daemon mon.smithi203 on smithi203 2024-05-25T06:15:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:15:57 smithi160 ceph-mon[112094]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-25T06:15:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:57 smithi203 systemd[1]: Stopping Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:15:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:58 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203[36012]: 2024-05-25T06:15:58.125+0000 7f881b1a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi203 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:15:58.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:58 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203[36012]: 2024-05-25T06:15:58.125+0000 7f881b1a5700 -1 mon.smithi203@1(peon) e2 *** Got Signal Terminated *** 2024-05-25T06:15:58.526 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:58 smithi203 podman[83350]: 2024-05-25 06:15:58.238252482 +0000 UTC m=+0.202716853 container died 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, RELEASE=quincy-7b01852, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-25T06:15:58.877 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:58 smithi203 podman[83350]: 2024-05-25 06:15:58.626068647 +0000 UTC m=+0.590533024 container cleanup 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-7b01852, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:15:58.877 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:58 smithi203 bash[83350]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203 2024-05-25T06:15:59.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:58 smithi203 podman[83362]: 2024-05-25 06:15:58.87744217 +0000 UTC m=+0.639030428 container remove 6f4175efcd45df73439e2167cf055cd111460da8042503a11ad50cc86b47956a (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-7b01852, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-25T06:15:59.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203.service: Deactivated successfully. 2024-05-25T06:15:59.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 systemd[1]: Stopped Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:15:59.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203.service: Consumed 7.125s CPU time. 2024-05-25T06:15:59.526 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 systemd[1]: Starting Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:15:59.526 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 podman[83489]: 2024-05-25 06:15:59.329729028 +0000 UTC m=+0.017721923 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:15:59.526 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 podman[83489]: 2024-05-25 06:15:59.437138805 +0000 UTC m=+0.125131723 container create bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:15:59.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 podman[83489]: 2024-05-25 06:15:59.598334725 +0000 UTC m=+0.286327594 container init bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 podman[83489]: 2024-05-25 06:15:59.601626846 +0000 UTC m=+0.289619723 container start bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: set uid:gid to 167:167 (ceph:ceph) 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc), process ceph-mon, pid 2 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: pidfile_write: ignore empty --pid-file 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: load: jerasure load: lrc 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: RocksDB version: 7.9.2 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Git sha 0 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Compile date 2024-05-23 20:42:41 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: DB SUMMARY 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: DB Session ID: R9HYG4HX2CUS8LXJ04AI 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: CURRENT file: CURRENT 2024-05-25T06:15:59.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: IDENTITY file: IDENTITY 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1091 Bytes 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi203/store.db dir, Total Num: 1, files: 000027.sst 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi203/store.db: 000025.log size: 5818238 ; 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.error_if_exists: 0 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.create_if_missing: 0 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.paranoid_checks: 1 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.env: 0x5615543d1c60 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.fs: PosixFileSystem 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.info_log: 0x561556c439e0 2024-05-25T06:15:59.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_file_opening_threads: 16 2024-05-25T06:15:59.903 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.statistics: (nil) 2024-05-25T06:15:59.903 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.use_fsync: 0 2024-05-25T06:15:59.903 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_log_file_size: 0 2024-05-25T06:15:59.903 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-25T06:15:59.903 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.keep_log_file_num: 1000 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.recycle_log_file_num: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.allow_fallocate: 1 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.allow_mmap_reads: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.allow_mmap_writes: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.use_direct_reads: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.create_missing_column_families: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.db_log_dir: 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.wal_dir: 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-25T06:15:59.904 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.advise_random_on_open: 1 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.db_write_buffer_size: 0 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.write_buffer_manager: 0x561556c141e0 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.rate_limiter: (nil) 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.wal_recovery_mode: 2 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.enable_thread_tracking: 0 2024-05-25T06:15:59.905 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.enable_pipelined_write: 0 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.unordered_write: 0 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.row_cache: None 2024-05-25T06:15:59.906 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.wal_filter: None 2024-05-25T06:15:59.907 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-25T06:15:59.908 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.allow_ingest_behind: 0 2024-05-25T06:15:59.908 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.two_write_queues: 0 2024-05-25T06:15:59.908 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.manual_wal_flush: 0 2024-05-25T06:15:59.908 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.wal_compression: 0 2024-05-25T06:15:59.908 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.atomic_flush: 0 2024-05-25T06:15:59.908 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-25T06:15:59.909 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-25T06:15:59.909 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-25T06:15:59.909 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.log_readahead_size: 0 2024-05-25T06:15:59.909 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.best_efforts_recovery: 0 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.allow_data_in_errors: 0 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.db_host_id: __hostname__ 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_background_jobs: 2 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_background_compactions: -1 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_subcompactions: 1 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-25T06:15:59.910 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_total_wal_size: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_open_files: -1 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bytes_per_sync: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_readahead_size: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_background_flushes: -1 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Compression algorithms supported: 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kZSTD supported: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kXpressCompression supported: 0 2024-05-25T06:15:59.911 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kLZ4HCCompression supported: 1 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kZlibCompression supported: 1 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kSnappyCompression supported: 1 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kLZ4Compression supported: 1 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: kBZip2Compression supported: 0 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi203/store.db/MANIFEST-000009 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.merge_operator: 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_filter: None 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_filter_factory: None 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.sst_partitioner_factory: None 2024-05-25T06:15:59.912 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561556c43ae0) 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: cache_index_and_filter_blocks: 1 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: pin_top_level_index_and_filter: 1 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: index_type: 0 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: data_block_index_type: 0 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: index_shortening: 1 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: checksum: 4 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: no_block_cache: 0 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache: 0x561556c45090 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache_name: BinnedLRUCache 2024-05-25T06:15:59.913 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache_options: 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: capacity : 536870912 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: num_shard_bits : 4 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: strict_capacity_limit : 0 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: high_pri_pool_ratio: 0.000 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_cache_compressed: (nil) 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: persistent_cache: (nil) 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_size: 4096 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_size_deviation: 10 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_restart_interval: 16 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: index_block_restart_interval: 1 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: metadata_block_size: 4096 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: partition_filters: 0 2024-05-25T06:15:59.914 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: use_delta_encoding: 1 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: filter_policy: bloomfilter 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: whole_key_filtering: 1 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: verify_compression: 0 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: read_amp_bytes_per_bit: 0 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: format_version: 5 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: enable_index_compression: 1 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: block_align: 0 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: max_auto_readahead_size: 262144 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: prepopulate_block_cache: 0 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: initial_auto_readahead_size: 8192 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: num_file_reads_for_auto_readahead: 2 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.write_buffer_size: 33554432 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_write_buffer_number: 2 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression: NoCompression 2024-05-25T06:15:59.915 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression: Disabled 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.prefix_extractor: nullptr 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.num_levels: 7 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-25T06:15:59.916 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.level: 32767 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.strategy: 0 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.enabled: false 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.target_file_size_base: 67108864 2024-05-25T06:15:59.917 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-25T06:15:59.918 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.arena_block_size: 1048576 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.disable_auto_compactions: 0 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-25T06:15:59.919 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.table_properties_collectors: 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.inplace_update_support: 0 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.bloom_locality: 0 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.max_successive_merges: 0 2024-05-25T06:15:59.920 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.paranoid_file_checks: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.force_consistency_checks: 1 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.report_bg_io_stats: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.ttl: 2592000 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.enable_blob_files: false 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.min_blob_size: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.blob_file_size: 268435456 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-25T06:15:59.921 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.blob_file_starting_level: 0 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi203/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12269, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1e92b2d3-c240-4f00-b8d7-320dab0d5694 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617759631468, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617759644689, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3557461, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12274, "largest_seqno": 14487, "table_properties": {"data_size": 3548268, "index_size": 5869, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2309, "raw_key_size": 21556, "raw_average_key_size": 23, "raw_value_size": 3528917, "raw_average_value_size": 3865, "num_data_blocks": 266, "num_entries": 913, "num_filter_entries": 913, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716617759, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1e92b2d3-c240-4f00-b8d7-320dab0d5694", "db_session_id": "R9HYG4HX2CUS8LXJ04AI", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716617759644803, "job": 1, "event": "recovery_finished"} 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-25T06:15:59.922 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi203/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561556d34000 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: DB pointer 0x561556d1e000 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** DB Stats ** 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-25T06:15:59.923 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: L0 1/0 3.39 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 667.7 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: L6 1/0 6.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Sum 2/0 10.29 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 667.7 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 667.7 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 667.7 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-25T06:15:59.924 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Cumulative compaction: 0.00 GB write, 189.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Interval compaction: 0.00 GB write, 189.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Block cache BinnedLRUCache@0x561556c45090#2 capacity: 512.00 MB usage: 8.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.53 KB,0.000482798%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: starting mon.smithi203 rank 1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi203 fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:15:59.925 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???) e2 preinit fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).mds e26 new map 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).mds e26 print_map 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: e26 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: legacy client fscid: 1 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: Filesystem 'cephfs' (1) 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: fs_name cephfs 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: epoch 26 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:15:59.926 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: tableserver 0 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: root 0 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: session_timeout 60 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: session_autoclose 300 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: max_file_size 1099511627776 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: max_xattr_size 65536 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: required_client_features {} 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: last_failure 0 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: last_failure_osd_epoch 46 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: max_mds 2 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: in 0,1 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: up {0=14480,1=24271} 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: failed 2024-05-25T06:15:59.927 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: damaged 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: stopped 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: data_pools [3] 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: metadata_pool 2 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: inline_data disabled 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: balancer 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: bal_rank_mask -1 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: standby_count_wanted 1 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: [mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: [mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: [mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: [mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout: 2024-05-25T06:15:59.928 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-25T06:15:59.929 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-25T06:15:59.929 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-25T06:15:59.929 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-25T06:15:59.929 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 ceph-mon[83507]: mon.smithi203@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-05-25T06:15:59.929 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 bash[83489]: bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb 2024-05-25T06:15:59.929 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:15:59 smithi203 systemd[1]: Started Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:16:00.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 -- 172.21.15.160:0/2484484292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c101cf0 msgr2=0x7f464c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 --2- 172.21.15.160:0/2484484292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c101cf0 0x7f464c1020c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f463c004940 tx=0x7f463c009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:00.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 -- 172.21.15.160:0/2484484292 shutdown_connections 2024-05-25T06:16:00.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 --2- 172.21.15.160:0/2484484292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c102690 0x7f464c0fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 --2- 172.21.15.160:0/2484484292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c101cf0 0x7f464c1020c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 -- 172.21.15.160:0/2484484292 >> 172.21.15.160:0/2484484292 conn(0x7f464c0f7860 msgr2=0x7f464c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 -- 172.21.15.160:0/2484484292 shutdown_connections 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.340+0000 7f465156d700 1 -- 172.21.15.160:0/2484484292 wait complete. 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f465156d700 1 Processor -- start 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f465156d700 1 -- start start 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f465156d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 0x7f464c10eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f465156d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 0x7f464c10f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f465156d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c10f560 con 0x7f464c102690 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f465156d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c10f6a0 con 0x7f464c101cf0 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 0x7f464c10f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 0x7f464c10f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36290/0 (socket says 172.21.15.160:36290) 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 -- 172.21.15.160:0/1933414833 learned_addr learned my addr 172.21.15.160:0/1933414833 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f464affd700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 0x7f464c10eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:00.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f464affd700 1 -- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 msgr2=0x7f464c10eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f464affd700 1 -- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 msgr2=0x7f464c10eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f464affd700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 0x7f464c10eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f464affd700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 0x7f464c10eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 -- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 msgr2=0x7f464c10eae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 0x7f464c10eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 -- 172.21.15.160:0/1933414833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c005040 con 0x7f464c102690 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.341+0000 7f4643fff700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 0x7f464c10f020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4634002a80 tx=0x7f4634004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.342+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4634007cc0 con 0x7f464c102690 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.342+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463400f040 con 0x7f464c102690 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.342+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463400db30 con 0x7f464c102690 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.342+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c117b50 con 0x7f464c102690 2024-05-25T06:16:00.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.342+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c117f70 con 0x7f464c102690 2024-05-25T06:16:00.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.343+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c0fd440 con 0x7f464c102690 2024-05-25T06:16:00.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.343+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f463400dcd0 con 0x7f464c102690 2024-05-25T06:16:00.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.344+0000 7f46437fe700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f463806e270 0x7f4638070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:00.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.344+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f463408d410 con 0x7f464c102690 2024-05-25T06:16:00.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.344+0000 7f464affd700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f463806e270 0x7f4638070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:00.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.344+0000 7f464affd700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f463806e270 0x7f4638070720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f463c004f90 tx=0x7f463c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:00.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.349+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f46340565c0 con 0x7f464c102690 2024-05-25T06:16:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.567+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f464c108e80 con 0x7f463806e270 2024-05-25T06:16:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.568+0000 7f46437fe700 1 -- 172.21.15.160:0/1933414833 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f464c108e80 con 0x7f463806e270 2024-05-25T06:16:00.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.570+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f463806e270 msgr2=0x7f4638070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.570+0000 7f465156d700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f463806e270 0x7f4638070720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f463c004f90 tx=0x7f463c009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.570+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 msgr2=0x7f464c10f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.570+0000 7f465156d700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 0x7f464c10f020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4634002a80 tx=0x7f4634004630 comp rx=0 tx=0).stop 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.570+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 shutdown_connections 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.571+0000 7f465156d700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f463806e270 0x7f4638070720 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.571+0000 7f465156d700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f464c101cf0 0x7f464c10eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.571+0000 7f465156d700 1 --2- 172.21.15.160:0/1933414833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f464c102690 0x7f464c10f020 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.571+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 >> 172.21.15.160:0/1933414833 conn(0x7f464c0f7860 msgr2=0x7f464c0f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.571+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 shutdown_connections 2024-05-25T06:16:00.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.571+0000 7f465156d700 1 -- 172.21.15.160:0/1933414833 wait complete. 2024-05-25T06:16:00.581 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 -- 172.21.15.160:0/697681493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8065640 msgr2=0x7fa9a8065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 --2- 172.21.15.160:0/697681493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8065640 0x7fa9a8065ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa994004940 tx=0x7fa994009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 -- 172.21.15.160:0/697681493 shutdown_connections 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 --2- 172.21.15.160:0/697681493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8065640 0x7fa9a8065ab0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 --2- 172.21.15.160:0/697681493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a8102e50 0x7fa9a8103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 -- 172.21.15.160:0/697681493 >> 172.21.15.160:0/697681493 conn(0x7fa9a8071d00 msgr2=0x7fa9a8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 -- 172.21.15.160:0/697681493 shutdown_connections 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.674+0000 7fa9af135700 1 -- 172.21.15.160:0/697681493 wait complete. 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9af135700 1 Processor -- start 2024-05-25T06:16:00.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9af135700 1 -- start start 2024-05-25T06:16:00.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9af135700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a8065640 0x7fa9a8113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9af135700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 0x7fa9a81140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9af135700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a8114700 con 0x7fa9a8102e50 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9af135700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a810daf0 con 0x7fa9a8065640 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 0x7fa9a81140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 0x7fa9a81140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36308/0 (socket says 172.21.15.160:36308) 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9a7fff700 1 -- 172.21.15.160:0/1089480572 learned_addr learned my addr 172.21.15.160:0/1089480572 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.675+0000 7fa9aced1700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a8065640 0x7fa9a8113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a7fff700 1 -- 172.21.15.160:0/1089480572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a8065640 msgr2=0x7fa9a8113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a7fff700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a8065640 0x7fa9a8113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a7fff700 1 -- 172.21.15.160:0/1089480572 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994005040 con 0x7fa9a8102e50 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a7fff700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 0x7fa9a81140b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b3b0 tx=0x7fa994009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa99400e040 con 0x7fa9a8102e50 2024-05-25T06:16:00.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa99400bbc0 con 0x7fa9a8102e50 2024-05-25T06:16:00.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a810dd70 con 0x7fa9a8102e50 2024-05-25T06:16:00.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a810e330 con 0x7fa9a8102e50 2024-05-25T06:16:00.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.676+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa99401c3f0 con 0x7fa9a8102e50 2024-05-25T06:16:00.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.677+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a8100100 con 0x7fa9a8102e50 2024-05-25T06:16:00.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.677+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fa99401c660 con 0x7fa9a8102e50 2024-05-25T06:16:00.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.678+0000 7fa9a57fa700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa99806e260 0x7fa998070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:00.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.678+0000 7fa9aced1700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa99806e260 0x7fa998070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:00.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.678+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa99408f020 con 0x7fa9a8102e50 2024-05-25T06:16:00.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.679+0000 7fa9aced1700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa99806e260 0x7fa998070710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa99c007fd0 tx=0x7fa99c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:00.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.683+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa99405bd00 con 0x7fa9a8102e50 2024-05-25T06:16:00.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.901+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9a8113840 con 0x7fa99806e260 2024-05-25T06:16:00.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.903+0000 7fa9a57fa700 1 -- 172.21.15.160:0/1089480572 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa9a8113840 con 0x7fa99806e260 2024-05-25T06:16:00.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa99806e260 msgr2=0x7fa998070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa99806e260 0x7fa998070710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa99c007fd0 tx=0x7fa99c009040 comp rx=0 tx=0).stop 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 msgr2=0x7fa9a81140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 0x7fa9a81140b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b3b0 tx=0x7fa994009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 shutdown_connections 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa99806e260 0x7fa998070710 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a8065640 0x7fa9a8113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 --2- 172.21.15.160:0/1089480572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9a8102e50 0x7fa9a81140b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 >> 172.21.15.160:0/1089480572 conn(0x7fa9a8071d00 msgr2=0x7fa9a80fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 shutdown_connections 2024-05-25T06:16:00.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:00.905+0000 7fa9af135700 1 -- 172.21.15.160:0/1089480572 wait complete. 2024-05-25T06:16:01.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 -- 172.21.15.160:0/3421923674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 msgr2=0x7f9f840fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/3421923674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f840fc1b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9f70004940 tx=0x7f9f70009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 -- 172.21.15.160:0/3421923674 shutdown_connections 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/3421923674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f840fc1b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/3421923674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 -- 172.21.15.160:0/3421923674 >> 172.21.15.160:0/3421923674 conn(0x7f9f84071d00 msgr2=0x7f9f84072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 -- 172.21.15.160:0/3421923674 shutdown_connections 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.009+0000 7f9f8ae83700 1 -- 172.21.15.160:0/3421923674 wait complete. 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f8ae83700 1 Processor -- start 2024-05-25T06:16:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f8ae83700 1 -- start start 2024-05-25T06:16:01.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f8ae83700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f84102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f8ae83700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f8ae83700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f841035f0 con 0x7f9f840fbd40 2024-05-25T06:16:01.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f8ae83700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f84103730 con 0x7f9f84101d90 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f88c1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f84102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f88c1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f84102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36332/0 (socket says 172.21.15.160:36332) 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f88c1f700 1 -- 172.21.15.160:0/1424916870 learned_addr learned my addr 172.21.15.160:0/1424916870 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f83fff700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f83fff700 1 -- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 msgr2=0x7f9f84103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f83fff700 1 -- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 msgr2=0x7f9f84103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f88c1f700 1 -- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 msgr2=0x7f9f84103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f83fff700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f83fff700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84103060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f88c1f700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84103060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.010+0000 7f9f88c1f700 1 -- 172.21.15.160:0/1424916870 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70005040 con 0x7f9f840fbd40 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.011+0000 7f9f88c1f700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f84102b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f78002a50 tx=0x7f9f780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.011+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f78007cc0 con 0x7f9f840fbd40 2024-05-25T06:16:01.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.011+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f7800f040 con 0x7f9f840fbd40 2024-05-25T06:16:01.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.011+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f84119eb0 con 0x7f9f840fbd40 2024-05-25T06:16:01.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.011+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f78009640 con 0x7f9f840fbd40 2024-05-25T06:16:01.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.011+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8411a330 con 0x7f9f840fbd40 2024-05-25T06:16:01.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.012+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f840fd430 con 0x7f9f840fbd40 2024-05-25T06:16:01.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.012+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f9f780097a0 con 0x7f9f840fbd40 2024-05-25T06:16:01.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.013+0000 7f9f817fa700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f7406e260 0x7f9f74070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.013+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9f7808db60 con 0x7f9f840fbd40 2024-05-25T06:16:01.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.013+0000 7f9f83fff700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f7406e260 0x7f9f74070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.013+0000 7f9f83fff700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f7406e260 0x7f9f74070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009fd0 tx=0x7f9f70009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:01.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.017+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9f78056d10 con 0x7f9f840fbd40 2024-05-25T06:16:01.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.152+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f78092020 con 0x7f9f840fbd40 2024-05-25T06:16:01.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.234+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f8404bed0 con 0x7f9f7406e260 2024-05-25T06:16:01.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.244+0000 7f9f817fa700 1 -- 172.21.15.160:0/1424916870 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9f8404bed0 con 0x7f9f7406e260 2024-05-25T06:16:01.244 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:16:01.244 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (114s) 12s ago 12m 17.7M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:16:01.244 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (13m) 12s ago 13m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:16:01.244 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (12m) 15s ago 12m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:16:01.244 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (89s) 12s ago 12m 74.2M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (7m) 12s ago 7m 61.2M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (7m) 12s ago 7m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (7m) 15s ago 7m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (7m) 15s ago 7m 116M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (3m) 12s ago 14m 475M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (2m) 15s ago 12m 449M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (28s) 12s ago 14m 27.6M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 starting - - - 2048M 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (2m) 12s ago 13m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (2m) 15s ago 12m 17.1M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (11m) 12s ago 11m 219M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (10m) 12s ago 10m 227M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:16:01.245 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (10m) 12s ago 10m 191M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:16:01.246 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (9m) 15s ago 9m 269M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:16:01.246 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (9m) 15s ago 9m 230M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:16:01.246 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (9m) 15s ago 9m 230M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:16:01.246 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (2m) 12s ago 12m 63.1M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.246+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f7406e260 msgr2=0x7f9f74070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.246+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f7406e260 0x7f9f74070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009fd0 tx=0x7f9f70009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.246+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 msgr2=0x7f9f84102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.246+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f84102b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f78002a50 tx=0x7f9f780046b0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 shutdown_connections 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f840fbd40 0x7f9f84102b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f7406e260 0x7f9f74070710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 --2- 172.21.15.160:0/1424916870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f84101d90 0x7f9f84103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 >> 172.21.15.160:0/1424916870 conn(0x7f9f84071d00 msgr2=0x7f9f840655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:01.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 shutdown_connections 2024-05-25T06:16:01.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.247+0000 7f9f8ae83700 1 -- 172.21.15.160:0/1424916870 wait complete. 2024-05-25T06:16:01.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 -- 172.21.15.160:0/3800631791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69780fb3c0 msgr2=0x7f69780fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 --2- 172.21.15.160:0/3800631791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69780fb3c0 0x7f69780fb830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6960004940 tx=0x7f6960009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:01.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 -- 172.21.15.160:0/3800631791 shutdown_connections 2024-05-25T06:16:01.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 --2- 172.21.15.160:0/3800631791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69780fb3c0 0x7f69780fb830 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 --2- 172.21.15.160:0/3800631791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6978102e60 0x7f6978103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 -- 172.21.15.160:0/3800631791 >> 172.21.15.160:0/3800631791 conn(0x7f6978072320 msgr2=0x7f6978072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:01.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 -- 172.21.15.160:0/3800631791 shutdown_connections 2024-05-25T06:16:01.350 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:01 smithi160 ceph-mon[112094]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:16:01.350 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:01 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.349+0000 7f697d7a8700 1 -- 172.21.15.160:0/3800631791 wait complete. 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f697d7a8700 1 Processor -- start 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f697d7a8700 1 -- start start 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f697d7a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 0x7f69780fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f697d7a8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 0x7f69780fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f697d7a8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69780fe4f0 con 0x7f6978102e60 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f697d7a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69780fd020 con 0x7f69780fb3c0 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f696ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 0x7f69780fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f696ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 0x7f69780fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36352/0 (socket says 172.21.15.160:36352) 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f696ffff700 1 -- 172.21.15.160:0/1828572236 learned_addr learned my addr 172.21.15.160:0/1828572236 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:01.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f6976ffd700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 0x7f69780fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f6976ffd700 1 -- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 msgr2=0x7f69780fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f6976ffd700 1 -- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 msgr2=0x7f69780fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f6976ffd700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 0x7f69780fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f6976ffd700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 0x7f69780fc5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f696ffff700 1 -- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 msgr2=0x7f69780fc5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.350+0000 7f696ffff700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 0x7f69780fc5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f696ffff700 1 -- 172.21.15.160:0/1828572236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960005040 con 0x7f6978102e60 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f696ffff700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 0x7f69780fcae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6960009150 tx=0x7f6960009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f696000e040 con 0x7f6978102e60 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6960013420 con 0x7f6978102e60 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f696001c630 con 0x7f6978102e60 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69780fd2a0 con 0x7f6978102e60 2024-05-25T06:16:01.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.351+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697811a0a0 con 0x7f6978102e60 2024-05-25T06:16:01.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.352+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6978100160 con 0x7f6978102e60 2024-05-25T06:16:01.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.353+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f6960013590 con 0x7f6978102e60 2024-05-25T06:16:01.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.353+0000 7f696f7fe700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f696406e270 0x7f6964070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.353+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f696008eec0 con 0x7f6978102e60 2024-05-25T06:16:01.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.353+0000 7f6976ffd700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f696406e270 0x7f6964070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.354+0000 7f6976ffd700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f696406e270 0x7f6964070720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6968009e50 tx=0x7f6968009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:01.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.361+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6960058190 con 0x7f6978102e60 2024-05-25T06:16:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:01 smithi203 ceph-mon[83507]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:16:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:01 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:01.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.638+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6978109f50 con 0x7f6978102e60 2024-05-25T06:16:01.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.639+0000 7f696f7fe700 1 -- 172.21.15.160:0/1828572236 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f696005b850 con 0x7f6978102e60 2024-05-25T06:16:01.639 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:16:01.639 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 10, 2024-05-25T06:16:01.640 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:16:01.641 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:16:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f696406e270 msgr2=0x7f6964070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f696406e270 0x7f6964070720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6968009e50 tx=0x7f6968009f10 comp rx=0 tx=0).stop 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 msgr2=0x7f69780fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 0x7f69780fcae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6960009150 tx=0x7f6960009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 shutdown_connections 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f696406e270 0x7f6964070720 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69780fb3c0 0x7f69780fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 --2- 172.21.15.160:0/1828572236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6978102e60 0x7f69780fcae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.641+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 >> 172.21.15.160:0/1828572236 conn(0x7f6978072320 msgr2=0x7f69780fa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.642+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 shutdown_connections 2024-05-25T06:16:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.642+0000 7f697d7a8700 1 -- 172.21.15.160:0/1828572236 wait complete. 2024-05-25T06:16:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 -- 172.21.15.160:0/1987996628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0101c80 msgr2=0x7fa8e0102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/1987996628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0101c80 0x7fa8e0102050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0004940 tx=0x7fa8d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 -- 172.21.15.160:0/1987996628 shutdown_connections 2024-05-25T06:16:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/1987996628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0102620 0x7fa8e00fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/1987996628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0101c80 0x7fa8e0102050 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 -- 172.21.15.160:0/1987996628 >> 172.21.15.160:0/1987996628 conn(0x7fa8e00f7850 msgr2=0x7fa8e00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 -- 172.21.15.160:0/1987996628 shutdown_connections 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.744+0000 7fa8e7f79700 1 -- 172.21.15.160:0/1987996628 wait complete. 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e7f79700 1 Processor -- start 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e7f79700 1 -- start start 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e7f79700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 0x7fa8e0113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e7f79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 0x7fa8e0114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e7f79700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01146c0 con 0x7fa8e0102620 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e7f79700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e010dab0 con 0x7fa8e0101c80 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5514700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 0x7fa8e0114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5514700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 0x7fa8e0114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36384/0 (socket says 172.21.15.160:36384) 2024-05-25T06:16:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5514700 1 -- 172.21.15.160:0/667697918 learned_addr learned my addr 172.21.15.160:0/667697918 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5d15700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 0x7fa8e0113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5d15700 1 -- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 msgr2=0x7fa8e0113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5d15700 1 -- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 msgr2=0x7fa8e0113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5d15700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 0x7fa8e0113b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5514700 1 -- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 msgr2=0x7fa8e0113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5d15700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 0x7fa8e0113b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5514700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 0x7fa8e0113b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.745+0000 7fa8e5514700 1 -- 172.21.15.160:0/667697918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0005040 con 0x7fa8e0102620 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.746+0000 7fa8e5514700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 0x7fa8e0114070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc002a50 tx=0x7fa8dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.746+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8dc007cc0 con 0x7fa8e0102620 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.746+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8dc00f040 con 0x7fa8e0102620 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.746+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e010dd90 con 0x7fa8e0102620 2024-05-25T06:16:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.746+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8dc00da40 con 0x7fa8e0102620 2024-05-25T06:16:01.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.746+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e010e350 con 0x7fa8e0102620 2024-05-25T06:16:01.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.747+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e00fd3d0 con 0x7fa8e0102620 2024-05-25T06:16:01.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.747+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fa8dc0097c0 con 0x7fa8e0102620 2024-05-25T06:16:01.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.748+0000 7fa8d67fc700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa8cc06e270 0x7fa8cc070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:01.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.748+0000 7fa8e5d15700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa8cc06e270 0x7fa8cc070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:01.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.748+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa8dc08dac0 con 0x7fa8e0102620 2024-05-25T06:16:01.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.748+0000 7fa8e5d15700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa8cc06e270 0x7fa8cc070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009fd0 tx=0x7fa8d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:01.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.753+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa8dc05a820 con 0x7fa8e0102620 2024-05-25T06:16:01.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.992+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa8e0108d70 con 0x7fa8e0102620 2024-05-25T06:16:01.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.992+0000 7fa8d67fc700 1 -- 172.21.15.160:0/667697918 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7fa8dc05a3b0 con 0x7fa8e0102620 2024-05-25T06:16:01.992 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:16:01.992 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:16:01.993 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:16:01.994 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:16:01.995 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:16:01.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa8cc06e270 msgr2=0x7fa8cc070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa8cc06e270 0x7fa8cc070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009fd0 tx=0x7fa8d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:01.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 msgr2=0x7fa8e0114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 0x7fa8e0114070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc002a50 tx=0x7fa8dc004600 comp rx=0 tx=0).stop 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 shutdown_connections 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa8cc06e270 0x7fa8cc070720 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e0101c80 0x7fa8e0113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 --2- 172.21.15.160:0/667697918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0102620 0x7fa8e0114070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 >> 172.21.15.160:0/667697918 conn(0x7fa8e00f7850 msgr2=0x7fa8e00f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 shutdown_connections 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:01.995+0000 7fa8e7f79700 1 -- 172.21.15.160:0/667697918 wait complete. 2024-05-25T06:16:01.997 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:16:02.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 -- 172.21.15.160:0/3351106131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 msgr2=0x7f10401020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 --2- 172.21.15.160:0/3351106131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f10401020e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1028004940 tx=0x7f1028009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:02.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 -- 172.21.15.160:0/3351106131 shutdown_connections 2024-05-25T06:16:02.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 --2- 172.21.15.160:0/3351106131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f10400fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 --2- 172.21.15.160:0/3351106131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f10401020e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 -- 172.21.15.160:0/3351106131 >> 172.21.15.160:0/3351106131 conn(0x7f10400f78e0 msgr2=0x7f10400f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 -- 172.21.15.160:0/3351106131 shutdown_connections 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.098+0000 7f1044fed700 1 -- 172.21.15.160:0/3351106131 wait complete. 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f1044fed700 1 Processor -- start 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f1044fed700 1 -- start start 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f1044fed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f1040112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f1044fed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f1040112db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f1044fed700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040113400 con 0x7f1040101d10 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f1044fed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104010ca00 con 0x7f10401026b0 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f1040112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f1040112870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36396/0 (socket says 172.21.15.160:36396) 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 -- 172.21.15.160:0/1100258268 learned_addr learned my addr 172.21.15.160:0/1100258268 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:02.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103dd9b700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f1040112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103dd9b700 1 -- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 msgr2=0x7f1040112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103dd9b700 1 -- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 msgr2=0x7f1040112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103dd9b700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f1040112db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 -- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 msgr2=0x7f1040112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103dd9b700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f1040112db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f1040112db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 -- 172.21.15.160:0/1100258268 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028005040 con 0x7f1040101d10 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.099+0000 7f103e59c700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f1040112870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f102800b3b0 tx=0x7f1028009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.100+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102800e040 con 0x7f1040101d10 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.100+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f102800bbc0 con 0x7f1040101d10 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.100+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104010cc80 con 0x7f1040101d10 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.100+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102801c3f0 con 0x7f1040101d10 2024-05-25T06:16:02.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.100+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104010d240 con 0x7f1040101d10 2024-05-25T06:16:02.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.101+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10400fd460 con 0x7f1040101d10 2024-05-25T06:16:02.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.101+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f102800bd30 con 0x7f1040101d10 2024-05-25T06:16:02.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.101+0000 7f1036ffd700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102c06e270 0x7f102c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:02.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.102+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f102808ef20 con 0x7f1040101d10 2024-05-25T06:16:02.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.102+0000 7f103dd9b700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102c06e270 0x7f102c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:02.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.102+0000 7f103dd9b700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102c06e270 0x7f102c070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f104010dbc0 tx=0x7f1030009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:02.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.106+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f10280c3850 con 0x7f1040101d10 2024-05-25T06:16:02.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.322+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1040108e30 con 0x7f102c06e270 2024-05-25T06:16:02.322 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: overall HEALTH_OK 2024-05-25T06:16:02.322 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:16:02.322 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: mon.smithi203 calling monitor election 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: mon.smithi160 calling monitor election 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: mon.smithi160 is new leader, mons smithi160,smithi203 in quorum (ranks 0,1) 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: monmap epoch 3 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: last_changed 2024-05-25T06:16:01.149994+0000 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: created 2024-05-25T06:01:25.425099+0000 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: min_mon_release 19 (squid) 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: election_strategy: 1 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: 0: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.smithi160 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.smithi203 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:16:02.323 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: osdmap e49: 6 total, 6 up, 6 in 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: mgrmap e39: smithi160.nalzlg(active, since 18s), standbys: smithi203.rotqex 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: overall HEALTH_OK 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1828572236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:02.324 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:02 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/667697918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:16:02.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.323+0000 7f1036ffd700 1 -- 172.21.15.160:0/1100258268 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f1040108e30 con 0x7f102c06e270 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "3/21 daemons upgraded", 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mon daemons", 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:16:02.325 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:16:02.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102c06e270 msgr2=0x7f102c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102c06e270 0x7f102c070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f104010dbc0 tx=0x7f1030009f70 comp rx=0 tx=0).stop 2024-05-25T06:16:02.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 msgr2=0x7f1040112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f1040112870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f102800b3b0 tx=0x7f1028009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 shutdown_connections 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1040101d10 0x7f1040112870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102c06e270 0x7f102c070720 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 --2- 172.21.15.160:0/1100258268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10401026b0 0x7f1040112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 >> 172.21.15.160:0/1100258268 conn(0x7f10400f78e0 msgr2=0x7f10400f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 shutdown_connections 2024-05-25T06:16:02.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.325+0000 7f1044fed700 1 -- 172.21.15.160:0/1100258268 wait complete. 2024-05-25T06:16:02.383 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: overall HEALTH_OK 2024-05-25T06:16:02.383 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi160"}]: dispatch 2024-05-25T06:16:02.383 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: mon.smithi203 calling monitor election 2024-05-25T06:16:02.383 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: mon.smithi160 calling monitor election 2024-05-25T06:16:02.383 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mon metadata", "id": "smithi203"}]: dispatch 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: mon.smithi160 is new leader, mons smithi160,smithi203 in quorum (ranks 0,1) 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: monmap epoch 3 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: last_changed 2024-05-25T06:16:01.149994+0000 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: created 2024-05-25T06:01:25.425099+0000 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: min_mon_release 19 (squid) 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: election_strategy: 1 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: 0: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.smithi160 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.smithi203 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2 up:standby-replay 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: osdmap e49: 6 total, 6 up, 6 in 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: mgrmap e39: smithi160.nalzlg(active, since 18s), standbys: smithi203.rotqex 2024-05-25T06:16:02.384 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: overall HEALTH_OK 2024-05-25T06:16:02.385 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:02.385 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:02.385 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:02.385 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:02.385 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1828572236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:02.385 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:02 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/667697918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:16:02.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.428+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/1724272765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 msgr2=0x7f3c981008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.428+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/1724272765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c981008a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c004940 tx=0x7f3c8c009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.428+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/1724272765 shutdown_connections 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.428+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/1724272765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c980fb5a0 0x7f3c980fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.428+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/1724272765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c981008a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.428+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/1724272765 >> 172.21.15.160:0/1724272765 conn(0x7f3c980714d0 msgr2=0x7f3c980718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/1724272765 shutdown_connections 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/1724272765 wait complete. 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 Processor -- start 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 -- start start 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c980fb5a0 0x7f3c980fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:02.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c980fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980ff0c0 con 0x7f3c981004d0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.429+0000 7f3c9f0e5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980fdbf0 con 0x7f3c980fb5a0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c980fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c980fd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36410/0 (socket says 172.21.15.160:36410) 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 -- 172.21.15.160:0/4200028677 learned_addr learned my addr 172.21.15.160:0/4200028677 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c9ce81700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c980fb5a0 0x7f3c980fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 -- 172.21.15.160:0/4200028677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c980fb5a0 msgr2=0x7f3c980fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c980fb5a0 0x7f3c980fd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 -- 172.21.15.160:0/4200028677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c005040 con 0x7f3c981004d0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c97fff700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c980fd6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400c830 tx=0x7f3c84004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c8400a760 con 0x7f3c981004d0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c980fded0 con 0x7f3c981004d0 2024-05-25T06:16:02.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c8400ad90 con 0x7f3c981004d0 2024-05-25T06:16:02.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c840113a0 con 0x7f3c981004d0 2024-05-25T06:16:02.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.430+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9811a0d0 con 0x7f3c981004d0 2024-05-25T06:16:02.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.431+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c98105a90 con 0x7f3c981004d0 2024-05-25T06:16:02.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.432+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f3c84012400 con 0x7f3c981004d0 2024-05-25T06:16:02.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.432+0000 7f3c957fa700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c8806e270 0x7f3c88070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:02.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.433+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3c8408dc80 con 0x7f3c981004d0 2024-05-25T06:16:02.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.433+0000 7f3c9ce81700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c8806e270 0x7f3c88070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:02.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.433+0000 7f3c9ce81700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c8806e270 0x7f3c88070720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c004f90 tx=0x7f3c8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:02.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.437+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3c84056fc0 con 0x7f3c981004d0 2024-05-25T06:16:02.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.713+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3c980fccb0 con 0x7f3c981004d0 2024-05-25T06:16:02.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.714+0000 7f3c957fa700 1 -- 172.21.15.160:0/4200028677 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3c8405a680 con 0x7f3c981004d0 2024-05-25T06:16:02.714 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:16:02.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c8806e270 msgr2=0x7f3c88070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c8806e270 0x7f3c88070720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c004f90 tx=0x7f3c8c009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:02.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 msgr2=0x7f3c980fd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:02.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c980fd6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400c830 tx=0x7f3c84004470 comp rx=0 tx=0).stop 2024-05-25T06:16:02.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 shutdown_connections 2024-05-25T06:16:02.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c8806e270 0x7f3c88070720 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c980fb5a0 0x7f3c980fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 --2- 172.21.15.160:0/4200028677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c981004d0 0x7f3c980fd6b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:02.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 >> 172.21.15.160:0/4200028677 conn(0x7f3c980714d0 msgr2=0x7f3c980f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:02.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 shutdown_connections 2024-05-25T06:16:02.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:02.716+0000 7f3c9f0e5700 1 -- 172.21.15.160:0/4200028677 wait complete. 2024-05-25T06:16:03.214 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:03 smithi203 ceph-mon[83507]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:03.214 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:03 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4200028677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:16:03.214 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:03 smithi203 ceph-mon[83507]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:03 smithi160 ceph-mon[112094]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:03 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4200028677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:16:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:03 smithi160 ceph-mon[112094]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:05 smithi203 ceph-mon[83507]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:05 smithi160 ceph-mon[112094]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:06.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:06.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:07.756 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:07 smithi203 ceph-mon[83507]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:07.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:07 smithi160 ceph-mon[112094]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:10.192 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: pgmap v16: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:10.192 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:16:10.193 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: pgmap v16: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:16:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:11.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:10 smithi160 ceph-mon[112094]: Detected new or changed devices on smithi203 2024-05-25T06:16:11.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:10 smithi160 ceph-mon[112094]: Reconfiguring mon.smithi160 (monmap changed)... 2024-05-25T06:16:11.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:10 smithi160 ceph-mon[112094]: Reconfiguring daemon mon.smithi160 on smithi160 2024-05-25T06:16:11.262 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:10 smithi203 ceph-mon[83507]: Detected new or changed devices on smithi203 2024-05-25T06:16:11.262 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:10 smithi203 ceph-mon[83507]: Reconfiguring mon.smithi160 (monmap changed)... 2024-05-25T06:16:11.262 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:10 smithi203 ceph-mon[83507]: Reconfiguring daemon mon.smithi160 on smithi160 2024-05-25T06:16:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:11 smithi160 ceph-mon[112094]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:16:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:12.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:16:12.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:16:12.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:11 smithi203 ceph-mon[83507]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:16:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi160.nalzlg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:16:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:16:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: Reconfiguring mgr.smithi160.nalzlg (monmap changed)... 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: Reconfiguring daemon mgr.smithi160.nalzlg on smithi160 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:13.213 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: Reconfiguring mgr.smithi160.nalzlg (monmap changed)... 2024-05-25T06:16:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: Reconfiguring daemon mgr.smithi160.nalzlg on smithi160 2024-05-25T06:16:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:13 smithi160 ceph-mon[112094]: Reconfiguring crash.smithi160 (monmap changed)... 2024-05-25T06:16:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:13 smithi160 ceph-mon[112094]: Reconfiguring daemon crash.smithi160 on smithi160 2024-05-25T06:16:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:13 smithi160 ceph-mon[112094]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:13 smithi203 ceph-mon[83507]: Reconfiguring crash.smithi160 (monmap changed)... 2024-05-25T06:16:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:13 smithi203 ceph-mon[83507]: Reconfiguring daemon crash.smithi160 on smithi160 2024-05-25T06:16:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:13 smithi203 ceph-mon[83507]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:15.313 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:15.313 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:15.314 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: Reconfiguring osd.0 (monmap changed)... 2024-05-25T06:16:15.314 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-25T06:16:15.314 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:15.314 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: Reconfiguring daemon osd.0 on smithi160 2024-05-25T06:16:15.314 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:15 smithi160 ceph-mon[112094]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:15.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: Reconfiguring osd.0 (monmap changed)... 2024-05-25T06:16:15.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-25T06:16:15.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:15.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: Reconfiguring daemon osd.0 on smithi160 2024-05-25T06:16:15.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:15 smithi203 ceph-mon[83507]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: Reconfiguring osd.1 (monmap changed)... 2024-05-25T06:16:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-25T06:16:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: Reconfiguring daemon osd.1 on smithi160 2024-05-25T06:16:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:17 smithi203 ceph-mon[83507]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: Reconfiguring osd.1 (monmap changed)... 2024-05-25T06:16:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-25T06:16:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: Reconfiguring daemon osd.1 on smithi160 2024-05-25T06:16:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:17 smithi160 ceph-mon[112094]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:18.932 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:18.932 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:18.932 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:18 smithi160 ceph-mon[112094]: Reconfiguring osd.2 (monmap changed)... 2024-05-25T06:16:18.932 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-25T06:16:18.932 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:18.932 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:18 smithi160 ceph-mon[112094]: Reconfiguring daemon osd.2 on smithi160 2024-05-25T06:16:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:18 smithi203 ceph-mon[83507]: Reconfiguring osd.2 (monmap changed)... 2024-05-25T06:16:19.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-25T06:16:19.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:19.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:18 smithi203 ceph-mon[83507]: Reconfiguring daemon osd.2 on smithi160 2024-05-25T06:16:20.052 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:19 smithi160 ceph-mon[112094]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:20.052 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:20.052 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:20.052 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:20.052 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:19 smithi203 ceph-mon[83507]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:20.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:21.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:20 smithi203 ceph-mon[83507]: Reconfiguring mds.cephfs.smithi160.rrhzig (monmap changed)... 2024-05-25T06:16:21.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:20 smithi203 ceph-mon[83507]: Reconfiguring daemon mds.cephfs.smithi160.rrhzig on smithi160 2024-05-25T06:16:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:20 smithi160 ceph-mon[112094]: Reconfiguring mds.cephfs.smithi160.rrhzig (monmap changed)... 2024-05-25T06:16:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:20 smithi160 ceph-mon[112094]: Reconfiguring daemon mds.cephfs.smithi160.rrhzig on smithi160 2024-05-25T06:16:21.933 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:21 smithi160 ceph-mon[112094]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:21.933 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:21.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:21.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:21.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:21 smithi203 ceph-mon[83507]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:22.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:22.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:22 smithi160 ceph-mon[112094]: Reconfiguring mds.cephfs.smithi160.tlctof (monmap changed)... 2024-05-25T06:16:22.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:22 smithi160 ceph-mon[112094]: Reconfiguring daemon mds.cephfs.smithi160.tlctof on smithi160 2024-05-25T06:16:22.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:22.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:22.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:22.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:23.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:22 smithi203 ceph-mon[83507]: Reconfiguring mds.cephfs.smithi160.tlctof (monmap changed)... 2024-05-25T06:16:23.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:22 smithi203 ceph-mon[83507]: Reconfiguring daemon mds.cephfs.smithi160.tlctof on smithi160 2024-05-25T06:16:23.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:23.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:23.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:23.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:23 smithi203 ceph-mon[83507]: Reconfiguring crash.smithi203 (monmap changed)... 2024-05-25T06:16:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:23 smithi203 ceph-mon[83507]: Reconfiguring daemon crash.smithi203 on smithi203 2024-05-25T06:16:24.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:23 smithi203 ceph-mon[83507]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:23 smithi160 ceph-mon[112094]: Reconfiguring crash.smithi203 (monmap changed)... 2024-05-25T06:16:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:23 smithi160 ceph-mon[112094]: Reconfiguring daemon crash.smithi203 on smithi203 2024-05-25T06:16:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:23 smithi160 ceph-mon[112094]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: Reconfiguring mgr.smithi203.rotqex (monmap changed)... 2024-05-25T06:16:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:16:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:16:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: Reconfiguring daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:16:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:25 smithi203 ceph-mon[83507]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: Reconfiguring mgr.smithi203.rotqex (monmap changed)... 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi203.rotqex", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: Reconfiguring daemon mgr.smithi203.rotqex on smithi203 2024-05-25T06:16:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:25 smithi160 ceph-mon[112094]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: Reconfiguring mon.smithi203 (monmap changed)... 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: Reconfiguring daemon mon.smithi203 on smithi203 2024-05-25T06:16:27.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:27 smithi203 ceph-mon[83507]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: Reconfiguring mon.smithi203 (monmap changed)... 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: Reconfiguring daemon mon.smithi203 on smithi203 2024-05-25T06:16:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:27 smithi160 ceph-mon[112094]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:28.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:28.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:28.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: Reconfiguring osd.3 (monmap changed)... 2024-05-25T06:16:28.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-25T06:16:28.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:28.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: Reconfiguring daemon osd.3 on smithi203 2024-05-25T06:16:28.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:28.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:28.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:28.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: Reconfiguring osd.3 (monmap changed)... 2024-05-25T06:16:28.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-25T06:16:28.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:28.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: Reconfiguring daemon osd.3 on smithi203 2024-05-25T06:16:28.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:29.959 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:29 smithi203 ceph-mon[83507]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:29.959 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:29.959 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:29.959 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-25T06:16:29.959 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:29 smithi160 ceph-mon[112094]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-25T06:16:29.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:30.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:30 smithi160 ceph-mon[112094]: Reconfiguring osd.4 (monmap changed)... 2024-05-25T06:16:30.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:30 smithi160 ceph-mon[112094]: Reconfiguring daemon osd.4 on smithi203 2024-05-25T06:16:30.973 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:30 smithi203 ceph-mon[83507]: Reconfiguring osd.4 (monmap changed)... 2024-05-25T06:16:30.973 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:30 smithi203 ceph-mon[83507]: Reconfiguring daemon osd.4 on smithi203 2024-05-25T06:16:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:31 smithi160 ceph-mon[112094]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-25T06:16:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:31.989 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:31 smithi203 ceph-mon[83507]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:31.989 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:31.989 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:31.989 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-25T06:16:31.989 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 -- 172.21.15.160:0/689370860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 msgr2=0x7f64001031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 --2- 172.21.15.160:0/689370860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001031b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63f4004940 tx=0x7f63f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 -- 172.21.15.160:0/689370860 shutdown_connections 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 --2- 172.21.15.160:0/689370860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6400103780 0x7f64000fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 --2- 172.21.15.160:0/689370860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001031b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 -- 172.21.15.160:0/689370860 >> 172.21.15.160:0/689370860 conn(0x7f64000f8990 msgr2=0x7f64000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 -- 172.21.15.160:0/689370860 shutdown_connections 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 -- 172.21.15.160:0/689370860 wait complete. 2024-05-25T06:16:32.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 Processor -- start 2024-05-25T06:16:32.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.835+0000 7f6406e29700 1 -- start start 2024-05-25T06:16:32.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6406e29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:32.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6406e29700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6400103780 0x7f6400113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:32.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6406e29700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6400114440 con 0x7f6400102de0 2024-05-25T06:16:32.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6406e29700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640010da90 con 0x7f6400103780 2024-05-25T06:16:32.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:32.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34080/0 (socket says 172.21.15.160:34080) 2024-05-25T06:16:32.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 -- 172.21.15.160:0/962260183 learned_addr learned my addr 172.21.15.160:0/962260183 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:32.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f63fffff700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6400103780 0x7f6400113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:32.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 -- 172.21.15.160:0/962260183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6400103780 msgr2=0x7f6400113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:32.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6400103780 0x7f6400113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 -- 172.21.15.160:0/962260183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4005040 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.836+0000 7f6404bc5700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001138b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b8b0 tx=0x7f63f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.837+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63f400e040 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.837+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63f4004290 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.837+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640010dd10 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.837+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63f401c3f0 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.837+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640010e2d0 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.838+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64001000f0 con 0x7f6400102de0 2024-05-25T06:16:32.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.838+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f63f4013420 con 0x7f6400102de0 2024-05-25T06:16:32.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.839+0000 7f63fd7fa700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63f006e270 0x7f63f0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:32.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.839+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f63f408ef80 con 0x7f6400102de0 2024-05-25T06:16:32.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.839+0000 7f63fffff700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63f006e270 0x7f63f0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:32.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.839+0000 7f63fffff700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63f006e270 0x7f63f0070720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f640010ec00 tx=0x7f63ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:32.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:32.843+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f63f405bce0 con 0x7f6400102de0 2024-05-25T06:16:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:32 smithi203 ceph-mon[83507]: Reconfiguring osd.5 (monmap changed)... 2024-05-25T06:16:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:32 smithi203 ceph-mon[83507]: Reconfiguring daemon osd.5 on smithi203 2024-05-25T06:16:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:32.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:32.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:32.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:32 smithi160 ceph-mon[112094]: Reconfiguring osd.5 (monmap changed)... 2024-05-25T06:16:32.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:32 smithi160 ceph-mon[112094]: Reconfiguring daemon osd.5 on smithi203 2024-05-25T06:16:32.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:32.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:32.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:32.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:33.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.062+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f640010ef00 con 0x7f63f006e270 2024-05-25T06:16:33.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.063+0000 7f63fd7fa700 1 -- 172.21.15.160:0/962260183 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f640010ef00 con 0x7f63f006e270 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.065+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63f006e270 msgr2=0x7f63f0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.065+0000 7f6406e29700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63f006e270 0x7f63f0070720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f640010ec00 tx=0x7f63ec00b040 comp rx=0 tx=0).stop 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.065+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 msgr2=0x7f64001138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.065+0000 7f6406e29700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001138b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b8b0 tx=0x7f63f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 shutdown_connections 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6400102de0 0x7f64001138b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63f006e270 0x7f63f0070720 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 --2- 172.21.15.160:0/962260183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6400103780 0x7f6400113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 >> 172.21.15.160:0/962260183 conn(0x7f64000f8990 msgr2=0x7f64000fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 shutdown_connections 2024-05-25T06:16:33.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.066+0000 7f6406e29700 1 -- 172.21.15.160:0/962260183 wait complete. 2024-05-25T06:16:33.077 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:16:33.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 -- 172.21.15.160:0/312405007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e41026b0 msgr2=0x7f01e40fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/312405007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e41026b0 0x7f01e40fbd20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01e0004940 tx=0x7f01e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:33.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 -- 172.21.15.160:0/312405007 shutdown_connections 2024-05-25T06:16:33.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/312405007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e41026b0 0x7f01e40fbd20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/312405007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01e4101d10 0x7f01e41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 -- 172.21.15.160:0/312405007 >> 172.21.15.160:0/312405007 conn(0x7f01e40f7860 msgr2=0x7f01e40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 -- 172.21.15.160:0/312405007 shutdown_connections 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.172+0000 7f01eb5e8700 1 -- 172.21.15.160:0/312405007 wait complete. 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01eb5e8700 1 Processor -- start 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01eb5e8700 1 -- start start 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01eb5e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 0x7f01e41128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01eb5e8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01e41026b0 0x7f01e4112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01eb5e8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4113430 con 0x7f01e4101d10 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01eb5e8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e410ca30 con 0x7f01e41026b0 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01e9384700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 0x7f01e41128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01e9384700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 0x7f01e41128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34110/0 (socket says 172.21.15.160:34110) 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01e9384700 1 -- 172.21.15.160:0/3131817079 learned_addr learned my addr 172.21.15.160:0/3131817079 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.173+0000 7f01e8b83700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01e41026b0 0x7f01e4112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01e9384700 1 -- 172.21.15.160:0/3131817079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01e41026b0 msgr2=0x7f01e4112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01e9384700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01e41026b0 0x7f01e4112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01e9384700 1 -- 172.21.15.160:0/3131817079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0005040 con 0x7f01e4101d10 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01e9384700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 0x7f01e41128a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01d4002a50 tx=0x7f01d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d4007cc0 con 0x7f01e4101d10 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01d400f040 con 0x7f01e4101d10 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e410cd10 con 0x7f01e4101d10 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d400da30 con 0x7f01e4101d10 2024-05-25T06:16:33.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.174+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e410d2a0 con 0x7f01e4101d10 2024-05-25T06:16:33.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.175+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e40fd460 con 0x7f01e4101d10 2024-05-25T06:16:33.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.176+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f01d400db90 con 0x7f01e4101d10 2024-05-25T06:16:33.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.176+0000 7f01d9ffb700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f01d006e270 0x7f01d0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.176+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f01d408db00 con 0x7f01e4101d10 2024-05-25T06:16:33.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.176+0000 7f01e8b83700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f01d006e270 0x7f01d0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.177+0000 7f01e8b83700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f01d006e270 0x7f01d0070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01e0004890 tx=0x7f01e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:33.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.181+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f01d405a7e0 con 0x7f01e4101d10 2024-05-25T06:16:33.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.396+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01e4108e40 con 0x7f01d006e270 2024-05-25T06:16:33.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.398+0000 7f01d9ffb700 1 -- 172.21.15.160:0/3131817079 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f01e4108e40 con 0x7f01d006e270 2024-05-25T06:16:33.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f01d006e270 msgr2=0x7f01d0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f01d006e270 0x7f01d0070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01e0004890 tx=0x7f01e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 msgr2=0x7f01e41128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 0x7f01e41128a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01d4002a50 tx=0x7f01d40046b0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 shutdown_connections 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e4101d10 0x7f01e41128a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f01d006e270 0x7f01d0070720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 --2- 172.21.15.160:0/3131817079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01e41026b0 0x7f01e4112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 >> 172.21.15.160:0/3131817079 conn(0x7f01e40f7860 msgr2=0x7f01e40f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 shutdown_connections 2024-05-25T06:16:33.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.401+0000 7f01eb5e8700 1 -- 172.21.15.160:0/3131817079 wait complete. 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 -- 172.21.15.160:0/1267046158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 msgr2=0x7fc0440fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/1267046158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc0440fc1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc034004940 tx=0x7fc034009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 -- 172.21.15.160:0/1267046158 shutdown_connections 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/1267046158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc0440fc1b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/1267046158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc044101d90 0x7fc044102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 -- 172.21.15.160:0/1267046158 >> 172.21.15.160:0/1267046158 conn(0x7fc044071d00 msgr2=0x7fc044072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 -- 172.21.15.160:0/1267046158 shutdown_connections 2024-05-25T06:16:33.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.508+0000 7fc04bb2a700 1 -- 172.21.15.160:0/1267046158 wait complete. 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc04bb2a700 1 Processor -- start 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc04bb2a700 1 -- start start 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc04bb2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc044113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc04bb2a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc044101d90 0x7fc044114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc04bb2a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044114680 con 0x7fc0440fbd40 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc04bb2a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04410da70 con 0x7fc044101d90 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc044113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc044113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34126/0 (socket says 172.21.15.160:34126) 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 -- 172.21.15.160:0/442013125 learned_addr learned my addr 172.21.15.160:0/442013125 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0490c5700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc044101d90 0x7fc044114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 -- 172.21.15.160:0/442013125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc044101d90 msgr2=0x7fc044114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc044101d90 0x7fc044114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 -- 172.21.15.160:0/442013125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034005040 con 0x7fc0440fbd40 2024-05-25T06:16:33.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc0498c6700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc044113af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04000eb40 tx=0x7fc04000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:33.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.509+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc040007cc0 con 0x7fc0440fbd40 2024-05-25T06:16:33.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.510+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04410dcf0 con 0x7fc0440fbd40 2024-05-25T06:16:33.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.510+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc04000f040 con 0x7fc0440fbd40 2024-05-25T06:16:33.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.510+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc040009620 con 0x7fc0440fbd40 2024-05-25T06:16:33.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.510+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04410e200 con 0x7fc0440fbd40 2024-05-25T06:16:33.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.511+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440fd430 con 0x7fc0440fbd40 2024-05-25T06:16:33.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.511+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fc040009780 con 0x7fc0440fbd40 2024-05-25T06:16:33.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.511+0000 7fc03a7fc700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc03006e260 0x7fc030070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.512+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc04008d940 con 0x7fc0440fbd40 2024-05-25T06:16:33.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.512+0000 7fc0490c5700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc03006e260 0x7fc030070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.512+0000 7fc0490c5700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc03006e260 0x7fc030070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc04410f420 tx=0x7fc034009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:33.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.516+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc04005a750 con 0x7fc0440fbd40 2024-05-25T06:16:33.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.731+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc04410de80 con 0x7fc03006e260 2024-05-25T06:16:33.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.741+0000 7fc03a7fc700 1 -- 172.21.15.160:0/442013125 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc04410de80 con 0x7fc03006e260 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (2m) 45s ago 13m 17.7M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (13m) 45s ago 13m 6660k - 17.2.7-958-g7b01852c baf1547092e7 eaa367bc8825 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (12m) 29s ago 12m 6660k - 17.2.7-958-g7b01852c baf1547092e7 6a0765dedea5 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (2m) 45s ago 13m 74.2M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (8m) 45s ago 8m 61.2M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (8m) 45s ago 8m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (8m) 29s ago 7m 61.4M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (8m) 29s ago 8m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:16:33.742 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (4m) 45s ago 14m 475M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (3m) 29s ago 12m 450M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (61s) 45s ago 15m 27.6M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (34s) 29s ago 12m 25.1M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (3m) 45s ago 13m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (3m) 29s ago 12m 17.0M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (11m) 45s ago 11m 219M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (11m) 45s ago 11m 227M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (10m) 45s ago 10m 191M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (10m) 29s ago 10m 269M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (10m) 29s ago 10m 230M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (9m) 29s ago 9m 230M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:16:33.743 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (2m) 45s ago 13m 63.1M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:16:33.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.743+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc03006e260 msgr2=0x7fc030070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.743+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc03006e260 0x7fc030070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc04410f420 tx=0x7fc034009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:33.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.743+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 msgr2=0x7fc044113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.743+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc044113af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04000eb40 tx=0x7fc04000b040 comp rx=0 tx=0).stop 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 shutdown_connections 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0440fbd40 0x7fc044113af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc03006e260 0x7fc030070710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 --2- 172.21.15.160:0/442013125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc044101d90 0x7fc044114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 >> 172.21.15.160:0/442013125 conn(0x7fc044071d00 msgr2=0x7fc0440fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 shutdown_connections 2024-05-25T06:16:33.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.744+0000 7fc04bb2a700 1 -- 172.21.15.160:0/442013125 wait complete. 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 -- 172.21.15.160:0/716299368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 msgr2=0x7f66641020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 --2- 172.21.15.160:0/716299368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641020d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6654004940 tx=0x7f6654009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 -- 172.21.15.160:0/716299368 shutdown_connections 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 --2- 172.21.15.160:0/716299368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 0x7f66640fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 --2- 172.21.15.160:0/716299368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641020d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 -- 172.21.15.160:0/716299368 >> 172.21.15.160:0/716299368 conn(0x7f66640f7810 msgr2=0x7f66640f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 -- 172.21.15.160:0/716299368 shutdown_connections 2024-05-25T06:16:33.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.864+0000 7f6669851700 1 -- 172.21.15.160:0/716299368 wait complete. 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6669851700 1 Processor -- start 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6669851700 1 -- start start 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6669851700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641128f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6669851700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 0x7f6664112e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6669851700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664113480 con 0x7f6664101d00 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6669851700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666410ca80 con 0x7f66641026a0 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34150/0 (socket says 172.21.15.160:34150) 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6662ffd700 1 -- 172.21.15.160:0/455495766 learned_addr learned my addr 172.21.15.160:0/455495766 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f66627fc700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 0x7f6664112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6662ffd700 1 -- 172.21.15.160:0/455495766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 msgr2=0x7f6664112e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6662ffd700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 0x7f6664112e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f6662ffd700 1 -- 172.21.15.160:0/455495766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654005040 con 0x7f6664101d00 2024-05-25T06:16:33.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.865+0000 7f66627fc700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 0x7f6664112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:16:33.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.866+0000 7f6662ffd700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641128f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f665400b3b0 tx=0x7f6654009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:33.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.866+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665400e040 con 0x7f6664101d00 2024-05-25T06:16:33.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.866+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666410cd00 con 0x7f6664101d00 2024-05-25T06:16:33.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.866+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666410d240 con 0x7f6664101d00 2024-05-25T06:16:33.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.866+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f665400bbc0 con 0x7f6664101d00 2024-05-25T06:16:33.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.866+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665401c3f0 con 0x7f6664101d00 2024-05-25T06:16:33.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.867+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66640fd450 con 0x7f6664101d00 2024-05-25T06:16:33.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.867+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f665400bd30 con 0x7f6664101d00 2024-05-25T06:16:33.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.868+0000 7f665b7fe700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f665006e270 0x7f6650070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:33.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.868+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f665408eff0 con 0x7f6664101d00 2024-05-25T06:16:33.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.868+0000 7f66627fc700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f665006e270 0x7f6650070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:33.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.868+0000 7f66627fc700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f665006e270 0x7f6650070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f664c007fd0 tx=0x7f664c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:33.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:33.873+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f665405bcd0 con 0x7f6664101d00 2024-05-25T06:16:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:33 smithi203 ceph-mon[83507]: Reconfiguring mds.cephfs.smithi203.yltymd (monmap changed)... 2024-05-25T06:16:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:33 smithi203 ceph-mon[83507]: Reconfiguring daemon mds.cephfs.smithi203.yltymd on smithi203 2024-05-25T06:16:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:33 smithi203 ceph-mon[83507]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:33.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:33 smithi160 ceph-mon[112094]: Reconfiguring mds.cephfs.smithi203.yltymd (monmap changed)... 2024-05-25T06:16:33.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:33 smithi160 ceph-mon[112094]: Reconfiguring daemon mds.cephfs.smithi203.yltymd on smithi203 2024-05-25T06:16:33.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:33 smithi160 ceph-mon[112094]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.170+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6664108e20 con 0x7f6664101d00 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.171+0000 7f665b7fe700 1 -- 172.21.15.160:0/455495766 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f665405b860 con 0x7f6664101d00 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:16:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 10, 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:16:34.172 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:16:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f665006e270 msgr2=0x7f6650070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f665006e270 0x7f6650070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f664c007fd0 tx=0x7f664c009040 comp rx=0 tx=0).stop 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 msgr2=0x7f66641128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641128f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f665400b3b0 tx=0x7f6654009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 shutdown_connections 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6664101d00 0x7f66641128f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f665006e270 0x7f6650070720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 --2- 172.21.15.160:0/455495766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66641026a0 0x7f6664112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 >> 172.21.15.160:0/455495766 conn(0x7f66640f7810 msgr2=0x7f66640f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 shutdown_connections 2024-05-25T06:16:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.173+0000 7f6669851700 1 -- 172.21.15.160:0/455495766 wait complete. 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.284+0000 7f922b284700 1 -- 172.21.15.160:0/4133572362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224101cd0 msgr2=0x7f92241020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.284+0000 7f922b284700 1 --2- 172.21.15.160:0/4133572362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224101cd0 0x7f92241020a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9220004940 tx=0x7f9220009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.284+0000 7f922b284700 1 -- 172.21.15.160:0/4133572362 shutdown_connections 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.284+0000 7f922b284700 1 --2- 172.21.15.160:0/4133572362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9224102670 0x7f92240fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.284+0000 7f922b284700 1 --2- 172.21.15.160:0/4133572362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224101cd0 0x7f92241020a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.284+0000 7f922b284700 1 -- 172.21.15.160:0/4133572362 >> 172.21.15.160:0/4133572362 conn(0x7f9224071460 msgr2=0x7f9224071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 -- 172.21.15.160:0/4133572362 shutdown_connections 2024-05-25T06:16:34.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 -- 172.21.15.160:0/4133572362 wait complete. 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 Processor -- start 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 -- start start 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9224101cd0 0x7f92240fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 0x7f92240fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92240fd260 con 0x7f9224102670 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922b284700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92240fd3a0 con 0x7f9224101cd0 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.285+0000 7f922881f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 0x7f92240fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922881f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 0x7f92240fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34172/0 (socket says 172.21.15.160:34172) 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922881f700 1 -- 172.21.15.160:0/2348994881 learned_addr learned my addr 172.21.15.160:0/2348994881 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:34.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f9229020700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9224101cd0 0x7f92240fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922881f700 1 -- 172.21.15.160:0/2348994881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9224101cd0 msgr2=0x7f92240fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922881f700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9224101cd0 0x7f92240fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922881f700 1 -- 172.21.15.160:0/2348994881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220005040 con 0x7f9224102670 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922881f700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 0x7f92240fcd20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92140044e0 tx=0x7f921400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9214007cc0 con 0x7f9224102670 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f921400f040 con 0x7f9224102670 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92240fd5f0 con 0x7f9224102670 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f921400ca50 con 0x7f9224102670 2024-05-25T06:16:34.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.286+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922411a190 con 0x7f9224102670 2024-05-25T06:16:34.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.287+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92241052e0 con 0x7f9224102670 2024-05-25T06:16:34.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.288+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f921400cbb0 con 0x7f9224102670 2024-05-25T06:16:34.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.289+0000 7f9219ffb700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f921006e260 0x7f9210070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.289+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f921408d5c0 con 0x7f9224102670 2024-05-25T06:16:34.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.289+0000 7f9229020700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f921006e260 0x7f9210070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.289+0000 7f9229020700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f921006e260 0x7f9210070710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9220009b30 tx=0x7f9220009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:34.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.293+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f92140568a0 con 0x7f9224102670 2024-05-25T06:16:34.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.543+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f92240fde80 con 0x7f9224102670 2024-05-25T06:16:34.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.544+0000 7f9219ffb700 1 -- 172.21.15.160:0/2348994881 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7f9214056030 con 0x7f9224102670 2024-05-25T06:16:34.544 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:16:34.544 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:16:34.545 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:16:34.546 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:34.547 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:34.548 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:16:34.548 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:16:34.548 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:16:34.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.546+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f921006e260 msgr2=0x7f9210070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.546+0000 7f922b284700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f921006e260 0x7f9210070710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9220009b30 tx=0x7f9220009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.546+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 msgr2=0x7f92240fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.546+0000 7f922b284700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 0x7f92240fcd20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92140044e0 tx=0x7f921400b040 comp rx=0 tx=0).stop 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 shutdown_connections 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f921006e260 0x7f9210070710 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9224101cd0 0x7f92240fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 --2- 172.21.15.160:0/2348994881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9224102670 0x7f92240fcd20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 >> 172.21.15.160:0/2348994881 conn(0x7f9224071460 msgr2=0x7f92240fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 shutdown_connections 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.547+0000 7f922b284700 1 -- 172.21.15.160:0/2348994881 wait complete. 2024-05-25T06:16:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 -- 172.21.15.160:0/1345708572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8065b80 msgr2=0x7f8fa8065ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 --2- 172.21.15.160:0/1345708572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8065b80 0x7f8fa8065ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f98004940 tx=0x7f8f98009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 -- 172.21.15.160:0/1345708572 shutdown_connections 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 --2- 172.21.15.160:0/1345708572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8065b80 0x7f8fa8065ff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 --2- 172.21.15.160:0/1345708572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8101c70 0x7f8fa8065640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 -- 172.21.15.160:0/1345708572 >> 172.21.15.160:0/1345708572 conn(0x7f8fa8071d00 msgr2=0x7f8fa8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 -- 172.21.15.160:0/1345708572 shutdown_connections 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 -- 172.21.15.160:0/1345708572 wait complete. 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 Processor -- start 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.652+0000 7f8fafc79700 1 -- start start 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fafc79700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8065b80 0x7f8fa811a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fafc79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 0x7f8fa811a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fafc79700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa811ad10 con 0x7f8fa8101c70 2024-05-25T06:16:34.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fafc79700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81140e0 con 0x7f8fa8065b80 2024-05-25T06:16:34.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 0x7f8fa811a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 0x7f8fa811a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34186/0 (socket says 172.21.15.160:34186) 2024-05-25T06:16:34.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 -- 172.21.15.160:0/51574985 learned_addr learned my addr 172.21.15.160:0/51574985 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:34.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fada15700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8065b80 0x7f8fa811a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 -- 172.21.15.160:0/51574985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8065b80 msgr2=0x7f8fa811a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8065b80 0x7f8fa811a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 -- 172.21.15.160:0/51574985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f98005040 con 0x7f8fa8101c70 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fada15700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8065b80 0x7f8fa811a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fad214700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 0x7f8fa811a6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009b30 tx=0x7f8f98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f98021070 con 0x7f8fa8101c70 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f98013c00 con 0x7f8fa8101c70 2024-05-25T06:16:34.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa8114360 con 0x7f8fa8101c70 2024-05-25T06:16:34.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.653+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f9800e070 con 0x7f8fa8101c70 2024-05-25T06:16:34.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.654+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa8114840 con 0x7f8fa8101c70 2024-05-25T06:16:34.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.655+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa80fc330 con 0x7f8fa8101c70 2024-05-25T06:16:34.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.655+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f8f9801cad0 con 0x7f8fa8101c70 2024-05-25T06:16:34.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.656+0000 7f8f9e7fc700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8f9406e260 0x7f8f94070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.656+0000 7f8fada15700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8f9406e260 0x7f8f94070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.656+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8f9808f100 con 0x7f8fa8101c70 2024-05-25T06:16:34.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.656+0000 7f8fada15700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8f9406e260 0x7f8f94070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009b80 tx=0x7f8fa400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:34.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.660+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8f9805be60 con 0x7f8fa8101c70 2024-05-25T06:16:34.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.876+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fa81104f0 con 0x7f8f9406e260 2024-05-25T06:16:34.876 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/455495766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:34.877 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:34 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2348994881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.877+0000 7f8f9e7fc700 1 -- 172.21.15.160:0/51574985 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f8fa81104f0 con 0x7f8f9406e260 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "4/21 daemons upgraded", 2024-05-25T06:16:34.878 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mon daemons", 2024-05-25T06:16:34.879 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:16:34.879 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8f9406e260 msgr2=0x7f8f94070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8f9406e260 0x7f8f94070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009b80 tx=0x7f8fa400b040 comp rx=0 tx=0).stop 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 msgr2=0x7f8fa811a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 0x7f8fa811a6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009b30 tx=0x7f8f98009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 shutdown_connections 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8f9406e260 0x7f8f94070710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fa8065b80 0x7f8fa811a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 --2- 172.21.15.160:0/51574985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fa8101c70 0x7f8fa811a6c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 >> 172.21.15.160:0/51574985 conn(0x7f8fa8071d00 msgr2=0x7f8fa80f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:34.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 shutdown_connections 2024-05-25T06:16:34.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.881+0000 7f8fafc79700 1 -- 172.21.15.160:0/51574985 wait complete. 2024-05-25T06:16:34.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 -- 172.21.15.160:0/3186394644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 msgr2=0x7fb774106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 --2- 172.21.15.160:0/3186394644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774106e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb768004940 tx=0x7fb768009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:34.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 -- 172.21.15.160:0/3186394644 shutdown_connections 2024-05-25T06:16:34.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 --2- 172.21.15.160:0/3186394644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774106e40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 --2- 172.21.15.160:0/3186394644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7740fde60 0x7fb7740fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 -- 172.21.15.160:0/3186394644 >> 172.21.15.160:0/3186394644 conn(0x7fb7740f7870 msgr2=0x7fb7740f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:34.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 -- 172.21.15.160:0/3186394644 shutdown_connections 2024-05-25T06:16:34.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.984+0000 7fb778f0e700 1 -- 172.21.15.160:0/3186394644 wait complete. 2024-05-25T06:16:34.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb778f0e700 1 Processor -- start 2024-05-25T06:16:34.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb778f0e700 1 -- start start 2024-05-25T06:16:34.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb778f0e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7740fde60 0x7fb774113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb778f0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb778f0e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741144d0 con 0x7fb7740fe800 2024-05-25T06:16:34.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb778f0e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77410dad0 con 0x7fb7740fde60 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34206/0 (socket says 172.21.15.160:34206) 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 -- 172.21.15.160:0/1235459415 learned_addr learned my addr 172.21.15.160:0/1235459415 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb77259c700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7740fde60 0x7fb774113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 -- 172.21.15.160:0/1235459415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7740fde60 msgr2=0x7fb774113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7740fde60 0x7fb774113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 -- 172.21.15.160:0/1235459415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb768005040 con 0x7fb7740fe800 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb771d9b700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774113e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b3b0 tx=0x7fb768009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76800e040 con 0x7fb7740fe800 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.985+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb768013420 con 0x7fb7740fe800 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.986+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76801c630 con 0x7fb7740fe800 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.986+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77410ddb0 con 0x7fb7740fe800 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.986+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77410e2f0 con 0x7fb7740fe800 2024-05-25T06:16:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.986+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb774066ad0 con 0x7fb7740fe800 2024-05-25T06:16:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.987+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fb76800baf0 con 0x7fb7740fe800 2024-05-25T06:16:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.987+0000 7fb762ffd700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb75c06e270 0x7fb75c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:16:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.987+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb76808ef60 con 0x7fb7740fe800 2024-05-25T06:16:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.988+0000 7fb77259c700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb75c06e270 0x7fb75c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:16:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.988+0000 7fb77259c700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb75c06e270 0x7fb75c070720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb764004d60 tx=0x7fb76400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:16:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:34.992+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb7680c3850 con 0x7fb7740fe800 2024-05-25T06:16:35.021 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:35.021 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:35.021 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:35.021 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/455495766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:35.021 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:35.021 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:35.022 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:16:35.022 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:35.022 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:34 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2348994881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:16:35.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.275+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb77410efc0 con 0x7fb7740fe800 2024-05-25T06:16:35.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.275+0000 7fb762ffd700 1 -- 172.21.15.160:0/1235459415 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb76805bd60 con 0x7fb7740fe800 2024-05-25T06:16:35.275 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:16:35.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb75c06e270 msgr2=0x7fb75c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:35.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb75c06e270 0x7fb75c070720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb764004d60 tx=0x7fb76400b040 comp rx=0 tx=0).stop 2024-05-25T06:16:35.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 msgr2=0x7fb774113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:16:35.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774113e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b3b0 tx=0x7fb768009e30 comp rx=0 tx=0).stop 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 shutdown_connections 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb75c06e270 0x7fb75c070720 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7740fde60 0x7fb774113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 --2- 172.21.15.160:0/1235459415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7740fe800 0x7fb774113e80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 >> 172.21.15.160:0/1235459415 conn(0x7fb7740f7870 msgr2=0x7fb7741016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 shutdown_connections 2024-05-25T06:16:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:16:35.277+0000 7fb778f0e700 1 -- 172.21.15.160:0/1235459415 wait complete. 2024-05-25T06:16:35.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:35 smithi203 ceph-mon[83507]: Reconfiguring mds.cephfs.smithi203.tndnrb (monmap changed)... 2024-05-25T06:16:35.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:35 smithi203 ceph-mon[83507]: Reconfiguring daemon mds.cephfs.smithi203.tndnrb on smithi203 2024-05-25T06:16:35.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:35 smithi203 ceph-mon[83507]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:35.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:35 smithi203 ceph-mon[83507]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:35.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:35 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1235459415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:16:35.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:35 smithi160 ceph-mon[112094]: Reconfiguring mds.cephfs.smithi203.tndnrb (monmap changed)... 2024-05-25T06:16:35.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:35 smithi160 ceph-mon[112094]: Reconfiguring daemon mds.cephfs.smithi203.tndnrb on smithi203 2024-05-25T06:16:35.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:35 smithi160 ceph-mon[112094]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:35.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:35 smithi160 ceph-mon[112094]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:16:35.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:35 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1235459415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:16:37.094 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:37.094 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all mon 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi160"}]: dispatch 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi160"}]': finished 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi203"}]: dispatch 2024-05-25T06:16:37.095 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi203"}]': finished 2024-05-25T06:16:37.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:37.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all mon 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi160"}]: dispatch 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi160"}]': finished 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi203"}]: dispatch 2024-05-25T06:16:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi203"}]': finished 2024-05-25T06:16:38.029 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:37 smithi160 ceph-mon[112094]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:38.029 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:37 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:38.029 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:37 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:38.029 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:37 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:37 smithi203 ceph-mon[83507]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:37 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:38.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:37 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:38.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:37 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:38.968 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:38 smithi160 ceph-mon[112094]: Upgrade: Updating crash.smithi160 (1/2) 2024-05-25T06:16:38.968 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:38 smithi160 ceph-mon[112094]: Deploying daemon crash.smithi160 on smithi160 2024-05-25T06:16:39.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:38 smithi203 ceph-mon[83507]: Upgrade: Updating crash.smithi160 (1/2) 2024-05-25T06:16:39.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:38 smithi203 ceph-mon[83507]: Deploying daemon crash.smithi160 on smithi160 2024-05-25T06:16:40.041 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:39 smithi160 ceph-mon[112094]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:40.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:39 smithi203 ceph-mon[83507]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:41 smithi160 ceph-mon[112094]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:41 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:42.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:41 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:41 smithi203 ceph-mon[83507]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:41 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:42.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:41 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:44.050 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: Upgrade: Updating crash.smithi203 (2/2) 2024-05-25T06:16:44.050 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:44.051 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:44.051 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:44.051 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: Deploying daemon crash.smithi203 on smithi203 2024-05-25T06:16:44.051 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:44.051 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: Upgrade: Updating crash.smithi203 (2/2) 2024-05-25T06:16:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi203", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-25T06:16:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: Deploying daemon crash.smithi203 on smithi203 2024-05-25T06:16:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:45 smithi160 ceph-mon[112094]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:46.250 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:45 smithi203 ceph-mon[83507]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:47.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:47 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:47.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:47 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:47.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:47 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:47.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:47 smithi203 ceph-mon[83507]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:47.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:47 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:47.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:47 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:47.935 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:47 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:47.935 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:47 smithi160 ceph-mon[112094]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:49 smithi160 ceph-mon[112094]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:49 smithi203 ceph-mon[83507]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:51 smithi203 ceph-mon[83507]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:51.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:51 smithi160 ceph-mon[112094]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:52 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:52 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:53.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:52 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:53.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:52 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:54.066 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:53 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:54.066 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:53 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:54.067 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:53 smithi203 ceph-mon[83507]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:54.165 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:53 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:54.165 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:53 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:54.165 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:53 smithi160 ceph-mon[112094]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:55 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:55 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:55.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:55 smithi203 ceph-mon[83507]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:55 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:55 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:55 smithi160 ceph-mon[112094]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:57.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all crash 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi160"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi160"}]': finished 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi203"}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi203"}]': finished 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-25T06:16:57.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: Upgrade: osd.0 is safe to restart 2024-05-25T06:16:57.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:57 smithi160 ceph-mon[112094]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:57.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all crash 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi160"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi160"}]': finished 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi203"}]: dispatch 2024-05-25T06:16:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi203"}]': finished 2024-05-25T06:16:57.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-25T06:16:57.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-25T06:16:57.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: Upgrade: osd.0 is safe to restart 2024-05-25T06:16:57.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:57 smithi203 ceph-mon[83507]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:16:58.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:58.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:58.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-25T06:16:58.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:58.933 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:16:58.933 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:16:58.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-25T06:16:58.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:16:59.835 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:59 smithi160 ceph-mon[112094]: Upgrade: Updating osd.0 2024-05-25T06:16:59.835 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:59 smithi160 ceph-mon[112094]: Deploying daemon osd.0 on smithi160 2024-05-25T06:16:59.835 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:16:59 smithi160 ceph-mon[112094]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:16:59.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:59 smithi203 ceph-mon[83507]: Upgrade: Updating osd.0 2024-05-25T06:16:59.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:59 smithi203 ceph-mon[83507]: Deploying daemon osd.0 on smithi160 2024-05-25T06:16:59.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:16:59 smithi203 ceph-mon[83507]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:17:01.525 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:01 smithi160 systemd[1]: Stopping Ceph osd.0 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:17:01.929 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:01 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[46870]: 2024-05-25T06:17:01.524+0000 7fcb5d0aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:17:01.930 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:01 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[46870]: 2024-05-25T06:17:01.524+0000 7fcb5d0aa700 -1 osd.0 49 *** Got signal Terminated *** 2024-05-25T06:17:01.930 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:01 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[46870]: 2024-05-25T06:17:01.524+0000 7fcb5d0aa700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:17:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:01 smithi160 ceph-mon[112094]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:01 smithi160 ceph-mon[112094]: osd.0 marked itself down and dead 2024-05-25T06:17:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:01 smithi203 ceph-mon[83507]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:01 smithi203 ceph-mon[83507]: osd.0 marked itself down and dead 2024-05-25T06:17:02.600 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:02 smithi160 podman[123038]: 2024-05-25 06:17:02.330930517 +0000 UTC m=+0.918839435 container died a030e70263e333776aa3a3a1e3390b885ae08e79b3f226b6978dd7d05c567ed2 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-7b01852, maintainer=Guillaume Abrioux ) 2024-05-25T06:17:02.935 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:02 smithi160 podman[123038]: 2024-05-25 06:17:02.76627064 +0000 UTC m=+1.354179608 container cleanup a030e70263e333776aa3a3a1e3390b885ae08e79b3f226b6978dd7d05c567ed2 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, RELEASE=quincy-7b01852, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:17:02.935 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:02 smithi160 bash[123038]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0 2024-05-25T06:17:03.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:02 smithi160 ceph-mon[112094]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:17:03.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:02 smithi160 ceph-mon[112094]: osdmap e50: 6 total, 5 up, 6 in 2024-05-25T06:17:03.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:02 smithi160 ceph-mon[112094]: osdmap e51: 6 total, 5 up, 6 in 2024-05-25T06:17:03.220 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:02 smithi160 podman[123055]: 2024-05-25 06:17:02.959349666 +0000 UTC m=+0.627253904 container remove a030e70263e333776aa3a3a1e3390b885ae08e79b3f226b6978dd7d05c567ed2 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=quincy-7b01852, maintainer=Guillaume Abrioux ) 2024-05-25T06:17:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:02 smithi203 ceph-mon[83507]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:17:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:02 smithi203 ceph-mon[83507]: osdmap e50: 6 total, 5 up, 6 in 2024-05-25T06:17:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:02 smithi203 ceph-mon[83507]: osdmap e51: 6 total, 5 up, 6 in 2024-05-25T06:17:03.502 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:03 smithi160 podman[123145]: 2024-05-25 06:17:03.377018445 +0000 UTC m=+0.101849652 container create 599cf5af5a70770ab092e6de427ad880d13ea432948ca4f6063ac8be2b168e81 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS) 2024-05-25T06:17:03.502 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:03 smithi160 podman[123145]: 2024-05-25 06:17:03.292678895 +0000 UTC m=+0.017510120 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:17:03.931 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:03 smithi160 podman[123145]: 2024-05-25 06:17:03.621408865 +0000 UTC m=+0.346240088 container init 599cf5af5a70770ab092e6de427ad880d13ea432948ca4f6063ac8be2b168e81 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521) 2024-05-25T06:17:03.931 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:03 smithi160 podman[123145]: 2024-05-25 06:17:03.624890352 +0000 UTC m=+0.349721575 container start 599cf5af5a70770ab092e6de427ad880d13ea432948ca4f6063ac8be2b168e81 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:17:03.931 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:03 smithi160 podman[123145]: 2024-05-25 06:17:03.704542503 +0000 UTC m=+0.429373720 container attach 599cf5af5a70770ab092e6de427ad880d13ea432948ca4f6063ac8be2b168e81 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c) 2024-05-25T06:17:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:03 smithi160 ceph-mon[112094]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:17:04.220 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:03 smithi160 podman[123145]: 2024-05-25 06:17:03.931904074 +0000 UTC m=+0.656735295 container died 599cf5af5a70770ab092e6de427ad880d13ea432948ca4f6063ac8be2b168e81 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-ea0684c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521) 2024-05-25T06:17:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:03 smithi203 ceph-mon[83507]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:17:04.845 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:04 smithi160 podman[123171]: 2024-05-25 06:17:04.59229813 +0000 UTC m=+0.654222952 container remove 599cf5af5a70770ab092e6de427ad880d13ea432948ca4f6063ac8be2b168e81 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:17:04.845 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:04 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-25T06:17:04.845 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:04 smithi160 systemd[1]: Stopped Ceph osd.0 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:17:04.845 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:04 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.0.service: Consumed 1min 955ms CPU time. 2024-05-25T06:17:05.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:04 smithi160 ceph-mon[112094]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:05.101 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:04 smithi160 ceph-mon[112094]: Health check failed: Degraded data redundancy: 63/324 objects degraded (19.444%), 19 pgs degraded (PG_DEGRADED) 2024-05-25T06:17:05.101 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:04 smithi160 systemd[1]: Starting Ceph osd.0 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:17:05.101 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 podman[123270]: 2024-05-25 06:17:04.984024626 +0000 UTC m=+0.021116079 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:17:05.102 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 podman[123270]: 2024-05-25 06:17:05.102021207 +0000 UTC m=+0.139112644 container create 90c062e23277256ae749b787746722b2f944e76d023dad1191233e9489d46858 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-25T06:17:05.390 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 podman[123270]: 2024-05-25 06:17:05.321652329 +0000 UTC m=+0.358743778 container init 90c062e23277256ae749b787746722b2f944e76d023dad1191233e9489d46858 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-05-25T06:17:05.390 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 podman[123270]: 2024-05-25 06:17:05.325375576 +0000 UTC m=+0.362467016 container start 90c062e23277256ae749b787746722b2f944e76d023dad1191233e9489d46858 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=) 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 -- 172.21.15.160:0/1405147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030102e50 msgr2=0x7f6030103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1405147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030102e50 0x7f6030103220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6024004940 tx=0x7f6024009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 -- 172.21.15.160:0/1405147316 shutdown_connections 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1405147316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6030065640 0x7f6030065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1405147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030102e50 0x7f6030103220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 -- 172.21.15.160:0/1405147316 >> 172.21.15.160:0/1405147316 conn(0x7f6030071d00 msgr2=0x7f6030072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:05.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 -- 172.21.15.160:0/1405147316 shutdown_connections 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.388+0000 7f6036cbf700 1 -- 172.21.15.160:0/1405147316 wait complete. 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.389+0000 7f6036cbf700 1 Processor -- start 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.389+0000 7f6036cbf700 1 -- start start 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.389+0000 7f6036cbf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 0x7f60300fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6036cbf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6030102e50 0x7f60300fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6036cbf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300fc910 con 0x7f6030065640 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6036cbf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300fca50 con 0x7f6030102e50 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 0x7f60300fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 0x7f60300fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46030/0 (socket says 172.21.15.160:46030) 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 -- 172.21.15.160:0/1989548092 learned_addr learned my addr 172.21.15.160:0/1989548092 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f602ffff700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6030102e50 0x7f60300fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 -- 172.21.15.160:0/1989548092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6030102e50 msgr2=0x7f60300fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6030102e50 0x7f60300fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 -- 172.21.15.160:0/1989548092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024005040 con 0x7f6030065640 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.390+0000 7f6034a5b700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 0x7f60300fbdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f602400b8b0 tx=0x7f6024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.391+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f602400e040 con 0x7f6030065640 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.391+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6024004290 con 0x7f6030065640 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.391+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6030119e80 con 0x7f6030065640 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.391+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f602401c3f0 con 0x7f6030065640 2024-05-25T06:17:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.391+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603011a360 con 0x7f6030065640 2024-05-25T06:17:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.392+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60300fe540 con 0x7f6030065640 2024-05-25T06:17:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.393+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f602401c620 con 0x7f6030065640 2024-05-25T06:17:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.393+0000 7f602d7fa700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f602006e260 0x7f6020070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.393+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f602408ef70 con 0x7f6030065640 2024-05-25T06:17:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.393+0000 7f602ffff700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f602006e260 0x7f6020070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.394+0000 7f602ffff700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f602006e260 0x7f6020070710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f601c006fd0 tx=0x7f601c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.398+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f60240c3850 con 0x7f6030065640 2024-05-25T06:17:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:04 smithi203 ceph-mon[83507]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:04 smithi203 ceph-mon[83507]: Health check failed: Degraded data redundancy: 63/324 objects degraded (19.444%), 19 pgs degraded (PG_DEGRADED) 2024-05-25T06:17:05.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.622+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60300fcbe0 con 0x7f602006e260 2024-05-25T06:17:05.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.624+0000 7f602d7fa700 1 -- 172.21.15.160:0/1989548092 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f60300fcbe0 con 0x7f602006e260 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f602006e260 msgr2=0x7f6020070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f602006e260 0x7f6020070710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f601c006fd0 tx=0x7f601c008040 comp rx=0 tx=0).stop 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 msgr2=0x7f60300fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 0x7f60300fbdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f602400b8b0 tx=0x7f6024009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 shutdown_connections 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6030065640 0x7f60300fbdb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f602006e260 0x7f6020070710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 --2- 172.21.15.160:0/1989548092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6030102e50 0x7f60300fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.626+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 >> 172.21.15.160:0/1989548092 conn(0x7f6030071d00 msgr2=0x7f60300fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.627+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 shutdown_connections 2024-05-25T06:17:05.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.627+0000 7f6036cbf700 1 -- 172.21.15.160:0/1989548092 wait complete. 2024-05-25T06:17:05.638 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:17:05.719 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 podman[123270]: 2024-05-25 06:17:05.416444594 +0000 UTC m=+0.453536060 container attach 90c062e23277256ae749b787746722b2f944e76d023dad1191233e9489d46858 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.738+0000 7f6923486700 1 -- 172.21.15.160:0/509967528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fde80 msgr2=0x7f691c0fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.738+0000 7f6923486700 1 --2- 172.21.15.160:0/509967528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fde80 0x7f691c0fe250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6918004970 tx=0x7f6918009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.738+0000 7f6923486700 1 -- 172.21.15.160:0/509967528 shutdown_connections 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.738+0000 7f6923486700 1 --2- 172.21.15.160:0/509967528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f691c0fe820 0x7f691c106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.738+0000 7f6923486700 1 --2- 172.21.15.160:0/509967528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fde80 0x7f691c0fe250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.738+0000 7f6923486700 1 -- 172.21.15.160:0/509967528 >> 172.21.15.160:0/509967528 conn(0x7f691c0f78e0 msgr2=0x7f691c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.739+0000 7f6923486700 1 -- 172.21.15.160:0/509967528 shutdown_connections 2024-05-25T06:17:05.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.739+0000 7f6923486700 1 -- 172.21.15.160:0/509967528 wait complete. 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.739+0000 7f6923486700 1 Processor -- start 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.739+0000 7f6923486700 1 -- start start 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6923486700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f691c0fde80 0x7f691c0fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6923486700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 0x7f691c0fc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6923486700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0fdc90 con 0x7f691c0fe820 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6923486700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0fc7c0 con 0x7f691c0fde80 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 0x7f691c0fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 0x7f691c0fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50202/0 (socket says 172.21.15.160:50202) 2024-05-25T06:17:05.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 -- 172.21.15.160:0/1653160181 learned_addr learned my addr 172.21.15.160:0/1653160181 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6921222700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f691c0fde80 0x7f691c0fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 -- 172.21.15.160:0/1653160181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f691c0fde80 msgr2=0x7f691c0fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f691c0fde80 0x7f691c0fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 -- 172.21.15.160:0/1653160181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6918005040 con 0x7f691c0fe820 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6920a21700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 0x7f691c0fc280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f690c00eb40 tx=0x7f690c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.740+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690c007cc0 con 0x7f691c0fe820 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.741+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f690c00f040 con 0x7f691c0fe820 2024-05-25T06:17:05.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.741+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c0fca40 con 0x7f691c0fe820 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.741+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690c00ca10 con 0x7f691c0fe820 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.741+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c119fc0 con 0x7f691c0fe820 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.742+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f690c00cc30 con 0x7f691c0fe820 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.743+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c066a30 con 0x7f691c0fe820 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.743+0000 7f6911ffb700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f690806e270 0x7f6908070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.743+0000 7f6921222700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f690806e270 0x7f6908070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:05.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.743+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f690c08d7e0 con 0x7f691c0fe820 2024-05-25T06:17:05.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.743+0000 7f6921222700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f690806e270 0x7f6908070720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6918000c00 tx=0x7f6918009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:05.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.752+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f690c092060 con 0x7f691c0fe820 2024-05-25T06:17:05.991 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-25T06:17:05.991 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 bash[123270]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-25T06:17:05.992 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate[123293]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-25T06:17:05.993 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:05 smithi160 podman[123270]: 2024-05-25 06:17:05.822459914 +0000 UTC m=+0.859551354 container died 90c062e23277256ae749b787746722b2f944e76d023dad1191233e9489d46858 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0) 2024-05-25T06:17:05.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.994+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f691c0fee70 con 0x7f690806e270 2024-05-25T06:17:05.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.995+0000 7f6911ffb700 1 -- 172.21.15.160:0/1653160181 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f691c0fee70 con 0x7f690806e270 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f690806e270 msgr2=0x7f6908070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f690806e270 0x7f6908070720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6918000c00 tx=0x7f6918009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 msgr2=0x7f691c0fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 0x7f691c0fc280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f690c00eb40 tx=0x7f690c00b040 comp rx=0 tx=0).stop 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 shutdown_connections 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f690806e270 0x7f6908070720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f691c0fde80 0x7f691c0fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 --2- 172.21.15.160:0/1653160181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f691c0fe820 0x7f691c0fc280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:05.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 >> 172.21.15.160:0/1653160181 conn(0x7f691c0f78e0 msgr2=0x7f691c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:05.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 shutdown_connections 2024-05-25T06:17:05.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:05.998+0000 7f6923486700 1 -- 172.21.15.160:0/1653160181 wait complete. 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 -- 172.21.15.160:0/545935217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7601026a0 msgr2=0x7fe7600fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 --2- 172.21.15.160:0/545935217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7601026a0 0x7fe7600fbd10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe750004940 tx=0x7fe750009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 -- 172.21.15.160:0/545935217 shutdown_connections 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 --2- 172.21.15.160:0/545935217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7601026a0 0x7fe7600fbd10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 --2- 172.21.15.160:0/545935217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760101d00 0x7fe7601020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 -- 172.21.15.160:0/545935217 >> 172.21.15.160:0/545935217 conn(0x7fe7600f7870 msgr2=0x7fe7600f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 -- 172.21.15.160:0/545935217 shutdown_connections 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 -- 172.21.15.160:0/545935217 wait complete. 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 Processor -- start 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.104+0000 7fe7681e3700 1 -- start start 2024-05-25T06:17:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7681e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 0x7fe7600fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7681e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7601026a0 0x7fe7600fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7681e3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600ff8d0 con 0x7fe760101d00 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7681e3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600fe400 con 0x7fe7601026a0 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 0x7fe7600fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 0x7fe7600fd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50218/0 (socket says 172.21.15.160:50218) 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 -- 172.21.15.160:0/1585814641 learned_addr learned my addr 172.21.15.160:0/1585814641 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe76577e700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7601026a0 0x7fe7600fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 -- 172.21.15.160:0/1585814641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7601026a0 msgr2=0x7fe7600fdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7601026a0 0x7fe7600fdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 -- 172.21.15.160:0/1585814641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750005040 con 0x7fe760101d00 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe765f7f700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 0x7fe7600fd980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe75c002a50 tx=0x7fe75c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe75c007cc0 con 0x7fe760101d00 2024-05-25T06:17:06.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe75c00f040 con 0x7fe760101d00 2024-05-25T06:17:06.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7600fe6e0 con 0x7fe760101d00 2024-05-25T06:17:06.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe75c00da30 con 0x7fe760101d00 2024-05-25T06:17:06.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.105+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76011a0e0 con 0x7fe760101d00 2024-05-25T06:17:06.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.106+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7600fd450 con 0x7fe760101d00 2024-05-25T06:17:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.107+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fe75c00db90 con 0x7fe760101d00 2024-05-25T06:17:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.107+0000 7fe7567fc700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe74c06e270 0x7fe74c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.107+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe75c08db10 con 0x7fe760101d00 2024-05-25T06:17:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.108+0000 7fe76577e700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe74c06e270 0x7fe74c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.108+0000 7fe76577e700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe74c06e270 0x7fe74c070720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe750004890 tx=0x7fe750009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:06.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.112+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe75c056cc0 con 0x7fe760101d00 2024-05-25T06:17:06.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.321+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7600f8f00 con 0x7fe74c06e270 2024-05-25T06:17:06.321 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:05 smithi160 ceph-mon[112094]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:06.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.332+0000 7fe7567fc700 1 -- 172.21.15.160:0/1585814641 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe7600f8f00 con 0x7fe74c06e270 2024-05-25T06:17:06.332 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (2m) 14s ago 14m 26.1M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (25s) 14s ago 14m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (20s) 16s ago 13m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (2m) 14s ago 13m 76.1M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (8m) 14s ago 8m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (8m) 14s ago 8m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (8m) 16s ago 8m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (8m) 16s ago 8m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (4m) 14s ago 15m 497M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (3m) 16s ago 13m 451M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (93s) 14s ago 15m 41.4M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:17:06.333 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (66s) 16s ago 13m 34.8M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (3m) 14s ago 14m 15.6M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (3m) 16s ago 13m 17.4M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (12m) 14s ago 12m 218M 1557M 17.2.7-958-g7b01852c baf1547092e7 a030e70263e3 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (11m) 14s ago 11m 227M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (11m) 14s ago 11m 191M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (11m) 16s ago 11m 269M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (10m) 16s ago 10m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (10m) 16s ago 10m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:17:06.334 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (3m) 14s ago 13m 65.3M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe74c06e270 msgr2=0x7fe74c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe74c06e270 0x7fe74c070720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe750004890 tx=0x7fe750009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 msgr2=0x7fe7600fd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 0x7fe7600fd980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe75c002a50 tx=0x7fe75c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 shutdown_connections 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760101d00 0x7fe7600fd980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe74c06e270 0x7fe74c070720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.334+0000 7fe7681e3700 1 --2- 172.21.15.160:0/1585814641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7601026a0 0x7fe7600fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.335+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 >> 172.21.15.160:0/1585814641 conn(0x7fe7600f7870 msgr2=0x7fe7600f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:06.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.335+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 shutdown_connections 2024-05-25T06:17:06.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.335+0000 7fe7681e3700 1 -- 172.21.15.160:0/1585814641 wait complete. 2024-05-25T06:17:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:05 smithi203 ceph-mon[83507]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3842218588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308102de0 msgr2=0x7fd3081031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3842218588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308102de0 0x7fd3081031b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8004940 tx=0x7fd2f8009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3842218588 shutdown_connections 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3842218588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd308103780 0x7fd3080fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3842218588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308102de0 0x7fd3081031b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3842218588 >> 172.21.15.160:0/3842218588 conn(0x7fd3080f8990 msgr2=0x7fd3080fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3842218588 shutdown_connections 2024-05-25T06:17:06.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3842218588 wait complete. 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.437+0000 7fd30d8a1700 1 Processor -- start 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd30d8a1700 1 -- start start 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd30d8a1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd308102de0 0x7fd308113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd30d8a1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 0x7fd308113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd30d8a1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308114550 con 0x7fd308103780 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd30d8a1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30810dac0 con 0x7fd308102de0 2024-05-25T06:17:06.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 0x7fd308113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 0x7fd308113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50230/0 (socket says 172.21.15.160:50230) 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 -- 172.21.15.160:0/3951903287 learned_addr learned my addr 172.21.15.160:0/3951903287 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd306ffd700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd308102de0 0x7fd308113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 -- 172.21.15.160:0/3951903287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd308102de0 msgr2=0x7fd308113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd308102de0 0x7fd308113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 -- 172.21.15.160:0/3951903287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8005040 con 0x7fd308103780 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.438+0000 7fd3067fc700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 0x7fd308113e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00b790 tx=0x7fd2fc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.439+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2fc00f6f0 con 0x7fd308103780 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.439+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30810dda0 con 0x7fd308103780 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.439+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2fc00fd20 con 0x7fd308103780 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.439+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2fc0103a0 con 0x7fd308103780 2024-05-25T06:17:06.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.439+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30810e360 con 0x7fd308103780 2024-05-25T06:17:06.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.440+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3081000f0 con 0x7fd308103780 2024-05-25T06:17:06.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.440+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fd2fc011400 con 0x7fd308103780 2024-05-25T06:17:06.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.440+0000 7fd2ef7fe700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2f006e270 0x7fd2f0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.441+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd2fc08da10 con 0x7fd308103780 2024-05-25T06:17:06.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.441+0000 7fd306ffd700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2f006e270 0x7fd2f0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.441+0000 7fd306ffd700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2f006e270 0x7fd2f0070720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8004f90 tx=0x7fd2f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:06.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.445+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2fc05a880 con 0x7fd308103780 2024-05-25T06:17:06.719 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:06 smithi160 podman[123466]: 2024-05-25 06:17:06.492430059 +0000 UTC m=+0.663066876 container remove 90c062e23277256ae749b787746722b2f944e76d023dad1191233e9489d46858 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-activate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-25T06:17:06.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.762+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd30810ef80 con 0x7fd308103780 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.763+0000 7fd2ef7fe700 1 -- 172.21.15.160:0/3951903287 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7fd2fc05a410 con 0x7fd308103780 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:17:06.763 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 5 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 9, 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:17:06.764 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:17:06.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2f006e270 msgr2=0x7fd2f0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2f006e270 0x7fd2f0070720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8004f90 tx=0x7fd2f8009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:06.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 msgr2=0x7fd308113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 0x7fd308113e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00b790 tx=0x7fd2fc009220 comp rx=0 tx=0).stop 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 shutdown_connections 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2f006e270 0x7fd2f0070720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd308102de0 0x7fd308113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 --2- 172.21.15.160:0/3951903287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd308103780 0x7fd308113e70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 >> 172.21.15.160:0/3951903287 conn(0x7fd3080f8990 msgr2=0x7fd3080fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 shutdown_connections 2024-05-25T06:17:06.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.765+0000 7fd30d8a1700 1 -- 172.21.15.160:0/3951903287 wait complete. 2024-05-25T06:17:06.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 -- 172.21.15.160:0/2268069132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 msgr2=0x7f797c0fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 --2- 172.21.15.160:0/2268069132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c0fbcf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f796c004940 tx=0x7f796c009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:06.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 -- 172.21.15.160:0/2268069132 shutdown_connections 2024-05-25T06:17:06.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 --2- 172.21.15.160:0/2268069132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c0fbcf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 --2- 172.21.15.160:0/2268069132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101ce0 0x7f797c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 -- 172.21.15.160:0/2268069132 >> 172.21.15.160:0/2268069132 conn(0x7f797c0f7850 msgr2=0x7f797c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 -- 172.21.15.160:0/2268069132 shutdown_connections 2024-05-25T06:17:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 -- 172.21.15.160:0/2268069132 wait complete. 2024-05-25T06:17:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 Processor -- start 2024-05-25T06:17:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.871+0000 7f79833ae700 1 -- start start 2024-05-25T06:17:06.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f79833ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101ce0 0x7f797c112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f79833ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f79833ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c113630 con 0x7f797c102680 2024-05-25T06:17:06.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f79833ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c10ca20 con 0x7f797c101ce0 2024-05-25T06:17:06.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50240/0 (socket says 172.21.15.160:50240) 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f798114a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101ce0 0x7f797c112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 -- 172.21.15.160:0/832760229 learned_addr learned my addr 172.21.15.160:0/832760229 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 -- 172.21.15.160:0/832760229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101ce0 msgr2=0x7f797c112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101ce0 0x7f797c112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 -- 172.21.15.160:0/832760229 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c005040 con 0x7f797c102680 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7980949700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c112fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f796c009100 tx=0x7f796c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.872+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f796c00e040 con 0x7f797c102680 2024-05-25T06:17:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.873+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c10cca0 con 0x7f797c102680 2024-05-25T06:17:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.873+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f796c004290 con 0x7f797c102680 2024-05-25T06:17:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.873+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f796c01c3f0 con 0x7f797c102680 2024-05-25T06:17:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.873+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c10d260 con 0x7f797c102680 2024-05-25T06:17:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.874+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c0fd430 con 0x7f797c102680 2024-05-25T06:17:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.875+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f796c013690 con 0x7f797c102680 2024-05-25T06:17:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.875+0000 7f7971ffb700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f796806e260 0x7f7968070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:06.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.875+0000 7f798114a700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f796806e260 0x7f7968070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:06.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.876+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f796c08edc0 con 0x7f797c102680 2024-05-25T06:17:06.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.876+0000 7f798114a700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f796806e260 0x7f7968070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7978004f40 tx=0x7f797800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:06.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:06.882+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f796c05bbc0 con 0x7f797c102680 2024-05-25T06:17:06.987 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:06 smithi160 podman[123574]: 2024-05-25 06:17:06.624806465 +0000 UTC m=+0.017581862 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:17:06.987 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:06 smithi160 podman[123574]: 2024-05-25 06:17:06.75169879 +0000 UTC m=+0.144474168 container create 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-25T06:17:07.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.138+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f797c10e0f0 con 0x7f797c102680 2024-05-25T06:17:07.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.140+0000 7f7971ffb700 1 -- 172.21.15.160:0/832760229 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7f796c05b750 con 0x7f797c102680 2024-05-25T06:17:07.140 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:17:07.141 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:17:07.142 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:17:07.143 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f796806e260 msgr2=0x7f7968070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f796806e260 0x7f7968070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7978004f40 tx=0x7f797800b040 comp rx=0 tx=0).stop 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 msgr2=0x7f797c112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c112fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f796c009100 tx=0x7f796c009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 shutdown_connections 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f796806e260 0x7f7968070710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101ce0 0x7f797c112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 --2- 172.21.15.160:0/832760229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f797c102680 0x7f797c112fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 >> 172.21.15.160:0/832760229 conn(0x7f797c0f7850 msgr2=0x7f797c0f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:07.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 shutdown_connections 2024-05-25T06:17:07.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.143+0000 7f79833ae700 1 -- 172.21.15.160:0/832760229 wait complete. 2024-05-25T06:17:07.145 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:17:07.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.252+0000 7fc79a8b0700 1 -- 172.21.15.160:0/3753544420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794102df0 msgr2=0x7fc7941031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.252+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/3753544420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794102df0 0x7fc7941031c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc784004940 tx=0x7fc784009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:07.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.252+0000 7fc79a8b0700 1 -- 172.21.15.160:0/3753544420 shutdown_connections 2024-05-25T06:17:07.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.252+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/3753544420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc794103790 0x7fc7940fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.252+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/3753544420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794102df0 0x7fc7941031c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.252+0000 7fc79a8b0700 1 -- 172.21.15.160:0/3753544420 >> 172.21.15.160:0/3753544420 conn(0x7fc7940f8980 msgr2=0x7fc7940fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:07.253 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:06 smithi160 ceph-mon[112094]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:07.254 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:06 smithi160 ceph-mon[112094]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:07.254 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:06 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3951903287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:07.254 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:06 smithi160 ceph-mon[112094]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:07.254 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:06 smithi160 podman[123574]: 2024-05-25 06:17:06.987690464 +0000 UTC m=+0.380465842 container init 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-25T06:17:07.254 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:06 smithi160 podman[123574]: 2024-05-25 06:17:06.991637522 +0000 UTC m=+0.384412907 container start 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-25T06:17:07.254 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:07 smithi160 ceph-osd[123616]: -- 172.21.15.160:0/3261095170 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55857f9f9680 con 0x55857f9c3000 2024-05-25T06:17:07.254 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:07 smithi160 bash[123574]: 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 2024-05-25T06:17:07.254 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:07 smithi160 systemd[1]: Started Ceph osd.0 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:17:07.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.253+0000 7fc79a8b0700 1 -- 172.21.15.160:0/3753544420 shutdown_connections 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.253+0000 7fc79a8b0700 1 -- 172.21.15.160:0/3753544420 wait complete. 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc79a8b0700 1 Processor -- start 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc79a8b0700 1 -- start start 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc79a8b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc794102df0 0x7fc794113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc79a8b0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 0x7fc7941140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc79a8b0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941147a0 con 0x7fc794103790 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc79a8b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79410db00 con 0x7fc794102df0 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 0x7fc7941140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 0x7fc7941140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50270/0 (socket says 172.21.15.160:50270) 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc7937fe700 1 -- 172.21.15.160:0/1102495115 learned_addr learned my addr 172.21.15.160:0/1102495115 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.254+0000 7fc793fff700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc794102df0 0x7fc794113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc7937fe700 1 -- 172.21.15.160:0/1102495115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc794102df0 msgr2=0x7fc794113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc7937fe700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc794102df0 0x7fc794113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc7937fe700 1 -- 172.21.15.160:0/1102495115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784005040 con 0x7fc794103790 2024-05-25T06:17:07.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc7937fe700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 0x7fc7941140c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b790 tx=0x7fc788009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:07.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78800f6f0 con 0x7fc794103790 2024-05-25T06:17:07.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc78800fd20 con 0x7fc794103790 2024-05-25T06:17:07.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79410dde0 con 0x7fc794103790 2024-05-25T06:17:07.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7880103a0 con 0x7fc794103790 2024-05-25T06:17:07.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.255+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79410e3a0 con 0x7fc794103790 2024-05-25T06:17:07.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.256+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7940fe540 con 0x7fc794103790 2024-05-25T06:17:07.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.257+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fc788010500 con 0x7fc794103790 2024-05-25T06:17:07.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.257+0000 7fc790ff9700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc77c06e260 0x7fc77c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:07.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.257+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc78808da80 con 0x7fc794103790 2024-05-25T06:17:07.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.257+0000 7fc793fff700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc77c06e260 0x7fc77c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:07.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.258+0000 7fc793fff700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc77c06e260 0x7fc77c070710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc784004f90 tx=0x7fc784009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:07.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.262+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc78805a8f0 con 0x7fc794103790 2024-05-25T06:17:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:06 smithi203 ceph-mon[83507]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:06 smithi203 ceph-mon[83507]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:06 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3951903287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:06 smithi203 ceph-mon[83507]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:07.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.486+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc794113850 con 0x7fc77c06e260 2024-05-25T06:17:07.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.487+0000 7fc790ff9700 1 -- 172.21.15.160:0/1102495115 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc794113850 con 0x7fc77c06e260 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "6/21 daemons upgraded", 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading osd daemons", 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:17:07.488 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc77c06e260 msgr2=0x7fc77c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc77c06e260 0x7fc77c070710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc784004f90 tx=0x7fc784009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 msgr2=0x7fc7941140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 0x7fc7941140c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b790 tx=0x7fc788009220 comp rx=0 tx=0).stop 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 shutdown_connections 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc77c06e260 0x7fc77c070710 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc794102df0 0x7fc794113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 --2- 172.21.15.160:0/1102495115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794103790 0x7fc7941140c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 >> 172.21.15.160:0/1102495115 conn(0x7fc7940f8980 msgr2=0x7fc7940f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:07.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 shutdown_connections 2024-05-25T06:17:07.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.490+0000 7fc79a8b0700 1 -- 172.21.15.160:0/1102495115 wait complete. 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 -- 172.21.15.160:0/629218550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e20101e20 msgr2=0x7f9e201021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 --2- 172.21.15.160:0/629218550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e20101e20 0x7f9e201021f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c004940 tx=0x7f9e1c009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 -- 172.21.15.160:0/629218550 shutdown_connections 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 --2- 172.21.15.160:0/629218550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e200fc710 0x7f9e200fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 --2- 172.21.15.160:0/629218550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e20101e20 0x7f9e201021f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 -- 172.21.15.160:0/629218550 >> 172.21.15.160:0/629218550 conn(0x7f9e20071460 msgr2=0x7f9e20071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 -- 172.21.15.160:0/629218550 shutdown_connections 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.601+0000 7f9e285c5700 1 -- 172.21.15.160:0/629218550 wait complete. 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e285c5700 1 Processor -- start 2024-05-25T06:17:07.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e285c5700 1 -- start start 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e285c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 0x7f9e20113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e285c5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e20101e20 0x7f9e20114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e285c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20114680 con 0x7f9e200fc710 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e285c5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2010da70 con 0x7f9e20101e20 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e26361700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 0x7f9e20113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e26361700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 0x7f9e20113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50284/0 (socket says 172.21.15.160:50284) 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e26361700 1 -- 172.21.15.160:0/3634897358 learned_addr learned my addr 172.21.15.160:0/3634897358 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.602+0000 7f9e25b60700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e20101e20 0x7f9e20114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e26361700 1 -- 172.21.15.160:0/3634897358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e20101e20 msgr2=0x7f9e20114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e26361700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e20101e20 0x7f9e20114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e26361700 1 -- 172.21.15.160:0/3634897358 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c005040 con 0x7f9e200fc710 2024-05-25T06:17:07.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e26361700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 0x7f9e20113af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009380 tx=0x7f9e1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:07.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e1c00e070 con 0x7f9e200fc710 2024-05-25T06:17:07.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2010dcf0 con 0x7f9e200fc710 2024-05-25T06:17:07.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e1c013c00 con 0x7f9e200fc710 2024-05-25T06:17:07.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e1c00b650 con 0x7f9e200fc710 2024-05-25T06:17:07.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.603+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2010e1d0 con 0x7f9e200fc710 2024-05-25T06:17:07.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.604+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e20105270 con 0x7f9e200fc710 2024-05-25T06:17:07.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.605+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 39) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f9e1c01c960 con 0x7f9e200fc710 2024-05-25T06:17:07.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.605+0000 7f9e16ffd700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9e0c06e260 0x7f9e0c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:07.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.606+0000 7f9e25b60700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9e0c06e260 0x7f9e0c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:07.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.606+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9e1c08f650 con 0x7f9e200fc710 2024-05-25T06:17:07.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.606+0000 7f9e25b60700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9e0c06e260 0x7f9e0c070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e2010ec30 tx=0x7f9e1000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:07.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.611+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9e1c05c3b0 con 0x7f9e200fc710 2024-05-25T06:17:07.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.925+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9e2010f140 con 0x7f9e200fc710 2024-05-25T06:17:07.925 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:07 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[123594]: 2024-05-25T06:17:07.539+0000 7f90e03fd7c0 -1 Falling back to public interface 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.925+0000 7f9e16ffd700 1 -- 172.21.15.160:0/3634897358 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7f9e1c05bf40 con 0x7f9e200fc710 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 63/324 objects degraded (19.444%), 19 pgs degraded 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout: osd.0 (root=default,host=smithi160) is down 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 63/324 objects degraded (19.444%), 19 pgs degraded 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-05-25T06:17:07.926 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-05-25T06:17:07.927 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-05-25T06:17:07.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9e0c06e260 msgr2=0x7f9e0c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9e0c06e260 0x7f9e0c070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e2010ec30 tx=0x7f9e1000b040 comp rx=0 tx=0).stop 2024-05-25T06:17:07.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 msgr2=0x7f9e20113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:07.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 0x7f9e20113af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009380 tx=0x7f9e1c009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 shutdown_connections 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e200fc710 0x7f9e20113af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9e0c06e260 0x7f9e0c070710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 --2- 172.21.15.160:0/3634897358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e20101e20 0x7f9e20114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 >> 172.21.15.160:0/3634897358 conn(0x7f9e20071460 msgr2=0x7f9e200fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 shutdown_connections 2024-05-25T06:17:07.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:07.927+0000 7f9e285c5700 1 -- 172.21.15.160:0/3634897358 wait complete. 2024-05-25T06:17:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:08 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:08.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/832760229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:17:08.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:08 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:08.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:08 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:08.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:08 smithi160 ceph-mon[112094]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:08.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3634897358' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:17:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:08 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/832760229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:17:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:08 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:08 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:08.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:08 smithi203 ceph-mon[83507]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:08.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3634897358' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:17:09.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:09 smithi160 ceph-mon[112094]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:09 smithi203 ceph-mon[83507]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:12.027 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:11 smithi160 ceph-mon[112094]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:11 smithi203 ceph-mon[83507]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:12.469 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:12 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[123594]: 2024-05-25T06:17:12.186+0000 7f90e03fd7c0 -1 osd.0 49 log_to_monitors true 2024-05-25T06:17:13.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:12 smithi160 ceph-mon[112094]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-25T06:17:13.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:13.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:13.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:13.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:12 smithi203 ceph-mon[83507]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-25T06:17:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:13.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:14.219 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:17:13 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[123594]: 2024-05-25T06:17:13.794+0000 7f90cd84c700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: osdmap e52: 6 total, 5 up, 6 in 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:13 smithi160 ceph-mon[112094]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' 2024-05-25T06:17:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/324 objects degraded (19.444%) 2024-05-25T06:17:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-25T06:17:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: osdmap e52: 6 total, 5 up, 6 in 2024-05-25T06:17:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:17:14.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:14.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:14.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:13 smithi203 ceph-mon[83507]: from='osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185]' entity='osd.0' 2024-05-25T06:17:15.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:14 smithi160 ceph-mon[112094]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:17:15.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:14 smithi160 ceph-mon[112094]: osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185] boot 2024-05-25T06:17:15.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:14 smithi160 ceph-mon[112094]: osdmap e53: 6 total, 6 up, 6 in 2024-05-25T06:17:15.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:17:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:14 smithi203 ceph-mon[83507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:17:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:14 smithi203 ceph-mon[83507]: osd.0 [v2:172.21.15.160:6802/793926185,v1:172.21.15.160:6803/793926185] boot 2024-05-25T06:17:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:14 smithi203 ceph-mon[83507]: osdmap e53: 6 total, 6 up, 6 in 2024-05-25T06:17:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-25T06:17:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:15 smithi160 ceph-mon[112094]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s; 63/324 objects degraded (19.444%) 2024-05-25T06:17:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:15 smithi160 ceph-mon[112094]: osdmap e54: 6 total, 6 up, 6 in 2024-05-25T06:17:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:15 smithi160 ceph-mon[112094]: mgrmap e40: smithi160.nalzlg(active, since 92s), standbys: smithi203.rotqex 2024-05-25T06:17:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:15 smithi203 ceph-mon[83507]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s; 63/324 objects degraded (19.444%) 2024-05-25T06:17:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:15 smithi203 ceph-mon[83507]: osdmap e54: 6 total, 6 up, 6 in 2024-05-25T06:17:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:15 smithi203 ceph-mon[83507]: mgrmap e40: smithi160.nalzlg(active, since 92s), standbys: smithi203.rotqex 2024-05-25T06:17:17.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s; 63/324 objects degraded (19.444%) 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:17.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s; 63/324 objects degraded (19.444%) 2024-05-25T06:17:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:19.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:18 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:19.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:18 smithi160 ceph-mon[112094]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-25T06:17:19.300 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:18 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:19.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:18 smithi203 ceph-mon[83507]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-25T06:17:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:19 smithi160 ceph-mon[112094]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s; 46 B/s, 0 objects/s recovering 2024-05-25T06:17:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:19 smithi160 ceph-mon[112094]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 63/324 objects degraded (19.444%), 19 pgs degraded) 2024-05-25T06:17:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:19 smithi160 ceph-mon[112094]: Cluster is now healthy 2024-05-25T06:17:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:19 smithi203 ceph-mon[83507]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s; 46 B/s, 0 objects/s recovering 2024-05-25T06:17:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:19 smithi203 ceph-mon[83507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 63/324 objects degraded (19.444%), 19 pgs degraded) 2024-05-25T06:17:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:19 smithi203 ceph-mon[83507]: Cluster is now healthy 2024-05-25T06:17:22.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:21 smithi160 ceph-mon[112094]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 773 B/s wr, 5 op/s; 35 B/s, 0 objects/s recovering 2024-05-25T06:17:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:21 smithi203 ceph-mon[83507]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 773 B/s wr, 5 op/s; 35 B/s, 0 objects/s recovering 2024-05-25T06:17:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:23 smithi160 ceph-mon[112094]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 34 B/s, 0 objects/s recovering 2024-05-25T06:17:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:23 smithi203 ceph-mon[83507]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 34 B/s, 0 objects/s recovering 2024-05-25T06:17:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:25 smithi160 ceph-mon[112094]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 28 B/s, 0 objects/s recovering 2024-05-25T06:17:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:25 smithi203 ceph-mon[83507]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 28 B/s, 0 objects/s recovering 2024-05-25T06:17:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:27 smithi160 ceph-mon[112094]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 25 B/s, 0 objects/s recovering 2024-05-25T06:17:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:28.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:27 smithi203 ceph-mon[83507]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 25 B/s, 0 objects/s recovering 2024-05-25T06:17:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:28.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:29 smithi160 ceph-mon[112094]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 23 B/s, 0 objects/s recovering 2024-05-25T06:17:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:29 smithi203 ceph-mon[83507]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 23 B/s, 0 objects/s recovering 2024-05-25T06:17:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:31 smithi203 ceph-mon[83507]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:31 smithi160 ceph-mon[112094]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:32.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:33.460 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:33 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:33.460 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:33 smithi160 ceph-mon[112094]: Upgrade: osd.1 is safe to restart 2024-05-25T06:17:33.460 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:33 smithi160 ceph-mon[112094]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:33 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-25T06:17:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:33 smithi203 ceph-mon[83507]: Upgrade: osd.1 is safe to restart 2024-05-25T06:17:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:33 smithi203 ceph-mon[83507]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:35.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:34 smithi160 ceph-mon[112094]: Upgrade: Updating osd.1 2024-05-25T06:17:35.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:35.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-25T06:17:35.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:34 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:17:35.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:34 smithi160 ceph-mon[112094]: Deploying daemon osd.1 on smithi160 2024-05-25T06:17:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:34 smithi203 ceph-mon[83507]: Upgrade: Updating osd.1 2024-05-25T06:17:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-25T06:17:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:34 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:17:35.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:34 smithi203 ceph-mon[83507]: Deploying daemon osd.1 on smithi160 2024-05-25T06:17:36.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:35 smithi160 ceph-mon[112094]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:17:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:35 smithi203 ceph-mon[83507]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:17:37.470 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:37 smithi160 systemd[1]: Stopping Ceph osd.1 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:17:37.919 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:37 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[54545]: 2024-05-25T06:17:37.478+0000 7feab82e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:17:37.919 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:37 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[54545]: 2024-05-25T06:17:37.478+0000 7feab82e8700 -1 osd.1 54 *** Got signal Terminated *** 2024-05-25T06:17:37.919 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:37 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[54545]: 2024-05-25T06:17:37.478+0000 7feab82e8700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:17:38.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 -- 172.21.15.160:0/651704274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88103780 msgr2=0x7f0d880fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 --2- 172.21.15.160:0/651704274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88103780 0x7f0d880fcdf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d70004940 tx=0x7f0d70009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 -- 172.21.15.160:0/651704274 shutdown_connections 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 --2- 172.21.15.160:0/651704274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88103780 0x7f0d880fcdf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 --2- 172.21.15.160:0/651704274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88102de0 0x7f0d881031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 -- 172.21.15.160:0/651704274 >> 172.21.15.160:0/651704274 conn(0x7f0d880f8990 msgr2=0x7f0d880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.038+0000 7f0d8e652700 1 -- 172.21.15.160:0/651704274 shutdown_connections 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 -- 172.21.15.160:0/651704274 wait complete. 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 Processor -- start 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 -- start start 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 0x7f0d88113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88103780 0x7f0d88113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881144e0 con 0x7f0d88102de0 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d8e652700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8810dae0 con 0x7f0d88103780 2024-05-25T06:17:38.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 0x7f0d88113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 0x7f0d88113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42988/0 (socket says 172.21.15.160:42988) 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d87fff700 1 -- 172.21.15.160:0/378053533 learned_addr learned my addr 172.21.15.160:0/378053533 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.039+0000 7f0d7ffff700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88103780 0x7f0d88113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d87fff700 1 -- 172.21.15.160:0/378053533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88103780 msgr2=0x7f0d88113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d87fff700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88103780 0x7f0d88113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d87fff700 1 -- 172.21.15.160:0/378053533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70005040 con 0x7f0d88102de0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d7ffff700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88103780 0x7f0d88113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d87fff700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 0x7f0d88113950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d78002a50 tx=0x7f0d780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d78007cc0 con 0x7f0d88102de0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d7800f040 con 0x7f0d88102de0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8810ddc0 con 0x7f0d88102de0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d78009640 con 0x7f0d88102de0 2024-05-25T06:17:38.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.040+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8810e380 con 0x7f0d88102de0 2024-05-25T06:17:38.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.041+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d880fe530 con 0x7f0d88102de0 2024-05-25T06:17:38.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.042+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0d780097a0 con 0x7f0d88102de0 2024-05-25T06:17:38.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.042+0000 7f0d857fa700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d7406e140 0x7f0d740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.042+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0d7808d8f0 con 0x7f0d88102de0 2024-05-25T06:17:38.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.042+0000 7f0d7ffff700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d7406e140 0x7f0d740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.042+0000 7f0d7ffff700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d7406e140 0x7f0d740705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d8810eca0 tx=0x7f0d70009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:38.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.047+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0d7805a7f0 con 0x7f0d88102de0 2024-05-25T06:17:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:37 smithi160 ceph-mon[112094]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:37 smithi160 ceph-mon[112094]: osd.1 marked itself down and dead 2024-05-25T06:17:38.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.266+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d88109ec0 con 0x7f0d7406e140 2024-05-25T06:17:38.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.268+0000 7f0d857fa700 1 -- 172.21.15.160:0/378053533 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0d88109ec0 con 0x7f0d7406e140 2024-05-25T06:17:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d7406e140 msgr2=0x7f0d740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d7406e140 0x7f0d740705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d8810eca0 tx=0x7f0d70009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 msgr2=0x7f0d88113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 0x7f0d88113950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d78002a50 tx=0x7f0d780046b0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 shutdown_connections 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d88102de0 0x7f0d88113950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d7406e140 0x7f0d740705f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 --2- 172.21.15.160:0/378053533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d88103780 0x7f0d88113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 >> 172.21.15.160:0/378053533 conn(0x7f0d880f8990 msgr2=0x7f0d880f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 shutdown_connections 2024-05-25T06:17:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.270+0000 7f0d8e652700 1 -- 172.21.15.160:0/378053533 wait complete. 2024-05-25T06:17:38.281 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 -- 172.21.15.160:0/741011276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fc4d0 msgr2=0x7fdb78109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/741011276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fc4d0 0x7fdb78109230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb64004940 tx=0x7fdb64009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 -- 172.21.15.160:0/741011276 shutdown_connections 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/741011276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fc4d0 0x7fdb78109230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/741011276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb780fbbc0 0x7fdb780fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 -- 172.21.15.160:0/741011276 >> 172.21.15.160:0/741011276 conn(0x7fdb780f7810 msgr2=0x7fdb780f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 -- 172.21.15.160:0/741011276 shutdown_connections 2024-05-25T06:17:38.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.376+0000 7fdb7eac0700 1 -- 172.21.15.160:0/741011276 wait complete. 2024-05-25T06:17:38.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7eac0700 1 Processor -- start 2024-05-25T06:17:38.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7eac0700 1 -- start start 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7eac0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 0x7fdb78115aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7eac0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb780fc4d0 0x7fdb78115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7eac0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb781166c0 con 0x7fdb780fbbc0 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7eac0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7810fc10 con 0x7fdb780fc4d0 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7c85c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 0x7fdb78115aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.377+0000 7fdb7c85c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 0x7fdb78115aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43014/0 (socket says 172.21.15.160:43014) 2024-05-25T06:17:38.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7c85c700 1 -- 172.21.15.160:0/2425074911 learned_addr learned my addr 172.21.15.160:0/2425074911 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb77fff700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb780fc4d0 0x7fdb78115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7c85c700 1 -- 172.21.15.160:0/2425074911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb780fc4d0 msgr2=0x7fdb78115fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7c85c700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb780fc4d0 0x7fdb78115fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7c85c700 1 -- 172.21.15.160:0/2425074911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb64005040 con 0x7fdb780fbbc0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7c85c700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 0x7fdb78115aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b790 tx=0x7fdb6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb6c00f6f0 con 0x7fdb780fbbc0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7810fe90 con 0x7fdb780fbbc0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb6c00fd20 con 0x7fdb780fbbc0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb78110450 con 0x7fdb780fbbc0 2024-05-25T06:17:38.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.378+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb6c0103a0 con 0x7fdb780fbbc0 2024-05-25T06:17:38.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.380+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb78066a70 con 0x7fdb780fbbc0 2024-05-25T06:17:38.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.380+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdb6c010500 con 0x7fdb780fbbc0 2024-05-25T06:17:38.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.381+0000 7fdb757fa700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdb6806e140 0x7fdb680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.381+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdb6c08d7f0 con 0x7fdb780fbbc0 2024-05-25T06:17:38.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.381+0000 7fdb77fff700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdb6806e140 0x7fdb680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.382+0000 7fdb77fff700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdb6806e140 0x7fdb680705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb78111340 tx=0x7fdb64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:38.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.386+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdb6c05a880 con 0x7fdb780fbbc0 2024-05-25T06:17:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:37 smithi203 ceph-mon[83507]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:17:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:37 smithi203 ceph-mon[83507]: osd.1 marked itself down and dead 2024-05-25T06:17:38.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.615+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb781111b0 con 0x7fdb6806e140 2024-05-25T06:17:38.616 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:38 smithi160 podman[128376]: 2024-05-25 06:17:38.395538241 +0000 UTC m=+1.040033530 container died 02d37c42b78d8c1869d6cf8ff19d422585023c001fe7f6473054cfed733f475f (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-7b01852, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-25T06:17:38.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.616+0000 7fdb757fa700 1 -- 172.21.15.160:0/2425074911 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdb781111b0 con 0x7fdb6806e140 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdb6806e140 msgr2=0x7fdb680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdb6806e140 0x7fdb680705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb78111340 tx=0x7fdb64009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 msgr2=0x7fdb78115aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 0x7fdb78115aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b790 tx=0x7fdb6c009250 comp rx=0 tx=0).stop 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 shutdown_connections 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb780fbbc0 0x7fdb78115aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdb6806e140 0x7fdb680705f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 --2- 172.21.15.160:0/2425074911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb780fc4d0 0x7fdb78115fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 >> 172.21.15.160:0/2425074911 conn(0x7fdb780f7810 msgr2=0x7fdb780f7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:38.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 shutdown_connections 2024-05-25T06:17:38.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.619+0000 7fdb7eac0700 1 -- 172.21.15.160:0/2425074911 wait complete. 2024-05-25T06:17:38.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 -- 172.21.15.160:0/4228166646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed00101d90 msgr2=0x7fed00102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 --2- 172.21.15.160:0/4228166646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed00101d90 0x7fed00102160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fecf4004940 tx=0x7fecf4009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:38.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 -- 172.21.15.160:0/4228166646 shutdown_connections 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 --2- 172.21.15.160:0/4228166646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000fbd40 0x7fed000fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 --2- 172.21.15.160:0/4228166646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed00101d90 0x7fed00102160 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 -- 172.21.15.160:0/4228166646 >> 172.21.15.160:0/4228166646 conn(0x7fed00071d00 msgr2=0x7fed00072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 -- 172.21.15.160:0/4228166646 shutdown_connections 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 -- 172.21.15.160:0/4228166646 wait complete. 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 Processor -- start 2024-05-25T06:17:38.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.725+0000 7fed0703b700 1 -- start start 2024-05-25T06:17:38.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed0703b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 0x7fed00112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed0703b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed00101d90 0x7fed00112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed0703b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed00113630 con 0x7fed000fbd40 2024-05-25T06:17:38.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed0703b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0010ca20 con 0x7fed00101d90 2024-05-25T06:17:38.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 0x7fed00112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 0x7fed00112aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43032/0 (socket says 172.21.15.160:43032) 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 -- 172.21.15.160:0/960478482 learned_addr learned my addr 172.21.15.160:0/960478482 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fecfffff700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed00101d90 0x7fed00112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 -- 172.21.15.160:0/960478482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed00101d90 msgr2=0x7fed00112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed00101d90 0x7fed00112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 -- 172.21.15.160:0/960478482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4005040 con 0x7fed000fbd40 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed04dd7700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 0x7fed00112aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fecf400b3b0 tx=0x7fecf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf400e040 con 0x7fed000fbd40 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.726+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0010cca0 con 0x7fed000fbd40 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.727+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecf4013420 con 0x7fed000fbd40 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.727+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf401c630 con 0x7fed000fbd40 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.727+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0010d260 con 0x7fed000fbd40 2024-05-25T06:17:38.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.727+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed000fd430 con 0x7fed000fbd40 2024-05-25T06:17:38.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.728+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fecf4013590 con 0x7fed000fbd40 2024-05-25T06:17:38.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.729+0000 7fecfd7fa700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf006e140 0x7fecf00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:38.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.729+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fecf408ed30 con 0x7fed000fbd40 2024-05-25T06:17:38.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.729+0000 7fecfffff700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf006e140 0x7fecf00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:38.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.729+0000 7fecfffff700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf006e140 0x7fecf00705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed0010dbe0 tx=0x7fecec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:38.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.733+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fecf405bcb0 con 0x7fed000fbd40 2024-05-25T06:17:38.920 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:38 smithi160 podman[128376]: 2024-05-25 06:17:38.854045891 +0000 UTC m=+1.498541162 container cleanup 02d37c42b78d8c1869d6cf8ff19d422585023c001fe7f6473054cfed733f475f (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-7b01852, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-25T06:17:38.920 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:38 smithi160 bash[128376]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1 2024-05-25T06:17:38.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.960+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed00108df0 con 0x7fecf006e140 2024-05-25T06:17:38.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.971+0000 7fecfd7fa700 1 -- 172.21.15.160:0/960478482 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fed00108df0 con 0x7fecf006e140 2024-05-25T06:17:38.971 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (3m) 26s ago 14m 26.1M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (58s) 26s ago 14m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (52s) 49s ago 13m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (3m) 26s ago 14m 76.3M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (9m) 26s ago 9m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (9m) 26s ago 9m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (9m) 49s ago 9m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (9m) 49s ago 9m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (5m) 26s ago 16m 499M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (4m) 49s ago 13m 451M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (2m) 26s ago 16m 43.1M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (99s) 49s ago 13m 34.8M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (4m) 26s ago 14m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:17:38.972 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (4m) 49s ago 13m 17.4M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (32s) 26s ago 12m 17.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (12m) 26s ago 12m 227M 1557M 17.2.7-958-g7b01852c baf1547092e7 02d37c42b78d 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (11m) 26s ago 11m 192M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (11m) 49s ago 11m 269M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (11m) 49s ago 11m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (10m) 49s ago 10m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:17:38.973 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (3m) 26s ago 14m 65.3M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:17:38.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf006e140 msgr2=0x7fecf00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf006e140 0x7fecf00705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed0010dbe0 tx=0x7fecec00b040 comp rx=0 tx=0).stop 2024-05-25T06:17:38.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 msgr2=0x7fed00112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 0x7fed00112aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fecf400b3b0 tx=0x7fecf4009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 shutdown_connections 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed000fbd40 0x7fed00112aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf006e140 0x7fecf00705f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 --2- 172.21.15.160:0/960478482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed00101d90 0x7fed00112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 >> 172.21.15.160:0/960478482 conn(0x7fed00071d00 msgr2=0x7fed000fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 shutdown_connections 2024-05-25T06:17:38.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:38.974+0000 7fed0703b700 1 -- 172.21.15.160:0/960478482 wait complete. 2024-05-25T06:17:39.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 -- 172.21.15.160:0/585005702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 msgr2=0x7fa8740fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 --2- 172.21.15.160:0/585005702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa8740fbcf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa868004940 tx=0x7fa868009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:39.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 -- 172.21.15.160:0/585005702 shutdown_connections 2024-05-25T06:17:39.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 --2- 172.21.15.160:0/585005702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa8740fbcf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 --2- 172.21.15.160:0/585005702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa874101ce0 0x7fa8741020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 -- 172.21.15.160:0/585005702 >> 172.21.15.160:0/585005702 conn(0x7fa8740f7850 msgr2=0x7fa8740f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:39.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 -- 172.21.15.160:0/585005702 shutdown_connections 2024-05-25T06:17:39.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.077+0000 7fa879048700 1 -- 172.21.15.160:0/585005702 wait complete. 2024-05-25T06:17:39.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa879048700 1 Processor -- start 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa879048700 1 -- start start 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa879048700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa874101ce0 0x7fa874112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa879048700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa874112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa879048700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874113630 con 0x7fa874102680 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa879048700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87410ca20 con 0x7fa874101ce0 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa87259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa874112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa87259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa874112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43054/0 (socket says 172.21.15.160:43054) 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa87259c700 1 -- 172.21.15.160:0/3080417013 learned_addr learned my addr 172.21.15.160:0/3080417013 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa872d9d700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa874101ce0 0x7fa874112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa87259c700 1 -- 172.21.15.160:0/3080417013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa874101ce0 msgr2=0x7fa874112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa87259c700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa874101ce0 0x7fa874112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.078+0000 7fa87259c700 1 -- 172.21.15.160:0/3080417013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868005040 con 0x7fa874102680 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.079+0000 7fa87259c700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa874112fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa868009100 tx=0x7fa868009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.079+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86800e040 con 0x7fa874102680 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.079+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87410cca0 con 0x7fa874102680 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.079+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa868009af0 con 0x7fa874102680 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.079+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86801c3f0 con 0x7fa874102680 2024-05-25T06:17:39.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.079+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87410d260 con 0x7fa874102680 2024-05-25T06:17:39.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.080+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8740fd430 con 0x7fa874102680 2024-05-25T06:17:39.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.080+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa86801c660 con 0x7fa874102680 2024-05-25T06:17:39.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.081+0000 7fa85b7fe700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa85c06e140 0x7fa85c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.081+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa86808ec60 con 0x7fa874102680 2024-05-25T06:17:39.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.081+0000 7fa872d9d700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa85c06e140 0x7fa85c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.081+0000 7fa872d9d700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa85c06e140 0x7fa85c0705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa864004eb0 tx=0x7fa86400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:39.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.086+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa86805bcf0 con 0x7fa874102680 2024-05-25T06:17:39.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:38 smithi160 ceph-mon[112094]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:17:39.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:38 smithi160 ceph-mon[112094]: osdmap e55: 6 total, 5 up, 6 in 2024-05-25T06:17:39.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:39 smithi160 podman[128440]: 2024-05-25 06:17:39.064423098 +0000 UTC m=+0.667103024 container remove 02d37c42b78d8c1869d6cf8ff19d422585023c001fe7f6473054cfed733f475f (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=quincy-7b01852, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-25T06:17:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:38 smithi203 ceph-mon[83507]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:17:39.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:38 smithi203 ceph-mon[83507]: osdmap e55: 6 total, 5 up, 6 in 2024-05-25T06:17:39.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.419+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa874108dc0 con 0x7fa874102680 2024-05-25T06:17:39.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.420+0000 7fa85b7fe700 1 -- 172.21.15.160:0/3080417013 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fa86805b880 con 0x7fa874102680 2024-05-25T06:17:39.420 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:17:39.420 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:17:39.420 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:17:39.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4, 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 8, 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 5 2024-05-25T06:17:39.421 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:17:39.422 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa85c06e140 msgr2=0x7fa85c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa85c06e140 0x7fa85c0705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa864004eb0 tx=0x7fa86400b040 comp rx=0 tx=0).stop 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 msgr2=0x7fa874112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa874112fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa868009100 tx=0x7fa868009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 shutdown_connections 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa85c06e140 0x7fa85c0705f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa874101ce0 0x7fa874112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 --2- 172.21.15.160:0/3080417013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa874102680 0x7fa874112fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 >> 172.21.15.160:0/3080417013 conn(0x7fa8740f7850 msgr2=0x7fa8740f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:39.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 shutdown_connections 2024-05-25T06:17:39.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.422+0000 7fa879048700 1 -- 172.21.15.160:0/3080417013 wait complete. 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.526+0000 7fe975a49700 1 -- 172.21.15.160:0/506802958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fbb90 msgr2=0x7fe9700fbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.526+0000 7fe975a49700 1 --2- 172.21.15.160:0/506802958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fbb90 0x7fe9700fbf60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe960004940 tx=0x7fe960009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.527+0000 7fe975a49700 1 -- 172.21.15.160:0/506802958 shutdown_connections 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.527+0000 7fe975a49700 1 --2- 172.21.15.160:0/506802958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fc4a0 0x7fe970109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.527+0000 7fe975a49700 1 --2- 172.21.15.160:0/506802958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fbb90 0x7fe9700fbf60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.527+0000 7fe975a49700 1 -- 172.21.15.160:0/506802958 >> 172.21.15.160:0/506802958 conn(0x7fe9700f7860 msgr2=0x7fe9700f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.527+0000 7fe975a49700 1 -- 172.21.15.160:0/506802958 shutdown_connections 2024-05-25T06:17:39.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.527+0000 7fe975a49700 1 -- 172.21.15.160:0/506802958 wait complete. 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe975a49700 1 Processor -- start 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe975a49700 1 -- start start 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe975a49700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fbb90 0x7fe9701159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe975a49700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 0x7fe970115ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe975a49700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970116530 con 0x7fe9700fc4a0 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe975a49700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97010fb50 con 0x7fe9700fbb90 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 0x7fe970115ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 0x7fe970115ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43076/0 (socket says 172.21.15.160:43076) 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96e7fc700 1 -- 172.21.15.160:0/1793908970 learned_addr learned my addr 172.21.15.160:0/1793908970 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96effd700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fbb90 0x7fe9701159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96e7fc700 1 -- 172.21.15.160:0/1793908970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fbb90 msgr2=0x7fe9701159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96e7fc700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fbb90 0x7fe9701159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96e7fc700 1 -- 172.21.15.160:0/1793908970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960005040 con 0x7fe9700fc4a0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.528+0000 7fe96effd700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fbb90 0x7fe9701159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.529+0000 7fe96e7fc700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 0x7fe970115ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe96400eb40 tx=0x7fe96400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.529+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe964007cc0 con 0x7fe9700fc4a0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.529+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97010fdd0 con 0x7fe9700fc4a0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.529+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe96400f040 con 0x7fe9700fc4a0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.529+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe964009620 con 0x7fe9700fc4a0 2024-05-25T06:17:39.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.529+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970110390 con 0x7fe9700fc4a0 2024-05-25T06:17:39.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.530+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970066b10 con 0x7fe9700fc4a0 2024-05-25T06:17:39.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.530+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe964009780 con 0x7fe9700fc4a0 2024-05-25T06:17:39.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.531+0000 7fe9577fe700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe95806e140 0x7fe9580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.531+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe96408d780 con 0x7fe9700fc4a0 2024-05-25T06:17:39.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.531+0000 7fe96effd700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe95806e140 0x7fe9580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.531+0000 7fe96effd700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe95806e140 0x7fe9580705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe96000b0d0 tx=0x7fe960009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:39.535 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.535+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe964056c00 con 0x7fe9700fc4a0 2024-05-25T06:17:39.719 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:39 smithi160 podman[128570]: 2024-05-25 06:17:39.41089932 +0000 UTC m=+0.020356560 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:17:39.719 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:39 smithi160 podman[128570]: 2024-05-25 06:17:39.52403791 +0000 UTC m=+0.133495127 container create e76d78e849788ceacee9bfbe47334b84906e91e4bb92b7cab1cc764e354b54d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-ea0684c) 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.796+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe970111020 con 0x7fe9700fc4a0 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.797+0000 7fe9577fe700 1 -- 172.21.15.160:0/1793908970 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7fe96405a2c0 con 0x7fe9700fc4a0 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:17:39.797 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:17:39.798 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:17:39.799 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:17:39.800 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe95806e140 msgr2=0x7fe9580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe95806e140 0x7fe9580705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe96000b0d0 tx=0x7fe960009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 msgr2=0x7fe970115ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 0x7fe970115ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe96400eb40 tx=0x7fe96400b040 comp rx=0 tx=0).stop 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 shutdown_connections 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe95806e140 0x7fe9580705f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9700fbb90 0x7fe9701159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 --2- 172.21.15.160:0/1793908970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700fc4a0 0x7fe970115ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 >> 172.21.15.160:0/1793908970 conn(0x7fe9700f7860 msgr2=0x7fe9700f7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 shutdown_connections 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.800+0000 7fe975a49700 1 -- 172.21.15.160:0/1793908970 wait complete. 2024-05-25T06:17:39.801 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 -- 172.21.15.160:0/3054537089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 msgr2=0x7f8eac0fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/3054537089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac0fc950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004940 tx=0x7f8ea0009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 -- 172.21.15.160:0/3054537089 shutdown_connections 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/3054537089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac0fc950 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/3054537089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac101cd0 0x7f8eac1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 -- 172.21.15.160:0/3054537089 >> 172.21.15.160:0/3054537089 conn(0x7f8eac0713f0 msgr2=0x7f8eac0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 -- 172.21.15.160:0/3054537089 shutdown_connections 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.907+0000 7f8eb1d18700 1 -- 172.21.15.160:0/3054537089 wait complete. 2024-05-25T06:17:39.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eb1d18700 1 Processor -- start 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eb1d18700 1 -- start start 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eb1d18700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac10eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eb1d18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac101cd0 0x7f8eac10efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eb1d18700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac10f630 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eb1d18700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac10f770 con 0x7f8eac101cd0 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eab7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac10eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eab7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac10eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43102/0 (socket says 172.21.15.160:43102) 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eab7fe700 1 -- 172.21.15.160:0/431297900 learned_addr learned my addr 172.21.15.160:0/431297900 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.908+0000 7f8eaaffd700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac101cd0 0x7f8eac10efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8eab7fe700 1 -- 172.21.15.160:0/431297900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac101cd0 msgr2=0x7f8eac10efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8eab7fe700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac101cd0 0x7f8eac10efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:39.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8eab7fe700 1 -- 172.21.15.160:0/431297900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0005040 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8eab7fe700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac10eaa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c002a50 tx=0x7f8e9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:39.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c007cc0 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e9c00f040 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac117b10 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c00da30 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.909+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac118070 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.910+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac0fd4f0 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.911+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8e9c00dbb0 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.911+0000 7f8e93fff700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8e9406e140 0x7f8e940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:39.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.912+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8e9c08d870 con 0x7f8eac0fc4e0 2024-05-25T06:17:39.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.912+0000 7f8eaaffd700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8e9406e140 0x7f8e940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:39.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.912+0000 7f8eaaffd700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8e9406e140 0x7f8e940705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0fbdd0 tx=0x7f8ea0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:39.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:39.916+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8e9c056c40 con 0x7f8eac0fc4e0 2024-05-25T06:17:40.081 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:39 smithi160 podman[128570]: 2024-05-25 06:17:39.768531791 +0000 UTC m=+0.377989030 container init e76d78e849788ceacee9bfbe47334b84906e91e4bb92b7cab1cc764e354b54d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-25T06:17:40.081 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:39 smithi160 podman[128570]: 2024-05-25 06:17:39.773132736 +0000 UTC m=+0.382589961 container start e76d78e849788ceacee9bfbe47334b84906e91e4bb92b7cab1cc764e354b54d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-ea0684c) 2024-05-25T06:17:40.081 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:39 smithi160 podman[128570]: 2024-05-25 06:17:39.885059489 +0000 UTC m=+0.494516720 container attach e76d78e849788ceacee9bfbe47334b84906e91e4bb92b7cab1cc764e354b54d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:17:40.081 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:40.082 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:40.082 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: pgmap v67: 65 pgs: 34 peering, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:17:40.082 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: Health check failed: Reduced data availability: 11 pgs peering (PG_AVAILABILITY) 2024-05-25T06:17:40.082 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: osdmap e56: 6 total, 5 up, 6 in 2024-05-25T06:17:40.082 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3080417013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:40.082 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:39 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1793908970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:17:40.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.147+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8eac04bed0 con 0x7f8e9406e140 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.148+0000 7f8e93fff700 1 -- 172.21.15.160:0/431297900 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8eac04bed0 con 0x7f8e9406e140 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:17:40.148 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:17:40.149 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:17:40.149 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:17:40.149 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "7/21 daemons upgraded", 2024-05-25T06:17:40.149 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading osd daemons", 2024-05-25T06:17:40.149 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:17:40.149 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8e9406e140 msgr2=0x7f8e940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8e9406e140 0x7f8e940705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0fbdd0 tx=0x7f8ea0009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 msgr2=0x7f8eac10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac10eaa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c002a50 tx=0x7f8e9c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 shutdown_connections 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eac0fc4e0 0x7f8eac10eaa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8e9406e140 0x7f8e940705f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 --2- 172.21.15.160:0/431297900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac101cd0 0x7f8eac10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.150+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 >> 172.21.15.160:0/431297900 conn(0x7f8eac0713f0 msgr2=0x7f8eac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.151+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 shutdown_connections 2024-05-25T06:17:40.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.151+0000 7f8eb1d18700 1 -- 172.21.15.160:0/431297900 wait complete. 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 -- 172.21.15.160:0/3136575508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 msgr2=0x7f85d4102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/3136575508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d4102160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85c4004940 tx=0x7f85c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 -- 172.21.15.160:0/3136575508 shutdown_connections 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/3136575508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d40fbd40 0x7f85d40fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/3136575508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d4102160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 -- 172.21.15.160:0/3136575508 >> 172.21.15.160:0/3136575508 conn(0x7f85d4071d00 msgr2=0x7f85d4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 -- 172.21.15.160:0/3136575508 shutdown_connections 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.255+0000 7f85d9ce0700 1 -- 172.21.15.160:0/3136575508 wait complete. 2024-05-25T06:17:40.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d9ce0700 1 Processor -- start 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d9ce0700 1 -- start start 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d9ce0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d40fbd40 0x7f85d410ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d9ce0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d410f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d9ce0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d410f760 con 0x7f85d4101d90 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d9ce0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d410f8a0 con 0x7f85d40fbd40 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85caffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d410f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85caffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d410f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43134/0 (socket says 172.21.15.160:43134) 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85caffd700 1 -- 172.21.15.160:0/47572704 learned_addr learned my addr 172.21.15.160:0/47572704 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.256+0000 7f85d37fe700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d40fbd40 0x7f85d410ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85caffd700 1 -- 172.21.15.160:0/47572704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d40fbd40 msgr2=0x7f85d410ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:40.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85caffd700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d40fbd40 0x7f85d410ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85caffd700 1 -- 172.21.15.160:0/47572704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4005040 con 0x7f85d4101d90 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85caffd700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d410f220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c8c0 tx=0x7f85bc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85bc00a760 con 0x7f85d4101d90 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85bc00ad90 con 0x7f85d4101d90 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d4117b90 con 0x7f85d4101d90 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85bc0113a0 con 0x7f85d4101d90 2024-05-25T06:17:40.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.257+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d4118120 con 0x7f85d4101d90 2024-05-25T06:17:40.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.258+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d40fd430 con 0x7f85d4101d90 2024-05-25T06:17:40.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.259+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f85bc012400 con 0x7f85d4101d90 2024-05-25T06:17:40.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.259+0000 7f85d0ff9700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f85c006e150 0x7f85c0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:17:40.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.259+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f85bc08db50 con 0x7f85d4101d90 2024-05-25T06:17:40.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.259+0000 7f85d37fe700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f85c006e150 0x7f85c0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:17:40.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.260+0000 7f85d37fe700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f85c006e150 0x7f85c0070600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f85c400b8b0 tx=0x7f85c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:17:40.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.264+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f85bc056fa0 con 0x7f85d4101d90 2024-05-25T06:17:40.388 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:40 smithi160 podman[128643]: 2024-05-25 06:17:40.115587067 +0000 UTC m=+0.018637337 container died e76d78e849788ceacee9bfbe47334b84906e91e4bb92b7cab1cc764e354b54d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5) 2024-05-25T06:17:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: pgmap v67: 65 pgs: 34 peering, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:17:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: Health check failed: Reduced data availability: 11 pgs peering (PG_AVAILABILITY) 2024-05-25T06:17:40.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: osdmap e56: 6 total, 5 up, 6 in 2024-05-25T06:17:40.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3080417013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:40.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:39 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1793908970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:17:40.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.549+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f85d4108df0 con 0x7f85d4101d90 2024-05-25T06:17:40.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.549+0000 7f85d0ff9700 1 -- 172.21.15.160:0/47572704 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1057 (secure 0 0 0) 0x7f85bc05a660 con 0x7f85d4101d90 2024-05-25T06:17:40.549 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 11 pgs peering 2024-05-25T06:17:40.549 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: osd.1 (root=default,host=smithi160) is down 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 11 pgs peering 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.4 is stuck peering for 5m, current state peering, last acting [0,4] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.6 is stuck peering for 4m, current state peering, last acting [3,4] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.9 is stuck peering for 4m, current state peering, last acting [4,0] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.a is stuck peering for 4m, current state peering, last acting [4,3] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.d is stuck peering for 5m, current state peering, last acting [3,2] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 2.1b is stuck peering for 6m, current state peering, last acting [0,5] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 3.0 is stuck peering for 4m, current state peering, last acting [2,4] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 3.2 is stuck peering for 4m, current state peering, last acting [5,3] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 3.4 is stuck peering for 4m, current state peering, last acting [2,3] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 3.b is stuck peering for 4m, current state peering, last acting [0,4] 2024-05-25T06:17:40.550 INFO:teuthology.orchestra.run.smithi160.stdout: pg 3.19 is stuck peering for 4m, current state peering, last acting [4,3] 2024-05-25T06:17:40.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f85c006e150 msgr2=0x7f85c0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:40.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f85c006e150 0x7f85c0070600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f85c400b8b0 tx=0x7f85c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 msgr2=0x7f85d410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d410f220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c8c0 tx=0x7f85bc0044c0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 shutdown_connections 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f85c006e150 0x7f85c0070600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d40fbd40 0x7f85d410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 --2- 172.21.15.160:0/47572704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d4101d90 0x7f85d410f220 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 >> 172.21.15.160:0/47572704 conn(0x7f85d4071d00 msgr2=0x7f85d40fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 shutdown_connections 2024-05-25T06:17:40.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:17:40.551+0000 7f85d9ce0700 1 -- 172.21.15.160:0/47572704 wait complete. 2024-05-25T06:17:41.216 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:40 smithi160 ceph-mon[112094]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:41.217 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:40 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/47572704' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:17:41.217 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:40 smithi160 podman[128643]: 2024-05-25 06:17:40.773316728 +0000 UTC m=+0.676367027 container remove e76d78e849788ceacee9bfbe47334b84906e91e4bb92b7cab1cc764e354b54d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c) 2024-05-25T06:17:41.217 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:40 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-25T06:17:41.217 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:40 smithi160 systemd[1]: Stopped Ceph osd.1 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:17:41.217 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:40 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.1.service: Consumed 1min 7.816s CPU time. 2024-05-25T06:17:41.217 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:40 smithi160 systemd[1]: Starting Ceph osd.1 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:17:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:40 smithi203 ceph-mon[83507]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:40 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/47572704' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:17:41.469 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 podman[128769]: 2024-05-25 06:17:41.11931651 +0000 UTC m=+0.017675482 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:17:41.469 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 podman[128769]: 2024-05-25 06:17:41.233005707 +0000 UTC m=+0.131364674 container create 47633d9292a3dd348a115abb477b20413e33110bec239301feebc11f0ffd0dbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD) 2024-05-25T06:17:41.470 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 podman[128769]: 2024-05-25 06:17:41.452372863 +0000 UTC m=+0.350731833 container init 47633d9292a3dd348a115abb477b20413e33110bec239301feebc11f0ffd0dbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-25T06:17:41.470 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 podman[128769]: 2024-05-25 06:17:41.456550628 +0000 UTC m=+0.354909596 container start 47633d9292a3dd348a115abb477b20413e33110bec239301feebc11f0ffd0dbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-25T06:17:41.908 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 podman[128769]: 2024-05-25 06:17:41.577219435 +0000 UTC m=+0.475578423 container attach 47633d9292a3dd348a115abb477b20413e33110bec239301feebc11f0ffd0dbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-25T06:17:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:41 smithi160 ceph-mon[112094]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:41 smithi160 ceph-mon[112094]: pgmap v69: 65 pgs: 34 peering, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 bash[128769]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-25T06:17:42.220 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-25T06:17:42.221 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-25T06:17:42.221 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-25T06:17:42.221 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate[128784]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-25T06:17:42.221 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:41 smithi160 podman[128769]: 2024-05-25 06:17:41.935058259 +0000 UTC m=+0.833417227 container died 47633d9292a3dd348a115abb477b20413e33110bec239301feebc11f0ffd0dbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate, RELEASE=squid-ea0684c, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-25T06:17:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:41 smithi203 ceph-mon[83507]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:17:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:41 smithi203 ceph-mon[83507]: pgmap v69: 65 pgs: 34 peering, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:17:42.990 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:42 smithi160 podman[128916]: 2024-05-25 06:17:42.740071135 +0000 UTC m=+0.799088870 container remove 47633d9292a3dd348a115abb477b20413e33110bec239301feebc11f0ffd0dbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-05-25T06:17:42.990 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:42 smithi160 podman[128974]: 2024-05-25 06:17:42.862082426 +0000 UTC m=+0.017888526 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:17:42.991 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:42 smithi160 podman[128974]: 2024-05-25 06:17:42.991251145 +0000 UTC m=+0.147057237 container create 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-25T06:17:43.250 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:43 smithi160 podman[128974]: 2024-05-25 06:17:43.226903891 +0000 UTC m=+0.382709991 container init 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, GIT_CLEAN=True) 2024-05-25T06:17:43.250 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:43 smithi160 podman[128974]: 2024-05-25 06:17:43.230323673 +0000 UTC m=+0.386129766 container start 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-25T06:17:43.719 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:43 smithi160 ceph-osd[128992]: -- 172.21.15.160:0/1272626043 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dfc3a5680 con 0x558dfc36ec00 2024-05-25T06:17:43.720 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:43 smithi160 bash[128974]: 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 2024-05-25T06:17:43.720 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:43 smithi160 systemd[1]: Started Ceph osd.1 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:17:44.219 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:43 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[128988]: 2024-05-25T06:17:43.772+0000 7fcf755367c0 -1 Falling back to public interface 2024-05-25T06:17:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:43 smithi160 ceph-mon[112094]: pgmap v70: 65 pgs: 34 peering, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:17:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:43 smithi203 ceph-mon[83507]: pgmap v70: 65 pgs: 34 peering, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-05-25T06:17:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:44.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:44 smithi160 ceph-mon[112094]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-05-25T06:17:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:44 smithi160 ceph-mon[112094]: Health check failed: Degraded data redundancy: 50/324 objects degraded (15.432%), 16 pgs degraded (PG_DEGRADED) 2024-05-25T06:17:45.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:44 smithi160 ceph-mon[112094]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 11 pgs peering) 2024-05-25T06:17:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:44 smithi203 ceph-mon[83507]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-05-25T06:17:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:44 smithi203 ceph-mon[83507]: Health check failed: Degraded data redundancy: 50/324 objects degraded (15.432%), 16 pgs degraded (PG_DEGRADED) 2024-05-25T06:17:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:44 smithi203 ceph-mon[83507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 11 pgs peering) 2024-05-25T06:17:48.206 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:47 smithi160 ceph-mon[112094]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-05-25T06:17:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:47 smithi203 ceph-mon[83507]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-05-25T06:17:49.219 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:48 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[128988]: 2024-05-25T06:17:48.810+0000 7fcf755367c0 -1 osd.1 54 log_to_monitors true 2024-05-25T06:17:49.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:49.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:49.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:49 smithi203 ceph-mon[83507]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-25T06:17:49.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:49 smithi203 ceph-mon[83507]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 50/324 objects degraded (15.432%) 2024-05-25T06:17:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:49 smithi160 ceph-mon[112094]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-25T06:17:49.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:49 smithi160 ceph-mon[112094]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 50/324 objects degraded (15.432%) 2024-05-25T06:17:50.681 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:17:50 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[128988]: 2024-05-25T06:17:50.395+0000 7fcf62985700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:17:50.682 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:50 smithi160 ceph-mon[112094]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-25T06:17:50.682 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:50 smithi160 ceph-mon[112094]: osdmap e57: 6 total, 5 up, 6 in 2024-05-25T06:17:50.682 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:50 smithi160 ceph-mon[112094]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:17:50.682 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:50 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:50.682 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:50 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:50.682 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:50 smithi160 ceph-mon[112094]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' 2024-05-25T06:17:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:50 smithi203 ceph-mon[83507]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-25T06:17:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:50 smithi203 ceph-mon[83507]: osdmap e57: 6 total, 5 up, 6 in 2024-05-25T06:17:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:50 smithi203 ceph-mon[83507]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:17:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:50 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:50 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:50 smithi203 ceph-mon[83507]: from='osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119]' entity='osd.1' 2024-05-25T06:17:51.802 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:51 smithi160 ceph-mon[112094]: pgmap v75: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 50/324 objects degraded (15.432%) 2024-05-25T06:17:51.802 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:51 smithi160 ceph-mon[112094]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:17:51.802 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:51 smithi160 ceph-mon[112094]: osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119] boot 2024-05-25T06:17:51.802 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:51 smithi160 ceph-mon[112094]: osdmap e58: 6 total, 6 up, 6 in 2024-05-25T06:17:51.802 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:17:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:51 smithi203 ceph-mon[83507]: pgmap v75: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 50/324 objects degraded (15.432%) 2024-05-25T06:17:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:51 smithi203 ceph-mon[83507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:17:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:51 smithi203 ceph-mon[83507]: osd.1 [v2:172.21.15.160:6810/3648042119,v1:172.21.15.160:6811/3648042119] boot 2024-05-25T06:17:51.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:51 smithi203 ceph-mon[83507]: osdmap e58: 6 total, 6 up, 6 in 2024-05-25T06:17:51.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-25T06:17:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:53 smithi203 ceph-mon[83507]: osdmap e59: 6 total, 6 up, 6 in 2024-05-25T06:17:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:53 smithi203 ceph-mon[83507]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 50/324 objects degraded (15.432%) 2024-05-25T06:17:53.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:53 smithi160 ceph-mon[112094]: osdmap e59: 6 total, 6 up, 6 in 2024-05-25T06:17:53.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:53 smithi160 ceph-mon[112094]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 50/324 objects degraded (15.432%) 2024-05-25T06:17:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:17:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:54 smithi160 ceph-mon[112094]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-05-25T06:17:54.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:54.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:54.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:17:54.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:17:54.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:54.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:17:54.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:54.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:54.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:17:54.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:17:54.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:17:54.902 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:54 smithi203 ceph-mon[83507]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-05-25T06:17:55.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:55 smithi160 ceph-mon[112094]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 op/s 2024-05-25T06:17:55.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:55 smithi203 ceph-mon[83507]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 op/s 2024-05-25T06:17:56.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:56 smithi160 ceph-mon[112094]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/324 objects degraded (15.432%), 16 pgs degraded) 2024-05-25T06:17:56.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:56 smithi160 ceph-mon[112094]: Cluster is now healthy 2024-05-25T06:17:56.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:56 smithi203 ceph-mon[83507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/324 objects degraded (15.432%), 16 pgs degraded) 2024-05-25T06:17:56.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:56 smithi203 ceph-mon[83507]: Cluster is now healthy 2024-05-25T06:17:57.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:57 smithi203 ceph-mon[83507]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-25T06:17:57.934 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:57 smithi160 ceph-mon[112094]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-25T06:17:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:17:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:17:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:00.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:17:59 smithi160 ceph-mon[112094]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-05-25T06:18:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:17:59 smithi203 ceph-mon[83507]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-05-25T06:18:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:01 smithi160 ceph-mon[112094]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-05-25T06:18:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:01 smithi203 ceph-mon[83507]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-05-25T06:18:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:03 smithi160 ceph-mon[112094]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-05-25T06:18:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:03 smithi203 ceph-mon[83507]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-05-25T06:18:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:05 smithi160 ceph-mon[112094]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-25T06:18:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:05 smithi203 ceph-mon[83507]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-25T06:18:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:07 smithi160 ceph-mon[112094]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:07 smithi203 ceph-mon[83507]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:09.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:08 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:18:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:08 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:18:10.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:09 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:18:10.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:09 smithi160 ceph-mon[112094]: Upgrade: osd.2 is safe to restart 2024-05-25T06:18:10.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:09 smithi160 ceph-mon[112094]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:18:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:09 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-25T06:18:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:09 smithi203 ceph-mon[83507]: Upgrade: osd.2 is safe to restart 2024-05-25T06:18:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:09 smithi203 ceph-mon[83507]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:18:10.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 -- 172.21.15.160:0/1320299281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060102680 msgr2=0x7fd0600fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:10.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 --2- 172.21.15.160:0/1320299281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060102680 0x7fd0600fbcf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd050004940 tx=0x7fd050009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:10.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 -- 172.21.15.160:0/1320299281 shutdown_connections 2024-05-25T06:18:10.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 --2- 172.21.15.160:0/1320299281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060102680 0x7fd0600fbcf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 --2- 172.21.15.160:0/1320299281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd060101ce0 0x7fd0601020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 -- 172.21.15.160:0/1320299281 >> 172.21.15.160:0/1320299281 conn(0x7fd0600f7890 msgr2=0x7fd0600f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:10.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 -- 172.21.15.160:0/1320299281 shutdown_connections 2024-05-25T06:18:10.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.661+0000 7fd065f30700 1 -- 172.21.15.160:0/1320299281 wait complete. 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.662+0000 7fd065f30700 1 Processor -- start 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.662+0000 7fd065f30700 1 -- start start 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.662+0000 7fd065f30700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 0x7fd060113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.662+0000 7fd065f30700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd060102680 0x7fd0601140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd065f30700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd060114710 con 0x7fd060101ce0 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd065f30700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06010db00 con 0x7fd060102680 2024-05-25T06:18:10.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 0x7fd060113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:10.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 0x7fd060113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55282/0 (socket says 172.21.15.160:55282) 2024-05-25T06:18:10.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 -- 172.21.15.160:0/1698673579 learned_addr learned my addr 172.21.15.160:0/1698673579 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:10.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05effd700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd060102680 0x7fd0601140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:10.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 -- 172.21.15.160:0/1698673579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd060102680 msgr2=0x7fd0601140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:10.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd060102680 0x7fd0601140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 -- 172.21.15.160:0/1698673579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd050005040 con 0x7fd060101ce0 2024-05-25T06:18:10.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd05f7fe700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 0x7fd060113b80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd048002a50 tx=0x7fd0480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:10.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd048007cc0 con 0x7fd060101ce0 2024-05-25T06:18:10.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06010dde0 con 0x7fd060101ce0 2024-05-25T06:18:10.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd04800f040 con 0x7fd060101ce0 2024-05-25T06:18:10.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd048009640 con 0x7fd060101ce0 2024-05-25T06:18:10.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.663+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06010e3a0 con 0x7fd060101ce0 2024-05-25T06:18:10.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.665+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0600fd430 con 0x7fd060101ce0 2024-05-25T06:18:10.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.665+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd0480097c0 con 0x7fd060101ce0 2024-05-25T06:18:10.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.665+0000 7fd057fff700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd04c06e140 0x7fd04c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:10.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.665+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd04808d870 con 0x7fd060101ce0 2024-05-25T06:18:10.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.666+0000 7fd05effd700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd04c06e140 0x7fd04c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:10.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.666+0000 7fd05effd700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd04c06e140 0x7fd04c0705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd06010ecc0 tx=0x7fd050009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:10.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.675+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd04805a770 con 0x7fd060101ce0 2024-05-25T06:18:10.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.906+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd060108dc0 con 0x7fd04c06e140 2024-05-25T06:18:10.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.907+0000 7fd057fff700 1 -- 172.21.15.160:0/1698673579 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd060108dc0 con 0x7fd04c06e140 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd04c06e140 msgr2=0x7fd04c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd04c06e140 0x7fd04c0705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd06010ecc0 tx=0x7fd050009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 msgr2=0x7fd060113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 0x7fd060113b80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd048002a50 tx=0x7fd0480046b0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 shutdown_connections 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd060101ce0 0x7fd060113b80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd04c06e140 0x7fd04c0705f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 --2- 172.21.15.160:0/1698673579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd060102680 0x7fd0601140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:10.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 >> 172.21.15.160:0/1698673579 conn(0x7fd0600f7890 msgr2=0x7fd060065ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:10.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 shutdown_connections 2024-05-25T06:18:10.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:10.910+0000 7fd065f30700 1 -- 172.21.15.160:0/1698673579 wait complete. 2024-05-25T06:18:10.923 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 -- 172.21.15.160:0/2121782068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba4101d00 msgr2=0x7f4ba41020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 --2- 172.21.15.160:0/2121782068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba4101d00 0x7f4ba41020d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b94004940 tx=0x7f4b94009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 -- 172.21.15.160:0/2121782068 shutdown_connections 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 --2- 172.21.15.160:0/2121782068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ba41026a0 0x7f4ba40fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 --2- 172.21.15.160:0/2121782068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba4101d00 0x7f4ba41020d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 -- 172.21.15.160:0/2121782068 >> 172.21.15.160:0/2121782068 conn(0x7f4ba40f7850 msgr2=0x7f4ba40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 -- 172.21.15.160:0/2121782068 shutdown_connections 2024-05-25T06:18:11.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 -- 172.21.15.160:0/2121782068 wait complete. 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.017+0000 7f4ba99da700 1 Processor -- start 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba99da700 1 -- start start 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba99da700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ba4101d00 0x7f4ba4112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba99da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 0x7f4ba4112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba99da700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4113420 con 0x7f4ba41026a0 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba99da700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba410ca20 con 0x7f4ba4101d00 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 0x7f4ba4112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 0x7f4ba4112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55304/0 (socket says 172.21.15.160:55304) 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 -- 172.21.15.160:0/63713087 learned_addr learned my addr 172.21.15.160:0/63713087 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba2ffd700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ba4101d00 0x7f4ba4112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 -- 172.21.15.160:0/63713087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ba4101d00 msgr2=0x7f4ba4112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ba4101d00 0x7f4ba4112890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 -- 172.21.15.160:0/63713087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94005040 con 0x7f4ba41026a0 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba27fc700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 0x7f4ba4112dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b98002a50 tx=0x7f4b98004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b98007cc0 con 0x7f4ba41026a0 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba410cd00 con 0x7f4ba41026a0 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b9800f040 con 0x7f4ba41026a0 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.018+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b9800da40 con 0x7f4ba41026a0 2024-05-25T06:18:11.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.019+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba410d2c0 con 0x7f4ba41026a0 2024-05-25T06:18:11.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.019+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba40fd450 con 0x7f4ba41026a0 2024-05-25T06:18:11.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.020+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4b980097c0 con 0x7f4ba41026a0 2024-05-25T06:18:11.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.020+0000 7f4b8b7fe700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4b8c06e140 0x7f4b8c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.021+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4b9808d900 con 0x7f4ba41026a0 2024-05-25T06:18:11.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.021+0000 7f4ba2ffd700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4b8c06e140 0x7f4b8c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.021+0000 7f4ba2ffd700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4b8c06e140 0x7f4b8c0705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b94004f90 tx=0x7f4b94009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:11.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.025+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4b9805a800 con 0x7f4ba41026a0 2024-05-25T06:18:11.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.257+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ba4112560 con 0x7f4b8c06e140 2024-05-25T06:18:11.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.258+0000 7f4b8b7fe700 1 -- 172.21.15.160:0/63713087 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4ba4112560 con 0x7f4b8c06e140 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4b8c06e140 msgr2=0x7f4b8c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4b8c06e140 0x7f4b8c0705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b94004f90 tx=0x7f4b94009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 msgr2=0x7f4ba4112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 0x7f4ba4112dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b98002a50 tx=0x7f4b98004600 comp rx=0 tx=0).stop 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 shutdown_connections 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4b8c06e140 0x7f4b8c0705f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ba4101d00 0x7f4ba4112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 --2- 172.21.15.160:0/63713087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ba41026a0 0x7f4ba4112dd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 >> 172.21.15.160:0/63713087 conn(0x7f4ba40f7850 msgr2=0x7f4ba40f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:11.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 shutdown_connections 2024-05-25T06:18:11.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.261+0000 7f4ba99da700 1 -- 172.21.15.160:0/63713087 wait complete. 2024-05-25T06:18:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 -- 172.21.15.160:0/81901727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c0702a0 msgr2=0x7f761c070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 --2- 172.21.15.160:0/81901727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c0702a0 0x7f761c070670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7618004970 tx=0x7f7618009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 -- 172.21.15.160:0/81901727 shutdown_connections 2024-05-25T06:18:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 --2- 172.21.15.160:0/81901727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c070bb0 0x7f761c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 --2- 172.21.15.160:0/81901727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c0702a0 0x7f761c070670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 -- 172.21.15.160:0/81901727 >> 172.21.15.160:0/81901727 conn(0x7f761c0f8990 msgr2=0x7f761c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 -- 172.21.15.160:0/81901727 shutdown_connections 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.366+0000 7f7622f94700 1 -- 172.21.15.160:0/81901727 wait complete. 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7622f94700 1 Processor -- start 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7622f94700 1 -- start start 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7622f94700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c0702a0 0x7f761c115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7622f94700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 0x7f761c116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7622f94700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1166a0 con 0x7f761c070bb0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7622f94700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c10fc80 con 0x7f761c0702a0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7613fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 0x7f761c116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7613fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 0x7f761c116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55320/0 (socket says 172.21.15.160:55320) 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7613fff700 1 -- 172.21.15.160:0/562880023 learned_addr learned my addr 172.21.15.160:0/562880023 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.367+0000 7f7620d30700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c0702a0 0x7f761c115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f7613fff700 1 -- 172.21.15.160:0/562880023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c0702a0 msgr2=0x7f761c115b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f7613fff700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c0702a0 0x7f761c115b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f7613fff700 1 -- 172.21.15.160:0/562880023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618005040 con 0x7f761c070bb0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f7613fff700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 0x7f761c116050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c830 tx=0x7f760c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f760c0106f0 con 0x7f761c070bb0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f760c010d20 con 0x7f761c070bb0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c10ff60 con 0x7f761c070bb0 2024-05-25T06:18:11.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f760c0113a0 con 0x7f761c070bb0 2024-05-25T06:18:11.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.368+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c1104a0 con 0x7f761c070bb0 2024-05-25T06:18:11.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.369+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c0719e0 con 0x7f761c070bb0 2024-05-25T06:18:11.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.370+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f760c012400 con 0x7f761c070bb0 2024-05-25T06:18:11.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.370+0000 7f76117fa700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f760406e140 0x7f76040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.370+0000 7f7620d30700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f760406e140 0x7f76040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.370+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f760c08da20 con 0x7f761c070bb0 2024-05-25T06:18:11.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.371+0000 7f7620d30700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f760406e140 0x7f76040705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7618004f50 tx=0x7f7618009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:11.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.375+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f760c05a9a0 con 0x7f761c070bb0 2024-05-25T06:18:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: Upgrade: Updating osd.2 2024-05-25T06:18:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-25T06:18:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:18:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: Deploying daemon osd.2 on smithi160 2024-05-25T06:18:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:11.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:11 smithi160 ceph-mon[112094]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:11.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.592+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f761c10c140 con 0x7f760406e140 2024-05-25T06:18:11.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.602+0000 7f76117fa700 1 -- 172.21.15.160:0/562880023 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f761c10c140 con 0x7f760406e140 2024-05-25T06:18:11.602 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:18:11.602 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (4m) 23s ago 15m 26.1M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (91s) 23s ago 15m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (85s) 82s ago 14m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (3m) 23s ago 14m 78.6M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (9m) 23s ago 9m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (9m) 23s ago 9m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (9m) 82s ago 9m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (9m) 82s ago 9m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (5m) 23s ago 16m 504M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (4m) 82s ago 14m 451M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (2m) 23s ago 16m 45.2M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (2m) 82s ago 14m 34.8M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (4m) 23s ago 15m 15.1M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:18:11.603 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (4m) 82s ago 14m 17.4M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (64s) 23s ago 13m 95.6M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (28s) 23s ago 12m 12.6M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (12m) 23s ago 12m 192M 1557M 17.2.7-958-g7b01852c baf1547092e7 13ee85a1fa9d 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (12m) 82s ago 12m 269M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (11m) 82s ago 11m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (11m) 82s ago 11m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:18:11.604 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (4m) 23s ago 14m 67.4M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f760406e140 msgr2=0x7f76040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f760406e140 0x7f76040705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7618004f50 tx=0x7f7618009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 msgr2=0x7f761c116050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 0x7f761c116050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c830 tx=0x7f760c004470 comp rx=0 tx=0).stop 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 shutdown_connections 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f760406e140 0x7f76040705f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c0702a0 0x7f761c115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 --2- 172.21.15.160:0/562880023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f761c070bb0 0x7f761c116050 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 >> 172.21.15.160:0/562880023 conn(0x7f761c0f8990 msgr2=0x7f761c0f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 shutdown_connections 2024-05-25T06:18:11.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.604+0000 7f7622f94700 1 -- 172.21.15.160:0/562880023 wait complete. 2024-05-25T06:18:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: Upgrade: Updating osd.2 2024-05-25T06:18:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-25T06:18:11.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:18:11.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: Deploying daemon osd.2 on smithi160 2024-05-25T06:18:11.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:11.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:11 smithi203 ceph-mon[83507]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:11.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.707+0000 7f7b24550700 1 -- 172.21.15.160:0/2183205968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c101e20 msgr2=0x7f7b1c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.707+0000 7f7b24550700 1 --2- 172.21.15.160:0/2183205968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c101e20 0x7f7b1c1021f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b18004940 tx=0x7f7b18009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.707+0000 7f7b24550700 1 -- 172.21.15.160:0/2183205968 shutdown_connections 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.707+0000 7f7b24550700 1 --2- 172.21.15.160:0/2183205968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c0fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.707+0000 7f7b24550700 1 --2- 172.21.15.160:0/2183205968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c101e20 0x7f7b1c1021f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.707+0000 7f7b24550700 1 -- 172.21.15.160:0/2183205968 >> 172.21.15.160:0/2183205968 conn(0x7f7b1c071460 msgr2=0x7f7b1c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 -- 172.21.15.160:0/2183205968 shutdown_connections 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 -- 172.21.15.160:0/2183205968 wait complete. 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 Processor -- start 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 -- start start 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c10ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c101e20 0x7f7b1c10f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c10f6e0 con 0x7f7b1c0fc710 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b24550700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c10f820 con 0x7f7b1c101e20 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b222ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c10ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b222ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c10ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55342/0 (socket says 172.21.15.160:55342) 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.708+0000 7f7b222ec700 1 -- 172.21.15.160:0/2770865711 learned_addr learned my addr 172.21.15.160:0/2770865711 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b222ec700 1 -- 172.21.15.160:0/2770865711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c101e20 msgr2=0x7f7b1c10f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b21aeb700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c101e20 0x7f7b1c10f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b222ec700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c101e20 0x7f7b1c10f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b222ec700 1 -- 172.21.15.160:0/2770865711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18005040 con 0x7f7b1c0fc710 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b21aeb700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c101e20 0x7f7b1c10f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b222ec700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c10ec60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b3b0 tx=0x7f7b18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1800e040 con 0x7f7b1c0fc710 2024-05-25T06:18:11.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b1800bbc0 con 0x7f7b1c0fc710 2024-05-25T06:18:11.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c117b10 con 0x7f7b1c0fc710 2024-05-25T06:18:11.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1801c3f0 con 0x7f7b1c0fc710 2024-05-25T06:18:11.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.709+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c1180a0 con 0x7f7b1c0fc710 2024-05-25T06:18:11.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.710+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c104260 con 0x7f7b1c0fc710 2024-05-25T06:18:11.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.711+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7b1800bd30 con 0x7f7b1c0fc710 2024-05-25T06:18:11.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.711+0000 7f7b12ffd700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b0806e140 0x7f7b080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:11.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.711+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7b1808ed20 con 0x7f7b1c0fc710 2024-05-25T06:18:11.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.711+0000 7f7b21aeb700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b0806e140 0x7f7b080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:11.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.711+0000 7f7b21aeb700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b0806e140 0x7f7b080705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009fd0 tx=0x7f7b0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:11.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:11.716+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7b180580f0 con 0x7f7b1c0fc710 2024-05-25T06:18:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.002+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b1c0652f0 con 0x7f7b1c0fc710 2024-05-25T06:18:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.003+0000 7f7b12ffd700 1 -- 172.21.15.160:0/2770865711 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f7b1805b7b0 con 0x7f7b1c0fc710 2024-05-25T06:18:12.003 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:18:12.003 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:18:12.003 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4, 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 8, 2024-05-25T06:18:12.004 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:18:12.005 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:18:12.005 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:18:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b0806e140 msgr2=0x7f7b080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b0806e140 0x7f7b080705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009fd0 tx=0x7f7b0c00b040 comp rx=0 tx=0).stop 2024-05-25T06:18:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 msgr2=0x7f7b1c10ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c10ec60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b3b0 tx=0x7f7b18009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 shutdown_connections 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b1c0fc710 0x7f7b1c10ec60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b0806e140 0x7f7b080705f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 --2- 172.21.15.160:0/2770865711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b1c101e20 0x7f7b1c10f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 >> 172.21.15.160:0/2770865711 conn(0x7f7b1c071460 msgr2=0x7f7b1c0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 shutdown_connections 2024-05-25T06:18:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.005+0000 7f7b24550700 1 -- 172.21.15.160:0/2770865711 wait complete. 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 -- 172.21.15.160:0/1773086704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 msgr2=0x7f1830070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 --2- 172.21.15.160:0/1773086704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830070cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1820004970 tx=0x7f1820009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 -- 172.21.15.160:0/1773086704 shutdown_connections 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 --2- 172.21.15.160:0/1773086704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830070cb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 --2- 172.21.15.160:0/1773086704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1830071f30 0x7f1830070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 -- 172.21.15.160:0/1773086704 >> 172.21.15.160:0/1773086704 conn(0x7f18300f8990 msgr2=0x7f18300fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 -- 172.21.15.160:0/1773086704 shutdown_connections 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 -- 172.21.15.160:0/1773086704 wait complete. 2024-05-25T06:18:12.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.115+0000 7f1837da5700 1 Processor -- start 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1837da5700 1 -- start start 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1837da5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1837da5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1830071f30 0x7f183011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1837da5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183011aba0 con 0x7f1830070840 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1837da5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18301140f0 con 0x7f1830071f30 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55368/0 (socket says 172.21.15.160:55368) 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 -- 172.21.15.160:0/4222838176 learned_addr learned my addr 172.21.15.160:0/4222838176 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835340700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1830071f30 0x7f183011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 -- 172.21.15.160:0/4222838176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1830071f30 msgr2=0x7f183011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1830071f30 0x7f183011a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 -- 172.21.15.160:0/4222838176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820005040 con 0x7f1830070840 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.116+0000 7f1835b41700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830119f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b790 tx=0x7f182c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:12.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.117+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f182c00f6f0 con 0x7f1830070840 2024-05-25T06:18:12.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.117+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830114370 con 0x7f1830070840 2024-05-25T06:18:12.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.117+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1830114880 con 0x7f1830070840 2024-05-25T06:18:12.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.117+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f182c00fd20 con 0x7f1830070840 2024-05-25T06:18:12.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.117+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f182c0103a0 con 0x7f1830070840 2024-05-25T06:18:12.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.118+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18140052b0 con 0x7f1830070840 2024-05-25T06:18:12.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.118+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f182c011400 con 0x7f1830070840 2024-05-25T06:18:12.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.119+0000 7f18267fc700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f181c06e140 0x7f181c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.119+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f182c08d890 con 0x7f1830070840 2024-05-25T06:18:12.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.119+0000 7f1835340700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f181c06e140 0x7f181c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.119+0000 7f1835340700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f181c06e140 0x7f181c0705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1820004f50 tx=0x7f1820009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:12.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.123+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f182c05a810 con 0x7f1830070840 2024-05-25T06:18:12.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.372+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1814005d70 con 0x7f1830070840 2024-05-25T06:18:12.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.373+0000 7f18267fc700 1 -- 172.21.15.160:0/4222838176 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7f182c05a3a0 con 0x7f1830070840 2024-05-25T06:18:12.373 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:18:12.373 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:18:12.373 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:18:12.373 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:18:12.374 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:18:12.375 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:18:12.376 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:12.377 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:12.377 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:12.377 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:12.377 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:18:12.377 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f181c06e140 msgr2=0x7f181c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f181c06e140 0x7f181c0705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1820004f50 tx=0x7f1820009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 msgr2=0x7f1830119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830119f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b790 tx=0x7f182c009250 comp rx=0 tx=0).stop 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 shutdown_connections 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830070840 0x7f1830119f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f181c06e140 0x7f181c0705f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 --2- 172.21.15.160:0/4222838176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1830071f30 0x7f183011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 >> 172.21.15.160:0/4222838176 conn(0x7f18300f8990 msgr2=0x7f18300f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:12.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 shutdown_connections 2024-05-25T06:18:12.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.375+0000 7f1837da5700 1 -- 172.21.15.160:0/4222838176 wait complete. 2024-05-25T06:18:12.379 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:18:12.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 -- 172.21.15.160:0/2235230725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8102670 msgr2=0x7fdaa80fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/2235230725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8102670 0x7fdaa80fb600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda90004940 tx=0x7fda90009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:12.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 -- 172.21.15.160:0/2235230725 shutdown_connections 2024-05-25T06:18:12.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/2235230725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8102670 0x7fdaa80fb600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/2235230725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa8101cd0 0x7fdaa81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 -- 172.21.15.160:0/2235230725 >> 172.21.15.160:0/2235230725 conn(0x7fdaa8071460 msgr2=0x7fdaa8071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 -- 172.21.15.160:0/2235230725 shutdown_connections 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.483+0000 7fdaacbd2700 1 -- 172.21.15.160:0/2235230725 wait complete. 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaacbd2700 1 Processor -- start 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaacbd2700 1 -- start start 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaacbd2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 0x7fdaa8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaacbd2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa8102670 0x7fdaa81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaacbd2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8114740 con 0x7fdaa8101cd0 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaacbd2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa810db30 con 0x7fdaa8102670 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 0x7fdaa8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 0x7fdaa8113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55398/0 (socket says 172.21.15.160:55398) 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaa659c700 1 -- 172.21.15.160:0/1310460624 learned_addr learned my addr 172.21.15.160:0/1310460624 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:12.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaa5d9b700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa8102670 0x7fdaa81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaa659c700 1 -- 172.21.15.160:0/1310460624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa8102670 msgr2=0x7fdaa81140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.484+0000 7fdaa659c700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa8102670 0x7fdaa81140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fdaa659c700 1 -- 172.21.15.160:0/1310460624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90005040 con 0x7fdaa8101cd0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fdaa659c700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 0x7fdaa8113bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0001d80 tx=0x7fdaa0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa0007d10 con 0x7fdaa8101cd0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa810de10 con 0x7fdaa8101cd0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaa0001e80 con 0x7fdaa8101cd0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa0014640 con 0x7fdaa8101cd0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.485+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa810e3a0 con 0x7fdaa8101cd0 2024-05-25T06:18:12.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.486+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa8065200 con 0x7fdaa8101cd0 2024-05-25T06:18:12.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.487+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdaa000e960 con 0x7fdaa8101cd0 2024-05-25T06:18:12.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.488+0000 7fda9effd700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fda9406e150 0x7fda94070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.488+0000 7fdaa5d9b700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fda9406e150 0x7fda94070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.488+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdaa008d800 con 0x7fdaa8101cd0 2024-05-25T06:18:12.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.488+0000 7fdaa5d9b700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fda9406e150 0x7fda94070600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b3b0 tx=0x7fda90009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:12.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.492+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdaa005a7b0 con 0x7fdaa8101cd0 2024-05-25T06:18:12.565 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:12 smithi160 ceph-mon[112094]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:12.565 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:12 smithi160 ceph-mon[112094]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:12.565 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:12 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2770865711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:12 smithi203 ceph-mon[83507]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:12 smithi203 ceph-mon[83507]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:12 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2770865711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:12.718 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.718+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdaa810dfa0 con 0x7fda9406e150 2024-05-25T06:18:12.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.719+0000 7fda9effd700 1 -- 172.21.15.160:0/1310460624 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdaa810dfa0 con 0x7fda9406e150 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "8/21 daemons upgraded", 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading osd daemons", 2024-05-25T06:18:12.720 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:18:12.721 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:18:12.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fda9406e150 msgr2=0x7fda94070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fda9406e150 0x7fda94070600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b3b0 tx=0x7fda90009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:12.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 msgr2=0x7fdaa8113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 0x7fdaa8113bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0001d80 tx=0x7fdaa0009ec0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 shutdown_connections 2024-05-25T06:18:12.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaa8101cd0 0x7fdaa8113bb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fda9406e150 0x7fda94070600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 --2- 172.21.15.160:0/1310460624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa8102670 0x7fdaa81140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 >> 172.21.15.160:0/1310460624 conn(0x7fdaa8071460 msgr2=0x7fdaa81009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:12.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 shutdown_connections 2024-05-25T06:18:12.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.722+0000 7fdaacbd2700 1 -- 172.21.15.160:0/1310460624 wait complete. 2024-05-25T06:18:12.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 -- 172.21.15.160:0/2200070322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 msgr2=0x7f341c0fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 --2- 172.21.15.160:0/2200070322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fe1d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f340c004940 tx=0x7f340c009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:12.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 -- 172.21.15.160:0/2200070322 shutdown_connections 2024-05-25T06:18:12.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 --2- 172.21.15.160:0/2200070322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 0x7f341c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 --2- 172.21.15.160:0/2200070322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fe1d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 -- 172.21.15.160:0/2200070322 >> 172.21.15.160:0/2200070322 conn(0x7f341c0f7870 msgr2=0x7f341c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 -- 172.21.15.160:0/2200070322 shutdown_connections 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.828+0000 7f3421735700 1 -- 172.21.15.160:0/2200070322 wait complete. 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.829+0000 7f3421735700 1 Processor -- start 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.829+0000 7f3421735700 1 -- start start 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.829+0000 7f3421735700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.829+0000 7f3421735700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 0x7f341c0fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.829+0000 7f3421735700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0fdc10 con 0x7f341c0fde00 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.829+0000 7f3421735700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0fc6b0 con 0x7f341c0fe7a0 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55416/0 (socket says 172.21.15.160:55416) 2024-05-25T06:18:12.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 -- 172.21.15.160:0/2137454589 learned_addr learned my addr 172.21.15.160:0/2137454589 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:12.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341a7fc700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 0x7f341c0fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 -- 172.21.15.160:0/2137454589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 msgr2=0x7f341c0fc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:12.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 0x7f341c0fc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:12.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 -- 172.21.15.160:0/2137454589 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c005040 con 0x7f341c0fde00 2024-05-25T06:18:12.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341a7fc700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 0x7f341c0fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:18:12.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f341affd700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fbc30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b3b0 tx=0x7f340c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:12.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f340c00e040 con 0x7f341c0fde00 2024-05-25T06:18:12.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f340c013420 con 0x7f341c0fde00 2024-05-25T06:18:12.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c0fc930 con 0x7f341c0fde00 2024-05-25T06:18:12.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f340c01c630 con 0x7f341c0fde00 2024-05-25T06:18:12.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.830+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c11a0f0 con 0x7f341c0fde00 2024-05-25T06:18:12.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.831+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c066b30 con 0x7f341c0fde00 2024-05-25T06:18:12.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.832+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f340c00baf0 con 0x7f341c0fde00 2024-05-25T06:18:12.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.832+0000 7f34037fe700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340406e140 0x7f34040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:12.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.832+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f340c08ecb0 con 0x7f341c0fde00 2024-05-25T06:18:12.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.832+0000 7f341a7fc700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340406e140 0x7f34040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:12.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.833+0000 7f341a7fc700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340406e140 0x7f34040705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f341c0fd360 tx=0x7f3410008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:12.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:12.837+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f340c0581a0 con 0x7f341c0fde00 2024-05-25T06:18:13.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.123+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f341c04c710 con 0x7f341c0fde00 2024-05-25T06:18:13.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.123+0000 7f34037fe700 1 -- 172.21.15.160:0/2137454589 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f340c05b860 con 0x7f341c0fde00 2024-05-25T06:18:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340406e140 msgr2=0x7f34040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340406e140 0x7f34040705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f341c0fd360 tx=0x7f3410008040 comp rx=0 tx=0).stop 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 msgr2=0x7f341c0fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fbc30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b3b0 tx=0x7f340c009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 shutdown_connections 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f341c0fde00 0x7f341c0fbc30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340406e140 0x7f34040705f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 --2- 172.21.15.160:0/2137454589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341c0fe7a0 0x7f341c0fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 >> 172.21.15.160:0/2137454589 conn(0x7f341c0f7870 msgr2=0x7f341c103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:13.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 shutdown_connections 2024-05-25T06:18:13.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:13.126+0000 7f3421735700 1 -- 172.21.15.160:0/2137454589 wait complete. 2024-05-25T06:18:13.411 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4222838176' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:18:13.411 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:13 smithi160 ceph-mon[112094]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:13.411 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:13 smithi160 ceph-mon[112094]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:13.412 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:13.412 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2137454589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:18:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4222838176' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:18:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:13 smithi203 ceph-mon[83507]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:13 smithi203 ceph-mon[83507]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2137454589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:18:14.116 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:13 smithi160 systemd[1]: Stopping Ceph osd.2 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:18:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:14 smithi160 ceph-mon[112094]: osd.2 marked itself down and dead 2024-05-25T06:18:14.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:14 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[61828]: 2024-05-25T06:18:14.115+0000 7fbedc64c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:18:14.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:14 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[61828]: 2024-05-25T06:18:14.116+0000 7fbedc64c700 -1 osd.2 59 *** Got signal Terminated *** 2024-05-25T06:18:14.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:14 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[61828]: 2024-05-25T06:18:14.116+0000 7fbedc64c700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:18:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:14 smithi203 ceph-mon[83507]: osd.2 marked itself down and dead 2024-05-25T06:18:14.965 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:14 smithi160 podman[134133]: 2024-05-25 06:18:14.687396348 +0000 UTC m=+0.694881400 container died 13ee85a1fa9d00a77cfa37956df9149e7ccc322e2d59c9ae4fec64a9c0b3451c (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, RELEASE=quincy-7b01852, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-25T06:18:14.965 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:14 smithi160 podman[134133]: 2024-05-25 06:18:14.965654826 +0000 UTC m=+0.973139871 container cleanup 13ee85a1fa9d00a77cfa37956df9149e7ccc322e2d59c9ae4fec64a9c0b3451c (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-7b01852, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-25T06:18:15.219 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:14 smithi160 bash[134133]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2 2024-05-25T06:18:15.219 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:15 smithi160 podman[134148]: 2024-05-25 06:18:15.19201963 +0000 UTC m=+0.499495222 container remove 13ee85a1fa9d00a77cfa37956df9149e7ccc322e2d59c9ae4fec64a9c0b3451c (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=quincy-7b01852, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:18:15.644 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:15 smithi160 ceph-mon[112094]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:18:15.644 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:15 smithi160 ceph-mon[112094]: osdmap e60: 6 total, 5 up, 6 in 2024-05-25T06:18:15.644 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:15 smithi160 ceph-mon[112094]: pgmap v90: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-05-25T06:18:15.645 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:15 smithi160 podman[134236]: 2024-05-25 06:18:15.427653601 +0000 UTC m=+0.017823130 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:18:15.645 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:15 smithi160 podman[134236]: 2024-05-25 06:18:15.526227475 +0000 UTC m=+0.116397042 container create f437ab2193481a56d05d8959913ef3015e1634ac6c8f5a09fee5014db3b12d6f (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-25T06:18:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:15 smithi203 ceph-mon[83507]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:18:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:15 smithi203 ceph-mon[83507]: osdmap e60: 6 total, 5 up, 6 in 2024-05-25T06:18:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:15 smithi203 ceph-mon[83507]: pgmap v90: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-05-25T06:18:15.969 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:15 smithi160 podman[134236]: 2024-05-25 06:18:15.72951435 +0000 UTC m=+0.319683879 container init f437ab2193481a56d05d8959913ef3015e1634ac6c8f5a09fee5014db3b12d6f (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-ea0684c, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:18:15.970 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:15 smithi160 podman[134236]: 2024-05-25 06:18:15.733092551 +0000 UTC m=+0.323262084 container start f437ab2193481a56d05d8959913ef3015e1634ac6c8f5a09fee5014db3b12d6f (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:18:15.970 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:15 smithi160 podman[134236]: 2024-05-25 06:18:15.803689806 +0000 UTC m=+0.393859360 container attach f437ab2193481a56d05d8959913ef3015e1634ac6c8f5a09fee5014db3b12d6f (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:16.314 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:16 smithi160 podman[134236]: 2024-05-25 06:18:16.044484292 +0000 UTC m=+0.634653818 container died f437ab2193481a56d05d8959913ef3015e1634ac6c8f5a09fee5014db3b12d6f (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:16 smithi203 ceph-mon[83507]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-25T06:18:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:16 smithi203 ceph-mon[83507]: osdmap e61: 6 total, 5 up, 6 in 2024-05-25T06:18:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:16 smithi160 ceph-mon[112094]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-25T06:18:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:16 smithi160 ceph-mon[112094]: osdmap e61: 6 total, 5 up, 6 in 2024-05-25T06:18:17.175 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:16 smithi160 podman[134262]: 2024-05-25 06:18:16.724076786 +0000 UTC m=+0.670354904 container remove f437ab2193481a56d05d8959913ef3015e1634ac6c8f5a09fee5014db3b12d6f (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=) 2024-05-25T06:18:17.175 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:16 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-25T06:18:17.175 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:16 smithi160 systemd[1]: Stopped Ceph osd.2 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:18:17.175 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:16 smithi160 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.2.service: Consumed 55.444s CPU time. 2024-05-25T06:18:17.176 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:16 smithi160 systemd[1]: Starting Ceph osd.2 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:18:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:17 smithi160 ceph-mon[112094]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-25T06:18:17.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 podman[134360]: 2024-05-25 06:18:17.175367169 +0000 UTC m=+0.113043902 container create 43605df216ac413aa90db8d8cc2d36ea54ed508d607b34931f54c64ae3f591b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c) 2024-05-25T06:18:17.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 podman[134360]: 2024-05-25 06:18:17.079844168 +0000 UTC m=+0.017520898 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:18:17.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 podman[134360]: 2024-05-25 06:18:17.411493824 +0000 UTC m=+0.349170528 container init 43605df216ac413aa90db8d8cc2d36ea54ed508d607b34931f54c64ae3f591b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-25T06:18:17.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 podman[134360]: 2024-05-25 06:18:17.415018809 +0000 UTC m=+0.352695522 container start 43605df216ac413aa90db8d8cc2d36ea54ed508d607b34931f54c64ae3f591b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-25T06:18:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:17 smithi203 ceph-mon[83507]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-25T06:18:17.866 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 podman[134360]: 2024-05-25 06:18:17.505575959 +0000 UTC m=+0.443252683 container attach 43605df216ac413aa90db8d8cc2d36ea54ed508d607b34931f54c64ae3f591b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c) 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-25T06:18:18.220 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 bash[134360]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-25T06:18:18.221 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-25T06:18:18.221 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-25T06:18:18.221 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-25T06:18:18.221 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-25T06:18:18.221 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate[134375]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-25T06:18:18.221 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:17 smithi160 podman[134360]: 2024-05-25 06:18:17.89285428 +0000 UTC m=+0.830530994 container died 43605df216ac413aa90db8d8cc2d36ea54ed508d607b34931f54c64ae3f591b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-25T06:18:18.711 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:18 smithi160 podman[134503]: 2024-05-25 06:18:18.498731937 +0000 UTC m=+0.595000090 container remove 43605df216ac413aa90db8d8cc2d36ea54ed508d607b34931f54c64ae3f591b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-ea0684c) 2024-05-25T06:18:18.969 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:18 smithi160 podman[134563]: 2024-05-25 06:18:18.614514796 +0000 UTC m=+0.017466023 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:18:18.969 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:18 smithi160 podman[134563]: 2024-05-25 06:18:18.725645747 +0000 UTC m=+0.128596960 container create 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:18 smithi203 ceph-mon[83507]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 27/324 objects degraded (8.333%) 2024-05-25T06:18:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:18 smithi203 ceph-mon[83507]: Health check failed: Degraded data redundancy: 27/324 objects degraded (8.333%), 9 pgs degraded (PG_DEGRADED) 2024-05-25T06:18:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:18 smithi160 ceph-mon[112094]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 27/324 objects degraded (8.333%) 2024-05-25T06:18:19.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:18 smithi160 ceph-mon[112094]: Health check failed: Degraded data redundancy: 27/324 objects degraded (8.333%), 9 pgs degraded (PG_DEGRADED) 2024-05-25T06:18:19.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:19 smithi160 podman[134563]: 2024-05-25 06:18:19.085585647 +0000 UTC m=+0.488536872 container init 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-ea0684c) 2024-05-25T06:18:19.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:19 smithi160 podman[134563]: 2024-05-25 06:18:19.088816124 +0000 UTC m=+0.491767342 container start 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-25T06:18:19.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:19 smithi160 ceph-osd[134583]: -- 172.21.15.160:0/3853784385 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557204481680 con 0x55720444b000 2024-05-25T06:18:19.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:19 smithi160 bash[134563]: 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc 2024-05-25T06:18:19.470 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:19 smithi160 systemd[1]: Started Ceph osd.2 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:18:19.969 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:19 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[134579]: 2024-05-25T06:18:19.631+0000 7f1a3d2287c0 -1 Falling back to public interface 2024-05-25T06:18:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:20 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:20 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:20 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:18:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:20 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:20 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:20 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:18:21.501 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:21 smithi160 ceph-mon[112094]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-05-25T06:18:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:21 smithi203 ceph-mon[83507]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-05-25T06:18:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:22 smithi203 ceph-mon[83507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-25T06:18:22.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:22 smithi160 ceph-mon[112094]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-25T06:18:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:23 smithi203 ceph-mon[83507]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-05-25T06:18:23.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:23 smithi160 ceph-mon[112094]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-05-25T06:18:24.157 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:24 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[134579]: 2024-05-25T06:18:24.066+0000 7f1a3d2287c0 -1 osd.2 59 log_to_monitors true 2024-05-25T06:18:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:24 smithi160 ceph-mon[112094]: from='osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-25T06:18:24.469 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:18:24 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[134579]: 2024-05-25T06:18:24.265+0000 7f1a2e67f700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:18:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:24 smithi203 ceph-mon[83507]: from='osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-25T06:18:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:25 smithi203 ceph-mon[83507]: from='osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-25T06:18:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:25 smithi203 ceph-mon[83507]: osdmap e62: 6 total, 5 up, 6 in 2024-05-25T06:18:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:25 smithi203 ceph-mon[83507]: from='osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:18:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:25.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:25 smithi203 ceph-mon[83507]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 39/324 objects degraded (12.037%) 2024-05-25T06:18:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:25 smithi160 ceph-mon[112094]: from='osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-25T06:18:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:25 smithi160 ceph-mon[112094]: osdmap e62: 6 total, 5 up, 6 in 2024-05-25T06:18:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:25 smithi160 ceph-mon[112094]: from='osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-25T06:18:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:25.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:25 smithi160 ceph-mon[112094]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 39/324 objects degraded (12.037%) 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666] boot 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: osdmap e63: 6 total, 6 up, 6 in 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:26.577 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:26 smithi160 ceph-mon[112094]: osdmap e64: 6 total, 6 up, 6 in 2024-05-25T06:18:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:18:26.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: osd.2 [v2:172.21.15.160:6818/1307550666,v1:172.21.15.160:6819/1307550666] boot 2024-05-25T06:18:26.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: osdmap e63: 6 total, 6 up, 6 in 2024-05-25T06:18:26.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-25T06:18:26.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:26.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:26.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:26 smithi203 ceph-mon[83507]: osdmap e64: 6 total, 6 up, 6 in 2024-05-25T06:18:27.456 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:27 smithi160 ceph-mon[112094]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 39/324 objects degraded (12.037%) 2024-05-25T06:18:27.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:27 smithi203 ceph-mon[83507]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 39/324 objects degraded (12.037%) 2024-05-25T06:18:28.308 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:28 smithi160 ceph-mon[112094]: Health check update: Degraded data redundancy: 39/324 objects degraded (12.037%), 12 pgs degraded (PG_DEGRADED) 2024-05-25T06:18:28.308 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:28.308 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:28 smithi203 ceph-mon[83507]: Health check update: Degraded data redundancy: 39/324 objects degraded (12.037%), 12 pgs degraded (PG_DEGRADED) 2024-05-25T06:18:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:29 smithi160 ceph-mon[112094]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 7 active+undersized, 53 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s; 17/324 objects degraded (5.247%) 2024-05-25T06:18:29.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:29 smithi203 ceph-mon[83507]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 7 active+undersized, 53 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s; 17/324 objects degraded (5.247%) 2024-05-25T06:18:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:30.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:30 smithi203 ceph-mon[83507]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-05-25T06:18:30.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:30.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:30 smithi160 ceph-mon[112094]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-05-25T06:18:31.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:31 smithi203 ceph-mon[83507]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 3 op/s 2024-05-25T06:18:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:31 smithi160 ceph-mon[112094]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 3 op/s 2024-05-25T06:18:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:32 smithi203 ceph-mon[83507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/324 objects degraded (5.247%), 5 pgs degraded) 2024-05-25T06:18:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:32 smithi203 ceph-mon[83507]: Cluster is now healthy 2024-05-25T06:18:32.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:32 smithi160 ceph-mon[112094]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/324 objects degraded (5.247%), 5 pgs degraded) 2024-05-25T06:18:32.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:32 smithi160 ceph-mon[112094]: Cluster is now healthy 2024-05-25T06:18:33.871 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:33 smithi160 ceph-mon[112094]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-05-25T06:18:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:33 smithi203 ceph-mon[83507]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-05-25T06:18:36.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:35 smithi160 ceph-mon[112094]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-25T06:18:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:35 smithi203 ceph-mon[83507]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-25T06:18:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:37 smithi160 ceph-mon[112094]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-25T06:18:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:37 smithi203 ceph-mon[83507]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-25T06:18:39.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:38 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:38 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:40.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:39 smithi160 ceph-mon[112094]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-25T06:18:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:39 smithi203 ceph-mon[83507]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-25T06:18:42.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:41 smithi160 ceph-mon[112094]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:41 smithi203 ceph-mon[83507]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 -- 172.21.15.160:0/3787904393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 msgr2=0x7f88840fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 --2- 172.21.15.160:0/3787904393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f88840fcdd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8880004940 tx=0x7f8880009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 -- 172.21.15.160:0/3787904393 shutdown_connections 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 --2- 172.21.15.160:0/3787904393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f88840fcdd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 --2- 172.21.15.160:0/3787904393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8884102dc0 0x7f8884103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 -- 172.21.15.160:0/3787904393 >> 172.21.15.160:0/3787904393 conn(0x7f88840f8930 msgr2=0x7f88840fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 -- 172.21.15.160:0/3787904393 shutdown_connections 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 -- 172.21.15.160:0/3787904393 wait complete. 2024-05-25T06:18:43.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 Processor -- start 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.242+0000 7f888b71d700 1 -- start start 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f888b71d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8884102dc0 0x7f8884115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f888b71d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f8884116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f888b71d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88841166e0 con 0x7f8884103760 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f888b71d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888410fcc0 con 0x7f8884102dc0 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f8884116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f8884116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54792/0 (socket says 172.21.15.160:54792) 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 -- 172.21.15.160:0/2731393457 learned_addr learned my addr 172.21.15.160:0/2731393457 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f88894b9700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8884102dc0 0x7f8884115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 -- 172.21.15.160:0/2731393457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8884102dc0 msgr2=0x7f8884115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8884102dc0 0x7f8884115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 -- 172.21.15.160:0/2731393457 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8880005040 con 0x7f8884103760 2024-05-25T06:18:43.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8888cb8700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f8884116090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8880009100 tx=0x7f8880009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:43.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f888000e040 con 0x7f8884103760 2024-05-25T06:18:43.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8880009af0 con 0x7f8884103760 2024-05-25T06:18:43.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888410ff40 con 0x7f8884103760 2024-05-25T06:18:43.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.243+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f888001c3f0 con 0x7f8884103760 2024-05-25T06:18:43.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.244+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8884110500 con 0x7f8884103760 2024-05-25T06:18:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.245+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88840fe510 con 0x7f8884103760 2024-05-25T06:18:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.245+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f888000baf0 con 0x7f8884103760 2024-05-25T06:18:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.246+0000 7f8875ffb700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f887006e140 0x7f88700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.246+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f888008ed10 con 0x7f8884103760 2024-05-25T06:18:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.246+0000 7f88894b9700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f887006e140 0x7f88700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.246+0000 7f88894b9700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f887006e140 0x7f88700705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8878004eb0 tx=0x7f887800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:43.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.250+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f888005bc90 con 0x7f8884103760 2024-05-25T06:18:43.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.466+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8884109f10 con 0x7f887006e140 2024-05-25T06:18:43.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.468+0000 7f8875ffb700 1 -- 172.21.15.160:0/2731393457 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8884109f10 con 0x7f887006e140 2024-05-25T06:18:43.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f887006e140 msgr2=0x7f88700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f887006e140 0x7f88700705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8878004eb0 tx=0x7f887800b040 comp rx=0 tx=0).stop 2024-05-25T06:18:43.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 msgr2=0x7f8884116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f8884116090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8880009100 tx=0x7f8880009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 shutdown_connections 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f887006e140 0x7f88700705f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8884102dc0 0x7f8884115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 --2- 172.21.15.160:0/2731393457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8884103760 0x7f8884116090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 >> 172.21.15.160:0/2731393457 conn(0x7f88840f8930 msgr2=0x7f88840f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 shutdown_connections 2024-05-25T06:18:43.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.470+0000 7f888b71d700 1 -- 172.21.15.160:0/2731393457 wait complete. 2024-05-25T06:18:43.481 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 -- 172.21.15.160:0/2582711134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fde40 msgr2=0x7f5c940fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2582711134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fde40 0x7f5c940fe210 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009ce0 tx=0x7f5c8c004d10 comp rx=0 tx=0).stop 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 -- 172.21.15.160:0/2582711134 shutdown_connections 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2582711134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fe7e0 0x7f5c94106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2582711134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fde40 0x7f5c940fe210 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 -- 172.21.15.160:0/2582711134 >> 172.21.15.160:0/2582711134 conn(0x7f5c940f7850 msgr2=0x7f5c940f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 -- 172.21.15.160:0/2582711134 shutdown_connections 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.574+0000 7f5c9b873700 1 -- 172.21.15.160:0/2582711134 wait complete. 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9b873700 1 Processor -- start 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9b873700 1 -- start start 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9b873700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fde40 0x7f5c94113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9b873700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 0x7f5c94114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9b873700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94114690 con 0x7f5c940fe7e0 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9b873700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9410da80 con 0x7f5c940fde40 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c98e0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 0x7f5c94114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c98e0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 0x7f5c94114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54812/0 (socket says 172.21.15.160:54812) 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c98e0e700 1 -- 172.21.15.160:0/2167106716 learned_addr learned my addr 172.21.15.160:0/2167106716 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.575+0000 7f5c9960f700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fde40 0x7f5c94113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c98e0e700 1 -- 172.21.15.160:0/2167106716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fde40 msgr2=0x7f5c94113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c98e0e700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fde40 0x7f5c94113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c98e0e700 1 -- 172.21.15.160:0/2167106716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c009940 con 0x7f5c940fe7e0 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c9960f700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fde40 0x7f5c94113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:18:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c98e0e700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 0x7f5c94114040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c830 tx=0x7f5c88004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:43.578 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8800a760 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c8800ad90 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c880113a0 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9410dd60 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.576+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9410e2a0 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.577+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c940666f0 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.578+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5c8800a8c0 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.578+0000 7f5c91ffb700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.578+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5c8808da40 con 0x7f5c940fe7e0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.578+0000 7f5c9960f700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.579+0000 7f5c9960f700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b740 tx=0x7f5c8c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.583+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c8805aa60 con 0x7f5c940fe7e0 2024-05-25T06:18:43.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.800+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c94109e00 con 0x7f5c7c06e140 2024-05-25T06:18:43.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.801+0000 7f5c91ffb700 1 -- 172.21.15.160:0/2167106716 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5c94109e00 con 0x7f5c7c06e140 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 msgr2=0x7f5c7c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b740 tx=0x7f5c8c004d10 comp rx=0 tx=0).stop 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 msgr2=0x7f5c94114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 0x7f5c94114040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c830 tx=0x7f5c88004470 comp rx=0 tx=0).stop 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 shutdown_connections 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.803+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c940fde40 0x7f5c94113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.804+0000 7f5c9b873700 1 --2- 172.21.15.160:0/2167106716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c940fe7e0 0x7f5c94114040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.804+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 >> 172.21.15.160:0/2167106716 conn(0x7f5c940f7850 msgr2=0x7f5c940f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:43.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.804+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 shutdown_connections 2024-05-25T06:18:43.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.804+0000 7f5c9b873700 1 -- 172.21.15.160:0/2167106716 wait complete. 2024-05-25T06:18:43.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 -- 172.21.15.160:0/855666194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fd0a0 msgr2=0x7f50c4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 --2- 172.21.15.160:0/855666194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fd0a0 0x7f50c4109350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f50b0004940 tx=0x7f50b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:43.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 -- 172.21.15.160:0/855666194 shutdown_connections 2024-05-25T06:18:43.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 --2- 172.21.15.160:0/855666194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fd0a0 0x7f50c4109350 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 --2- 172.21.15.160:0/855666194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c40fc790 0x7f50c40fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 -- 172.21.15.160:0/855666194 >> 172.21.15.160:0/855666194 conn(0x7f50c4071d00 msgr2=0x7f50c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:43.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 -- 172.21.15.160:0/855666194 shutdown_connections 2024-05-25T06:18:43.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.917+0000 7f50cab9b700 1 -- 172.21.15.160:0/855666194 wait complete. 2024-05-25T06:18:43.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50cab9b700 1 Processor -- start 2024-05-25T06:18:43.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50cab9b700 1 -- start start 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50cab9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 0x7f50c4115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50cab9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c40fd0a0 0x7f50c41162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50cab9b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4116930 con 0x7f50c40fc790 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50cab9b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c410fd00 con 0x7f50c40fd0a0 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50c8937700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 0x7f50c4115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50c8937700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 0x7f50c4115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54834/0 (socket says 172.21.15.160:54834) 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50c8937700 1 -- 172.21.15.160:0/1449843259 learned_addr learned my addr 172.21.15.160:0/1449843259 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.918+0000 7f50c3fff700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c40fd0a0 0x7f50c41162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c8937700 1 -- 172.21.15.160:0/1449843259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c40fd0a0 msgr2=0x7f50c41162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c8937700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c40fd0a0 0x7f50c41162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c8937700 1 -- 172.21.15.160:0/1449843259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0005040 con 0x7f50c40fc790 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c8937700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 0x7f50c4115da0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50b800eb40 tx=0x7f50b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:43.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b8007cc0 con 0x7f50c40fc790 2024-05-25T06:18:43.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50b800f040 con 0x7f50c40fc790 2024-05-25T06:18:43.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c410ff80 con 0x7f50c40fc790 2024-05-25T06:18:43.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b800ca10 con 0x7f50c40fc790 2024-05-25T06:18:43.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.919+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c41104c0 con 0x7f50c40fc790 2024-05-25T06:18:43.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.920+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c40fc330 con 0x7f50c40fc790 2024-05-25T06:18:43.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.921+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f50b800cb70 con 0x7f50c40fc790 2024-05-25T06:18:43.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.921+0000 7f50c17fa700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f50b406e150 0x7f50b4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:43.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.921+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f50b808d5c0 con 0x7f50c40fc790 2024-05-25T06:18:43.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.921+0000 7f50c3fff700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f50b406e150 0x7f50b4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:43.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.922+0000 7f50c3fff700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f50b406e150 0x7f50b4070600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009000 tx=0x7f50b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:43.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:43.926+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f50b805a5f0 con 0x7f50c40fc790 2024-05-25T06:18:44.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.138+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c410c1a0 con 0x7f50b406e150 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.148+0000 7f50c17fa700 1 -- 172.21.15.160:0/1449843259 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f50b0008e80 con 0x7f50b406e150 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (4m) 19s ago 15m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (2m) 19s ago 15m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (117s) 114s ago 14m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (4m) 19s ago 15m 80.2M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (10m) 19s ago 10m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (10m) 19s ago 10m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (10m) 114s ago 10m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:18:44.149 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (10m) 114s ago 10m 117M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (6m) 19s ago 17m 505M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (5m) 114s ago 14m 451M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (3m) 19s ago 17m 45.1M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (2m) 114s ago 14m 34.8M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (5m) 19s ago 15m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (5m) 114s ago 14m 17.4M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (97s) 19s ago 13m 96.8M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (61s) 19s ago 13m 98.9M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (25s) 19s ago 13m 12.3M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (12m) 114s ago 12m 269M 2581M 17.2.7-958-g7b01852c baf1547092e7 428c4573a7fc 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (12m) 114s ago 12m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (11m) 114s ago 11m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:18:44.150 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (4m) 19s ago 15m 67.5M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:18:44.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f50b406e150 msgr2=0x7f50b4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f50b406e150 0x7f50b4070600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009000 tx=0x7f50b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:44.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 msgr2=0x7f50c4115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 0x7f50c4115da0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50b800eb40 tx=0x7f50b800b040 comp rx=0 tx=0).stop 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 shutdown_connections 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c40fc790 0x7f50c4115da0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f50b406e150 0x7f50b4070600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 --2- 172.21.15.160:0/1449843259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c40fd0a0 0x7f50c41162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 >> 172.21.15.160:0/1449843259 conn(0x7f50c4071d00 msgr2=0x7f50c40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 shutdown_connections 2024-05-25T06:18:44.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.151+0000 7f50cab9b700 1 -- 172.21.15.160:0/1449843259 wait complete. 2024-05-25T06:18:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:43 smithi160 ceph-mon[112094]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:44.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.254+0000 7f420be66700 1 -- 172.21.15.160:0/1029972310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 msgr2=0x7f4204065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.254+0000 7f420be66700 1 --2- 172.21.15.160:0/1029972310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204065ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41f4004940 tx=0x7f41f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:44.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- 172.21.15.160:0/1029972310 shutdown_connections 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 --2- 172.21.15.160:0/1029972310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204065ab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 --2- 172.21.15.160:0/1029972310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4204102e50 0x7f4204103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- 172.21.15.160:0/1029972310 >> 172.21.15.160:0/1029972310 conn(0x7f4204071d00 msgr2=0x7f4204072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- 172.21.15.160:0/1029972310 shutdown_connections 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- 172.21.15.160:0/1029972310 wait complete. 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 Processor -- start 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- start start 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4204102e50 0x7f4204114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4204114660 con 0x7f4204065640 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.255+0000 7f420be66700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420410da50 con 0x7f4204102e50 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54854/0 (socket says 172.21.15.160:54854) 2024-05-25T06:18:44.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 -- 172.21.15.160:0/1313033025 learned_addr learned my addr 172.21.15.160:0/1313033025 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209401700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4204102e50 0x7f4204114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 -- 172.21.15.160:0/1313033025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4204102e50 msgr2=0x7f4204114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4204102e50 0x7f4204114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 -- 172.21.15.160:0/1313033025 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4005040 con 0x7f4204065640 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f4209c02700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204113ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4200002a80 tx=0x7f42000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4200007cc0 con 0x7f4204065640 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420410dd30 con 0x7f4204065640 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f420000f040 con 0x7f4204065640 2024-05-25T06:18:44.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42000096f0 con 0x7f4204065640 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.256+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420410e2f0 con 0x7f4204065640 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.257+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4204100100 con 0x7f4204065640 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.258+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f420000d690 con 0x7f4204065640 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.258+0000 7f41fa7fc700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f41f006e140 0x7f41f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.258+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f420008d280 con 0x7f4204065640 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.258+0000 7f4209401700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f41f006e140 0x7f41f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:44.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.259+0000 7f4209401700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f41f006e140 0x7f41f00705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41f400b3b0 tx=0x7f41f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:44.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.263+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f420005a180 con 0x7f4204065640 2024-05-25T06:18:44.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:43 smithi203 ceph-mon[83507]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:44.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:44.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:44.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.542+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f420405f5f0 con 0x7f4204065640 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.543+0000 7f41fa7fc700 1 -- 172.21.15.160:0/1313033025 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f4200059d10 con 0x7f4204065640 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:18:44.543 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 3, 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 3 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 7, 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 7 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:18:44.544 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:18:44.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f41f006e140 msgr2=0x7f41f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f41f006e140 0x7f41f00705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41f400b3b0 tx=0x7f41f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 msgr2=0x7f4204113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204113ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4200002a80 tx=0x7f42000046b0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 shutdown_connections 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4204065640 0x7f4204113ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f41f006e140 0x7f41f00705f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 --2- 172.21.15.160:0/1313033025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4204102e50 0x7f4204114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 >> 172.21.15.160:0/1313033025 conn(0x7f4204071d00 msgr2=0x7f42040fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 shutdown_connections 2024-05-25T06:18:44.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.546+0000 7f420be66700 1 -- 172.21.15.160:0/1313033025 wait complete. 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 -- 172.21.15.160:0/1239604747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 msgr2=0x7f59bc065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 --2- 172.21.15.160:0/1239604747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc065ab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59a8004940 tx=0x7f59a8009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 -- 172.21.15.160:0/1239604747 shutdown_connections 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 --2- 172.21.15.160:0/1239604747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc065ab0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 --2- 172.21.15.160:0/1239604747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc102e50 0x7f59bc103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 -- 172.21.15.160:0/1239604747 >> 172.21.15.160:0/1239604747 conn(0x7f59bc071d00 msgr2=0x7f59bc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 -- 172.21.15.160:0/1239604747 shutdown_connections 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.649+0000 7f59c322e700 1 -- 172.21.15.160:0/1239604747 wait complete. 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c322e700 1 Processor -- start 2024-05-25T06:18:44.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c322e700 1 -- start start 2024-05-25T06:18:44.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c322e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:44.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c322e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc102e50 0x7f59bc0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:44.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c322e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0fc880 con 0x7f59bc065640 2024-05-25T06:18:44.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c322e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0fc9c0 con 0x7f59bc102e50 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c0fca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c0fca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc0fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54880/0 (socket says 172.21.15.160:54880) 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.650+0000 7f59c0fca700 1 -- 172.21.15.160:0/1541454571 learned_addr learned my addr 172.21.15.160:0/1541454571 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59bbfff700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc102e50 0x7f59bc0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59c0fca700 1 -- 172.21.15.160:0/1541454571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc102e50 msgr2=0x7f59bc0fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59c0fca700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc102e50 0x7f59bc0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59c0fca700 1 -- 172.21.15.160:0/1541454571 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8005040 con 0x7f59bc065640 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59c0fca700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc0fbdb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c830 tx=0x7f59b0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b000a760 con 0x7f59bc065640 2024-05-25T06:18:44.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59b000ad90 con 0x7f59bc065640 2024-05-25T06:18:44.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc119e70 con 0x7f59bc065640 2024-05-25T06:18:44.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b00113a0 con 0x7f59bc065640 2024-05-25T06:18:44.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.651+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc11a2f0 con 0x7f59bc065640 2024-05-25T06:18:44.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.652+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc100100 con 0x7f59bc065640 2024-05-25T06:18:44.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.653+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f59b0012400 con 0x7f59bc065640 2024-05-25T06:18:44.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.653+0000 7f59b97fa700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f59ac06e140 0x7f59ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:44.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.653+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f59b008daf0 con 0x7f59bc065640 2024-05-25T06:18:44.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.653+0000 7f59bbfff700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f59ac06e140 0x7f59ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:44.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.654+0000 7f59bbfff700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f59ac06e140 0x7f59ac0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59a8000c00 tx=0x7f59a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:44.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.658+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f59b0056f40 con 0x7f59bc065640 2024-05-25T06:18:44.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.903+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f59bc04bed0 con 0x7f59bc065640 2024-05-25T06:18:44.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.903+0000 7f59b97fa700 1 -- 172.21.15.160:0/1541454571 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7f59b005a600 con 0x7f59bc065640 2024-05-25T06:18:44.903 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:18:44.903 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:18:44.903 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:18:44.904 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:18:44.905 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:18:44.906 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.905+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f59ac06e140 msgr2=0x7f59ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.905+0000 7f59c322e700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f59ac06e140 0x7f59ac0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59a8000c00 tx=0x7f59a8009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.905+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 msgr2=0x7f59bc0fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.905+0000 7f59c322e700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc0fbdb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c830 tx=0x7f59b0004470 comp rx=0 tx=0).stop 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 shutdown_connections 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59bc065640 0x7f59bc0fbdb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f59ac06e140 0x7f59ac0705f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 --2- 172.21.15.160:0/1541454571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc102e50 0x7f59bc0fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:44.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 >> 172.21.15.160:0/1541454571 conn(0x7f59bc071d00 msgr2=0x7f59bc0fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:44.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 shutdown_connections 2024-05-25T06:18:44.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:44.906+0000 7f59c322e700 1 -- 172.21.15.160:0/1541454571 wait complete. 2024-05-25T06:18:44.908 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.010+0000 7f462003b700 1 -- 172.21.15.160:0/2846916317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065e60 msgr2=0x7f4618101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.010+0000 7f462003b700 1 --2- 172.21.15.160:0/2846916317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065e60 0x7f4618101a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4608004940 tx=0x7f4608009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- 172.21.15.160:0/2846916317 shutdown_connections 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 --2- 172.21.15.160:0/2846916317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065e60 0x7f4618101a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 --2- 172.21.15.160:0/2846916317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4618065a00 0x7f46181014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- 172.21.15.160:0/2846916317 >> 172.21.15.160:0/2846916317 conn(0x7f4618073350 msgr2=0x7f4618073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- 172.21.15.160:0/2846916317 shutdown_connections 2024-05-25T06:18:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- 172.21.15.160:0/2846916317 wait complete. 2024-05-25T06:18:45.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 Processor -- start 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- start start 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 0x7f4618119eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4618065e60 0x7f461811a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461811aa40 con 0x7f4618065a00 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.011+0000 7f462003b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618114020 con 0x7f4618065e60 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 0x7f4618119eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 0x7f4618119eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54906/0 (socket says 172.21.15.160:54906) 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 -- 172.21.15.160:0/1984789809 learned_addr learned my addr 172.21.15.160:0/1984789809 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461d5d6700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4618065e60 0x7f461811a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 -- 172.21.15.160:0/1984789809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4618065e60 msgr2=0x7f461811a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4618065e60 0x7f461811a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 -- 172.21.15.160:0/1984789809 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608005040 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f461ddd7700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 0x7f4618119eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4614002a80 tx=0x7f46140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4614007cc0 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f461400f040 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4618114300 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461400dac0 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.012+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46181148c0 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.013+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46180fcbd0 con 0x7f4618065a00 2024-05-25T06:18:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.014+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f461400dc20 con 0x7f4618065a00 2024-05-25T06:18:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.014+0000 7f460e7fc700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f460406e140 0x7f46040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.014+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f461408d730 con 0x7f4618065a00 2024-05-25T06:18:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.014+0000 7f461d5d6700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f460406e140 0x7f46040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.015+0000 7f461d5d6700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f460406e140 0x7f46040705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4608004f90 tx=0x7f4608009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:45.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.019+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f461405a630 con 0x7f4618065a00 2024-05-25T06:18:45.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:44 smithi160 ceph-mon[112094]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:45.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:44 smithi160 ceph-mon[112094]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:45.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:44 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1313033025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:45.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:44 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:45.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:44 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1541454571' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:18:45.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.235+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4618115570 con 0x7f460406e140 2024-05-25T06:18:45.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.236+0000 7f460e7fc700 1 -- 172.21.15.160:0/1984789809 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4618115570 con 0x7f460406e140 2024-05-25T06:18:45.236 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:18:45.236 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:18:45.236 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:18:45.236 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:18:45.236 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "9/21 daemons upgraded", 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading osd daemons", 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:18:45.237 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:18:45.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f460406e140 msgr2=0x7f46040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f460406e140 0x7f46040705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4608004f90 tx=0x7f4608009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:45.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 msgr2=0x7f4618119eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 0x7f4618119eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4614002a80 tx=0x7f46140046b0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 shutdown_connections 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4618065a00 0x7f4618119eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f460406e140 0x7f46040705f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 --2- 172.21.15.160:0/1984789809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4618065e60 0x7f461811a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 >> 172.21.15.160:0/1984789809 conn(0x7f4618073350 msgr2=0x7f46180fabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 shutdown_connections 2024-05-25T06:18:45.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.238+0000 7f462003b700 1 -- 172.21.15.160:0/1984789809 wait complete. 2024-05-25T06:18:45.291 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:44 smithi203 ceph-mon[83507]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:45.291 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:44 smithi203 ceph-mon[83507]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:45.291 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:44 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1313033025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:18:45.291 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:44 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:45.291 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:44 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1541454571' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:18:45.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.341+0000 7fc544128700 1 -- 172.21.15.160:0/4027387468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c102690 msgr2=0x7fc53c0fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.341+0000 7fc544128700 1 --2- 172.21.15.160:0/4027387468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c102690 0x7fc53c0fbd00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc538004940 tx=0x7fc538009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:45.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.341+0000 7fc544128700 1 -- 172.21.15.160:0/4027387468 shutdown_connections 2024-05-25T06:18:45.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.341+0000 7fc544128700 1 --2- 172.21.15.160:0/4027387468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c102690 0x7fc53c0fbd00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.341+0000 7fc544128700 1 --2- 172.21.15.160:0/4027387468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53c101cf0 0x7fc53c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.341+0000 7fc544128700 1 -- 172.21.15.160:0/4027387468 >> 172.21.15.160:0/4027387468 conn(0x7fc53c0f7860 msgr2=0x7fc53c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:45.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 -- 172.21.15.160:0/4027387468 shutdown_connections 2024-05-25T06:18:45.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 -- 172.21.15.160:0/4027387468 wait complete. 2024-05-25T06:18:45.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 Processor -- start 2024-05-25T06:18:45.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 -- start start 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 0x7fc53c10ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53c102690 0x7fc53c10f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c10f5e0 con 0x7fc53c101cf0 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc544128700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c10f750 con 0x7fc53c102690 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc541ec4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 0x7fc53c10ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc541ec4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 0x7fc53c10ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54912/0 (socket says 172.21.15.160:54912) 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.342+0000 7fc541ec4700 1 -- 172.21.15.160:0/2541938012 learned_addr learned my addr 172.21.15.160:0/2541938012 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc5416c3700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53c102690 0x7fc53c10f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc541ec4700 1 -- 172.21.15.160:0/2541938012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53c102690 msgr2=0x7fc53c10f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc541ec4700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53c102690 0x7fc53c10f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc541ec4700 1 -- 172.21.15.160:0/2541938012 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538005040 con 0x7fc53c101cf0 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc541ec4700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 0x7fc53c10ead0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b790 tx=0x7fc52c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52c00f6f0 con 0x7fc53c101cf0 2024-05-25T06:18:45.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc52c00fd20 con 0x7fc53c101cf0 2024-05-25T06:18:45.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52c0103a0 con 0x7fc53c101cf0 2024-05-25T06:18:45.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c117b40 con 0x7fc53c101cf0 2024-05-25T06:18:45.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.343+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c117fa0 con 0x7fc53c101cf0 2024-05-25T06:18:45.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.344+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53c0fd440 con 0x7fc53c101cf0 2024-05-25T06:18:45.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.345+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc52c011400 con 0x7fc53c101cf0 2024-05-25T06:18:45.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.345+0000 7fc5327fc700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc52806e140 0x7fc5280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:18:45.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.345+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc52c08d7c0 con 0x7fc53c101cf0 2024-05-25T06:18:45.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.345+0000 7fc5416c3700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc52806e140 0x7fc5280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:18:45.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.346+0000 7fc5416c3700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc52806e140 0x7fc5280705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc53c066ac0 tx=0x7fc538009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:18:45.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.350+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc52c056cb0 con 0x7fc53c101cf0 2024-05-25T06:18:45.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.630+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc53c04c710 con 0x7fc53c101cf0 2024-05-25T06:18:45.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.630+0000 7fc5327fc700 1 -- 172.21.15.160:0/2541938012 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc52c05a370 con 0x7fc53c101cf0 2024-05-25T06:18:45.631 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:18:45.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc52806e140 msgr2=0x7fc5280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc52806e140 0x7fc5280705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc53c066ac0 tx=0x7fc538009e30 comp rx=0 tx=0).stop 2024-05-25T06:18:45.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 msgr2=0x7fc53c10ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:18:45.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 0x7fc53c10ead0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b790 tx=0x7fc52c009250 comp rx=0 tx=0).stop 2024-05-25T06:18:45.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 shutdown_connections 2024-05-25T06:18:45.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc53c101cf0 0x7fc53c10ead0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc52806e140 0x7fc5280705f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 --2- 172.21.15.160:0/2541938012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53c102690 0x7fc53c10f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:18:45.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 >> 172.21.15.160:0/2541938012 conn(0x7fc53c0f7860 msgr2=0x7fc53c0f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:18:45.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 shutdown_connections 2024-05-25T06:18:45.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:18:45.633+0000 7fc544128700 1 -- 172.21.15.160:0/2541938012 wait complete. 2024-05-25T06:18:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:45 smithi160 ceph-mon[112094]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:45 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:45 smithi160 ceph-mon[112094]: Upgrade: osd.3 is safe to restart 2024-05-25T06:18:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:45 smithi160 ceph-mon[112094]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:18:46.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:45 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2541938012' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:18:46.280 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:45 smithi203 ceph-mon[83507]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:46.281 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:45 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-25T06:18:46.281 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:45 smithi203 ceph-mon[83507]: Upgrade: osd.3 is safe to restart 2024-05-25T06:18:46.281 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:45 smithi203 ceph-mon[83507]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:18:46.281 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:45 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2541938012' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:18:47.185 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:46 smithi203 ceph-mon[83507]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:47.185 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:46 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:47.185 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:46 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-25T06:18:47.185 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:46 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:18:47.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:46 smithi160 ceph-mon[112094]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:18:47.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:47.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-25T06:18:47.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:18:48.146 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:47 smithi203 ceph-mon[83507]: Upgrade: Updating osd.3 2024-05-25T06:18:48.146 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:47 smithi203 ceph-mon[83507]: Deploying daemon osd.3 on smithi203 2024-05-25T06:18:48.146 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:47 smithi203 ceph-mon[83507]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:47 smithi160 ceph-mon[112094]: Upgrade: Updating osd.3 2024-05-25T06:18:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:47 smithi160 ceph-mon[112094]: Deploying daemon osd.3 on smithi203 2024-05-25T06:18:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:47 smithi160 ceph-mon[112094]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:18:49.869 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:49 smithi203 systemd[1]: Stopping Ceph osd.3 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:18:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:49 smithi203 ceph-mon[83507]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:18:50.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:49 smithi203 ceph-mon[83507]: osd.3 marked itself down and dead 2024-05-25T06:18:50.149 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:49 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[41551]: 2024-05-25T06:18:49.869+0000 7fe965b19700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:18:50.150 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:49 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[41551]: 2024-05-25T06:18:49.870+0000 7fe965b19700 -1 osd.3 64 *** Got signal Terminated *** 2024-05-25T06:18:50.150 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:49 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[41551]: 2024-05-25T06:18:49.870+0000 7fe965b19700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:18:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:49 smithi160 ceph-mon[112094]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-25T06:18:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:49 smithi160 ceph-mon[112094]: osd.3 marked itself down and dead 2024-05-25T06:18:50.649 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:50 smithi203 podman[92317]: 2024-05-25 06:18:50.368766545 +0000 UTC m=+0.601798797 container died 428c4573a7fc34896914cbb29670f16d69a806fbde665ca5e075233a8c1cb613 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=quincy-7b01852, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:50.942 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:50 smithi203 podman[92317]: 2024-05-25 06:18:50.678108162 +0000 UTC m=+0.911140406 container cleanup 428c4573a7fc34896914cbb29670f16d69a806fbde665ca5e075233a8c1cb613 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=quincy-7b01852, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-25T06:18:50.942 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:50 smithi203 bash[92317]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3 2024-05-25T06:18:51.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:50 smithi160 ceph-mon[112094]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:18:51.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:50 smithi160 ceph-mon[112094]: osdmap e65: 6 total, 5 up, 6 in 2024-05-25T06:18:51.313 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:50 smithi203 ceph-mon[83507]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:18:51.313 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:50 smithi203 ceph-mon[83507]: osdmap e65: 6 total, 5 up, 6 in 2024-05-25T06:18:51.314 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:50 smithi203 podman[92343]: 2024-05-25 06:18:50.963220899 +0000 UTC m=+0.590527581 container remove 428c4573a7fc34896914cbb29670f16d69a806fbde665ca5e075233a8c1cb613 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=quincy-7b01852, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:51.633 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:51 smithi203 podman[92429]: 2024-05-25 06:18:51.314345919 +0000 UTC m=+0.108321757 container create 67b494b56f2958278fb19c1e60b559c44c2df9f07b531f4ae7f3678fa72469ff (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:18:51.633 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:51 smithi203 podman[92429]: 2024-05-25 06:18:51.223121904 +0000 UTC m=+0.017097721 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:18:51.633 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:51 smithi203 podman[92429]: 2024-05-25 06:18:51.559262443 +0000 UTC m=+0.353238267 container init 67b494b56f2958278fb19c1e60b559c44c2df9f07b531f4ae7f3678fa72469ff (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-05-25T06:18:51.633 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:51 smithi203 podman[92429]: 2024-05-25 06:18:51.562924045 +0000 UTC m=+0.356899871 container start 67b494b56f2958278fb19c1e60b559c44c2df9f07b531f4ae7f3678fa72469ff (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c) 2024-05-25T06:18:51.633 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:51 smithi203 podman[92429]: 2024-05-25 06:18:51.633914202 +0000 UTC m=+0.427890025 container attach 67b494b56f2958278fb19c1e60b559c44c2df9f07b531f4ae7f3678fa72469ff (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:18:51.899 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:51 smithi203 podman[92429]: 2024-05-25 06:18:51.853258021 +0000 UTC m=+0.647233845 container died 67b494b56f2958278fb19c1e60b559c44c2df9f07b531f4ae7f3678fa72469ff (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-25T06:18:52.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:51 smithi160 ceph-mon[112094]: pgmap v113: 65 pgs: 16 stale+active+clean, 49 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:18:52.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:51 smithi160 ceph-mon[112094]: osdmap e66: 6 total, 5 up, 6 in 2024-05-25T06:18:52.316 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:51 smithi203 ceph-mon[83507]: pgmap v113: 65 pgs: 16 stale+active+clean, 49 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:18:52.316 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:51 smithi203 ceph-mon[83507]: osdmap e66: 6 total, 5 up, 6 in 2024-05-25T06:18:52.631 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 podman[92455]: 2024-05-25 06:18:52.412991368 +0000 UTC m=+0.553316955 container remove 67b494b56f2958278fb19c1e60b559c44c2df9f07b531f4ae7f3678fa72469ff (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:52.632 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-25T06:18:52.632 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 systemd[1]: Stopped Ceph osd.3 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:18:52.632 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.3.service: Consumed 1min 18.697s CPU time. 2024-05-25T06:18:52.899 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 systemd[1]: Starting Ceph osd.3 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:18:52.900 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 podman[92554]: 2024-05-25 06:18:52.752866115 +0000 UTC m=+0.018118795 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:18:52.900 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:52 smithi203 podman[92554]: 2024-05-25 06:18:52.855746959 +0000 UTC m=+0.120999660 container create 5bf947e45b1e321b48bf265ee5efc7d709ad302b65c655e2cd5eaf3b9321d824 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS) 2024-05-25T06:18:53.399 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 podman[92554]: 2024-05-25 06:18:53.1171184 +0000 UTC m=+0.382371092 container init 5bf947e45b1e321b48bf265ee5efc7d709ad302b65c655e2cd5eaf3b9321d824 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, ceph=True) 2024-05-25T06:18:53.399 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 podman[92554]: 2024-05-25 06:18:53.120548833 +0000 UTC m=+0.385801522 container start 5bf947e45b1e321b48bf265ee5efc7d709ad302b65c655e2cd5eaf3b9321d824 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-25T06:18:53.400 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 podman[92554]: 2024-05-25 06:18:53.192050362 +0000 UTC m=+0.457303055 container attach 5bf947e45b1e321b48bf265ee5efc7d709ad302b65c655e2cd5eaf3b9321d824 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-25T06:18:53.887 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-25T06:18:53.887 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-25T06:18:53.887 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 bash[92554]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate[92568]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-25T06:18:53.888 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:53 smithi203 podman[92554]: 2024-05-25 06:18:53.601635857 +0000 UTC m=+0.866888548 container died 5bf947e45b1e321b48bf265ee5efc7d709ad302b65c655e2cd5eaf3b9321d824 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-25T06:18:54.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:53 smithi203 ceph-mon[83507]: pgmap v115: 65 pgs: 16 stale+active+clean, 49 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-25T06:18:54.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:53 smithi160 ceph-mon[112094]: pgmap v115: 65 pgs: 16 stale+active+clean, 49 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-25T06:18:54.505 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 podman[92702]: 2024-05-25 06:18:54.196582352 +0000 UTC m=+0.588726251 container remove 5bf947e45b1e321b48bf265ee5efc7d709ad302b65c655e2cd5eaf3b9321d824 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:18:54.505 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 podman[92764]: 2024-05-25 06:18:54.406037046 +0000 UTC m=+0.111161427 container create 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-25T06:18:54.505 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 podman[92764]: 2024-05-25 06:18:54.31260193 +0000 UTC m=+0.017726275 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:18:54.899 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 podman[92764]: 2024-05-25 06:18:54.641899121 +0000 UTC m=+0.347023464 container init 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-25T06:18:54.900 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 podman[92764]: 2024-05-25 06:18:54.645616635 +0000 UTC m=+0.350740971 container start 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:18:54.900 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 ceph-osd[92783]: -- 172.21.15.203:0/1284032239 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c14ea53680 con 0x55c14ea1d000 2024-05-25T06:18:54.900 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 bash[92764]: 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 2024-05-25T06:18:54.900 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:54 smithi203 systemd[1]: Started Ceph osd.3 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:18:55.580 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:18:55 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:18:55.198+0000 7fbb979f37c0 -1 Falling back to public interface 2024-05-25T06:18:55.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:55 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:55.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:55 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:55.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:55 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:18:55.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:55 smithi203 ceph-mon[83507]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:18:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:55 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:55 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:55 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:18:56.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:55 smithi160 ceph-mon[112094]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:18:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:56 smithi203 ceph-mon[83507]: Health check failed: Degraded data redundancy: 68/324 objects degraded (20.988%), 19 pgs degraded (PG_DEGRADED) 2024-05-25T06:18:57.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:56 smithi160 ceph-mon[112094]: Health check failed: Degraded data redundancy: 68/324 objects degraded (20.988%), 19 pgs degraded (PG_DEGRADED) 2024-05-25T06:18:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:57 smithi203 ceph-mon[83507]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:18:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:58.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:57 smithi160 ceph-mon[112094]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:18:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:59.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:18:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:18:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:00.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:18:59 smithi203 ceph-mon[83507]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:19:00.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:18:59 smithi160 ceph-mon[112094]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:19:00.574 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:19:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:19:00.337+0000 7fbb979f37c0 -1 osd.3 64 log_to_monitors true 2024-05-25T06:19:01.164 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:00 smithi203 ceph-mon[83507]: from='osd.3 [v2:172.21.15.203:6800/316097287,v1:172.21.15.203:6801/316097287]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:19:01.164 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:00 smithi203 ceph-mon[83507]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:19:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:00 smithi160 ceph-mon[112094]: from='osd.3 [v2:172.21.15.203:6800/316097287,v1:172.21.15.203:6801/316097287]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:19:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:00 smithi160 ceph-mon[112094]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-25T06:19:02.149 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:19:01 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:19:01.821+0000 7fbb84e42700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:19:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:01 smithi203 ceph-mon[83507]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:19:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:01 smithi203 ceph-mon[83507]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-25T06:19:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:01 smithi203 ceph-mon[83507]: from='osd.3 [v2:172.21.15.203:6800/316097287,v1:172.21.15.203:6801/316097287]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:01 smithi203 ceph-mon[83507]: osdmap e67: 6 total, 5 up, 6 in 2024-05-25T06:19:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:01 smithi203 ceph-mon[83507]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:01 smithi203 ceph-mon[83507]: from='osd.3 ' entity='osd.3' 2024-05-25T06:19:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:01 smithi160 ceph-mon[112094]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-05-25T06:19:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:01 smithi160 ceph-mon[112094]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-25T06:19:02.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:01 smithi160 ceph-mon[112094]: from='osd.3 [v2:172.21.15.203:6800/316097287,v1:172.21.15.203:6801/316097287]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:02.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:01 smithi160 ceph-mon[112094]: osdmap e67: 6 total, 5 up, 6 in 2024-05-25T06:19:02.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:01 smithi160 ceph-mon[112094]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:02.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:01 smithi160 ceph-mon[112094]: from='osd.3 ' entity='osd.3' 2024-05-25T06:19:03.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:03.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:03.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: osd.3 [v2:172.21.15.203:6800/316097287,v1:172.21.15.203:6801/316097287] boot 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: osdmap e68: 6 total, 6 up, 6 in 2024-05-25T06:19:03.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:19:03.901 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:03 smithi203 ceph-mon[83507]: pgmap v122: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 68/324 objects degraded (20.988%) 2024-05-25T06:19:03.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:03.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: osd.3 [v2:172.21.15.203:6800/316097287,v1:172.21.15.203:6801/316097287] boot 2024-05-25T06:19:03.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: osdmap e68: 6 total, 6 up, 6 in 2024-05-25T06:19:03.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-25T06:19:03.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:03 smithi160 ceph-mon[112094]: pgmap v122: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 68/324 objects degraded (20.988%) 2024-05-25T06:19:05.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:04 smithi203 ceph-mon[83507]: osdmap e69: 6 total, 6 up, 6 in 2024-05-25T06:19:05.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:04 smithi160 ceph-mon[112094]: osdmap e69: 6 total, 6 up, 6 in 2024-05-25T06:19:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:05 smithi203 ceph-mon[83507]: pgmap v124: 65 pgs: 4 peering, 10 active+undersized+degraded, 6 active+undersized, 45 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 40/324 objects degraded (12.346%) 2024-05-25T06:19:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:05 smithi160 ceph-mon[112094]: pgmap v124: 65 pgs: 4 peering, 10 active+undersized+degraded, 6 active+undersized, 45 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 40/324 objects degraded (12.346%) 2024-05-25T06:19:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:06 smithi203 ceph-mon[83507]: Health check update: Degraded data redundancy: 40/324 objects degraded (12.346%), 10 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:07.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:06 smithi160 ceph-mon[112094]: Health check update: Degraded data redundancy: 40/324 objects degraded (12.346%), 10 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:07 smithi203 ceph-mon[83507]: pgmap v125: 65 pgs: 4 peering, 61 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-05-25T06:19:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:07 smithi160 ceph-mon[112094]: pgmap v125: 65 pgs: 4 peering, 61 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-05-25T06:19:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:08 smithi203 ceph-mon[83507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/324 objects degraded (12.346%), 10 pgs degraded) 2024-05-25T06:19:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:08 smithi203 ceph-mon[83507]: Cluster is now healthy 2024-05-25T06:19:09.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:08 smithi160 ceph-mon[112094]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/324 objects degraded (12.346%), 10 pgs degraded) 2024-05-25T06:19:09.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:08 smithi160 ceph-mon[112094]: Cluster is now healthy 2024-05-25T06:19:10.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:09 smithi203 ceph-mon[83507]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-05-25T06:19:10.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:09 smithi160 ceph-mon[112094]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-05-25T06:19:12.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:11 smithi160 ceph-mon[112094]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:19:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:11 smithi203 ceph-mon[83507]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:19:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:13 smithi160 ceph-mon[112094]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-25T06:19:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:13 smithi203 ceph-mon[83507]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-25T06:19:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:14.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:15.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 -- 172.21.15.160:0/1526541823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf301037c0 msgr2=0x7fdf300fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:15.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 --2- 172.21.15.160:0/1526541823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf301037c0 0x7fdf300fb3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf24004940 tx=0x7fdf24009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:15.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 -- 172.21.15.160:0/1526541823 shutdown_connections 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 --2- 172.21.15.160:0/1526541823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf301037c0 0x7fdf300fb3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 --2- 172.21.15.160:0/1526541823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf30102e20 0x7fdf301031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 -- 172.21.15.160:0/1526541823 >> 172.21.15.160:0/1526541823 conn(0x7fdf30072380 msgr2=0x7fdf30072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 -- 172.21.15.160:0/1526541823 shutdown_connections 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 -- 172.21.15.160:0/1526541823 wait complete. 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 Processor -- start 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.749+0000 7fdf372b1700 1 -- start start 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf372b1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 0x7fdf30113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf372b1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf301037c0 0x7fdf30113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf372b1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30114510 con 0x7fdf30102e20 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf372b1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3010db30 con 0x7fdf301037c0 2024-05-25T06:19:15.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 0x7fdf30113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 0x7fdf30113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57406/0 (socket says 172.21.15.160:57406) 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 -- 172.21.15.160:0/3747986937 learned_addr learned my addr 172.21.15.160:0/3747986937 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3484c700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf301037c0 0x7fdf30113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 -- 172.21.15.160:0/3747986937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf301037c0 msgr2=0x7fdf30113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf301037c0 0x7fdf30113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 -- 172.21.15.160:0/3747986937 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24005040 con 0x7fdf30102e20 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf3504d700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 0x7fdf30113980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf28001d00 tx=0x7fdf28009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf28007d10 con 0x7fdf30102e20 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3010de10 con 0x7fdf30102e20 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf28007e70 con 0x7fdf30102e20 2024-05-25T06:19:15.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.750+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf28014640 con 0x7fdf30102e20 2024-05-25T06:19:15.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.751+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3010e350 con 0x7fdf30102e20 2024-05-25T06:19:15.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.752+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf300fe6e0 con 0x7fdf30102e20 2024-05-25T06:19:15.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.752+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdf2800e900 con 0x7fdf30102e20 2024-05-25T06:19:15.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.753+0000 7fdf2dffb700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdf1806e140 0x7fdf180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:15.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.753+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdf2808daa0 con 0x7fdf30102e20 2024-05-25T06:19:15.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.753+0000 7fdf3484c700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdf1806e140 0x7fdf180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:15.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.753+0000 7fdf3484c700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdf1806e140 0x7fdf180705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400b3b0 tx=0x7fdf24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:15.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.757+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdf28056e70 con 0x7fdf30102e20 2024-05-25T06:19:15.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.974+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf3010efd0 con 0x7fdf1806e140 2024-05-25T06:19:15.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.975+0000 7fdf2dffb700 1 -- 172.21.15.160:0/3747986937 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fdf3010efd0 con 0x7fdf1806e140 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.977+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdf1806e140 msgr2=0x7fdf180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.977+0000 7fdf372b1700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdf1806e140 0x7fdf180705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400b3b0 tx=0x7fdf24009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.977+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 msgr2=0x7fdf30113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.977+0000 7fdf372b1700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 0x7fdf30113980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf28001d00 tx=0x7fdf28009e40 comp rx=0 tx=0).stop 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 shutdown_connections 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf30102e20 0x7fdf30113980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:15.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fdf1806e140 0x7fdf180705f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:15.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 --2- 172.21.15.160:0/3747986937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf301037c0 0x7fdf30113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:15.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 >> 172.21.15.160:0/3747986937 conn(0x7fdf30072380 msgr2=0x7fdf300fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:15.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 shutdown_connections 2024-05-25T06:19:15.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:15.978+0000 7fdf372b1700 1 -- 172.21.15.160:0/3747986937 wait complete. 2024-05-25T06:19:15.988 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:19:16.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 -- 172.21.15.160:0/2470447627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d0101d90 msgr2=0x7f62d0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2470447627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d0101d90 0x7f62d0102160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62cc004940 tx=0x7f62cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:16.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 -- 172.21.15.160:0/2470447627 shutdown_connections 2024-05-25T06:19:16.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2470447627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d00fbd40 0x7f62d00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2470447627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d0101d90 0x7f62d0102160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 -- 172.21.15.160:0/2470447627 >> 172.21.15.160:0/2470447627 conn(0x7f62d0071d00 msgr2=0x7f62d0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:16.083 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:15 smithi160 ceph-mon[112094]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 -- 172.21.15.160:0/2470447627 shutdown_connections 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 -- 172.21.15.160:0/2470447627 wait complete. 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.082+0000 7f62d7b25700 1 Processor -- start 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d7b25700 1 -- start start 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d7b25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 0x7f62d010eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d7b25700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d0101d90 0x7f62d010f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d7b25700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d010f660 con 0x7f62d00fbd40 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d7b25700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d010f7d0 con 0x7f62d0101d90 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 0x7f62d010eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 0x7f62d010eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57416/0 (socket says 172.21.15.160:57416) 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 -- 172.21.15.160:0/2357993755 learned_addr learned my addr 172.21.15.160:0/2357993755 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:16.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d50c0700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d0101d90 0x7f62d010f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 -- 172.21.15.160:0/2357993755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d0101d90 msgr2=0x7f62d010f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d0101d90 0x7f62d010f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 -- 172.21.15.160:0/2357993755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc005040 con 0x7f62d00fbd40 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62d58c1700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 0x7f62d010eb20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009150 tx=0x7f62cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62cc00e040 con 0x7f62d00fbd40 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.083+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62cc00bbc0 con 0x7f62d00fbd40 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.084+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d0117b90 con 0x7f62d00fbd40 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.084+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62cc01c630 con 0x7f62d00fbd40 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.084+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d0118070 con 0x7f62d00fbd40 2024-05-25T06:19:16.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.084+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d00fd430 con 0x7f62d00fbd40 2024-05-25T06:19:16.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.085+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f62cc00bd30 con 0x7f62d00fbd40 2024-05-25T06:19:16.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.086+0000 7f62c67fc700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62bc06e150 0x7f62bc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.086+0000 7f62d50c0700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62bc06e150 0x7f62bc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.086+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f62cc08ec40 con 0x7f62d00fbd40 2024-05-25T06:19:16.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.086+0000 7f62d50c0700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62bc06e150 0x7f62bc070600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62d0065fb0 tx=0x7f62c0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:16.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.090+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f62cc058130 con 0x7f62d00fbd40 2024-05-25T06:19:16.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.309+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62d004bf40 con 0x7f62bc06e150 2024-05-25T06:19:16.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.310+0000 7f62c67fc700 1 -- 172.21.15.160:0/2357993755 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f62d004bf40 con 0x7f62bc06e150 2024-05-25T06:19:16.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62bc06e150 msgr2=0x7f62bc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62bc06e150 0x7f62bc070600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62d0065fb0 tx=0x7f62c0009ee0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 msgr2=0x7f62d010eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 0x7f62d010eb20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009150 tx=0x7f62cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 shutdown_connections 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d00fbd40 0x7f62d010eb20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62bc06e150 0x7f62bc070600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 --2- 172.21.15.160:0/2357993755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d0101d90 0x7f62d010f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 >> 172.21.15.160:0/2357993755 conn(0x7f62d0071d00 msgr2=0x7f62d0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 shutdown_connections 2024-05-25T06:19:16.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.312+0000 7f62d7b25700 1 -- 172.21.15.160:0/2357993755 wait complete. 2024-05-25T06:19:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:15 smithi203 ceph-mon[83507]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-25T06:19:16.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 -- 172.21.15.160:0/2850290769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 msgr2=0x7f100c065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 --2- 172.21.15.160:0/2850290769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c065ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc004940 tx=0x7f0ffc009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:16.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 -- 172.21.15.160:0/2850290769 shutdown_connections 2024-05-25T06:19:16.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 --2- 172.21.15.160:0/2850290769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c065ab0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 --2- 172.21.15.160:0/2850290769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f100c102e50 0x7f100c103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 -- 172.21.15.160:0/2850290769 >> 172.21.15.160:0/2850290769 conn(0x7f100c071d00 msgr2=0x7f100c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:16.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 -- 172.21.15.160:0/2850290769 shutdown_connections 2024-05-25T06:19:16.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.415+0000 7f10146c1700 1 -- 172.21.15.160:0/2850290769 wait complete. 2024-05-25T06:19:16.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f10146c1700 1 Processor -- start 2024-05-25T06:19:16.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f10146c1700 1 -- start start 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f10146c1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f10146c1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f100c102e50 0x7f100c1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f10146c1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c114720 con 0x7f100c065640 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f10146c1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c10db10 con 0x7f100c102e50 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f101245d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f101245d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57420/0 (socket says 172.21.15.160:57420) 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.416+0000 7f101245d700 1 -- 172.21.15.160:0/1342035460 learned_addr learned my addr 172.21.15.160:0/1342035460 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:16.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f1011c5c700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f100c102e50 0x7f100c1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f101245d700 1 -- 172.21.15.160:0/1342035460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f100c102e50 msgr2=0x7f100c1140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f101245d700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f100c102e50 0x7f100c1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f101245d700 1 -- 172.21.15.160:0/1342035460 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc005040 con 0x7f100c065640 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f101245d700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c113b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f100800c810 tx=0x7f1008004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1008012810 con 0x7f100c065640 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c10ddf0 con 0x7f100c065640 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1008012e40 con 0x7f100c065640 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100800a370 con 0x7f100c065640 2024-05-25T06:19:16.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.417+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c10e3b0 con 0x7f100c065640 2024-05-25T06:19:16.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.418+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c100100 con 0x7f100c065640 2024-05-25T06:19:16.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.419+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1008011810 con 0x7f100c065640 2024-05-25T06:19:16.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.419+0000 7f1002ffd700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ff806e140 0x7f0ff80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.419+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f100808dee0 con 0x7f100c065640 2024-05-25T06:19:16.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.419+0000 7f1011c5c700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ff806e140 0x7f0ff80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.420+0000 7f1011c5c700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ff806e140 0x7f0ff80705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b3b0 tx=0x7f0ffc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:16.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.424+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f100805ade0 con 0x7f100c065640 2024-05-25T06:19:16.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.630+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f100c10f1e0 con 0x7f0ff806e140 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.641+0000 7f1002ffd700 1 -- 172.21.15.160:0/1342035460 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f100c10f1e0 con 0x7f0ff806e140 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (5m) 52s ago 16m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (2m) 52s ago 16m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (2m) 18s ago 15m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (4m) 52s ago 15m 80.2M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (10m) 52s ago 10m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (10m) 52s ago 10m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:19:16.641 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (10m) 18s ago 10m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (10m) 18s ago 10m 116M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (6m) 52s ago 17m 505M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (6m) 18s ago 15m 453M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (3m) 52s ago 17m 45.1M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (3m) 18s ago 15m 38.5M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (5m) 52s ago 16m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (5m) 18s ago 15m 16.8M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (2m) 52s ago 14m 96.8M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (93s) 52s ago 14m 98.9M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (58s) 52s ago 13m 12.3M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (22s) 18s ago 13m 12.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (12m) 18s ago 12m 231M 2581M 17.2.7-958-g7b01852c baf1547092e7 c42942c7475b 2024-05-25T06:19:16.642 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (12m) 18s ago 12m 232M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:19:16.643 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (5m) 52s ago 15m 67.5M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:19:16.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ff806e140 msgr2=0x7f0ff80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ff806e140 0x7f0ff80705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b3b0 tx=0x7f0ffc009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:16.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 msgr2=0x7f100c113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c113b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f100800c810 tx=0x7f1008004470 comp rx=0 tx=0).stop 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 shutdown_connections 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f100c065640 0x7f100c113b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ff806e140 0x7f0ff80705f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 --2- 172.21.15.160:0/1342035460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f100c102e50 0x7f100c1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 >> 172.21.15.160:0/1342035460 conn(0x7f100c071d00 msgr2=0x7f100c0faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 shutdown_connections 2024-05-25T06:19:16.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.643+0000 7f10146c1700 1 -- 172.21.15.160:0/1342035460 wait complete. 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 -- 172.21.15.160:0/2636158935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 msgr2=0x7fa7a41090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/2636158935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a41090f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa794009d10 tx=0x7fa794004d10 comp rx=0 tx=0).stop 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 -- 172.21.15.160:0/2636158935 shutdown_connections 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/2636158935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a41090f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/2636158935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7a4070300 0x7fa7a40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 -- 172.21.15.160:0/2636158935 >> 172.21.15.160:0/2636158935 conn(0x7fa7a40f8950 msgr2=0x7fa7a40fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 -- 172.21.15.160:0/2636158935 shutdown_connections 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.746+0000 7fa7aac7f700 1 -- 172.21.15.160:0/2636158935 wait complete. 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7aac7f700 1 Processor -- start 2024-05-25T06:19:16.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7aac7f700 1 -- start start 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7aac7f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7a4070300 0x7fa7a4115f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7aac7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a4116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7aac7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4116a90 con 0x7fa7a4070c10 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7aac7f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a410fe60 con 0x7fa7a4070300 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a4116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a4116440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57444/0 (socket says 172.21.15.160:57444) 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a3fff700 1 -- 172.21.15.160:0/1893946221 learned_addr learned my addr 172.21.15.160:0/1893946221 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a8a1b700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7a4070300 0x7fa7a4115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a3fff700 1 -- 172.21.15.160:0/1893946221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7a4070300 msgr2=0x7fa7a4115f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a3fff700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7a4070300 0x7fa7a4115f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.747+0000 7fa7a3fff700 1 -- 172.21.15.160:0/1893946221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794009940 con 0x7fa7a4070c10 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.748+0000 7fa7a3fff700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a4116440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa794008df0 tx=0x7fa794004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.748+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79400e040 con 0x7fa7a4070c10 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.748+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa794004610 con 0x7fa7a4070c10 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.748+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a41100e0 con 0x7fa7a4070c10 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.748+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79401c7e0 con 0x7fa7a4070c10 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.748+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a4110620 con 0x7fa7a4070c10 2024-05-25T06:19:16.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.749+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4071a40 con 0x7fa7a4070c10 2024-05-25T06:19:16.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.749+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa794012410 con 0x7fa7a4070c10 2024-05-25T06:19:16.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.750+0000 7fa7a17fa700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa79006e140 0x7fa7900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:16.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.750+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa79408f7e0 con 0x7fa7a4070c10 2024-05-25T06:19:16.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.750+0000 7fa7a8a1b700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa79006e140 0x7fa7900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:16.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.750+0000 7fa7a8a1b700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa79006e140 0x7fa7900705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa79c007fd0 tx=0x7fa79c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:16.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:16.755+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa79405c6e0 con 0x7fa7a4070c10 2024-05-25T06:19:17.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.036+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa7a404bed0 con 0x7fa7a4070c10 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.037+0000 7fa7a17fa700 1 -- 172.21.15.160:0/1893946221 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fa79405c270 con 0x7fa7a4070c10 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:17.037 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 2, 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 6, 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 8 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:19:17.038 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:19:17.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa79006e140 msgr2=0x7fa7900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa79006e140 0x7fa7900705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa79c007fd0 tx=0x7fa79c009040 comp rx=0 tx=0).stop 2024-05-25T06:19:17.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 msgr2=0x7fa7a4116440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a4116440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa794008df0 tx=0x7fa794004d10 comp rx=0 tx=0).stop 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 shutdown_connections 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa79006e140 0x7fa7900705f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7a4070300 0x7fa7a4115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 --2- 172.21.15.160:0/1893946221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7a4070c10 0x7fa7a4116440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.039+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 >> 172.21.15.160:0/1893946221 conn(0x7fa7a40f8950 msgr2=0x7fa7a40f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.040+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 shutdown_connections 2024-05-25T06:19:17.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.040+0000 7fa7aac7f700 1 -- 172.21.15.160:0/1893946221 wait complete. 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 -- 172.21.15.160:0/849198039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 msgr2=0x7f562c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 --2- 172.21.15.160:0/849198039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c0fbd20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5614004940 tx=0x7f5614009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 -- 172.21.15.160:0/849198039 shutdown_connections 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 --2- 172.21.15.160:0/849198039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c0fbd20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 --2- 172.21.15.160:0/849198039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f562c101d10 0x7f562c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 -- 172.21.15.160:0/849198039 >> 172.21.15.160:0/849198039 conn(0x7f562c0f7860 msgr2=0x7f562c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 -- 172.21.15.160:0/849198039 shutdown_connections 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.143+0000 7f5630bfa700 1 -- 172.21.15.160:0/849198039 wait complete. 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5630bfa700 1 Processor -- start 2024-05-25T06:19:17.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5630bfa700 1 -- start start 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5630bfa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f562c101d10 0x7f562c10eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5630bfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c10f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5630bfa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c10f540 con 0x7f562c1026b0 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5630bfa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c10f680 con 0x7f562c101d10 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c10f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c10f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57466/0 (socket says 172.21.15.160:57466) 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 -- 172.21.15.160:0/713723628 learned_addr learned my addr 172.21.15.160:0/713723628 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f562a59c700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f562c101d10 0x7f562c10eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 -- 172.21.15.160:0/713723628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f562c101d10 msgr2=0x7f562c10eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f562c101d10 0x7f562c10eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 -- 172.21.15.160:0/713723628 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614005040 con 0x7f562c1026b0 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.144+0000 7f5623fff700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c10f000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5614000c00 tx=0x7f5614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:17.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.145+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561400e070 con 0x7f562c1026b0 2024-05-25T06:19:17.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.145+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c10f900 con 0x7f562c1026b0 2024-05-25T06:19:17.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.145+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5614013c00 con 0x7f562c1026b0 2024-05-25T06:19:17.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.145+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561400b650 con 0x7f562c1026b0 2024-05-25T06:19:17.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.145+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c117e00 con 0x7f562c1026b0 2024-05-25T06:19:17.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.146+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c0fd460 con 0x7f562c1026b0 2024-05-25T06:19:17.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.146+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f561401d9c0 con 0x7f562c1026b0 2024-05-25T06:19:17.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.147+0000 7f5622ffd700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f561806e140 0x7f56180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.147+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f561408ed40 con 0x7f562c1026b0 2024-05-25T06:19:17.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.147+0000 7f562a59c700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f561806e140 0x7f56180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.147+0000 7f562a59c700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f561806e140 0x7f56180705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f561c009b80 tx=0x7f561c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:17.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.151+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5614058240 con 0x7f562c1026b0 2024-05-25T06:19:17.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.393+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f562c117ff0 con 0x7f562c1026b0 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.394+0000 7f5622ffd700 1 -- 172.21.15.160:0/713723628 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7f5614022070 con 0x7f562c1026b0 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:19:17.394 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:19:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:19:17.396 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:19:17.397 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f561806e140 msgr2=0x7f56180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f561806e140 0x7f56180705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f561c009b80 tx=0x7f561c00b040 comp rx=0 tx=0).stop 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 msgr2=0x7f562c10f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c10f000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5614000c00 tx=0x7f5614009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 shutdown_connections 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f561806e140 0x7f56180705f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f562c101d10 0x7f562c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 --2- 172.21.15.160:0/713723628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f562c1026b0 0x7f562c10f000 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 >> 172.21.15.160:0/713723628 conn(0x7f562c0f7860 msgr2=0x7f562c0f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:17.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 shutdown_connections 2024-05-25T06:19:17.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.396+0000 7f5630bfa700 1 -- 172.21.15.160:0/713723628 wait complete. 2024-05-25T06:19:17.399 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.499+0000 7f0dca670700 1 -- 172.21.15.160:0/2047535931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc40fb520 msgr2=0x7f0dc40fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.499+0000 7f0dca670700 1 --2- 172.21.15.160:0/2047535931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc40fb520 0x7f0dc40fb990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0db8004940 tx=0x7f0db8009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 -- 172.21.15.160:0/2047535931 shutdown_connections 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 --2- 172.21.15.160:0/2047535931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc40fb520 0x7f0dc40fb990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 --2- 172.21.15.160:0/2047535931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4102f60 0x7f0dc4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 -- 172.21.15.160:0/2047535931 >> 172.21.15.160:0/2047535931 conn(0x7f0dc40714d0 msgr2=0x7f0dc40718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 -- 172.21.15.160:0/2047535931 shutdown_connections 2024-05-25T06:19:17.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 -- 172.21.15.160:0/2047535931 wait complete. 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 Processor -- start 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.500+0000 7f0dca670700 1 -- start start 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dca670700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc40fb520 0x7f0dc4113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dca670700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 0x7f0dc4113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dca670700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4114640 con 0x7f0dc4102f60 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dca670700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc410db90 con 0x7f0dc40fb520 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 0x7f0dc4113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 0x7f0dc4113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57488/0 (socket says 172.21.15.160:57488) 2024-05-25T06:19:17.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 -- 172.21.15.160:0/1202429930 learned_addr learned my addr 172.21.15.160:0/1202429930 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc3fff700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc40fb520 0x7f0dc4113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 -- 172.21.15.160:0/1202429930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc40fb520 msgr2=0x7f0dc4113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc40fb520 0x7f0dc4113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 -- 172.21.15.160:0/1202429930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8005040 con 0x7f0dc4102f60 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc3fff700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc40fb520 0x7f0dc4113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc37fe700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 0x7f0dc4113f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b3b0 tx=0x7f0db8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db800e040 con 0x7f0dc4102f60 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0db800bbc0 con 0x7f0dc4102f60 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc410ddb0 con 0x7f0dc4102f60 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.501+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db801c3f0 con 0x7f0dc4102f60 2024-05-25T06:19:17.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.502+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc410e2f0 con 0x7f0dc4102f60 2024-05-25T06:19:17.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.502+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc4100250 con 0x7f0dc4102f60 2024-05-25T06:19:17.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.503+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0db800bd30 con 0x7f0dc4102f60 2024-05-25T06:19:17.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.503+0000 7f0dc0ff9700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.504+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0db808ed60 con 0x7f0dc4102f60 2024-05-25T06:19:17.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.504+0000 7f0dc3fff700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.504+0000 7f0dc3fff700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009b10 tx=0x7f0db4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:17.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.508+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0db805bc60 con 0x7f0dc4102f60 2024-05-25T06:19:17.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.723+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dc410f0e0 con 0x7f0dac06e140 2024-05-25T06:19:17.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.724+0000 7f0dc0ff9700 1 -- 172.21.15.160:0/1202429930 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0dc410f0e0 con 0x7f0dac06e140 2024-05-25T06:19:17.724 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:19:17.724 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:19:17.724 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "10/21 daemons upgraded", 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading osd daemons", 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:19:17.725 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:19:17.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 msgr2=0x7f0dac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009b10 tx=0x7f0db4008040 comp rx=0 tx=0).stop 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 msgr2=0x7f0dc4113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 0x7f0dc4113f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b3b0 tx=0x7f0db8009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 shutdown_connections 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc40fb520 0x7f0dc4113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 --2- 172.21.15.160:0/1202429930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102f60 0x7f0dc4113f60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 >> 172.21.15.160:0/1202429930 conn(0x7f0dc40714d0 msgr2=0x7f0dc40fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 shutdown_connections 2024-05-25T06:19:17.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.727+0000 7f0dca670700 1 -- 172.21.15.160:0/1202429930 wait complete. 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 -- 172.21.15.160:0/1836234949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c102670 msgr2=0x7fd70c0fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 --2- 172.21.15.160:0/1836234949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c102670 0x7fd70c0fb600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd700004940 tx=0x7fd700009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 -- 172.21.15.160:0/1836234949 shutdown_connections 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 --2- 172.21.15.160:0/1836234949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c102670 0x7fd70c0fb600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 --2- 172.21.15.160:0/1836234949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd70c101cd0 0x7fd70c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 -- 172.21.15.160:0/1836234949 >> 172.21.15.160:0/1836234949 conn(0x7fd70c071460 msgr2=0x7fd70c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 -- 172.21.15.160:0/1836234949 shutdown_connections 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.837+0000 7fd7137d7700 1 -- 172.21.15.160:0/1836234949 wait complete. 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd7137d7700 1 Processor -- start 2024-05-25T06:19:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd7137d7700 1 -- start start 2024-05-25T06:19:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd7137d7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 0x7fd70c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd7137d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd70c102670 0x7fd70c1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd7137d7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c114700 con 0x7fd70c101cd0 2024-05-25T06:19:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd7137d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c10daf0 con 0x7fd70c102670 2024-05-25T06:19:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd711573700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 0x7fd70c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd711573700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 0x7fd70c113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57498/0 (socket says 172.21.15.160:57498) 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd711573700 1 -- 172.21.15.160:0/3669775076 learned_addr learned my addr 172.21.15.160:0/3669775076 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd710d72700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd70c102670 0x7fd70c1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd711573700 1 -- 172.21.15.160:0/3669775076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd70c102670 msgr2=0x7fd70c1140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd711573700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd70c102670 0x7fd70c1140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.838+0000 7fd711573700 1 -- 172.21.15.160:0/3669775076 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700005040 con 0x7fd70c101cd0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.839+0000 7fd711573700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 0x7fd70c113b70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd704001d80 tx=0x7fd704009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.839+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd704007d10 con 0x7fd70c101cd0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.839+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd704001e80 con 0x7fd70c101cd0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.839+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c10ddd0 con 0x7fd70c101cd0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.839+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd704014640 con 0x7fd70c101cd0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.839+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c10e360 con 0x7fd70c101cd0 2024-05-25T06:19:17.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.840+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70c065200 con 0x7fd70c101cd0 2024-05-25T06:19:17.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.840+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd70400e960 con 0x7fd70c101cd0 2024-05-25T06:19:17.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.841+0000 7fd709ffb700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd6f406e140 0x7fd6f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:17.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.841+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd70408d850 con 0x7fd70c101cd0 2024-05-25T06:19:17.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.841+0000 7fd710d72700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd6f406e140 0x7fd6f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:17.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.841+0000 7fd710d72700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd6f406e140 0x7fd6f40705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd70000b3b0 tx=0x7fd700009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:17.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:17.846+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd70405a800 con 0x7fd70c101cd0 2024-05-25T06:19:18.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.131+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd70c109ed0 con 0x7fd70c101cd0 2024-05-25T06:19:18.131 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:18.131 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:18.131 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:18.131 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: pgmap v130: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:18.132 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1893946221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:18.132 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/713723628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:19:18.132 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:18.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.131+0000 7fd709ffb700 1 -- 172.21.15.160:0/3669775076 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd7040097e0 con 0x7fd70c101cd0 2024-05-25T06:19:18.132 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd6f406e140 msgr2=0x7fd6f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd6f406e140 0x7fd6f40705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd70000b3b0 tx=0x7fd700009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 msgr2=0x7fd70c113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 0x7fd70c113b70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd704001d80 tx=0x7fd704009ec0 comp rx=0 tx=0).stop 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 shutdown_connections 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd70c101cd0 0x7fd70c113b70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd6f406e140 0x7fd6f40705f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 --2- 172.21.15.160:0/3669775076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd70c102670 0x7fd70c1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 >> 172.21.15.160:0/3669775076 conn(0x7fd70c071460 msgr2=0x7fd70c1009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 shutdown_connections 2024-05-25T06:19:18.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:18.134+0000 7fd7137d7700 1 -- 172.21.15.160:0/3669775076 wait complete. 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: pgmap v130: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1893946221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/713723628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:19:18.355 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:19.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:18 smithi160 ceph-mon[112094]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:19.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:18 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:19.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:18 smithi160 ceph-mon[112094]: Upgrade: osd.4 is safe to restart 2024-05-25T06:19:19.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:19.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:18 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3669775076' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:19:19.267 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:18 smithi203 ceph-mon[83507]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:19.268 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:18 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-25T06:19:19.268 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:18 smithi203 ceph-mon[83507]: Upgrade: osd.4 is safe to restart 2024-05-25T06:19:19.268 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:19.268 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:18 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3669775076' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:19:20.163 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:19 smithi203 ceph-mon[83507]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:20.163 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:20.163 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-25T06:19:20.163 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:19 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:19 smithi160 ceph-mon[112094]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:20.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-25T06:19:20.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:19 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:20 smithi160 ceph-mon[112094]: Upgrade: Updating osd.4 2024-05-25T06:19:21.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:20 smithi160 ceph-mon[112094]: Deploying daemon osd.4 on smithi203 2024-05-25T06:19:21.277 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:20 smithi203 ceph-mon[83507]: Upgrade: Updating osd.4 2024-05-25T06:19:21.277 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:20 smithi203 ceph-mon[83507]: Deploying daemon osd.4 on smithi203 2024-05-25T06:19:22.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:21 smithi160 ceph-mon[112094]: pgmap v132: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:19:22.293 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:21 smithi203 ceph-mon[83507]: pgmap v132: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:19:22.899 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:22 smithi203 systemd[1]: Stopping Ceph osd.4 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:19:23.399 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:22 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[48383]: 2024-05-25T06:19:22.915+0000 7f473b7f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:19:23.400 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:22 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[48383]: 2024-05-25T06:19:22.915+0000 7f473b7f7700 -1 osd.4 69 *** Got signal Terminated *** 2024-05-25T06:19:23.400 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:22 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[48383]: 2024-05-25T06:19:22.915+0000 7f473b7f7700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:19:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:23 smithi203 ceph-mon[83507]: pgmap v133: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:19:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:23 smithi203 ceph-mon[83507]: osd.4 marked itself down and dead 2024-05-25T06:19:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:23 smithi160 ceph-mon[112094]: pgmap v133: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:19:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:23 smithi160 ceph-mon[112094]: osd.4 marked itself down and dead 2024-05-25T06:19:24.540 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:24 smithi203 podman[97602]: 2024-05-25 06:19:24.26672726 +0000 UTC m=+1.460134092 container died c42942c7475bf0965f9f0fbd3b96ee99e39c379a44a87202fe567d95ce33ba47 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=quincy-7b01852, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-25T06:19:24.899 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:24 smithi203 podman[97602]: 2024-05-25 06:19:24.640549189 +0000 UTC m=+1.833956023 container cleanup c42942c7475bf0965f9f0fbd3b96ee99e39c379a44a87202fe567d95ce33ba47 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, RELEASE=quincy-7b01852, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-25T06:19:24.899 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:24 smithi203 bash[97602]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4 2024-05-25T06:19:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:24 smithi160 ceph-mon[112094]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:19:25.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:24 smithi160 ceph-mon[112094]: osdmap e70: 6 total, 5 up, 6 in 2024-05-25T06:19:25.264 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:24 smithi203 ceph-mon[83507]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:19:25.264 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:24 smithi203 ceph-mon[83507]: osdmap e70: 6 total, 5 up, 6 in 2024-05-25T06:19:25.264 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:24 smithi203 podman[97619]: 2024-05-25 06:19:24.900228748 +0000 UTC m=+0.630427495 container remove c42942c7475bf0965f9f0fbd3b96ee99e39c379a44a87202fe567d95ce33ba47 (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=quincy-7b01852, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-25T06:19:25.546 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:25 smithi203 podman[97709]: 2024-05-25 06:19:25.167578024 +0000 UTC m=+0.017326270 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:19:25.546 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:25 smithi203 podman[97709]: 2024-05-25 06:19:25.301653693 +0000 UTC m=+0.151401917 container create 47a9e003230a5e61e1611f713bca29f3d2645bb8a3171b9ade1887e3ad9fd0f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:19:25.546 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:25 smithi203 podman[97709]: 2024-05-25 06:19:25.545977332 +0000 UTC m=+0.395725580 container init 47a9e003230a5e61e1611f713bca29f3d2645bb8a3171b9ade1887e3ad9fd0f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-25T06:19:25.842 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:25 smithi203 podman[97709]: 2024-05-25 06:19:25.550692606 +0000 UTC m=+0.400440839 container start 47a9e003230a5e61e1611f713bca29f3d2645bb8a3171b9ade1887e3ad9fd0f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:19:25.843 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:25 smithi203 podman[97709]: 2024-05-25 06:19:25.646084609 +0000 UTC m=+0.495832841 container attach 47a9e003230a5e61e1611f713bca29f3d2645bb8a3171b9ade1887e3ad9fd0f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-25T06:19:26.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:25 smithi203 ceph-mon[83507]: pgmap v135: 65 pgs: 10 peering, 4 stale+active+clean, 51 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:26.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:25 smithi203 ceph-mon[83507]: osdmap e71: 6 total, 5 up, 6 in 2024-05-25T06:19:26.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:25 smithi203 ceph-mon[83507]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-25T06:19:26.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:25 smithi203 podman[97709]: 2024-05-25 06:19:25.843549222 +0000 UTC m=+0.693297463 container died 47a9e003230a5e61e1611f713bca29f3d2645bb8a3171b9ade1887e3ad9fd0f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:19:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:25 smithi160 ceph-mon[112094]: pgmap v135: 65 pgs: 10 peering, 4 stale+active+clean, 51 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:25 smithi160 ceph-mon[112094]: osdmap e71: 6 total, 5 up, 6 in 2024-05-25T06:19:26.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:25 smithi160 ceph-mon[112094]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-25T06:19:26.872 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 podman[97735]: 2024-05-25 06:19:26.434197829 +0000 UTC m=+0.583908836 container remove 47a9e003230a5e61e1611f713bca29f3d2645bb8a3171b9ade1887e3ad9fd0f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD) 2024-05-25T06:19:26.872 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-25T06:19:26.872 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 systemd[1]: Stopped Ceph osd.4 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:19:26.872 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.4.service: Consumed 1min 4.050s CPU time. 2024-05-25T06:19:26.872 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 systemd[1]: Starting Ceph osd.4 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:19:27.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:26 smithi203 ceph-mon[83507]: Health check failed: Degraded data redundancy: 25/324 objects degraded (7.716%), 6 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:27.150 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 podman[97832]: 2024-05-25 06:19:26.775130221 +0000 UTC m=+0.017497042 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:19:27.150 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:26 smithi203 podman[97832]: 2024-05-25 06:19:26.902246153 +0000 UTC m=+0.144612954 container create 55651b5a89b62d65bab8ed4e723e254115fa6996d524b48fc16eec1e981b202d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:19:27.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:26 smithi160 ceph-mon[112094]: Health check failed: Degraded data redundancy: 25/324 objects degraded (7.716%), 6 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:27.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 podman[97832]: 2024-05-25 06:19:27.188709576 +0000 UTC m=+0.431076402 container init 55651b5a89b62d65bab8ed4e723e254115fa6996d524b48fc16eec1e981b202d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD) 2024-05-25T06:19:27.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 podman[97832]: 2024-05-25 06:19:27.192617162 +0000 UTC m=+0.434983978 container start 55651b5a89b62d65bab8ed4e723e254115fa6996d524b48fc16eec1e981b202d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-25T06:19:27.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 podman[97832]: 2024-05-25 06:19:27.280134828 +0000 UTC m=+0.522501645 container attach 55651b5a89b62d65bab8ed4e723e254115fa6996d524b48fc16eec1e981b202d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-25T06:19:27.963 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 bash[97832]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate[97847]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-25T06:19:27.964 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:27 smithi203 podman[97832]: 2024-05-25 06:19:27.676846049 +0000 UTC m=+0.919212858 container died 55651b5a89b62d65bab8ed4e723e254115fa6996d524b48fc16eec1e981b202d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-25T06:19:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:27 smithi160 ceph-mon[112094]: pgmap v137: 65 pgs: 6 active+undersized+degraded, 10 peering, 3 stale+active+clean, 5 active+undersized, 41 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25/324 objects degraded (7.716%) 2024-05-25T06:19:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:28.326 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:27 smithi203 ceph-mon[83507]: pgmap v137: 65 pgs: 6 active+undersized+degraded, 10 peering, 3 stale+active+clean, 5 active+undersized, 41 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25/324 objects degraded (7.716%) 2024-05-25T06:19:28.326 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:28.326 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:28.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 podman[97981]: 2024-05-25 06:19:28.326567981 +0000 UTC m=+0.644040110 container remove 55651b5a89b62d65bab8ed4e723e254115fa6996d524b48fc16eec1e981b202d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-25T06:19:28.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 podman[98042]: 2024-05-25 06:19:28.44546263 +0000 UTC m=+0.017426387 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:19:28.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 podman[98042]: 2024-05-25 06:19:28.561020942 +0000 UTC m=+0.132984687 container create 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-25T06:19:29.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 podman[98042]: 2024-05-25 06:19:28.805344786 +0000 UTC m=+0.377308530 container init 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:19:29.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 podman[98042]: 2024-05-25 06:19:28.808630783 +0000 UTC m=+0.380594528 container start 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-25T06:19:29.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 ceph-osd[98061]: -- 172.21.15.203:0/2573602985 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b91981b680 con 0x55b9197e5000 2024-05-25T06:19:29.150 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 bash[98042]: 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac 2024-05-25T06:19:29.150 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:28 smithi203 systemd[1]: Started Ceph osd.4 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:19:29.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:29 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:19:29.353+0000 7f59eff657c0 -1 Falling back to public interface 2024-05-25T06:19:30.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:29 smithi203 ceph-mon[83507]: pgmap v138: 65 pgs: 12 active+undersized+degraded, 10 peering, 12 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-05-25T06:19:30.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:30.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:30.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:29 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:19:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:29 smithi160 ceph-mon[112094]: pgmap v138: 65 pgs: 12 active+undersized+degraded, 10 peering, 12 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-05-25T06:19:30.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:30.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:30.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:29 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:19:31.348 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:31 smithi203 ceph-mon[83507]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-05-25T06:19:31.348 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:31 smithi203 ceph-mon[83507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-25T06:19:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:31 smithi160 ceph-mon[112094]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-05-25T06:19:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:31 smithi160 ceph-mon[112094]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-25T06:19:33.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:33.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:32 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:33.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:33.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:32 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:33 smithi203 ceph-mon[83507]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:33.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:33 smithi203 ceph-mon[83507]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-05-25T06:19:33.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:33 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:33.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:33 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:34.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:33 smithi160 ceph-mon[112094]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:34.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:33 smithi160 ceph-mon[112094]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-05-25T06:19:34.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:33 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:34.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:33 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:34.812 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:34 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:19:34.421+0000 7f59eff657c0 -1 osd.4 69 log_to_monitors true 2024-05-25T06:19:35.064 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:19:34 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:19:34.891+0000 7f59e13bc700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:19:35.064 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:34 smithi203 ceph-mon[83507]: from='osd.4 [v2:172.21.15.203:6808/2420151889,v1:172.21.15.203:6809/2420151889]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:19:35.065 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:34 smithi203 ceph-mon[83507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:19:35.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:34 smithi160 ceph-mon[112094]: from='osd.4 [v2:172.21.15.203:6808/2420151889,v1:172.21.15.203:6809/2420151889]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:19:35.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:34 smithi160 ceph-mon[112094]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-25T06:19:36.134 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:35 smithi203 ceph-mon[83507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-25T06:19:36.134 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:35 smithi203 ceph-mon[83507]: osdmap e72: 6 total, 5 up, 6 in 2024-05-25T06:19:36.134 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:35 smithi203 ceph-mon[83507]: from='osd.4 [v2:172.21.15.203:6808/2420151889,v1:172.21.15.203:6809/2420151889]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:36.134 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:35 smithi203 ceph-mon[83507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:36.134 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:35 smithi203 ceph-mon[83507]: pgmap v142: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-05-25T06:19:36.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:35 smithi160 ceph-mon[112094]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-25T06:19:36.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:35 smithi160 ceph-mon[112094]: osdmap e72: 6 total, 5 up, 6 in 2024-05-25T06:19:36.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:35 smithi160 ceph-mon[112094]: from='osd.4 [v2:172.21.15.203:6808/2420151889,v1:172.21.15.203:6809/2420151889]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:36.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:35 smithi160 ceph-mon[112094]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:19:36.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:35 smithi160 ceph-mon[112094]: pgmap v142: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: osd.4 [v2:172.21.15.203:6808/2420151889,v1:172.21.15.203:6809/2420151889] boot 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: osdmap e73: 6 total, 6 up, 6 in 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:36.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:19:36.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:36.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:19:36.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:36.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:36.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:36.971 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:36 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: osd.4 [v2:172.21.15.203:6808/2420151889,v1:172.21.15.203:6809/2420151889] boot 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: osdmap e73: 6 total, 6 up, 6 in 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:37.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:19:37.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:37.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:37.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:37.151 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:36 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:37 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:37 smithi203 ceph-mon[83507]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-25T06:19:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:37 smithi203 ceph-mon[83507]: osdmap e74: 6 total, 6 up, 6 in 2024-05-25T06:19:38.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:37 smithi203 ceph-mon[83507]: pgmap v145: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 31/324 objects degraded (9.568%) 2024-05-25T06:19:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:37 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:38.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:37 smithi160 ceph-mon[112094]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-25T06:19:38.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:37 smithi160 ceph-mon[112094]: osdmap e74: 6 total, 6 up, 6 in 2024-05-25T06:19:38.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:37 smithi160 ceph-mon[112094]: pgmap v145: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 31/324 objects degraded (9.568%) 2024-05-25T06:19:39.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:38 smithi203 ceph-mon[83507]: Health check update: Degraded data redundancy: 31/324 objects degraded (9.568%), 11 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:39.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:38 smithi160 ceph-mon[112094]: Health check update: Degraded data redundancy: 31/324 objects degraded (9.568%), 11 pgs degraded (PG_DEGRADED) 2024-05-25T06:19:40.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:39 smithi160 ceph-mon[112094]: pgmap v146: 65 pgs: 10 peering, 7 active+undersized+degraded, 7 active+undersized, 41 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s; 17/324 objects degraded (5.247%) 2024-05-25T06:19:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:39 smithi203 ceph-mon[83507]: pgmap v146: 65 pgs: 10 peering, 7 active+undersized+degraded, 7 active+undersized, 41 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s; 17/324 objects degraded (5.247%) 2024-05-25T06:19:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:41 smithi203 ceph-mon[83507]: pgmap v147: 65 pgs: 10 peering, 55 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-25T06:19:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:41 smithi203 ceph-mon[83507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/324 objects degraded (5.247%), 7 pgs degraded) 2024-05-25T06:19:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:41 smithi203 ceph-mon[83507]: Cluster is now healthy 2024-05-25T06:19:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:41 smithi160 ceph-mon[112094]: pgmap v147: 65 pgs: 10 peering, 55 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-25T06:19:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:41 smithi160 ceph-mon[112094]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/324 objects degraded (5.247%), 7 pgs degraded) 2024-05-25T06:19:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:41 smithi160 ceph-mon[112094]: Cluster is now healthy 2024-05-25T06:19:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:43 smithi160 ceph-mon[112094]: pgmap v148: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-25T06:19:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:44.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:43 smithi203 ceph-mon[83507]: pgmap v148: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-25T06:19:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:45 smithi160 ceph-mon[112094]: pgmap v149: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-05-25T06:19:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:45 smithi203 ceph-mon[83507]: pgmap v149: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-05-25T06:19:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:47 smithi160 ceph-mon[112094]: pgmap v150: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 -- 172.21.15.160:0/1628538373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738102de0 msgr2=0x7fa7381031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1628538373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738102de0 0x7fa7381031b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa734004940 tx=0x7fa734009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 -- 172.21.15.160:0/1628538373 shutdown_connections 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1628538373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa738103780 0x7fa7380fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1628538373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738102de0 0x7fa7381031b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 -- 172.21.15.160:0/1628538373 >> 172.21.15.160:0/1628538373 conn(0x7fa7380f8990 msgr2=0x7fa7380fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 -- 172.21.15.160:0/1628538373 shutdown_connections 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 -- 172.21.15.160:0/1628538373 wait complete. 2024-05-25T06:19:48.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 Processor -- start 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.252+0000 7fa7407d4700 1 -- start start 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa7407d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa738102de0 0x7fa738070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa7407d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 0x7fa7380707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa7407d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738072160 con 0x7fa738103780 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa7407d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738070ce0 con 0x7fa738102de0 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 0x7fa7380707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 0x7fa7380707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52368/0 (socket says 172.21.15.160:52368) 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 -- 172.21.15.160:0/1034200168 learned_addr learned my addr 172.21.15.160:0/1034200168 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73e570700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa738102de0 0x7fa738070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 -- 172.21.15.160:0/1034200168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa738102de0 msgr2=0x7fa738070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa738102de0 0x7fa738070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 -- 172.21.15.160:0/1034200168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa734005040 con 0x7fa738103780 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa73dd6f700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 0x7fa7380707a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa728002a50 tx=0x7fa728004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa728007cc0 con 0x7fa738103780 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa72800f040 con 0x7fa738103780 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa738070fc0 con 0x7fa738103780 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.253+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa72800d9c0 con 0x7fa738103780 2024-05-25T06:19:48.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.254+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73811a160 con 0x7fa738103780 2024-05-25T06:19:48.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.254+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7381000f0 con 0x7fa738103780 2024-05-25T06:19:48.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.255+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa728009730 con 0x7fa738103780 2024-05-25T06:19:48.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.255+0000 7fa72effd700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa72406e140 0x7fa7240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.256+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa72808d730 con 0x7fa738103780 2024-05-25T06:19:48.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.256+0000 7fa73e570700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa72406e140 0x7fa7240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.256+0000 7fa73e570700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa72406e140 0x7fa7240705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa73400b3b0 tx=0x7fa734009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:48.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.260+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa728056b80 con 0x7fa738103780 2024-05-25T06:19:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:47 smithi203 ceph-mon[83507]: pgmap v150: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-05-25T06:19:48.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.475+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa738071760 con 0x7fa72406e140 2024-05-25T06:19:48.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.477+0000 7fa72effd700 1 -- 172.21.15.160:0/1034200168 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa738071760 con 0x7fa72406e140 2024-05-25T06:19:48.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa72406e140 msgr2=0x7fa7240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa72406e140 0x7fa7240705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa73400b3b0 tx=0x7fa734009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:48.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 msgr2=0x7fa7380707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 0x7fa7380707a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa728002a50 tx=0x7fa728004600 comp rx=0 tx=0).stop 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 shutdown_connections 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa72406e140 0x7fa7240705f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa738102de0 0x7fa738070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 --2- 172.21.15.160:0/1034200168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa738103780 0x7fa7380707a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 >> 172.21.15.160:0/1034200168 conn(0x7fa7380f8990 msgr2=0x7fa738107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 shutdown_connections 2024-05-25T06:19:48.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.479+0000 7fa7407d4700 1 -- 172.21.15.160:0/1034200168 wait complete. 2024-05-25T06:19:48.489 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:19:48.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 -- 172.21.15.160:0/3290226969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 msgr2=0x7f30d80fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/3290226969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d80fe1b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004940 tx=0x7f30c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 -- 172.21.15.160:0/3290226969 shutdown_connections 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/3290226969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30d80fe780 0x7f30d8106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/3290226969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d80fe1b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 -- 172.21.15.160:0/3290226969 >> 172.21.15.160:0/3290226969 conn(0x7f30d80f7890 msgr2=0x7f30d80f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 -- 172.21.15.160:0/3290226969 shutdown_connections 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.582+0000 7f30dd6f4700 1 -- 172.21.15.160:0/3290226969 wait complete. 2024-05-25T06:19:48.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30dd6f4700 1 Processor -- start 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30dd6f4700 1 -- start start 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30dd6f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d8113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30dd6f4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30d80fe780 0x7f30d81140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30dd6f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8114780 con 0x7f30d80fdde0 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30dd6f4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d810dae0 con 0x7f30d80fe780 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d8113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d8113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52388/0 (socket says 172.21.15.160:52388) 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.583+0000 7f30d6ffd700 1 -- 172.21.15.160:0/2155588859 learned_addr learned my addr 172.21.15.160:0/2155588859 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30d6ffd700 1 -- 172.21.15.160:0/2155588859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30d80fe780 msgr2=0x7f30d81140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30d6ffd700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30d80fe780 0x7f30d81140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30d6ffd700 1 -- 172.21.15.160:0/2155588859 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8005040 con 0x7f30d80fdde0 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30d6ffd700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d8113b60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009380 tx=0x7f30c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:48.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c800e040 con 0x7f30d80fdde0 2024-05-25T06:19:48.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d810dd60 con 0x7f30d80fdde0 2024-05-25T06:19:48.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30c8009af0 con 0x7f30d80fdde0 2024-05-25T06:19:48.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c801c630 con 0x7f30d80fdde0 2024-05-25T06:19:48.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.584+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d810e2a0 con 0x7f30d80fdde0 2024-05-25T06:19:48.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.585+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d8101790 con 0x7f30d80fdde0 2024-05-25T06:19:48.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.586+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f30c800bbc0 con 0x7f30d80fdde0 2024-05-25T06:19:48.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.586+0000 7f30cffff700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30c406e140 0x7f30c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.587+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f30c808ecd0 con 0x7f30d80fdde0 2024-05-25T06:19:48.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.587+0000 7f30ce7fc700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30c406e140 0x7f30c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.587+0000 7f30ce7fc700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30c406e140 0x7f30c40705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:48.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.591+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30c805bcf0 con 0x7f30d80fdde0 2024-05-25T06:19:48.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.809+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d810f1b0 con 0x7f30c406e140 2024-05-25T06:19:48.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.810+0000 7f30cffff700 1 -- 172.21.15.160:0/2155588859 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f30d810f1b0 con 0x7f30c406e140 2024-05-25T06:19:48.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30c406e140 msgr2=0x7f30c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30c406e140 0x7f30c40705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006db0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 msgr2=0x7f30d8113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d8113b60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009380 tx=0x7f30c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 shutdown_connections 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80fdde0 0x7f30d8113b60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30c406e140 0x7f30c40705f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 --2- 172.21.15.160:0/2155588859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30d80fe780 0x7f30d81140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 >> 172.21.15.160:0/2155588859 conn(0x7f30d80f7890 msgr2=0x7f30d80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.812+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 shutdown_connections 2024-05-25T06:19:48.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.813+0000 7f30dd6f4700 1 -- 172.21.15.160:0/2155588859 wait complete. 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.914+0000 7f336d07f700 1 -- 172.21.15.160:0/2234987394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fe790 msgr2=0x7f3368106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.914+0000 7f336d07f700 1 --2- 172.21.15.160:0/2234987394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fe790 0x7f3368106e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f335c004940 tx=0x7f335c009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 -- 172.21.15.160:0/2234987394 shutdown_connections 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 --2- 172.21.15.160:0/2234987394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fe790 0x7f3368106e60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 --2- 172.21.15.160:0/2234987394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fddf0 0x7f33680fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 -- 172.21.15.160:0/2234987394 >> 172.21.15.160:0/2234987394 conn(0x7f33680f7870 msgr2=0x7f33680f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:48.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 -- 172.21.15.160:0/2234987394 shutdown_connections 2024-05-25T06:19:48.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 -- 172.21.15.160:0/2234987394 wait complete. 2024-05-25T06:19:48.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 Processor -- start 2024-05-25T06:19:48.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.915+0000 7f336d07f700 1 -- start start 2024-05-25T06:19:48.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f336d07f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 0x7f3368115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f336d07f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fe790 0x7f33681160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f336d07f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33681167a0 con 0x7f33680fddf0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f336d07f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336810fcf0 con 0x7f33680fe790 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 0x7f3368115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 0x7f3368115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52400/0 (socket says 172.21.15.160:52400) 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 -- 172.21.15.160:0/3556169401 learned_addr learned my addr 172.21.15.160:0/3556169401 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f336659c700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fe790 0x7f33681160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 -- 172.21.15.160:0/3556169401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fe790 msgr2=0x7f33681160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fe790 0x7f33681160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 -- 172.21.15.160:0/3556169401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c005040 con 0x7f33680fddf0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f336659c700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fe790 0x7f33681160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.916+0000 7f3366d9d700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 0x7f3368115b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f335800b790 tx=0x7f3358009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.917+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335800f770 con 0x7f33680fddf0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.917+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f335800fda0 con 0x7f33680fddf0 2024-05-25T06:19:48.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.917+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335800d310 con 0x7f33680fddf0 2024-05-25T06:19:48.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.917+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336810ff70 con 0x7f33680fddf0 2024-05-25T06:19:48.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.917+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33681104b0 con 0x7f33680fddf0 2024-05-25T06:19:48.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.918+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3368106920 con 0x7f33680fddf0 2024-05-25T06:19:48.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.918+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3358010420 con 0x7f33680fddf0 2024-05-25T06:19:48.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.919+0000 7f334f7fe700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f335006e140 0x7f33500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:48.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.919+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f335808da00 con 0x7f33680fddf0 2024-05-25T06:19:48.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.919+0000 7f336659c700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f335006e140 0x7f33500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:48.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.919+0000 7f336659c700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f335006e140 0x7f33500705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f335c000c00 tx=0x7f335c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:48.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:48.924+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f335805aa20 con 0x7f33680fddf0 2024-05-25T06:19:49.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.133+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f33681111d0 con 0x7f335006e140 2024-05-25T06:19:49.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.143+0000 7f334f7fe700 1 -- 172.21.15.160:0/3556169401 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f33681111d0 con 0x7f335006e140 2024-05-25T06:19:49.144 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:19:49.144 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (5m) 84s ago 16m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:19:49.144 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (3m) 84s ago 16m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:19:49.144 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (3m) 17s ago 16m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:19:49.144 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (5m) 84s ago 16m 80.2M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:19:49.144 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (11m) 84s ago 11m 61.1M - 17.2.7-958-g7b01852c baf1547092e7 1cf6502282a3 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (11m) 84s ago 11m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (11m) 17s ago 11m 61.3M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (11m) 17s ago 11m 116M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (7m) 84s ago 18m 505M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (6m) 17s ago 15m 453M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (4m) 84s ago 18m 45.1M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (3m) 17s ago 15m 39.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:19:49.145 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (6m) 84s ago 16m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:19:49.146 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (6m) 17s ago 15m 16.8M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:19:49.146 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (2m) 84s ago 15m 96.8M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:19:49.146 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (2m) 84s ago 14m 98.9M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:19:49.146 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (90s) 84s ago 14m 12.3M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:19:49.146 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (54s) 17s ago 13m 111M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:19:49.146 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (20s) 17s ago 13m 15.1M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:19:49.147 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (12m) 17s ago 12m 233M 2581M 17.2.7-958-g7b01852c baf1547092e7 5d9ea2c92911 2024-05-25T06:19:49.147 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (5m) 84s ago 16m 67.5M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:19:49.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f335006e140 msgr2=0x7f33500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f335006e140 0x7f33500705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f335c000c00 tx=0x7f335c009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:49.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 msgr2=0x7f3368115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 0x7f3368115b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f335800b790 tx=0x7f3358009250 comp rx=0 tx=0).stop 2024-05-25T06:19:49.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 shutdown_connections 2024-05-25T06:19:49.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33680fddf0 0x7f3368115b80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f335006e140 0x7f33500705f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 --2- 172.21.15.160:0/3556169401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33680fe790 0x7f33681160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 >> 172.21.15.160:0/3556169401 conn(0x7f33680f7870 msgr2=0x7f33680fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:49.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 shutdown_connections 2024-05-25T06:19:49.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.146+0000 7f336d07f700 1 -- 172.21.15.160:0/3556169401 wait complete. 2024-05-25T06:19:49.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:48 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:49.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 -- 172.21.15.160:0/1966582697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d10 msgr2=0x7f7b681020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 --2- 172.21.15.160:0/1966582697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d10 0x7f7b681020e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004940 tx=0x7f7b58009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:49.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 -- 172.21.15.160:0/1966582697 shutdown_connections 2024-05-25T06:19:49.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 --2- 172.21.15.160:0/1966582697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b681026b0 0x7f7b680fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 --2- 172.21.15.160:0/1966582697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d10 0x7f7b681020e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 -- 172.21.15.160:0/1966582697 >> 172.21.15.160:0/1966582697 conn(0x7f7b680f7820 msgr2=0x7f7b680f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 -- 172.21.15.160:0/1966582697 shutdown_connections 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.248+0000 7f7b700ad700 1 -- 172.21.15.160:0/1966582697 wait complete. 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b700ad700 1 Processor -- start 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b700ad700 1 -- start start 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b700ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b68101d10 0x7f7b68112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b700ad700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 0x7f7b681130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b700ad700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68113720 con 0x7f7b681026b0 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b700ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6810cd00 con 0x7f7b68101d10 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6d648700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 0x7f7b681130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6d648700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 0x7f7b681130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52422/0 (socket says 172.21.15.160:52422) 2024-05-25T06:19:49.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6d648700 1 -- 172.21.15.160:0/3938075238 learned_addr learned my addr 172.21.15.160:0/3938075238 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6de49700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b68101d10 0x7f7b68112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6d648700 1 -- 172.21.15.160:0/3938075238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b68101d10 msgr2=0x7f7b68112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6d648700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b68101d10 0x7f7b68112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.249+0000 7f7b6d648700 1 -- 172.21.15.160:0/3938075238 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58005040 con 0x7f7b681026b0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.250+0000 7f7b6d648700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 0x7f7b681130d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b64002a50 tx=0x7f7b64004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.250+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b64007cc0 con 0x7f7b681026b0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.250+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b6400f040 con 0x7f7b681026b0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.250+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6810cfe0 con 0x7f7b681026b0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.250+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6400d9c0 con 0x7f7b681026b0 2024-05-25T06:19:49.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.250+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6810d5a0 con 0x7f7b681026b0 2024-05-25T06:19:49.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.251+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b680fd460 con 0x7f7b681026b0 2024-05-25T06:19:49.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.251+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7b64009730 con 0x7f7b681026b0 2024-05-25T06:19:49.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.252+0000 7f7b5e7fc700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5406e140 0x7f7b540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.252+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7b6408d840 con 0x7f7b681026b0 2024-05-25T06:19:49.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.252+0000 7f7b6de49700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5406e140 0x7f7b540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.252+0000 7f7b6de49700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5406e140 0x7f7b540705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004f90 tx=0x7f7b58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:49.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.257+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7b6405a740 con 0x7f7b681026b0 2024-05-25T06:19:49.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:48 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:49.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.533+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b68108e40 con 0x7f7b681026b0 2024-05-25T06:19:49.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.533+0000 7f7b5e7fc700 1 -- 172.21.15.160:0/3938075238 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f7b6405a2d0 con 0x7f7b681026b0 2024-05-25T06:19:49.533 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 1, 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 5 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 4 2024-05-25T06:19:49.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:19:49.535 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:19:49.535 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 5, 2024-05-25T06:19:49.535 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 9 2024-05-25T06:19:49.535 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:19:49.535 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5406e140 msgr2=0x7f7b540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5406e140 0x7f7b540705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004f90 tx=0x7f7b58009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 msgr2=0x7f7b681130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 0x7f7b681130d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b64002a50 tx=0x7f7b64004600 comp rx=0 tx=0).stop 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 shutdown_connections 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5406e140 0x7f7b540705f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b68101d10 0x7f7b68112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 --2- 172.21.15.160:0/3938075238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b681026b0 0x7f7b681130d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 >> 172.21.15.160:0/3938075238 conn(0x7f7b680f7820 msgr2=0x7f7b680f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:49.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 shutdown_connections 2024-05-25T06:19:49.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.536+0000 7f7b700ad700 1 -- 172.21.15.160:0/3938075238 wait complete. 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 -- 172.21.15.160:0/4015752366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214103780 msgr2=0x7f52140fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 --2- 172.21.15.160:0/4015752366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214103780 0x7f52140fcdf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5208004940 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 -- 172.21.15.160:0/4015752366 shutdown_connections 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 --2- 172.21.15.160:0/4015752366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214103780 0x7f52140fcdf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 --2- 172.21.15.160:0/4015752366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5214102de0 0x7f52141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 -- 172.21.15.160:0/4015752366 >> 172.21.15.160:0/4015752366 conn(0x7f52140f89b0 msgr2=0x7f52140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:49.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 -- 172.21.15.160:0/4015752366 shutdown_connections 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.639+0000 7f521b8f4700 1 -- 172.21.15.160:0/4015752366 wait complete. 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f521b8f4700 1 Processor -- start 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f521b8f4700 1 -- start start 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f521b8f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 0x7f5214115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f521b8f4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5214103780 0x7f5214116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f521b8f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5214116950 con 0x7f5214102de0 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f521b8f4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521410fd60 con 0x7f5214103780 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 0x7f5214115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 0x7f5214115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52440/0 (socket says 172.21.15.160:52440) 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 -- 172.21.15.160:0/2839090250 learned_addr learned my addr 172.21.15.160:0/2839090250 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5218e8f700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5214103780 0x7f5214116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 -- 172.21.15.160:0/2839090250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5214103780 msgr2=0x7f5214116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5214103780 0x7f5214116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 -- 172.21.15.160:0/2839090250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208005040 con 0x7f5214102de0 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.640+0000 7f5219690700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 0x7f5214115dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f520c00ec90 tx=0x7f520c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.641+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520c007b30 con 0x7f5214102de0 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.641+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f520c00f040 con 0x7f5214102de0 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.641+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5214110040 con 0x7f5214102de0 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.641+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520c00cc80 con 0x7f5214102de0 2024-05-25T06:19:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.641+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5214110550 con 0x7f5214102de0 2024-05-25T06:19:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.642+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52140fe530 con 0x7f5214102de0 2024-05-25T06:19:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.642+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f520c007c90 con 0x7f5214102de0 2024-05-25T06:19:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.643+0000 7f5211ffb700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51fc06e140 0x7f51fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.643+0000 7f5218e8f700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51fc06e140 0x7f51fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.643+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f520c08df60 con 0x7f5214102de0 2024-05-25T06:19:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.643+0000 7f5218e8f700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51fc06e140 0x7f51fc0705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5208004890 tx=0x7f5208009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:49.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.647+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f520c0c2850 con 0x7f5214102de0 2024-05-25T06:19:49.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.887+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f521404bed0 con 0x7f5214102de0 2024-05-25T06:19:49.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.887+0000 7f5211ffb700 1 -- 172.21.15.160:0/2839090250 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1923 (secure 0 0 0) 0x7f520c057330 con 0x7f5214102de0 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:e26 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 26 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:15:56.474429+0000 2024-05-25T06:19:49.888 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 46 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=14480,1=24271} 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:19:49.889 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 14480 members: 24271,14480 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.160:6828/3783773637,v1:172.21.15.160:6829/3783773637] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/2201510143,v1:172.21.15.160:6827/2201510143] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{1:24271} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.203:6824/3325193142,v1:172.21.15.203:6825/3325193142] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:24283} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.203:6826/1913747634,v1:172.21.15.203:6827/1913747634] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:19:49.890 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:19:49.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51fc06e140 msgr2=0x7f51fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51fc06e140 0x7f51fc0705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5208004890 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:49.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 msgr2=0x7f5214115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 0x7f5214115dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f520c00ec90 tx=0x7f520c00b040 comp rx=0 tx=0).stop 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 shutdown_connections 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5214102de0 0x7f5214115dc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51fc06e140 0x7f51fc0705f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.890+0000 7f521b8f4700 1 --2- 172.21.15.160:0/2839090250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5214103780 0x7f5214116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.891+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 >> 172.21.15.160:0/2839090250 conn(0x7f52140f89b0 msgr2=0x7f5214071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.891+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 shutdown_connections 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.891+0000 7f521b8f4700 1 -- 172.21.15.160:0/2839090250 wait complete. 2024-05-25T06:19:49.892 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1967619524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 msgr2=0x7f0b440fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1967619524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b440fcdf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0b40004940 tx=0x7f0b40009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1967619524 shutdown_connections 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1967619524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b440fcdf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1967619524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 0x7f0b441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1967619524 >> 172.21.15.160:0/1967619524 conn(0x7f0b440f8990 msgr2=0x7f0b440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.994+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1967619524 shutdown_connections 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1967619524 wait complete. 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 Processor -- start 2024-05-25T06:19:49.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 -- start start 2024-05-25T06:19:49.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 0x7f0b44113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b44113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b44114590 con 0x7f0b44103780 2024-05-25T06:19:49.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b4bfa7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4410db20 con 0x7f0b44102de0 2024-05-25T06:19:49.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b49542700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b44113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b49542700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b44113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52458/0 (socket says 172.21.15.160:52458) 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b49542700 1 -- 172.21.15.160:0/1741144101 learned_addr learned my addr 172.21.15.160:0/1741144101 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.995+0000 7f0b49d43700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 0x7f0b44113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b49542700 1 -- 172.21.15.160:0/1741144101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 msgr2=0x7f0b44113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b49542700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 0x7f0b44113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b49542700 1 -- 172.21.15.160:0/1741144101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40005040 con 0x7f0b44103780 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b49d43700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 0x7f0b44113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b49542700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b44113eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009100 tx=0x7f0b40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4000e040 con 0x7f0b44103780 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b40013420 con 0x7f0b44103780 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4410dda0 con 0x7f0b44103780 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4001c610 con 0x7f0b44103780 2024-05-25T06:19:49.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.996+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4410e360 con 0x7f0b44103780 2024-05-25T06:19:49.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.997+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b441000f0 con 0x7f0b44103780 2024-05-25T06:19:49.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.998+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0b40013590 con 0x7f0b44103780 2024-05-25T06:19:49.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.998+0000 7f0b367fc700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b3006e140 0x7f0b300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:49.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.998+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0b4008ed60 con 0x7f0b44103780 2024-05-25T06:19:50.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.998+0000 7f0b49d43700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b3006e140 0x7f0b300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:50.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:49.999+0000 7f0b49d43700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b3006e140 0x7f0b300705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009a80 tx=0x7f0b38008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:50.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.003+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0b400581b0 con 0x7f0b44103780 2024-05-25T06:19:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:49 smithi160 ceph-mon[112094]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:49 smithi160 ceph-mon[112094]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:50.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:49 smithi160 ceph-mon[112094]: pgmap v151: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-25T06:19:50.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:49 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3938075238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:50.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:49 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2839090250' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:19:50.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.222+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b4410efc0 con 0x7f0b3006e140 2024-05-25T06:19:50.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.223+0000 7f0b367fc700 1 -- 172.21.15.160:0/1741144101 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0b4410efc0 con 0x7f0b3006e140 2024-05-25T06:19:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr" 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "11/21 daemons upgraded", 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading osd daemons", 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:19:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:19:50.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.225+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b3006e140 msgr2=0x7f0b300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.225+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b3006e140 0x7f0b300705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009a80 tx=0x7f0b38008040 comp rx=0 tx=0).stop 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.225+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 msgr2=0x7f0b44113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.225+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b44113eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009100 tx=0x7f0b40009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.225+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 shutdown_connections 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.226+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b3006e140 0x7f0b300705f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.226+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b44102de0 0x7f0b44113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.226+0000 7f0b4bfa7700 1 --2- 172.21.15.160:0/1741144101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b44103780 0x7f0b44113eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.226+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 >> 172.21.15.160:0/1741144101 conn(0x7f0b440f8990 msgr2=0x7f0b440fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.226+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 shutdown_connections 2024-05-25T06:19:50.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.226+0000 7f0b4bfa7700 1 -- 172.21.15.160:0/1741144101 wait complete. 2024-05-25T06:19:50.320 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:49 smithi203 ceph-mon[83507]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:50.320 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:49 smithi203 ceph-mon[83507]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:50.321 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:49 smithi203 ceph-mon[83507]: pgmap v151: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-25T06:19:50.321 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:49 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3938075238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:19:50.321 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:49 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2839090250' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:19:50.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 -- 172.21.15.160:0/3211217620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664071f30 msgr2=0x7fd664070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:50.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/3211217620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664071f30 0x7fd664070300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd660004970 tx=0x7fd660009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:50.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 -- 172.21.15.160:0/3211217620 shutdown_connections 2024-05-25T06:19:50.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/3211217620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd664070840 0x7fd664070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/3211217620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664071f30 0x7fd664070300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 -- 172.21.15.160:0/3211217620 >> 172.21.15.160:0/3211217620 conn(0x7fd6640f8990 msgr2=0x7fd6640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 -- 172.21.15.160:0/3211217620 shutdown_connections 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.328+0000 7fd66b8a9700 1 -- 172.21.15.160:0/3211217620 wait complete. 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd66b8a9700 1 Processor -- start 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd66b8a9700 1 -- start start 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd66b8a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 0x7fd664119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd66b8a9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd664071f30 0x7fd66411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd66b8a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66411aad0 con 0x7fd664070840 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd66b8a9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641140b0 con 0x7fd664071f30 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 0x7fd664119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 0x7fd664119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52476/0 (socket says 172.21.15.160:52476) 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 -- 172.21.15.160:0/2430203757 learned_addr learned my addr 172.21.15.160:0/2430203757 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd668e44700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd664071f30 0x7fd66411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:50.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 -- 172.21.15.160:0/2430203757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd664071f30 msgr2=0x7fd66411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd664071f30 0x7fd66411a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 -- 172.21.15.160:0/2430203757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660005040 con 0x7fd664070840 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd669645700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 0x7fd664119f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd660000c00 tx=0x7fd660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.329+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66000e040 con 0x7fd664070840 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.330+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd664114330 con 0x7fd664070840 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.330+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd66000bc10 con 0x7fd664070840 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.330+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66001c630 con 0x7fd664070840 2024-05-25T06:19:50.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.330+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd664114870 con 0x7fd664070840 2024-05-25T06:19:50.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.331+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd664071a40 con 0x7fd664070840 2024-05-25T06:19:50.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.331+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd660013420 con 0x7fd664070840 2024-05-25T06:19:50.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.332+0000 7fd659ffb700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd65006e140 0x7fd6500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:19:50.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.332+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd66008edb0 con 0x7fd664070840 2024-05-25T06:19:50.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.332+0000 7fd668e44700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd65006e140 0x7fd6500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:19:50.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.332+0000 7fd668e44700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd65006e140 0x7fd6500705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6641157e0 tx=0x7fd65400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:19:50.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.336+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd66005bd30 con 0x7fd664070840 2024-05-25T06:19:50.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.614+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd664115590 con 0x7fd664070840 2024-05-25T06:19:50.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.615+0000 7fd659ffb700 1 -- 172.21.15.160:0/2430203757 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd660004de0 con 0x7fd664070840 2024-05-25T06:19:50.615 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:19:50.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd65006e140 msgr2=0x7fd6500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:50.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd65006e140 0x7fd6500705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6641157e0 tx=0x7fd65400b040 comp rx=0 tx=0).stop 2024-05-25T06:19:50.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 msgr2=0x7fd664119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 0x7fd664119f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd660000c00 tx=0x7fd660009e30 comp rx=0 tx=0).stop 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 shutdown_connections 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd664070840 0x7fd664119f40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd65006e140 0x7fd6500705f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 --2- 172.21.15.160:0/2430203757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd664071f30 0x7fd66411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 >> 172.21.15.160:0/2430203757 conn(0x7fd6640f8990 msgr2=0x7fd6640f9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 shutdown_connections 2024-05-25T06:19:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:19:50.617+0000 7fd66b8a9700 1 -- 172.21.15.160:0/2430203757 wait complete. 2024-05-25T06:19:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:50 smithi203 ceph-mon[83507]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:50 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2430203757' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:19:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:50 smithi160 ceph-mon[112094]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:50 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2430203757' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:19:52.254 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:51 smithi203 ceph-mon[83507]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:52.254 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:51 smithi203 ceph-mon[83507]: pgmap v152: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:19:52.254 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:52.254 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:51 smithi203 ceph-mon[83507]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:52.255 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:51 smithi203 ceph-mon[83507]: Upgrade: osd.5 is safe to restart 2024-05-25T06:19:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:51 smithi160 ceph-mon[112094]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:19:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:51 smithi160 ceph-mon[112094]: pgmap v152: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-25T06:19:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:52.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:51 smithi160 ceph-mon[112094]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-25T06:19:52.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:51 smithi160 ceph-mon[112094]: Upgrade: osd.5 is safe to restart 2024-05-25T06:19:53.307 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:52 smithi203 ceph-mon[83507]: pgmap v153: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:19:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:52 smithi160 ceph-mon[112094]: pgmap v153: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:19:54.527 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:54 smithi203 ceph-mon[83507]: Upgrade: Updating osd.5 2024-05-25T06:19:54.527 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:54.527 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-25T06:19:54.527 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:54 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:54.528 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:54 smithi203 ceph-mon[83507]: Deploying daemon osd.5 on smithi203 2024-05-25T06:19:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:54 smithi160 ceph-mon[112094]: Upgrade: Updating osd.5 2024-05-25T06:19:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-25T06:19:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:54 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:19:54.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:54 smithi160 ceph-mon[112094]: Deploying daemon osd.5 on smithi203 2024-05-25T06:19:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:55 smithi203 ceph-mon[83507]: pgmap v154: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:19:55.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:55 smithi160 ceph-mon[112094]: pgmap v154: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:19:56.694 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:56 smithi203 systemd[1]: Stopping Ceph osd.5 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:19:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:56 smithi203 ceph-mon[83507]: osd.5 marked itself down and dead 2024-05-25T06:19:57.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[55269]: 2024-05-25T06:19:56.694+0000 7f37a9d52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:19:57.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[55269]: 2024-05-25T06:19:56.694+0000 7f37a9d52700 -1 osd.5 74 *** Got signal Terminated *** 2024-05-25T06:19:57.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[55269]: 2024-05-25T06:19:56.694+0000 7f37a9d52700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:19:57.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:56 smithi160 ceph-mon[112094]: osd.5 marked itself down and dead 2024-05-25T06:19:57.605 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:57 smithi203 podman[102834]: 2024-05-25 06:19:57.317529441 +0000 UTC m=+0.731506877 container died 5d9ea2c9291151736bd7db6ae516ca0ade46559105d7a665a5167c1cf7efed7f (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=quincy-7b01852, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-25T06:19:57.899 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:57 smithi203 podman[102834]: 2024-05-25 06:19:57.687030968 +0000 UTC m=+1.101008401 container cleanup 5d9ea2c9291151736bd7db6ae516ca0ade46559105d7a665a5167c1cf7efed7f (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-7b01852, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-25T06:19:57.899 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:57 smithi203 bash[102834]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5 2024-05-25T06:19:57.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:57 smithi203 ceph-mon[83507]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:19:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:57 smithi203 ceph-mon[83507]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-25T06:19:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:57 smithi203 ceph-mon[83507]: osdmap e75: 6 total, 5 up, 6 in 2024-05-25T06:19:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:57 smithi203 ceph-mon[83507]: pgmap v156: 65 pgs: 13 stale+active+clean, 52 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:57.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:57 smithi203 ceph-mon[83507]: osdmap e76: 6 total, 5 up, 6 in 2024-05-25T06:19:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:57 smithi160 ceph-mon[112094]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-25T06:19:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:57 smithi160 ceph-mon[112094]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-25T06:19:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:57 smithi160 ceph-mon[112094]: osdmap e75: 6 total, 5 up, 6 in 2024-05-25T06:19:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:57 smithi160 ceph-mon[112094]: pgmap v156: 65 pgs: 13 stale+active+clean, 52 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:19:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:57 smithi160 ceph-mon[112094]: osdmap e76: 6 total, 5 up, 6 in 2024-05-25T06:19:58.282 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:57 smithi203 podman[102853]: 2024-05-25 06:19:57.963977782 +0000 UTC m=+0.649392046 container remove 5d9ea2c9291151736bd7db6ae516ca0ade46559105d7a665a5167c1cf7efed7f (image=quay.ceph.io/ceph-ci/ceph@sha256:310f49aadef44fa432a2c7bb53a87480af2cd68a365532f51086859b03d32c45, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=quincy-7b01852, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:19:58.542 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:58 smithi203 podman[102941]: 2024-05-25 06:19:58.185247011 +0000 UTC m=+0.017143590 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:19:58.542 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:58 smithi203 podman[102941]: 2024-05-25 06:19:58.290136839 +0000 UTC m=+0.122033477 container create 24ff7e122e62f0ba030690024c03bac3a4c3fc72c8a988ab0cde3f90f54c0abd (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=) 2024-05-25T06:19:58.843 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:58 smithi203 podman[102941]: 2024-05-25 06:19:58.543162553 +0000 UTC m=+0.375059117 container init 24ff7e122e62f0ba030690024c03bac3a4c3fc72c8a988ab0cde3f90f54c0abd (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-25T06:19:58.843 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:58 smithi203 podman[102941]: 2024-05-25 06:19:58.546970617 +0000 UTC m=+0.378867186 container start 24ff7e122e62f0ba030690024c03bac3a4c3fc72c8a988ab0cde3f90f54c0abd (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-25T06:19:58.844 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:58 smithi203 podman[102941]: 2024-05-25 06:19:58.651710117 +0000 UTC m=+0.483606695 container attach 24ff7e122e62f0ba030690024c03bac3a4c3fc72c8a988ab0cde3f90f54c0abd (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:19:59.146 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:59.146 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:59.146 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:58 smithi203 podman[102941]: 2024-05-25 06:19:58.844564414 +0000 UTC m=+0.676460991 container died 24ff7e122e62f0ba030690024c03bac3a4c3fc72c8a988ab0cde3f90f54c0abd (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:19:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:19:59.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:19:59.859 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 podman[102967]: 2024-05-25 06:19:59.414193711 +0000 UTC m=+0.560802024 container remove 24ff7e122e62f0ba030690024c03bac3a4c3fc72c8a988ab0cde3f90f54c0abd (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-25T06:19:59.859 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-25T06:19:59.859 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 systemd[1]: Stopped Ceph osd.5 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:19:59.859 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.5.service: Consumed 1min 8.510s CPU time. 2024-05-25T06:19:59.859 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 systemd[1]: Starting Ceph osd.5 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:20:00.143 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:19:59 smithi203 ceph-mon[83507]: pgmap v158: 65 pgs: 11 stale+active+clean, 3 active+undersized, 51 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-25T06:20:00.143 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 podman[103068]: 2024-05-25 06:19:59.762436638 +0000 UTC m=+0.017591213 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:20:00.143 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:19:59 smithi203 podman[103068]: 2024-05-25 06:19:59.899413308 +0000 UTC m=+0.154567875 container create 3e5ca0b7f03126e8dd5b3d049f4c4ddd219946b0e8db67131c1f6d1cbac7da0d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate, ceph=True, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:20:00.144 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 podman[103068]: 2024-05-25 06:20:00.143967755 +0000 UTC m=+0.399122322 container init 3e5ca0b7f03126e8dd5b3d049f4c4ddd219946b0e8db67131c1f6d1cbac7da0d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-25T06:20:00.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:19:59 smithi160 ceph-mon[112094]: pgmap v158: 65 pgs: 11 stale+active+clean, 3 active+undersized, 51 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-25T06:20:00.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 podman[103068]: 2024-05-25 06:20:00.147284396 +0000 UTC m=+0.402438971 container start 3e5ca0b7f03126e8dd5b3d049f4c4ddd219946b0e8db67131c1f6d1cbac7da0d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-25T06:20:00.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 podman[103068]: 2024-05-25 06:20:00.227393874 +0000 UTC m=+0.482548449 container attach 3e5ca0b7f03126e8dd5b3d049f4c4ddd219946b0e8db67131c1f6d1cbac7da0d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-25T06:20:00.899 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 bash[103068]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-25T06:20:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-25T06:20:00.901 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate[103084]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-25T06:20:00.901 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:00 smithi203 podman[103068]: 2024-05-25 06:20:00.625502677 +0000 UTC m=+0.880657244 container died 3e5ca0b7f03126e8dd5b3d049f4c4ddd219946b0e8db67131c1f6d1cbac7da0d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-25T06:20:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:00 smithi160 ceph-mon[112094]: Health detail: HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:20:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:00 smithi160 ceph-mon[112094]: [WRN] OSD_DOWN: 1 osds down 2024-05-25T06:20:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:00 smithi160 ceph-mon[112094]: osd.5 (root=default,host=smithi203) is down 2024-05-25T06:20:01.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:00 smithi160 ceph-mon[112094]: [WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:20:01.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:00 smithi160 ceph-mon[112094]: all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:20:01.256 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:00 smithi203 ceph-mon[83507]: Health detail: HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:20:01.256 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:00 smithi203 ceph-mon[83507]: [WRN] OSD_DOWN: 1 osds down 2024-05-25T06:20:01.256 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:00 smithi203 ceph-mon[83507]: osd.5 (root=default,host=smithi203) is down 2024-05-25T06:20:01.256 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:00 smithi203 ceph-mon[83507]: [WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:20:01.256 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:00 smithi203 ceph-mon[83507]: all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:20:01.583 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 podman[103203]: 2024-05-25 06:20:01.256998362 +0000 UTC m=+0.625970149 container remove 3e5ca0b7f03126e8dd5b3d049f4c4ddd219946b0e8db67131c1f6d1cbac7da0d (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True) 2024-05-25T06:20:01.583 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 podman[103265]: 2024-05-25 06:20:01.373490171 +0000 UTC m=+0.016927556 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:20:01.583 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 podman[103265]: 2024-05-25 06:20:01.474974038 +0000 UTC m=+0.118411410 container create 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-25T06:20:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:01 smithi203 ceph-mon[83507]: pgmap v159: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 44 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/324 objects degraded (6.481%) 2024-05-25T06:20:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:01 smithi203 ceph-mon[83507]: Health check failed: Degraded data redundancy: 21/324 objects degraded (6.481%), 8 pgs degraded (PG_DEGRADED) 2024-05-25T06:20:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:01 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:01 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:02.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:01 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:02.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 podman[103265]: 2024-05-25 06:20:01.68591512 +0000 UTC m=+0.329352507 container init 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-25T06:20:02.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 podman[103265]: 2024-05-25 06:20:01.689124925 +0000 UTC m=+0.332562298 container start 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0) 2024-05-25T06:20:02.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 ceph-osd[103284]: -- 172.21.15.203:0/3087905637 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a427a7680 con 0x560a42771000 2024-05-25T06:20:02.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 bash[103265]: 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 2024-05-25T06:20:02.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:01 smithi203 systemd[1]: Started Ceph osd.5 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:20:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:01 smithi160 ceph-mon[112094]: pgmap v159: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 44 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/324 objects degraded (6.481%) 2024-05-25T06:20:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:01 smithi160 ceph-mon[112094]: Health check failed: Degraded data redundancy: 21/324 objects degraded (6.481%), 8 pgs degraded (PG_DEGRADED) 2024-05-25T06:20:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:01 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:01 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:02.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:01 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:02.649 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:02 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:20:02.231+0000 7f0a0138d7c0 -1 Falling back to public interface 2024-05-25T06:20:04.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:03 smithi160 ceph-mon[112094]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:04.304 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:03 smithi203 ceph-mon[83507]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:06.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:05 smithi203 ceph-mon[83507]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:06.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:06.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:05 smithi160 ceph-mon[112094]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:07.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:07.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:07.581 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:07 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:20:07.161+0000 7f0a0138d7c0 -1 osd.5 74 log_to_monitors true 2024-05-25T06:20:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:07 smithi160 ceph-mon[112094]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:07 smithi160 ceph-mon[112094]: from='osd.5 [v2:172.21.15.203:6816/2019898829,v1:172.21.15.203:6817/2019898829]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:20:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:07 smithi160 ceph-mon[112094]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:20:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:07 smithi160 ceph-mon[112094]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-05-25T06:20:08.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:07 smithi203 ceph-mon[83507]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:08.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:07 smithi203 ceph-mon[83507]: from='osd.5 [v2:172.21.15.203:6816/2019898829,v1:172.21.15.203:6817/2019898829]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:20:08.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:07 smithi203 ceph-mon[83507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-25T06:20:08.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:07 smithi203 ceph-mon[83507]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-05-25T06:20:08.401 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:20:07 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:20:07.988+0000 7f09f27e4700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-25T06:20:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:08 smithi203 ceph-mon[83507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-25T06:20:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:08 smithi203 ceph-mon[83507]: from='osd.5 [v2:172.21.15.203:6816/2019898829,v1:172.21.15.203:6817/2019898829]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:20:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:08 smithi203 ceph-mon[83507]: osdmap e77: 6 total, 5 up, 6 in 2024-05-25T06:20:09.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:08 smithi203 ceph-mon[83507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:20:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:08 smithi160 ceph-mon[112094]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-25T06:20:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:08 smithi160 ceph-mon[112094]: from='osd.5 [v2:172.21.15.203:6816/2019898829,v1:172.21.15.203:6817/2019898829]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:20:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:08 smithi160 ceph-mon[112094]: osdmap e77: 6 total, 5 up, 6 in 2024-05-25T06:20:09.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:08 smithi160 ceph-mon[112094]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-25T06:20:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: osd.5 [v2:172.21.15.203:6816/2019898829,v1:172.21.15.203:6817/2019898829] boot 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: osdmap e78: 6 total, 6 up, 6 in 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-25T06:20:10.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-25T06:20:10.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-25T06:20:10.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-25T06:20:10.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-25T06:20:10.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:09 smithi203 ceph-mon[83507]: osdmap e79: 6 total, 6 up, 6 in 2024-05-25T06:20:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-05-25T06:20:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-25T06:20:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: osd.5 [v2:172.21.15.203:6816/2019898829,v1:172.21.15.203:6817/2019898829] boot 2024-05-25T06:20:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: osdmap e78: 6 total, 6 up, 6 in 2024-05-25T06:20:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-25T06:20:10.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-25T06:20:10.472 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-25T06:20:10.473 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:09 smithi160 ceph-mon[112094]: osdmap e79: 6 total, 6 up, 6 in 2024-05-25T06:20:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:10 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all osd 2024-05-25T06:20:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:10 smithi203 ceph-mon[83507]: Upgrade: Setting require_osd_release to 19 squid 2024-05-25T06:20:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:10 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:10 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-25T06:20:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:10 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all osd 2024-05-25T06:20:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:10 smithi160 ceph-mon[112094]: Upgrade: Setting require_osd_release to 19 squid 2024-05-25T06:20:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:10 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:10 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-25T06:20:11.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:20:11.152+0000 7fd61985b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-25T06:20:12.014 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-25T06:20:12.014 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: pgmap v167: 65 pgs: 7 peering, 10 active+undersized+degraded, 11 active+undersized, 37 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 41/324 objects degraded (12.654%) 2024-05-25T06:20:12.014 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: osdmap e80: 6 total, 6 up, 6 in 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-25T06:20:12.015 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: osdmap e81: 6 total, 6 up, 6 in 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.203:6826/2242985365,v1:172.21.15.203:6827/2242985365] up:boot 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6826/2550259626,v1:172.21.15.160:6827/2550259626] up:boot 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:20:12.016 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:11 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:20:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: pgmap v167: 65 pgs: 7 peering, 10 active+undersized+degraded, 11 active+undersized, 37 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 41/324 objects degraded (12.654%) 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: osdmap e80: 6 total, 6 up, 6 in 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: fsmap cephfs:2 {0=cephfs.smithi160.tlctof=up:active,1=cephfs.smithi203.yltymd=up:active} 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: osdmap e81: 6 total, 6 up, 6 in 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.203:6826/2242985365,v1:172.21.15.203:6827/2242985365] up:boot 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6826/2550259626,v1:172.21.15.160:6827/2550259626] up:boot 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:20:12.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:11 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:20:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.203:6824/1521635049,v1:172.21.15.203:6825/1521635049] up:boot 2024-05-25T06:20:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6828/3633650351,v1:172.21.15.160:6829/3633650351] up:boot 2024-05-25T06:20:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:20:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:20:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: Upgrade: Updating mds.cephfs.smithi160.rrhzig 2024-05-25T06:20:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:20:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: Deploying daemon mds.cephfs.smithi160.rrhzig on smithi160 2024-05-25T06:20:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: Health check update: Degraded data redundancy: 41/324 objects degraded (12.654%), 10 pgs degraded (PG_DEGRADED) 2024-05-25T06:20:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: pgmap v170: 65 pgs: 7 peering, 5 active+undersized+degraded, 9 active+undersized, 44 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s; 15/324 objects degraded (4.630%) 2024-05-25T06:20:13.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:13.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.203:6824/1521635049,v1:172.21.15.203:6825/1521635049] up:boot 2024-05-25T06:20:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6828/3633650351,v1:172.21.15.160:6829/3633650351] up:boot 2024-05-25T06:20:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:20:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:20:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: Upgrade: Updating mds.cephfs.smithi160.rrhzig 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.rrhzig", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: Deploying daemon mds.cephfs.smithi160.rrhzig on smithi160 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: Health check update: Degraded data redundancy: 41/324 objects degraded (12.654%), 10 pgs degraded (PG_DEGRADED) 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: pgmap v170: 65 pgs: 7 peering, 5 active+undersized+degraded, 9 active+undersized, 44 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s; 15/324 objects degraded (4.630%) 2024-05-25T06:20:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:13.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:15.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:15 smithi160 ceph-mon[112094]: pgmap v171: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-25T06:20:15.222 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:15 smithi160 ceph-mon[112094]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/324 objects degraded (4.630%), 5 pgs degraded) 2024-05-25T06:20:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:15 smithi203 ceph-mon[83507]: pgmap v171: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-25T06:20:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:15 smithi203 ceph-mon[83507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/324 objects degraded (4.630%), 5 pgs degraded) 2024-05-25T06:20:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:17 smithi203 ceph-mon[83507]: osdmap e82: 6 total, 6 up, 6 in 2024-05-25T06:20:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:17 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:20:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:20:17.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:17 smithi203 ceph-mon[83507]: pgmap v173: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:17 smithi160 ceph-mon[112094]: osdmap e82: 6 total, 6 up, 6 in 2024-05-25T06:20:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:17 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:20:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:20:17.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:17 smithi160 ceph-mon[112094]: pgmap v173: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:20.059 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:19 smithi160 ceph-mon[112094]: pgmap v174: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:20.373 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:19 smithi203 ceph-mon[83507]: pgmap v174: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:20.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 -- 172.21.15.160:0/1127356935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f680fbd40 msgr2=0x7f9f680fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:20.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 --2- 172.21.15.160:0/1127356935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f680fbd40 0x7f9f680fc1b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f58004940 tx=0x7f9f58009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 -- 172.21.15.160:0/1127356935 shutdown_connections 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 --2- 172.21.15.160:0/1127356935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f680fbd40 0x7f9f680fc1b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 --2- 172.21.15.160:0/1127356935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f68101d90 0x7f9f68102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 -- 172.21.15.160:0/1127356935 >> 172.21.15.160:0/1127356935 conn(0x7f9f68071d00 msgr2=0x7f9f68072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 -- 172.21.15.160:0/1127356935 shutdown_connections 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.737+0000 7f9f703b3700 1 -- 172.21.15.160:0/1127356935 wait complete. 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f703b3700 1 Processor -- start 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f703b3700 1 -- start start 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f703b3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 0x7f9f68113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f703b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f68101d90 0x7f9f68114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f703b3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f681146b0 con 0x7f9f68101d90 2024-05-25T06:20:20.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f703b3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6810daa0 con 0x7f9f680fbd40 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f6e14f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 0x7f9f68113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f6e14f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 0x7f9f68113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.160:35516/0 (socket says 172.21.15.160:35516) 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f6e14f700 1 -- 172.21.15.160:0/594017482 learned_addr learned my addr 172.21.15.160:0/594017482 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.738+0000 7f9f6d94e700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f68101d90 0x7f9f68114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f6e14f700 1 -- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f68101d90 msgr2=0x7f9f68114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f6e14f700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f68101d90 0x7f9f68114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f6e14f700 1 -- 172.21.15.160:0/594017482 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58005040 con 0x7f9f680fbd40 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f6e14f700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 0x7f9f68113b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400eb40 tx=0x7f9f6400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f64007cc0 con 0x7f9f680fbd40 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6810dd20 con 0x7f9f680fbd40 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f6400f040 con 0x7f9f680fbd40 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f6400ca10 con 0x7f9f680fbd40 2024-05-25T06:20:20.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.739+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6810e230 con 0x7f9f680fbd40 2024-05-25T06:20:20.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.740+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f681063c0 con 0x7f9f680fbd40 2024-05-25T06:20:20.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.741+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9f6400cc80 con 0x7f9f680fbd40 2024-05-25T06:20:20.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.741+0000 7f9f5effd700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f5406e140 0x7f9f540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:20.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.741+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9f6408d7e0 con 0x7f9f680fbd40 2024-05-25T06:20:20.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.741+0000 7f9f6d94e700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f5406e140 0x7f9f540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:20.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.742+0000 7f9f6d94e700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f5406e140 0x7f9f540705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009b30 tx=0x7f9f58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:20.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.746+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9f6405a670 con 0x7f9f680fbd40 2024-05-25T06:20:20.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.982+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f6810f1e0 con 0x7f9f5406e140 2024-05-25T06:20:20.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.985+0000 7f9f5effd700 1 -- 172.21.15.160:0/594017482 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f6810f1e0 con 0x7f9f5406e140 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f5406e140 msgr2=0x7f9f540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f5406e140 0x7f9f540705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009b30 tx=0x7f9f58009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 msgr2=0x7f9f68113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 0x7f9f68113b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400eb40 tx=0x7f9f6400b040 comp rx=0 tx=0).stop 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 shutdown_connections 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9f5406e140 0x7f9f540705f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f680fbd40 0x7f9f68113b20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.987+0000 7f9f703b3700 1 --2- 172.21.15.160:0/594017482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f68101d90 0x7f9f68114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.988+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 >> 172.21.15.160:0/594017482 conn(0x7f9f68071d00 msgr2=0x7f9f68065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.988+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 shutdown_connections 2024-05-25T06:20:20.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:20.988+0000 7f9f703b3700 1 -- 172.21.15.160:0/594017482 wait complete. 2024-05-25T06:20:21.000 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.098+0000 7f18dbbf2700 1 -- 172.21.15.160:0/3690982377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 msgr2=0x7f18d40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.098+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/3690982377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d40fc1b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18d0004940 tx=0x7f18d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 -- 172.21.15.160:0/3690982377 shutdown_connections 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/3690982377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d40fc1b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/3690982377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18d4101d90 0x7f18d4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 -- 172.21.15.160:0/3690982377 >> 172.21.15.160:0/3690982377 conn(0x7f18d4071d00 msgr2=0x7f18d4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 -- 172.21.15.160:0/3690982377 shutdown_connections 2024-05-25T06:20:21.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 -- 172.21.15.160:0/3690982377 wait complete. 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 Processor -- start 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.099+0000 7f18dbbf2700 1 -- start start 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18dbbf2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d410eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18dbbf2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18d4101d90 0x7f18d410f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18dbbf2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d410f720 con 0x7f18d40fbd40 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18dbbf2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d410f860 con 0x7f18d4101d90 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d410eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d410eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58730/0 (socket says 172.21.15.160:58730) 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 -- 172.21.15.160:0/7772411 learned_addr learned my addr 172.21.15.160:0/7772411 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d918d700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18d4101d90 0x7f18d410f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 -- 172.21.15.160:0/7772411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18d4101d90 msgr2=0x7f18d410f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18d4101d90 0x7f18d410f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 -- 172.21.15.160:0/7772411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0005040 con 0x7f18d40fbd40 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18d998e700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d410eca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f18c400eb70 tx=0x7f18c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4007cc0 con 0x7f18d40fbd40 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d4117b50 con 0x7f18d40fbd40 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18c400f040 con 0x7f18d40fbd40 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c400ca90 con 0x7f18d40fbd40 2024-05-25T06:20:21.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.100+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d4118000 con 0x7f18d40fbd40 2024-05-25T06:20:21.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.101+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d40fd430 con 0x7f18d40fbd40 2024-05-25T06:20:21.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.102+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f18c400cbf0 con 0x7f18d40fbd40 2024-05-25T06:20:21.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.102+0000 7f18ca7fc700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18c006e140 0x7f18c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.103+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f18c408d6d0 con 0x7f18d40fbd40 2024-05-25T06:20:21.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.103+0000 7f18d918d700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18c006e140 0x7f18c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.103+0000 7f18d918d700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18c006e140 0x7f18c00705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009b30 tx=0x7f18d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:21.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.107+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f18c4056a00 con 0x7f18d40fbd40 2024-05-25T06:20:21.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.333+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18d4065df0 con 0x7f18c006e140 2024-05-25T06:20:21.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.334+0000 7f18ca7fc700 1 -- 172.21.15.160:0/7772411 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f18d4065df0 con 0x7f18c006e140 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18c006e140 msgr2=0x7f18c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18c006e140 0x7f18c00705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009b30 tx=0x7f18d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 msgr2=0x7f18d410eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d410eca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f18c400eb70 tx=0x7f18c400b040 comp rx=0 tx=0).stop 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 shutdown_connections 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d40fbd40 0x7f18d410eca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18c006e140 0x7f18c00705f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 --2- 172.21.15.160:0/7772411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18d4101d90 0x7f18d410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 >> 172.21.15.160:0/7772411 conn(0x7f18d4071d00 msgr2=0x7f18d40f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:21.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 shutdown_connections 2024-05-25T06:20:21.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.338+0000 7f18dbbf2700 1 -- 172.21.15.160:0/7772411 wait complete. 2024-05-25T06:20:21.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 -- 172.21.15.160:0/3089058545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c101cd0 msgr2=0x7fce5c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 --2- 172.21.15.160:0/3089058545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c101cd0 0x7fce5c1020a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce58004940 tx=0x7fce58009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 -- 172.21.15.160:0/3089058545 shutdown_connections 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 --2- 172.21.15.160:0/3089058545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce5c102670 0x7fce5c0fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 --2- 172.21.15.160:0/3089058545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c101cd0 0x7fce5c1020a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 -- 172.21.15.160:0/3089058545 >> 172.21.15.160:0/3089058545 conn(0x7fce5c071460 msgr2=0x7fce5c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 -- 172.21.15.160:0/3089058545 shutdown_connections 2024-05-25T06:20:21.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.452+0000 7fce638b6700 1 -- 172.21.15.160:0/3089058545 wait complete. 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce638b6700 1 Processor -- start 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce638b6700 1 -- start start 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce638b6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce5c101cd0 0x7fce5c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce638b6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 0x7fce5c114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce638b6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c114680 con 0x7fce5c102670 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce638b6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c10dac0 con 0x7fce5c101cd0 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce60e51700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 0x7fce5c114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce60e51700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 0x7fce5c114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58738/0 (socket says 172.21.15.160:58738) 2024-05-25T06:20:21.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.453+0000 7fce60e51700 1 -- 172.21.15.160:0/3650355463 learned_addr learned my addr 172.21.15.160:0/3650355463 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce61652700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce5c101cd0 0x7fce5c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce60e51700 1 -- 172.21.15.160:0/3650355463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce5c101cd0 msgr2=0x7fce5c113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce60e51700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce5c101cd0 0x7fce5c113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce60e51700 1 -- 172.21.15.160:0/3650355463 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58005040 con 0x7fce5c102670 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce60e51700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 0x7fce5c114030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0044e0 tx=0x7fce4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce4c007cc0 con 0x7fce5c102670 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce4c00f040 con 0x7fce5c102670 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c10dd40 con 0x7fce5c102670 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce4c00cb00 con 0x7fce5c102670 2024-05-25T06:20:21.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.454+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c10e280 con 0x7fce5c102670 2024-05-25T06:20:21.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.455+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c065200 con 0x7fce5c102670 2024-05-25T06:20:21.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.456+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fce4c00cc60 con 0x7fce5c102670 2024-05-25T06:20:21.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.456+0000 7fce51ffb700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fce4806e140 0x7fce480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.457+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fce4c08d6c0 con 0x7fce5c102670 2024-05-25T06:20:21.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.457+0000 7fce61652700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fce4806e140 0x7fce480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.457+0000 7fce61652700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fce4806e140 0x7fce480705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce5800b0d0 tx=0x7fce58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:21.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.461+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fce4c05a550 con 0x7fce5c102670 2024-05-25T06:20:21.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.684+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fce5c1137c0 con 0x7fce4806e140 2024-05-25T06:20:21.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.695+0000 7fce51ffb700 1 -- 172.21.15.160:0/3650355463 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fce5c1137c0 con 0x7fce4806e140 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (6m) 117s ago 17m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (3m) 117s ago 17m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (3m) 16s ago 16m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (5m) 117s ago 17m 80.2M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 starting - - - - 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (11m) 117s ago 11m 117M - 17.2.7-958-g7b01852c baf1547092e7 0054167d677a 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (11m) 16s ago 11m 61.4M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (11m) 16s ago 11m 116M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:20:21.695 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (7m) 117s ago 18m 505M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (7m) 16s ago 16m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (4m) 117s ago 18m 45.1M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (4m) 16s ago 16m 39.6M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (6m) 117s ago 17m 15.2M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (6m) 16s ago 16m 16.9M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (3m) 117s ago 15m 96.8M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (2m) 117s ago 15m 98.9M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (2m) 117s ago 14m 12.3M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (87s) 16s ago 14m 113M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (53s) 16s ago 13m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:20:21.696 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (20s) 16s ago 13m 12.6M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:20:21.697 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (6m) 117s ago 16m 67.5M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:20:21.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fce4806e140 msgr2=0x7fce480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fce4806e140 0x7fce480705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce5800b0d0 tx=0x7fce58009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 msgr2=0x7fce5c114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 0x7fce5c114030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0044e0 tx=0x7fce4c00b040 comp rx=0 tx=0).stop 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 shutdown_connections 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fce4806e140 0x7fce480705f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce5c101cd0 0x7fce5c113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 --2- 172.21.15.160:0/3650355463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce5c102670 0x7fce5c114030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.697+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 >> 172.21.15.160:0/3650355463 conn(0x7fce5c071460 msgr2=0x7fce5c0fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.698+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 shutdown_connections 2024-05-25T06:20:21.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.698+0000 7fce638b6700 1 -- 172.21.15.160:0/3650355463 wait complete. 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.803+0000 7f87f35ce700 1 -- 172.21.15.160:0/2383421155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec103780 msgr2=0x7f87ec0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.803+0000 7f87f35ce700 1 --2- 172.21.15.160:0/2383421155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec103780 0x7f87ec0fcdf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f87dc004940 tx=0x7f87dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.803+0000 7f87f35ce700 1 -- 172.21.15.160:0/2383421155 shutdown_connections 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.803+0000 7f87f35ce700 1 --2- 172.21.15.160:0/2383421155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec103780 0x7f87ec0fcdf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.803+0000 7f87f35ce700 1 --2- 172.21.15.160:0/2383421155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec102de0 0x7f87ec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.803+0000 7f87f35ce700 1 -- 172.21.15.160:0/2383421155 >> 172.21.15.160:0/2383421155 conn(0x7f87ec0f8950 msgr2=0x7f87ec0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:21.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 -- 172.21.15.160:0/2383421155 shutdown_connections 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 -- 172.21.15.160:0/2383421155 wait complete. 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 Processor -- start 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 -- start start 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 0x7f87ec070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec103780 0x7f87ec0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec070d30 con 0x7f87ec102de0 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f35ce700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec070e70 con 0x7f87ec103780 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f136a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 0x7f87ec070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f136a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 0x7f87ec070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58746/0 (socket says 172.21.15.160:58746) 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.804+0000 7f87f136a700 1 -- 172.21.15.160:0/3565999850 learned_addr learned my addr 172.21.15.160:0/3565999850 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f0b69700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec103780 0x7f87ec0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f136a700 1 -- 172.21.15.160:0/3565999850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec103780 msgr2=0x7f87ec0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f136a700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec103780 0x7f87ec0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f136a700 1 -- 172.21.15.160:0/3565999850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc005040 con 0x7f87ec102de0 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f0b69700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec103780 0x7f87ec0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f136a700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 0x7f87ec070260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87e8002a80 tx=0x7f87e80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e8007cc0 con 0x7f87ec102de0 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e800f040 con 0x7f87ec102de0 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec119df0 con 0x7f87ec102de0 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e80096f0 con 0x7f87ec102de0 2024-05-25T06:20:21.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.805+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec11a270 con 0x7f87ec102de0 2024-05-25T06:20:21.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.806+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0fe530 con 0x7f87ec102de0 2024-05-25T06:20:21.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.807+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f87e800d690 con 0x7f87ec102de0 2024-05-25T06:20:21.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.807+0000 7f87e1ffb700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f87d806e140 0x7f87d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:21.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.808+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f87e808d310 con 0x7f87ec102de0 2024-05-25T06:20:21.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.808+0000 7f87f0b69700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f87d806e140 0x7f87d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:21.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.808+0000 7f87f0b69700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f87d806e140 0x7f87d80705f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00b3b0 tx=0x7f87dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:21.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:21.816+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f87e8056640 con 0x7f87ec102de0 2024-05-25T06:20:21.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: pgmap v175: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:21.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:21.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:21.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:21.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:boot 2024-05-25T06:20:21.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:21.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:20:22.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.113+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f87ec071950 con 0x7f87ec102de0 2024-05-25T06:20:22.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.114+0000 7f87e1ffb700 1 -- 172.21.15.160:0/3565999850 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+832 (secure 0 0 0) 0x7f87e8059d00 con 0x7f87ec102de0 2024-05-25T06:20:22.114 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:20:22.115 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 3, 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 1 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 3, 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 11 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:20:22.116 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:20:22.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f87d806e140 msgr2=0x7f87d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f87d806e140 0x7f87d80705f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00b3b0 tx=0x7f87dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:22.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 msgr2=0x7f87ec070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 0x7f87ec070260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87e8002a80 tx=0x7f87e80046b0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 shutdown_connections 2024-05-25T06:20:22.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87ec102de0 0x7f87ec070260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f87d806e140 0x7f87d80705f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 --2- 172.21.15.160:0/3565999850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec103780 0x7f87ec0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 >> 172.21.15.160:0/3565999850 conn(0x7f87ec0f8950 msgr2=0x7f87ec0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:22.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 shutdown_connections 2024-05-25T06:20:22.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.117+0000 7f87f35ce700 1 -- 172.21.15.160:0/3565999850 wait complete. 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.239+0000 7f103dbe2700 1 -- 172.21.15.160:0/1435203207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 msgr2=0x7f10380fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.239+0000 7f103dbe2700 1 --2- 172.21.15.160:0/1435203207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380fce10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f102c004940 tx=0x7f102c009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.239+0000 7f103dbe2700 1 -- 172.21.15.160:0/1435203207 shutdown_connections 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.239+0000 7f103dbe2700 1 --2- 172.21.15.160:0/1435203207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380fce10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.239+0000 7f103dbe2700 1 --2- 172.21.15.160:0/1435203207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1038102e00 0x7f10381031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.240+0000 7f103dbe2700 1 -- 172.21.15.160:0/1435203207 >> 172.21.15.160:0/1435203207 conn(0x7f10380f89b0 msgr2=0x7f10380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:22.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.240+0000 7f103dbe2700 1 -- 172.21.15.160:0/1435203207 shutdown_connections 2024-05-25T06:20:22.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.240+0000 7f103dbe2700 1 -- 172.21.15.160:0/1435203207 wait complete. 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f103dbe2700 1 Processor -- start 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f103dbe2700 1 -- start start 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f103dbe2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1038102e00 0x7f1038070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f103dbe2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f103dbe2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038070d30 con 0x7f10381037a0 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f103dbe2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038070e70 con 0x7f1038102e00 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f1036ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f1036ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58760/0 (socket says 172.21.15.160:58760) 2024-05-25T06:20:22.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f1036ffd700 1 -- 172.21.15.160:0/658204036 learned_addr learned my addr 172.21.15.160:0/658204036 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:22.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.241+0000 7f10377fe700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1038102e00 0x7f1038070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:22.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f1036ffd700 1 -- 172.21.15.160:0/658204036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1038102e00 msgr2=0x7f1038070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f1036ffd700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1038102e00 0x7f1038070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f1036ffd700 1 -- 172.21.15.160:0/658204036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c005040 con 0x7f10381037a0 2024-05-25T06:20:22.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f1036ffd700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380707a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f102c009100 tx=0x7f102c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:22.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102c00e040 con 0x7f10381037a0 2024-05-25T06:20:22.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10380710c0 con 0x7f10381037a0 2024-05-25T06:20:22.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103811a2e0 con 0x7f10381037a0 2024-05-25T06:20:22.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f102c009af0 con 0x7f10381037a0 2024-05-25T06:20:22.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.242+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102c013d10 con 0x7f10381037a0 2024-05-25T06:20:22.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.243+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10240052b0 con 0x7f10381037a0 2024-05-25T06:20:22.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.244+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f102c025400 con 0x7f10381037a0 2024-05-25T06:20:22.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.245+0000 7f101ffff700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102006e140 0x7f10200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:22.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.245+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f102c08ed20 con 0x7f10381037a0 2024-05-25T06:20:22.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.245+0000 7f10377fe700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102006e140 0x7f10200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:22.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.245+0000 7f10377fe700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102006e140 0x7f10200705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1028004eb0 tx=0x7f102800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:22.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.253+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f102c058140 con 0x7f10381037a0 2024-05-25T06:20:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: pgmap v175: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:boot 2024-05-25T06:20:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:20:22.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.532+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1024005d70 con 0x7f10381037a0 2024-05-25T06:20:22.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.532+0000 7f101ffff700 1 -- 172.21.15.160:0/658204036 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f102c05b800 con 0x7f10381037a0 2024-05-25T06:20:22.533 INFO:teuthology.orchestra.run.smithi160.stdout:e31 2024-05-25T06:20:22.533 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:20:22.533 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:20:22.534 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:20:22.535 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:20:22.536 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:22.537 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/3633650351,v1:172.21.15.160:6829/3633650351] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:20:22.537 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:20:22.537 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/2242985365,v1:172.21.15.203:6827/2242985365] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:20:22.537 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1521635049,v1:172.21.15.203:6825/1521635049] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102006e140 msgr2=0x7f10200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102006e140 0x7f10200705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1028004eb0 tx=0x7f102800b040 comp rx=0 tx=0).stop 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 msgr2=0x7f10380707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380707a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f102c009100 tx=0x7f102c009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 shutdown_connections 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f102006e140 0x7f10200705f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1038102e00 0x7f1038070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 --2- 172.21.15.160:0/658204036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10381037a0 0x7f10380707a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.539 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 >> 172.21.15.160:0/658204036 conn(0x7f10380f89b0 msgr2=0x7f10381021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:22.539 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 shutdown_connections 2024-05-25T06:20:22.539 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.535+0000 7f103dbe2700 1 -- 172.21.15.160:0/658204036 wait complete. 2024-05-25T06:20:22.539 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 31 2024-05-25T06:20:22.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 -- 172.21.15.160:0/1416148380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400102df0 msgr2=0x7f14001031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 --2- 172.21.15.160:0/1416148380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400102df0 0x7f14001031c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f13f0004940 tx=0x7f13f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:22.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 -- 172.21.15.160:0/1416148380 shutdown_connections 2024-05-25T06:20:22.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 --2- 172.21.15.160:0/1416148380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400103790 0x7f14000fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 --2- 172.21.15.160:0/1416148380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400102df0 0x7f14001031c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 -- 172.21.15.160:0/1416148380 >> 172.21.15.160:0/1416148380 conn(0x7f14000f8980 msgr2=0x7f14000fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 -- 172.21.15.160:0/1416148380 shutdown_connections 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.643+0000 7f14078c9700 1 -- 172.21.15.160:0/1416148380 wait complete. 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f14078c9700 1 Processor -- start 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f14078c9700 1 -- start start 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f14078c9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400102df0 0x7f1400113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f14078c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 0x7f1400113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f14078c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1400114610 con 0x7f1400103790 2024-05-25T06:20:22.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f14078c9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140010da00 con 0x7f1400102df0 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f1404e64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 0x7f1400113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f1404e64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 0x7f1400113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58790/0 (socket says 172.21.15.160:58790) 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.644+0000 7f1404e64700 1 -- 172.21.15.160:0/1313818729 learned_addr learned my addr 172.21.15.160:0/1313818729 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f1405665700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400102df0 0x7f1400113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f1404e64700 1 -- 172.21.15.160:0/1313818729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400102df0 msgr2=0x7f1400113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f1404e64700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400102df0 0x7f1400113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f1404e64700 1 -- 172.21.15.160:0/1313818729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0005040 con 0x7f1400103790 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f1404e64700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 0x7f1400113fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00eb40 tx=0x7f13fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13fc007cc0 con 0x7f1400103790 2024-05-25T06:20:22.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13fc00f040 con 0x7f1400103790 2024-05-25T06:20:22.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13fc00c9d0 con 0x7f1400103790 2024-05-25T06:20:22.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140010dc80 con 0x7f1400103790 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.645+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140010e1c0 con 0x7f1400103790 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.646+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400100100 con 0x7f1400103790 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.647+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f13fc00cc40 con 0x7f1400103790 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.648+0000 7f13f5ffb700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f13ec06e140 0x7f13ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.648+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f13fc08d5c0 con 0x7f1400103790 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.648+0000 7f1405665700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f13ec06e140 0x7f13ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:22.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.649+0000 7f1405665700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f13ec06e140 0x7f13ec0705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009b30 tx=0x7f13f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:22.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.654+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f13fc05a4d0 con 0x7f1400103790 2024-05-25T06:20:22.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.875+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f140010f140 con 0x7f13ec06e140 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.877+0000 7f13f5ffb700 1 -- 172.21.15.160:0/1313818729 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f140010f140 con 0x7f13ec06e140 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:20:22.877 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:20:22.878 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "12/21 daemons upgraded", 2024-05-25T06:20:22.878 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:20:22.878 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:20:22.878 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:20:22.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f13ec06e140 msgr2=0x7f13ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f13ec06e140 0x7f13ec0705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009b30 tx=0x7f13f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 msgr2=0x7f1400113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 0x7f1400113fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00eb40 tx=0x7f13fc00b040 comp rx=0 tx=0).stop 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 shutdown_connections 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f13ec06e140 0x7f13ec0705f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400102df0 0x7f1400113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 --2- 172.21.15.160:0/1313818729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1400103790 0x7f1400113fc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 >> 172.21.15.160:0/1313818729 conn(0x7f14000f8980 msgr2=0x7f1400107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 shutdown_connections 2024-05-25T06:20:22.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.879+0000 7f14078c9700 1 -- 172.21.15.160:0/1313818729 wait complete. 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.997+0000 7f5943353700 1 -- 172.21.15.160:0/860200478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070c60 msgr2=0x7f593c109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.997+0000 7f5943353700 1 --2- 172.21.15.160:0/860200478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070c60 0x7f593c109170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f592c004970 tx=0x7f592c009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.997+0000 7f5943353700 1 -- 172.21.15.160:0/860200478 shutdown_connections 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.997+0000 7f5943353700 1 --2- 172.21.15.160:0/860200478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070c60 0x7f593c109170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.997+0000 7f5943353700 1 --2- 172.21.15.160:0/860200478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f593c070350 0x7f593c070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.997+0000 7f5943353700 1 -- 172.21.15.160:0/860200478 >> 172.21.15.160:0/860200478 conn(0x7f593c0f89b0 msgr2=0x7f593c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.998+0000 7f5943353700 1 -- 172.21.15.160:0/860200478 shutdown_connections 2024-05-25T06:20:22.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.998+0000 7f5943353700 1 -- 172.21.15.160:0/860200478 wait complete. 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.998+0000 7f5943353700 1 Processor -- start 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.998+0000 7f5943353700 1 -- start start 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f5943353700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 0x7f593c115a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f5943353700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f593c070c60 0x7f593c115f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f5943353700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c116590 con 0x7f593c070350 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f5943353700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c10fb90 con 0x7f593c070c60 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 0x7f593c115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 0x7f593c115a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58806/0 (socket says 172.21.15.160:58806) 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 -- 172.21.15.160:0/1713433525 learned_addr learned my addr 172.21.15.160:0/1713433525 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59408ee700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f593c070c60 0x7f593c115f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 -- 172.21.15.160:0/1713433525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f593c070c60 msgr2=0x7f593c115f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:23.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f593c070c60 0x7f593c115f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 -- 172.21.15.160:0/1713433525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c005040 con 0x7f593c070350 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:22.999+0000 7f59410ef700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 0x7f593c115a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5938002a50 tx=0x7f59380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.000+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5938007cc0 con 0x7f593c070350 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.000+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f593800f040 con 0x7f593c070350 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.000+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c10fe70 con 0x7f593c070350 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.000+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f593800da30 con 0x7f593c070350 2024-05-25T06:20:23.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.000+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c110380 con 0x7f593c070350 2024-05-25T06:20:23.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.001+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593c0719e0 con 0x7f593c070350 2024-05-25T06:20:23.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.001+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f593800db90 con 0x7f593c070350 2024-05-25T06:20:23.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.002+0000 7f5931ffb700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f592806e140 0x7f59280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:23.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.002+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f593808d840 con 0x7f593c070350 2024-05-25T06:20:23.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.002+0000 7f59408ee700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f592806e140 0x7f59280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:23.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.003+0000 7f59408ee700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f592806e140 0x7f59280705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f592c00b370 tx=0x7f592c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:23.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.008+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f593805a720 con 0x7f593c070350 2024-05-25T06:20:23.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:22 smithi160 ceph-mon[112094]: from='client.44279 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:23.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:22 smithi160 ceph-mon[112094]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:23.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:22 smithi160 ceph-mon[112094]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:23.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:22 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3565999850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:23.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:22 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/658204036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:20:23.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.321+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f593c110fe0 con 0x7f593c070350 2024-05-25T06:20:23.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.322+0000 7f5931ffb700 1 -- 172.21.15.160:0/1713433525 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f593805a2b0 con 0x7f593c070350 2024-05-25T06:20:23.322 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:20:23.322 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:20:23.323 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:20:23.323 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:20:23.323 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:20:23.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f592806e140 msgr2=0x7f59280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:23.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f592806e140 0x7f59280705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f592c00b370 tx=0x7f592c009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:23.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 msgr2=0x7f593c115a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:23.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 0x7f593c115a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5938002a50 tx=0x7f59380046b0 comp rx=0 tx=0).stop 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 shutdown_connections 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f593c070350 0x7f593c115a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f592806e140 0x7f59280705f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 --2- 172.21.15.160:0/1713433525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f593c070c60 0x7f593c115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.324+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 >> 172.21.15.160:0/1713433525 conn(0x7f593c0f89b0 msgr2=0x7f593c0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.325+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 shutdown_connections 2024-05-25T06:20:23.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:23.325+0000 7f5943353700 1 -- 172.21.15.160:0/1713433525 wait complete. 2024-05-25T06:20:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:22 smithi203 ceph-mon[83507]: from='client.44279 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:22 smithi203 ceph-mon[83507]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:22 smithi203 ceph-mon[83507]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:22 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3565999850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:22 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/658204036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:20:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:23 smithi160 ceph-mon[112094]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:24.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:23 smithi160 ceph-mon[112094]: pgmap v176: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:23 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:24.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:23 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1713433525' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:20:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:23 smithi203 ceph-mon[83507]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:23 smithi203 ceph-mon[83507]: pgmap v176: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:23 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:23 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1713433525' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:20:26.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:25 smithi203 ceph-mon[83507]: pgmap v177: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:26.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:26.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:25 smithi160 ceph-mon[112094]: pgmap v177: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:27 smithi160 ceph-mon[112094]: pgmap v178: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:27 smithi203 ceph-mon[83507]: pgmap v178: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:28.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:29 smithi203 ceph-mon[83507]: pgmap v179: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:29.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:29 smithi160 ceph-mon[112094]: pgmap v179: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:31.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:31 smithi203 ceph-mon[83507]: pgmap v180: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:31.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:31.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:31 smithi160 ceph-mon[112094]: pgmap v180: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:33 smithi203 ceph-mon[83507]: Upgrade: Updating mds.cephfs.smithi160.tlctof 2024-05-25T06:20:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:33 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:33.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:33 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:20:33.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:33 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:33.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:33 smithi203 ceph-mon[83507]: Deploying daemon mds.cephfs.smithi160.tlctof on smithi160 2024-05-25T06:20:33.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:33 smithi203 ceph-mon[83507]: pgmap v181: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:33.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:33 smithi160 ceph-mon[112094]: Upgrade: Updating mds.cephfs.smithi160.tlctof 2024-05-25T06:20:33.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:33 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:33.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:33 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi160.tlctof", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:20:33.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:33 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:33.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:33 smithi160 ceph-mon[112094]: Deploying daemon mds.cephfs.smithi160.tlctof on smithi160 2024-05-25T06:20:33.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:33 smithi160 ceph-mon[112094]: pgmap v181: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:35.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:35 smithi203 ceph-mon[83507]: osdmap e83: 6 total, 6 up, 6 in 2024-05-25T06:20:35.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:35 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:20:35.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:35 smithi203 ceph-mon[83507]: pgmap v183: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:35.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:35 smithi160 ceph-mon[112094]: osdmap e83: 6 total, 6 up, 6 in 2024-05-25T06:20:35.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:35 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:20:35.970 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:35 smithi160 ceph-mon[112094]: pgmap v183: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:37 smithi160 ceph-mon[112094]: pgmap v184: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:37 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:20:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:37 smithi203 ceph-mon[83507]: pgmap v184: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:38.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:37 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:20:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:39 smithi203 ceph-mon[83507]: pgmap v185: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:40.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:39 smithi160 ceph-mon[112094]: pgmap v185: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:41.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:41.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: pgmap v186: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:41.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:boot 2024-05-25T06:20:41.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:41.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:41 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:20:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:41.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: pgmap v186: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:41.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:boot 2024-05-25T06:20:41.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:41.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:41 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:20:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:44.224 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:43 smithi160 ceph-mon[112094]: pgmap v187: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:43 smithi203 ceph-mon[83507]: pgmap v187: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:45 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:45 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:46.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:45 smithi203 ceph-mon[83507]: pgmap v188: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:45 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:45 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:46.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:45 smithi160 ceph-mon[112094]: pgmap v188: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:47.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:46 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:47.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:46 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:47.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:47.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:46 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:47.935 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:47 smithi160 ceph-mon[112094]: pgmap v189: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:48.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:47 smithi203 ceph-mon[83507]: pgmap v189: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:50.185 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: pgmap v190: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.186 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:49 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: pgmap v190: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:50.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:49 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:52.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:51 smithi203 ceph-mon[83507]: pgmap v191: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:51 smithi203 ceph-mon[83507]: Upgrade: Updating mds.cephfs.smithi203.yltymd 2024-05-25T06:20:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:20:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:51 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:52.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:51 smithi203 ceph-mon[83507]: Deploying daemon mds.cephfs.smithi203.yltymd on smithi203 2024-05-25T06:20:52.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:51 smithi160 ceph-mon[112094]: pgmap v191: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:52.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:51 smithi160 ceph-mon[112094]: Upgrade: Updating mds.cephfs.smithi203.yltymd 2024-05-25T06:20:52.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:52.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.yltymd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:20:52.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:51 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:20:52.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:51 smithi160 ceph-mon[112094]: Deploying daemon mds.cephfs.smithi203.yltymd on smithi203 2024-05-25T06:20:53.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 -- 172.21.15.160:0/2383956926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 msgr2=0x7fddf0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:53.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 --2- 172.21.15.160:0/2383956926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf0101a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdde4004940 tx=0x7fdde4009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:53.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 -- 172.21.15.160:0/2383956926 shutdown_connections 2024-05-25T06:20:53.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 --2- 172.21.15.160:0/2383956926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf0101a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 --2- 172.21.15.160:0/2383956926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf00fb7f0 0x7fddf00fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 -- 172.21.15.160:0/2383956926 >> 172.21.15.160:0/2383956926 conn(0x7fddf0071460 msgr2=0x7fddf0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 -- 172.21.15.160:0/2383956926 shutdown_connections 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.441+0000 7fddf68ce700 1 -- 172.21.15.160:0/2383956926 wait complete. 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddf68ce700 1 Processor -- start 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddf68ce700 1 -- start start 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddf68ce700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf00fb7f0 0x7fddf00fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddf68ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf00fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddf68ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00fe6d0 con 0x7fddf00fc190 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddf68ce700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00fd200 con 0x7fddf00fb7f0 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf00fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf00fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47422/0 (socket says 172.21.15.160:47422) 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddef7fe700 1 -- 172.21.15.160:0/1793645242 learned_addr learned my addr 172.21.15.160:0/1793645242 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:53.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.442+0000 7fddeffff700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf00fb7f0 0x7fddf00fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddef7fe700 1 -- 172.21.15.160:0/1793645242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf00fb7f0 msgr2=0x7fddf00fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddef7fe700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf00fb7f0 0x7fddf00fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddef7fe700 1 -- 172.21.15.160:0/1793645242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4005040 con 0x7fddf00fc190 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddef7fe700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf00fccc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009030 tx=0x7fdde4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde400e040 con 0x7fddf00fc190 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf00fd480 con 0x7fddf00fc190 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdde4004290 con 0x7fddf00fc190 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde4013ea0 con 0x7fddf00fc190 2024-05-25T06:20:53.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.443+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf011a0b0 con 0x7fddf00fc190 2024-05-25T06:20:53.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.444+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf0065f30 con 0x7fddf00fc190 2024-05-25T06:20:53.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.445+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdde40136b0 con 0x7fddf00fc190 2024-05-25T06:20:53.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.445+0000 7fddecff9700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fddd806e140 0x7fddd80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:53.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.445+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdde408eca0 con 0x7fddf00fc190 2024-05-25T06:20:53.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.445+0000 7fddeffff700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fddd806e140 0x7fddd80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:53.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.446+0000 7fddeffff700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fddd806e140 0x7fddd80705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdde0004f40 tx=0x7fdde000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:53.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.450+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdde4058050 con 0x7fddf00fc190 2024-05-25T06:20:53.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.664+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddf0101160 con 0x7fddd806e140 2024-05-25T06:20:53.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.666+0000 7fddecff9700 1 -- 172.21.15.160:0/1793645242 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fddf0101160 con 0x7fddd806e140 2024-05-25T06:20:53.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fddd806e140 msgr2=0x7fddd80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:53.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fddd806e140 0x7fddd80705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdde0004f40 tx=0x7fdde000b040 comp rx=0 tx=0).stop 2024-05-25T06:20:53.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 msgr2=0x7fddf00fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:53.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf00fccc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009030 tx=0x7fdde4009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:53.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 shutdown_connections 2024-05-25T06:20:53.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fddd806e140 0x7fddd80705f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf00fb7f0 0x7fddf00fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 --2- 172.21.15.160:0/1793645242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf00fc190 0x7fddf00fccc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 >> 172.21.15.160:0/1793645242 conn(0x7fddf0071460 msgr2=0x7fddf0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:53.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 shutdown_connections 2024-05-25T06:20:53.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.668+0000 7fddf68ce700 1 -- 172.21.15.160:0/1793645242 wait complete. 2024-05-25T06:20:53.679 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:20:53.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 -- 172.21.15.160:0/3125053463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4065640 msgr2=0x7f68e4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 --2- 172.21.15.160:0/3125053463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4065640 0x7f68e4065ab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68cc004940 tx=0x7f68cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 -- 172.21.15.160:0/3125053463 shutdown_connections 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 --2- 172.21.15.160:0/3125053463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4065640 0x7f68e4065ab0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 --2- 172.21.15.160:0/3125053463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4102e50 0x7f68e4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 -- 172.21.15.160:0/3125053463 >> 172.21.15.160:0/3125053463 conn(0x7f68e4071d00 msgr2=0x7f68e4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 -- 172.21.15.160:0/3125053463 shutdown_connections 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 -- 172.21.15.160:0/3125053463 wait complete. 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 Processor -- start 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.773+0000 7f68e9232700 1 -- start start 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e9232700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4065640 0x7f68e4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e9232700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 0x7f68e41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e9232700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4114720 con 0x7f68e4102e50 2024-05-25T06:20:53.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e9232700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e410db10 con 0x7f68e4065640 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 0x7f68e41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 0x7f68e41140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47442/0 (socket says 172.21.15.160:47442) 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 -- 172.21.15.160:0/3789015982 learned_addr learned my addr 172.21.15.160:0/3789015982 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e2d9d700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4065640 0x7f68e4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 -- 172.21.15.160:0/3789015982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4065640 msgr2=0x7f68e4113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4065640 0x7f68e4113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 -- 172.21.15.160:0/3789015982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc005040 con 0x7f68e4102e50 2024-05-25T06:20:53.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e2d9d700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4065640 0x7f68e4113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e259c700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 0x7f68e41140d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b3b0 tx=0x7f68cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68cc00e040 con 0x7f68e4102e50 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e410dd90 con 0x7f68e4102e50 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.774+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68cc00bbc0 con 0x7f68e4102e50 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.775+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e410e350 con 0x7f68e4102e50 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.775+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68cc025400 con 0x7f68e4102e50 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.775+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e4100100 con 0x7f68e4102e50 2024-05-25T06:20:53.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.776+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f68cc00bd30 con 0x7f68e4102e50 2024-05-25T06:20:53.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.777+0000 7f68db7fe700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68d006e140 0x7f68d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:53.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.777+0000 7f68e2d9d700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68d006e140 0x7f68d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:53.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.777+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f68cc08edf0 con 0x7f68e4102e50 2024-05-25T06:20:53.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.777+0000 7f68e2d9d700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68d006e140 0x7f68d00705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009fd0 tx=0x7f68dc009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:53.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:53.781+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f68cc05bc50 con 0x7f68e4102e50 2024-05-25T06:20:54.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.000+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68e410f230 con 0x7f68d006e140 2024-05-25T06:20:54.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.001+0000 7f68db7fe700 1 -- 172.21.15.160:0/3789015982 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f68e410f230 con 0x7f68d006e140 2024-05-25T06:20:54.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68d006e140 msgr2=0x7f68d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68d006e140 0x7f68d00705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009fd0 tx=0x7f68dc009f20 comp rx=0 tx=0).stop 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 msgr2=0x7f68e41140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 0x7f68e41140d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b3b0 tx=0x7f68cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 shutdown_connections 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68d006e140 0x7f68d00705f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e4065640 0x7f68e4113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 --2- 172.21.15.160:0/3789015982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e4102e50 0x7f68e41140d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 >> 172.21.15.160:0/3789015982 conn(0x7f68e4071d00 msgr2=0x7f68e40faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 shutdown_connections 2024-05-25T06:20:54.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.004+0000 7f68e9232700 1 -- 172.21.15.160:0/3789015982 wait complete. 2024-05-25T06:20:54.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 -- 172.21.15.160:0/4226649501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298102df0 msgr2=0x7fe2981031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 --2- 172.21.15.160:0/4226649501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298102df0 0x7fe2981031c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe294009ce0 tx=0x7fe294004d10 comp rx=0 tx=0).stop 2024-05-25T06:20:54.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 -- 172.21.15.160:0/4226649501 shutdown_connections 2024-05-25T06:20:54.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 --2- 172.21.15.160:0/4226649501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe298103790 0x7fe2980fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 --2- 172.21.15.160:0/4226649501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298102df0 0x7fe2981031c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 -- 172.21.15.160:0/4226649501 >> 172.21.15.160:0/4226649501 conn(0x7fe2980f8980 msgr2=0x7fe2980fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:54.108 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:53 smithi160 ceph-mon[112094]: pgmap v192: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 -- 172.21.15.160:0/4226649501 shutdown_connections 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.107+0000 7fe29fde1700 1 -- 172.21.15.160:0/4226649501 wait complete. 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29fde1700 1 Processor -- start 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29fde1700 1 -- start start 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29fde1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe298102df0 0x7fe298070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29fde1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 0x7fe2980707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29fde1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298070dc0 con 0x7fe298103790 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29fde1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298070f00 con 0x7fe298102df0 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29d37c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 0x7fe2980707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29d37c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 0x7fe2980707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47458/0 (socket says 172.21.15.160:47458) 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29d37c700 1 -- 172.21.15.160:0/3788347395 learned_addr learned my addr 172.21.15.160:0/3788347395 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:54.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29db7d700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe298102df0 0x7fe298070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29d37c700 1 -- 172.21.15.160:0/3788347395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe298102df0 msgr2=0x7fe298070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29d37c700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe298102df0 0x7fe298070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.108+0000 7fe29d37c700 1 -- 172.21.15.160:0/3788347395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe294009940 con 0x7fe298103790 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.109+0000 7fe29d37c700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 0x7fe2980707a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe28800b790 tx=0x7fe288009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.109+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe28800f6f0 con 0x7fe298103790 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.109+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe28800fd20 con 0x7fe298103790 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.109+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2880103a0 con 0x7fe298103790 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.109+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29811a000 con 0x7fe298103790 2024-05-25T06:20:54.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.109+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29811a510 con 0x7fe298103790 2024-05-25T06:20:54.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.110+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2980fe540 con 0x7fe298103790 2024-05-25T06:20:54.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.111+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe288011400 con 0x7fe298103790 2024-05-25T06:20:54.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.111+0000 7fe28e7fc700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe28406e140 0x7fe2840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.111+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe28808d840 con 0x7fe298103790 2024-05-25T06:20:54.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.111+0000 7fe29db7d700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe28406e140 0x7fe2840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.112+0000 7fe29db7d700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe28406e140 0x7fe2840705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b740 tx=0x7fe294004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:54.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.116+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe288056cb0 con 0x7fe298103790 2024-05-25T06:20:54.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.321+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe29810a000 con 0x7fe28406e140 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.332+0000 7fe28e7fc700 1 -- 172.21.15.160:0/3788347395 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe294009780 con 0x7fe28406e140 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (6m) 9s ago 17m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (4m) 9s ago 18m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (4m) 49s ago 17m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (6m) 9s ago 17m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (33s) 9s ago 12m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (14s) 9s ago 12m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (12m) 49s ago 12m 61.4M - 17.2.7-958-g7b01852c baf1547092e7 3c40673ea435 2024-05-25T06:20:54.332 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (12m) 49s ago 12m 116M - 17.2.7-958-g7b01852c baf1547092e7 07627014550b 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (8m) 9s ago 19m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (7m) 49s ago 16m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (5m) 9s ago 19m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (4m) 49s ago 16m 39.6M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (7m) 9s ago 17m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (7m) 49s ago 17m 16.9M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (3m) 9s ago 16m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (3m) 9s ago 15m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (2m) 9s ago 15m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (2m) 49s ago 14m 113M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (85s) 49s ago 14m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (52s) 49s ago 13m 12.6M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:20:54.333 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (7m) 9s ago 17m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:20:54.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe28406e140 msgr2=0x7fe2840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe28406e140 0x7fe2840705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b740 tx=0x7fe294004d10 comp rx=0 tx=0).stop 2024-05-25T06:20:54.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 msgr2=0x7fe2980707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 0x7fe2980707a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe28800b790 tx=0x7fe288009220 comp rx=0 tx=0).stop 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 shutdown_connections 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe28406e140 0x7fe2840705f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe298102df0 0x7fe298070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 --2- 172.21.15.160:0/3788347395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe298103790 0x7fe2980707a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 >> 172.21.15.160:0/3788347395 conn(0x7fe2980f8980 msgr2=0x7fe2980f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 shutdown_connections 2024-05-25T06:20:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.334+0000 7fe29fde1700 1 -- 172.21.15.160:0/3788347395 wait complete. 2024-05-25T06:20:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:53 smithi203 ceph-mon[83507]: pgmap v192: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 -- 172.21.15.160:0/1626259364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 msgr2=0x7ff20c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 --2- 172.21.15.160:0/1626259364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c106e40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff200004940 tx=0x7ff200009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 -- 172.21.15.160:0/1626259364 shutdown_connections 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 --2- 172.21.15.160:0/1626259364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c106e40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 --2- 172.21.15.160:0/1626259364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff20c0fde60 0x7ff20c0fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 -- 172.21.15.160:0/1626259364 >> 172.21.15.160:0/1626259364 conn(0x7ff20c0f7870 msgr2=0x7ff20c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:54.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 -- 172.21.15.160:0/1626259364 shutdown_connections 2024-05-25T06:20:54.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 -- 172.21.15.160:0/1626259364 wait complete. 2024-05-25T06:20:54.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.435+0000 7ff2111d1700 1 Processor -- start 2024-05-25T06:20:54.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff2111d1700 1 -- start start 2024-05-25T06:20:54.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff2111d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff20c0fde60 0x7ff20c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff2111d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff2111d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c114550 con 0x7ff20c0fe800 2024-05-25T06:20:54.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff2111d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c10dac0 con 0x7ff20c0fde60 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47478/0 (socket says 172.21.15.160:47478) 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20a59c700 1 -- 172.21.15.160:0/2521742570 learned_addr learned my addr 172.21.15.160:0/2521742570 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20ad9d700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff20c0fde60 0x7ff20c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20a59c700 1 -- 172.21.15.160:0/2521742570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff20c0fde60 msgr2=0x7ff20c113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20a59c700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff20c0fde60 0x7ff20c113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.436+0000 7ff20a59c700 1 -- 172.21.15.160:0/2521742570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200005040 con 0x7ff20c0fe800 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.437+0000 7ff20a59c700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c113e70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b3b0 tx=0x7ff200009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.437+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20000e040 con 0x7ff20c0fe800 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.437+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c10dda0 con 0x7ff20c0fe800 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.437+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff200013420 con 0x7ff20c0fe800 2024-05-25T06:20:54.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.437+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20001c630 con 0x7ff20c0fe800 2024-05-25T06:20:54.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.437+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c10e2e0 con 0x7ff20c0fe800 2024-05-25T06:20:54.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.438+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c066ad0 con 0x7ff20c0fe800 2024-05-25T06:20:54.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.438+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff200013590 con 0x7ff20c0fe800 2024-05-25T06:20:54.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.439+0000 7ff1f37fe700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff1f406e140 0x7ff1f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.439+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff20008ede0 con 0x7ff20c0fe800 2024-05-25T06:20:54.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.439+0000 7ff20ad9d700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff1f406e140 0x7ff1f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.440+0000 7ff20ad9d700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff1f406e140 0x7ff1f40705f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009bf0 tx=0x7ff1fc0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:54.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.444+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff20005bcc0 con 0x7ff20c0fe800 2024-05-25T06:20:54.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.722+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff20c04c710 con 0x7ff20c0fe800 2024-05-25T06:20:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.723+0000 7ff1f37fe700 1 -- 172.21.15.160:0/2521742570 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+832 (secure 0 0 0) 0x7ff20005b850 con 0x7ff20c0fe800 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:54.799 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 1, 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 17.2.7-958-g7b01852c (7b01852c69b416ba2d4f5e12d093f476d16043d8) quincy (stable)": 1, 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 12 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:20:54.800 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff1f406e140 msgr2=0x7ff1f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff1f406e140 0x7ff1f40705f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009bf0 tx=0x7ff1fc0091a0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 msgr2=0x7ff20c113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c113e70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b3b0 tx=0x7ff200009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 shutdown_connections 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff1f406e140 0x7ff1f40705f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff20c0fde60 0x7ff20c113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 --2- 172.21.15.160:0/2521742570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff20c0fe800 0x7ff20c113e70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 >> 172.21.15.160:0/2521742570 conn(0x7ff20c0f7870 msgr2=0x7ff20c0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 shutdown_connections 2024-05-25T06:20:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.725+0000 7ff2111d1700 1 -- 172.21.15.160:0/2521742570 wait complete. 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 -- 172.21.15.160:0/281512999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 msgr2=0x7ffa58102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 --2- 172.21.15.160:0/281512999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102140 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa54004940 tx=0x7ffa54009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 -- 172.21.15.160:0/281512999 shutdown_connections 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 --2- 172.21.15.160:0/281512999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 0x7ffa580fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 --2- 172.21.15.160:0/281512999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102140 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 -- 172.21.15.160:0/281512999 >> 172.21.15.160:0/281512999 conn(0x7ffa580f78e0 msgr2=0x7ffa580f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 -- 172.21.15.160:0/281512999 shutdown_connections 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.828+0000 7ffa60616700 1 -- 172.21.15.160:0/281512999 wait complete. 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa60616700 1 Processor -- start 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa60616700 1 -- start start 2024-05-25T06:20:54.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa60616700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa60616700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 0x7ffa58103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa60616700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58104b20 con 0x7ffa58101d70 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa60616700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58103650 con 0x7ffa58102710 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47498/0 (socket says 172.21.15.160:47498) 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 -- 172.21.15.160:0/1718576244 learned_addr learned my addr 172.21.15.160:0/1718576244 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5dbb1700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 0x7ffa58103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 -- 172.21.15.160:0/1718576244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 msgr2=0x7ffa58103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 0x7ffa58103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 -- 172.21.15.160:0/1718576244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54005040 con 0x7ffa58101d70 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5dbb1700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 0x7ffa58103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.829+0000 7ffa5e3b2700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009150 tx=0x7ffa54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:54.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.830+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa5400e040 con 0x7ffa58101d70 2024-05-25T06:20:54.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.830+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa54013420 con 0x7ffa58101d70 2024-05-25T06:20:54.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.830+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa581038d0 con 0x7ffa58101d70 2024-05-25T06:20:54.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.830+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa5401c630 con 0x7ffa58101d70 2024-05-25T06:20:54.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.830+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5811a050 con 0x7ffa58101d70 2024-05-25T06:20:54.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.831+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa580fd4c0 con 0x7ffa58101d70 2024-05-25T06:20:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.831+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ffa54013590 con 0x7ffa58101d70 2024-05-25T06:20:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.832+0000 7ffa4effd700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ffa4406e140 0x7ffa440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.832+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffa5408edf0 con 0x7ffa58101d70 2024-05-25T06:20:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.832+0000 7ffa5dbb1700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ffa4406e140 0x7ffa440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.832+0000 7ffa5dbb1700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ffa4406e140 0x7ffa440705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa58104300 tx=0x7ffa48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:54.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:54.836+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ffa54058160 con 0x7ffa58101d70 2024-05-25T06:20:55.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.076+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa5811a240 con 0x7ffa58101d70 2024-05-25T06:20:55.076 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:54 smithi160 ceph-mon[112094]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:55.077 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:54 smithi160 ceph-mon[112094]: osdmap e84: 6 total, 6 up, 6 in 2024-05-25T06:20:55.077 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:54 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:20:55.077 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:54 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2521742570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:55.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.076+0000 7ffa4effd700 1 -- 172.21.15.160:0/1718576244 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1701 (secure 0 0 0) 0x7ffa5405b820 con 0x7ffa58101d70 2024-05-25T06:20:55.077 INFO:teuthology.orchestra.run.smithi160.stdout:e34 2024-05-25T06:20:55.077 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:20:55.077 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:20:55.078 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:20:55.079 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:55.080 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:20:55.081 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:20:55.081 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:20:55.081 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:20:55.081 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/2242985365,v1:172.21.15.203:6827/2242985365] compat {c=[1],r=[1],i=[7ff]}] 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ffa4406e140 msgr2=0x7ffa440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ffa4406e140 0x7ffa440705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa58104300 tx=0x7ffa48009040 comp rx=0 tx=0).stop 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 msgr2=0x7ffa58102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009150 tx=0x7ffa54009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 shutdown_connections 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa58101d70 0x7ffa58102bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ffa4406e140 0x7ffa440705f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 --2- 172.21.15.160:0/1718576244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58102710 0x7ffa58103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.078+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 >> 172.21.15.160:0/1718576244 conn(0x7ffa580f78e0 msgr2=0x7ffa58066530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.079+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 shutdown_connections 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.079+0000 7ffa60616700 1 -- 172.21.15.160:0/1718576244 wait complete. 2024-05-25T06:20:55.082 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 34 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 -- 172.21.15.160:0/1549809193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 msgr2=0x7f30c01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 --2- 172.21.15.160:0/1549809193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c01021f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30b0004940 tx=0x7f30b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 -- 172.21.15.160:0/1549809193 shutdown_connections 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 --2- 172.21.15.160:0/1549809193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30c00fc710 0x7f30c00fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 --2- 172.21.15.160:0/1549809193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c01021f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 -- 172.21.15.160:0/1549809193 >> 172.21.15.160:0/1549809193 conn(0x7f30c0071460 msgr2=0x7f30c0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 -- 172.21.15.160:0/1549809193 shutdown_connections 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.183+0000 7f30c66bc700 1 -- 172.21.15.160:0/1549809193 wait complete. 2024-05-25T06:20:55.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30c66bc700 1 Processor -- start 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30c66bc700 1 -- start start 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30c66bc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30c00fc710 0x7f30c0115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30c66bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c0116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30c66bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01168b0 con 0x7f30c0101e20 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30c66bc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c010fc80 con 0x7f30c00fc710 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30b7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c0116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30b7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c0116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47514/0 (socket says 172.21.15.160:47514) 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30b7fff700 1 -- 172.21.15.160:0/2756315886 learned_addr learned my addr 172.21.15.160:0/2756315886 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30bffff700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30c00fc710 0x7f30c0115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30b7fff700 1 -- 172.21.15.160:0/2756315886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30c00fc710 msgr2=0x7f30c0115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30b7fff700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30c00fc710 0x7f30c0115d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.184+0000 7f30b7fff700 1 -- 172.21.15.160:0/2756315886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0005040 con 0x7f30c0101e20 2024-05-25T06:20:55.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.185+0000 7f30b7fff700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c0116260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c8c0 tx=0x7f30a80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:55.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.185+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30a800a980 con 0x7f30c0101e20 2024-05-25T06:20:55.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.185+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c010ff60 con 0x7f30c0101e20 2024-05-25T06:20:55.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.185+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30a800ce60 con 0x7f30c0101e20 2024-05-25T06:20:55.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.185+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30a80115e0 con 0x7f30c0101e20 2024-05-25T06:20:55.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.185+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c0110520 con 0x7f30c0101e20 2024-05-25T06:20:55.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.186+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c00652f0 con 0x7f30c0101e20 2024-05-25T06:20:55.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.186+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f30a8012400 con 0x7f30c0101e20 2024-05-25T06:20:55.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.187+0000 7f30bd7fa700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30ac06e150 0x7f30ac070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:55.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.187+0000 7f30bffff700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30ac06e150 0x7f30ac070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:55.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.187+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f30a808dc40 con 0x7f30c0101e20 2024-05-25T06:20:55.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.187+0000 7f30bffff700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30ac06e150 0x7f30ac070600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b3b0 tx=0x7f30b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:55.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.192+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30a8093020 con 0x7f30c0101e20 2024-05-25T06:20:55.215 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:54 smithi203 ceph-mon[83507]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:55.215 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:54 smithi203 ceph-mon[83507]: osdmap e84: 6 total, 6 up, 6 in 2024-05-25T06:20:55.215 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:54 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:20:55.216 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:54 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2521742570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:20:55.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.406+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30c0108f20 con 0x7f30ac06e150 2024-05-25T06:20:55.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.408+0000 7f30bd7fa700 1 -- 172.21.15.160:0/2756315886 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f30c0108f20 con 0x7f30ac06e150 2024-05-25T06:20:55.408 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:20:55.408 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:20:55.408 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "14/21 daemons upgraded", 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:20:55.409 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.410+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30ac06e150 msgr2=0x7f30ac070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.410+0000 7f30c66bc700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30ac06e150 0x7f30ac070600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b3b0 tx=0x7f30b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 msgr2=0x7f30c0116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c0116260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c8c0 tx=0x7f30a80044c0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 shutdown_connections 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30ac06e150 0x7f30ac070600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30c00fc710 0x7f30c0115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 --2- 172.21.15.160:0/2756315886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30c0101e20 0x7f30c0116260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 >> 172.21.15.160:0/2756315886 conn(0x7f30c0071460 msgr2=0x7f30c00fb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 shutdown_connections 2024-05-25T06:20:55.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.411+0000 7f30c66bc700 1 -- 172.21.15.160:0/2756315886 wait complete. 2024-05-25T06:20:55.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 -- 172.21.15.160:0/1184142697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fc620 msgr2=0x7f66d8100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 --2- 172.21.15.160:0/1184142697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fc620 0x7f66d8100c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66c8004940 tx=0x7f66c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:55.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 -- 172.21.15.160:0/1184142697 shutdown_connections 2024-05-25T06:20:55.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 --2- 172.21.15.160:0/1184142697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fc620 0x7f66d8100c40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 --2- 172.21.15.160:0/1184142697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d80fbc80 0x7f66d80fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 -- 172.21.15.160:0/1184142697 >> 172.21.15.160:0/1184142697 conn(0x7f66d80f7940 msgr2=0x7f66d80f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 -- 172.21.15.160:0/1184142697 shutdown_connections 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 -- 172.21.15.160:0/1184142697 wait complete. 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 Processor -- start 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.514+0000 7f66df4fc700 1 -- start start 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66df4fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 0x7f66d811a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66df4fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d80fc620 0x7f66d811a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66df4fc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d811aba0 con 0x7f66d80fbc80 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66df4fc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8114180 con 0x7f66d80fc620 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 0x7f66d811a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 0x7f66d811a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56042/0 (socket says 172.21.15.160:56042) 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 -- 172.21.15.160:0/2724537027 learned_addr learned my addr 172.21.15.160:0/2724537027 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dca97700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d80fc620 0x7f66d811a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:55.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 -- 172.21.15.160:0/2724537027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d80fc620 msgr2=0x7f66d811a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d80fc620 0x7f66d811a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 -- 172.21.15.160:0/2724537027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8005040 con 0x7f66d80fbc80 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66dd298700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 0x7f66d811a010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66d4002a50 tx=0x7f66d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d4007cc0 con 0x7f66d80fbc80 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d8114460 con 0x7f66d80fbc80 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66d400f040 con 0x7f66d80fbc80 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.515+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d400da30 con 0x7f66d80fbc80 2024-05-25T06:20:55.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.516+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d81149a0 con 0x7f66d80fbc80 2024-05-25T06:20:55.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.516+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d8066b70 con 0x7f66d80fbc80 2024-05-25T06:20:55.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.517+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f66d400db90 con 0x7f66d80fbc80 2024-05-25T06:20:55.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.517+0000 7f66cdffb700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f66c406e140 0x7f66c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:20:55.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.517+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f66d408d910 con 0x7f66d80fbc80 2024-05-25T06:20:55.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.518+0000 7f66dca97700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f66c406e140 0x7f66c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:20:55.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.518+0000 7f66dca97700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f66c406e140 0x7f66c40705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66d81158b0 tx=0x7f66c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:20:55.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.522+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f66d405a770 con 0x7f66d80fbc80 2024-05-25T06:20:55.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.804+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f66d8115660 con 0x7f66d80fbc80 2024-05-25T06:20:55.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.805+0000 7f66cdffb700 1 -- 172.21.15.160:0/2724537027 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f66d405a300 con 0x7f66d80fbc80 2024-05-25T06:20:55.805 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:20:55.805 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:20:55.806 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:20:55.806 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:20:55.806 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:20:55.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f66c406e140 msgr2=0x7f66c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f66c406e140 0x7f66c40705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66d81158b0 tx=0x7f66c8009e30 comp rx=0 tx=0).stop 2024-05-25T06:20:55.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 msgr2=0x7f66d811a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 0x7f66d811a010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66d4002a50 tx=0x7f66d40046b0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 shutdown_connections 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66d80fbc80 0x7f66d811a010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f66c406e140 0x7f66c40705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 --2- 172.21.15.160:0/2724537027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d80fc620 0x7f66d811a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 >> 172.21.15.160:0/2724537027 conn(0x7f66d80f7940 msgr2=0x7f66d810aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 shutdown_connections 2024-05-25T06:20:55.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:20:55.807+0000 7f66df4fc700 1 -- 172.21.15.160:0/2724537027 wait complete. 2024-05-25T06:20:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:55 smithi160 ceph-mon[112094]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:55 smithi160 ceph-mon[112094]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:55 smithi160 ceph-mon[112094]: pgmap v194: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:55 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1718576244' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:20:56.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:55 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2724537027' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:20:56.286 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:55 smithi203 ceph-mon[83507]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:56.286 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:55 smithi203 ceph-mon[83507]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:56.286 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:55 smithi203 ceph-mon[83507]: pgmap v194: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:56.286 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:55 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1718576244' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:20:56.286 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:55 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2724537027' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:20:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:56 smithi203 ceph-mon[83507]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:56 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:56 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:57.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:56 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:56 smithi160 ceph-mon[112094]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:20:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:56 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:56 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:20:57.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:56 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:20:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:57 smithi160 ceph-mon[112094]: pgmap v195: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:57 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] up:boot 2024-05-25T06:20:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:57 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:20:58.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:20:58.397 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:57 smithi203 ceph-mon[83507]: pgmap v195: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:20:58.397 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:57 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] up:boot 2024-05-25T06:20:58.397 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:57 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:20:58.397 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:20:58.397 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:59 smithi203 ceph-mon[83507]: pgmap v196: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:59 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:20:59 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:59 smithi160 ceph-mon[112094]: pgmap v196: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:59 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:20:59 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:01 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:01 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:02.150 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:01 smithi203 ceph-mon[83507]: pgmap v197: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:01 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:01 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:02.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:01 smithi160 ceph-mon[112094]: pgmap v197: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:04.271 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:03 smithi203 ceph-mon[83507]: pgmap v198: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:04.297 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:03 smithi160 ceph-mon[112094]: pgmap v198: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:05 smithi203 ceph-mon[83507]: pgmap v199: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:05.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:05 smithi160 ceph-mon[112094]: pgmap v199: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:06 smithi203 ceph-mon[83507]: Upgrade: Updating mds.cephfs.smithi203.tndnrb 2024-05-25T06:21:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:21:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:06 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:21:07.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:06 smithi203 ceph-mon[83507]: Deploying daemon mds.cephfs.smithi203.tndnrb on smithi203 2024-05-25T06:21:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:06 smithi160 ceph-mon[112094]: Upgrade: Updating mds.cephfs.smithi203.tndnrb 2024-05-25T06:21:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi203.tndnrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-25T06:21:07.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:06 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:21:07.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:06 smithi160 ceph-mon[112094]: Deploying daemon mds.cephfs.smithi203.tndnrb on smithi203 2024-05-25T06:21:08.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:07 smithi203 ceph-mon[83507]: pgmap v200: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:07 smithi160 ceph-mon[112094]: pgmap v200: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:09 smithi203 ceph-mon[83507]: pgmap v201: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:09 smithi203 ceph-mon[83507]: osdmap e85: 6 total, 6 up, 6 in 2024-05-25T06:21:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:09 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:21:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:09 smithi160 ceph-mon[112094]: pgmap v201: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:09 smithi160 ceph-mon[112094]: osdmap e85: 6 total, 6 up, 6 in 2024-05-25T06:21:10.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:09 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-25T06:21:12.370 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:11 smithi203 ceph-mon[83507]: pgmap v203: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:12.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:11 smithi160 ceph-mon[112094]: pgmap v203: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:13.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:21:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: pgmap v204: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] up:boot 2024-05-25T06:21:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:21:13.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:21:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: pgmap v204: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] up:boot 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-25T06:21:14.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:21:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:15 smithi203 ceph-mon[83507]: pgmap v205: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:15 smithi160 ceph-mon[112094]: pgmap v205: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:17 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:17 smithi203 ceph-mon[83507]: pgmap v206: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:17 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:21:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:17 smithi160 ceph-mon[112094]: pgmap v206: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:20.301 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:19 smithi203 ceph-mon[83507]: pgmap v207: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:19 smithi160 ceph-mon[112094]: pgmap v207: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:21 smithi203 ceph-mon[83507]: pgmap v208: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:21 smithi160 ceph-mon[112094]: pgmap v208: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:23 smithi203 ceph-mon[83507]: pgmap v209: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:23 smithi160 ceph-mon[112094]: pgmap v209: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:25.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.925+0000 7fd51fe9c700 1 -- 172.21.15.160:0/172657037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 msgr2=0x7fd5181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:25.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.925+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/172657037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd5181031b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd514004940 tx=0x7fd514009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:25.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.925+0000 7fd51fe9c700 1 -- 172.21.15.160:0/172657037 shutdown_connections 2024-05-25T06:21:25.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.925+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/172657037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd518103780 0x7fd5180fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:25.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.925+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/172657037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd5181031b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.925+0000 7fd51fe9c700 1 -- 172.21.15.160:0/172657037 >> 172.21.15.160:0/172657037 conn(0x7fd5180f8990 msgr2=0x7fd5180fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 -- 172.21.15.160:0/172657037 shutdown_connections 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 -- 172.21.15.160:0/172657037 wait complete. 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 Processor -- start 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 -- start start 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd518070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd518103780 0x7fd5180707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518072160 con 0x7fd518102de0 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.926+0000 7fd51fe9c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518070ce0 con 0x7fd518103780 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd518070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:25.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd518070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52924/0 (socket says 172.21.15.160:52924) 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 -- 172.21.15.160:0/2058200952 learned_addr learned my addr 172.21.15.160:0/2058200952 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51d437700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd518103780 0x7fd5180707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 -- 172.21.15.160:0/2058200952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd518103780 msgr2=0x7fd5180707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd518103780 0x7fd5180707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 -- 172.21.15.160:0/2058200952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd514005040 con 0x7fd518102de0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51dc38700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd518070260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd51400b3b0 tx=0x7fd514009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51400e040 con 0x7fd518102de0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd514013420 con 0x7fd518102de0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd518070f60 con 0x7fd518102de0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51401c630 con 0x7fd518102de0 2024-05-25T06:21:25.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.927+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51811a110 con 0x7fd518102de0 2024-05-25T06:21:25.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.928+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5181000f0 con 0x7fd518102de0 2024-05-25T06:21:25.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.929+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd514013590 con 0x7fd518102de0 2024-05-25T06:21:25.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.929+0000 7fd50e7fc700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd50406e140 0x7fd5040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:25.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.929+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd51408ec30 con 0x7fd518102de0 2024-05-25T06:21:25.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.929+0000 7fd51d437700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd50406e140 0x7fd5040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:25.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.930+0000 7fd51d437700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd50406e140 0x7fd5040705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd518071940 tx=0x7fd50800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:25.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:25.934+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd5140580a0 con 0x7fd518102de0 2024-05-25T06:21:26.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.153+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5180717b0 con 0x7fd50406e140 2024-05-25T06:21:26.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.154+0000 7fd50e7fc700 1 -- 172.21.15.160:0/2058200952 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd5180717b0 con 0x7fd50406e140 2024-05-25T06:21:26.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd50406e140 msgr2=0x7fd5040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd50406e140 0x7fd5040705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd518071940 tx=0x7fd50800b040 comp rx=0 tx=0).stop 2024-05-25T06:21:26.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 msgr2=0x7fd518070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd518070260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd51400b3b0 tx=0x7fd514009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:26.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 shutdown_connections 2024-05-25T06:21:26.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd518102de0 0x7fd518070260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd50406e140 0x7fd5040705f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 --2- 172.21.15.160:0/2058200952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd518103780 0x7fd5180707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 >> 172.21.15.160:0/2058200952 conn(0x7fd5180f8990 msgr2=0x7fd518107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:26.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 shutdown_connections 2024-05-25T06:21:26.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.157+0000 7fd51fe9c700 1 -- 172.21.15.160:0/2058200952 wait complete. 2024-05-25T06:21:26.167 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:21:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:25 smithi160 ceph-mon[112094]: pgmap v210: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 -- 172.21.15.160:0/101039358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 msgr2=0x7fd144102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 --2- 172.21.15.160:0/101039358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144102160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd134004940 tx=0x7fd134009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 -- 172.21.15.160:0/101039358 shutdown_connections 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 --2- 172.21.15.160:0/101039358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1440fbd40 0x7fd1440fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 --2- 172.21.15.160:0/101039358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144102160 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 -- 172.21.15.160:0/101039358 >> 172.21.15.160:0/101039358 conn(0x7fd144071d00 msgr2=0x7fd144072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 -- 172.21.15.160:0/101039358 shutdown_connections 2024-05-25T06:21:26.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 -- 172.21.15.160:0/101039358 wait complete. 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 Processor -- start 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.261+0000 7fd148a25700 1 -- start start 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd148a25700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1440fbd40 0x7fd144102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd148a25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd148a25700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441035f0 con 0x7fd144101d90 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd148a25700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd144103730 con 0x7fd1440fbd40 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52946/0 (socket says 172.21.15.160:52946) 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 -- 172.21.15.160:0/2410201173 learned_addr learned my addr 172.21.15.160:0/2410201173 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd14259c700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1440fbd40 0x7fd144102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 -- 172.21.15.160:0/2410201173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1440fbd40 msgr2=0x7fd144102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1440fbd40 0x7fd144102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 -- 172.21.15.160:0/2410201173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134005040 con 0x7fd144101d90 2024-05-25T06:21:26.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd141d9b700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144103060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd138002a50 tx=0x7fd1380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd138007cc0 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd13800f040 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13800da30 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd144119e80 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.262+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14411a2a0 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.263+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1440fd430 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.264+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd13800db90 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.264+0000 7fd132ffd700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd12c06e150 0x7fd12c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.265+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd13808d920 con 0x7fd144101d90 2024-05-25T06:21:26.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.265+0000 7fd14259c700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd12c06e150 0x7fd12c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.265+0000 7fd14259c700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd12c06e150 0x7fd12c070600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd13400b3b0 tx=0x7fd134009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:26.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.269+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd138056c50 con 0x7fd144101d90 2024-05-25T06:21:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:25 smithi203 ceph-mon[83507]: pgmap v210: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:26.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.481+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd14404bed0 con 0x7fd12c06e150 2024-05-25T06:21:26.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.483+0000 7fd132ffd700 1 -- 172.21.15.160:0/2410201173 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd14404bed0 con 0x7fd12c06e150 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd12c06e150 msgr2=0x7fd12c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd12c06e150 0x7fd12c070600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd13400b3b0 tx=0x7fd134009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 msgr2=0x7fd144103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144103060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd138002a50 tx=0x7fd1380046b0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 shutdown_connections 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd12c06e150 0x7fd12c070600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1440fbd40 0x7fd144102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 --2- 172.21.15.160:0/2410201173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd144101d90 0x7fd144103060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 >> 172.21.15.160:0/2410201173 conn(0x7fd144071d00 msgr2=0x7fd1440fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:26.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 shutdown_connections 2024-05-25T06:21:26.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.485+0000 7fd148a25700 1 -- 172.21.15.160:0/2410201173 wait complete. 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.588+0000 7f982f63a700 1 -- 172.21.15.160:0/715550643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fbf20 msgr2=0x7f982810d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.588+0000 7f982f63a700 1 --2- 172.21.15.160:0/715550643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fbf20 0x7f982810d310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9818004970 tx=0x7f9818009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.588+0000 7f982f63a700 1 -- 172.21.15.160:0/715550643 shutdown_connections 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.588+0000 7f982f63a700 1 --2- 172.21.15.160:0/715550643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fbf20 0x7f982810d310 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.588+0000 7f982f63a700 1 --2- 172.21.15.160:0/715550643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98280fb610 0x7f98280fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.588+0000 7f982f63a700 1 -- 172.21.15.160:0/715550643 >> 172.21.15.160:0/715550643 conn(0x7f9828072380 msgr2=0x7f9828072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 -- 172.21.15.160:0/715550643 shutdown_connections 2024-05-25T06:21:26.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 -- 172.21.15.160:0/715550643 wait complete. 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 Processor -- start 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 -- start start 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 0x7f9828119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98280fbf20 0x7f982811a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982811abf0 con 0x7f98280fb610 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.589+0000 7f982f63a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828114140 con 0x7f98280fbf20 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 0x7f9828119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 0x7f9828119fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52956/0 (socket says 172.21.15.160:52956) 2024-05-25T06:21:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 -- 172.21.15.160:0/1277376918 learned_addr learned my addr 172.21.15.160:0/1277376918 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982cbd5700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98280fbf20 0x7f982811a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 -- 172.21.15.160:0/1277376918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98280fbf20 msgr2=0x7f982811a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98280fbf20 0x7f982811a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 -- 172.21.15.160:0/1277376918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818005040 con 0x7f98280fb610 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982d3d6700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 0x7f9828119fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f982400b790 tx=0x7f9824009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f982400f6f0 con 0x7f98280fb610 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f982400fd20 con 0x7f98280fb610 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98240103a0 con 0x7f98280fb610 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.590+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98281143c0 con 0x7f98280fb610 2024-05-25T06:21:26.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.591+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828114880 con 0x7f98280fb610 2024-05-25T06:21:26.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.592+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98280fc3d0 con 0x7f98280fb610 2024-05-25T06:21:26.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.593+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9824011400 con 0x7f98280fb610 2024-05-25T06:21:26.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.593+0000 7f981dffb700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f981406e140 0x7f98140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.593+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f982408d960 con 0x7f98280fb610 2024-05-25T06:21:26.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.593+0000 7f982cbd5700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f981406e140 0x7f98140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.594+0000 7f982cbd5700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f981406e140 0x7f98140705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9818004f50 tx=0x7f9818009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:26.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.598+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f982405a840 con 0x7f98280fb610 2024-05-25T06:21:26.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.801+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f98281156e0 con 0x7f981406e140 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.812+0000 7f981dffb700 1 -- 172.21.15.160:0/1277376918 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f98281156e0 con 0x7f981406e140 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (7m) 42s ago 18m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (4m) 42s ago 18m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (4m) 11s ago 17m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (6m) 42s ago 18m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (66s) 42s ago 13m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (47s) 42s ago 12m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:21:26.812 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (14s) 11s ago 12m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (30s) 11s ago 12m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (8m) 42s ago 19m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (8m) 11s ago 17m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (5m) 42s ago 19m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (5m) 11s ago 17m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (7m) 42s ago 18m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (7m) 11s ago 17m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (4m) 42s ago 16m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (3m) 42s ago 16m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (3m) 42s ago 15m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (2m) 11s ago 15m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (118s) 11s ago 14m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:21:26.813 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (85s) 11s ago 14m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:21:26.814 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (7m) 42s ago 17m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:21:26.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f981406e140 msgr2=0x7f98140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f981406e140 0x7f98140705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9818004f50 tx=0x7f9818009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:26.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 msgr2=0x7f9828119fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 0x7f9828119fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f982400b790 tx=0x7f9824009250 comp rx=0 tx=0).stop 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 shutdown_connections 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98280fb610 0x7f9828119fd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f981406e140 0x7f98140705f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 --2- 172.21.15.160:0/1277376918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98280fbf20 0x7f982811a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 >> 172.21.15.160:0/1277376918 conn(0x7f9828072380 msgr2=0x7f98280fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 shutdown_connections 2024-05-25T06:21:26.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.814+0000 7f982f63a700 1 -- 172.21.15.160:0/1277376918 wait complete. 2024-05-25T06:21:26.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 -- 172.21.15.160:0/637063674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038102de0 msgr2=0x7f40381031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 --2- 172.21.15.160:0/637063674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038102de0 0x7f40381031b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4034004940 tx=0x7f4034009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:26.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 -- 172.21.15.160:0/637063674 shutdown_connections 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 --2- 172.21.15.160:0/637063674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4038103780 0x7f40380fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 --2- 172.21.15.160:0/637063674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038102de0 0x7f40381031b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 -- 172.21.15.160:0/637063674 >> 172.21.15.160:0/637063674 conn(0x7f40380f8990 msgr2=0x7f40380fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 -- 172.21.15.160:0/637063674 shutdown_connections 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 -- 172.21.15.160:0/637063674 wait complete. 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 Processor -- start 2024-05-25T06:21:26.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.916+0000 7f40400f9700 1 -- start start 2024-05-25T06:21:26.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f40400f9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f40400f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f40400f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038114500 con 0x7f4038103780 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f40400f9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403810db50 con 0x7f4038102de0 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52978/0 (socket says 172.21.15.160:52978) 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 -- 172.21.15.160:0/2400649811 learned_addr learned my addr 172.21.15.160:0/2400649811 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403de95700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 -- 172.21.15.160:0/2400649811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4038102de0 msgr2=0x7f4038113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 -- 172.21.15.160:0/2400649811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034005040 con 0x7f4038103780 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.917+0000 7f403d694700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 0x7f4038113eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f402800c800 tx=0x7f4028004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.918+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402800a760 con 0x7f4038103780 2024-05-25T06:21:26.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.918+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f402800ad90 con 0x7f4038103780 2024-05-25T06:21:26.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.918+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403810de30 con 0x7f4038103780 2024-05-25T06:21:26.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.918+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40280113a0 con 0x7f4038103780 2024-05-25T06:21:26.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.918+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403810e3f0 con 0x7f4038103780 2024-05-25T06:21:26.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.919+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40381000f0 con 0x7f4038103780 2024-05-25T06:21:26.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.919+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4028012400 con 0x7f4038103780 2024-05-25T06:21:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.920+0000 7f402e7fc700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f402406e140 0x7f40240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.920+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f402808db90 con 0x7f4038103780 2024-05-25T06:21:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.920+0000 7f403de95700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f402406e140 0x7f40240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.920+0000 7f403de95700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f402406e140 0x7f40240705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f403400b3b0 tx=0x7f4034009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:26.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:26.924+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f402805aa70 con 0x7f4038103780 2024-05-25T06:21:27.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.207+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4038109ee0 con 0x7f4038103780 2024-05-25T06:21:27.208 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:27 smithi160 ceph-mon[112094]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:27.208 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:27 smithi160 ceph-mon[112094]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:27.208 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:27 smithi160 ceph-mon[112094]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:27.208 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:27 smithi160 ceph-mon[112094]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:27.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.208+0000 7f402e7fc700 1 -- 172.21.15.160:0/2400649811 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f402805a600 con 0x7f4038103780 2024-05-25T06:21:27.208 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:27.209 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:21:27.210 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:21:27.210 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:21:27.210 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:21:27.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.210+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f402406e140 msgr2=0x7f40240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.210+0000 7f40400f9700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f402406e140 0x7f40240705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f403400b3b0 tx=0x7f4034009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.210+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 msgr2=0x7f4038113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.210+0000 7f40400f9700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 0x7f4038113eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f402800c800 tx=0x7f4028004470 comp rx=0 tx=0).stop 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 shutdown_connections 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f402406e140 0x7f40240705f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 --2- 172.21.15.160:0/2400649811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 >> 172.21.15.160:0/2400649811 conn(0x7f40380f8990 msgr2=0x7f4038107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 shutdown_connections 2024-05-25T06:21:27.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.211+0000 7f40400f9700 1 -- 172.21.15.160:0/2400649811 wait complete. 2024-05-25T06:21:27.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 -- 172.21.15.160:0/645431958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 msgr2=0x7f98bc0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 --2- 172.21.15.160:0/645431958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc0fbce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98b0004940 tx=0x7f98b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 -- 172.21.15.160:0/645431958 shutdown_connections 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 --2- 172.21.15.160:0/645431958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc0fbce0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 --2- 172.21.15.160:0/645431958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98bc101cd0 0x7f98bc1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 -- 172.21.15.160:0/645431958 >> 172.21.15.160:0/645431958 conn(0x7f98bc0f7860 msgr2=0x7f98bc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 -- 172.21.15.160:0/645431958 shutdown_connections 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.314+0000 7f98c3974700 1 -- 172.21.15.160:0/645431958 wait complete. 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c3974700 1 Processor -- start 2024-05-25T06:21:27.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c3974700 1 -- start start 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c3974700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98bc101cd0 0x7f98bc1127d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c3974700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c3974700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc113360 con 0x7f98bc102670 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c3974700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc10c960 con 0x7f98bc101cd0 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52994/0 (socket says 172.21.15.160:52994) 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 -- 172.21.15.160:0/3868765978 learned_addr learned my addr 172.21.15.160:0/3868765978 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 -- 172.21.15.160:0/3868765978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98bc101cd0 msgr2=0x7f98bc1127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c1710700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98bc101cd0 0x7f98bc1127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98bc101cd0 0x7f98bc1127d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 -- 172.21.15.160:0/3868765978 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b0005040 con 0x7f98bc102670 2024-05-25T06:21:27.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.315+0000 7f98c0f0f700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc112d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c00 tx=0x7f98b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:27.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.316+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b0021070 con 0x7f98bc102670 2024-05-25T06:21:27.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.316+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98b0013cd0 con 0x7f98bc102670 2024-05-25T06:21:27.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.316+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc10cbe0 con 0x7f98bc102670 2024-05-25T06:21:27.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.316+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b000e070 con 0x7f98bc102670 2024-05-25T06:21:27.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.316+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc10d0c0 con 0x7f98bc102670 2024-05-25T06:21:27.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.317+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc0fd420 con 0x7f98bc102670 2024-05-25T06:21:27.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.317+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f98b001caf0 con 0x7f98bc102670 2024-05-25T06:21:27.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.318+0000 7f98b9ffb700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f98a406e140 0x7f98a40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:27.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.318+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f98b008ef60 con 0x7f98bc102670 2024-05-25T06:21:27.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.318+0000 7f98c1710700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f98a406e140 0x7f98a40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:27.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.318+0000 7f98c1710700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f98a406e140 0x7f98a40705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009f10 tx=0x7f98b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:27.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.322+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f98b00c3850 con 0x7f98bc102670 2024-05-25T06:21:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:27 smithi203 ceph-mon[83507]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:27 smithi203 ceph-mon[83507]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:27 smithi203 ceph-mon[83507]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:27 smithi203 ceph-mon[83507]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:27.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.562+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f98bc108db0 con 0x7f98bc102670 2024-05-25T06:21:27.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.563+0000 7f98b9ffb700 1 -- 172.21.15.160:0/3868765978 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f98b005bf00 con 0x7f98bc102670 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:21:27.563 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:21:27.564 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:21:27.565 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:21:27.566 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:21:27.566 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:21:27.566 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:21:27.566 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:21:27.566 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:21:27.566 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f98a406e140 msgr2=0x7f98a40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f98a406e140 0x7f98a40705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009f10 tx=0x7f98b400b040 comp rx=0 tx=0).stop 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 msgr2=0x7f98bc112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc112d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c00 tx=0x7f98b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 shutdown_connections 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f98a406e140 0x7f98a40705f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98bc101cd0 0x7f98bc1127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 --2- 172.21.15.160:0/3868765978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98bc102670 0x7f98bc112d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 >> 172.21.15.160:0/3868765978 conn(0x7f98bc0f7860 msgr2=0x7f98bc0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 shutdown_connections 2024-05-25T06:21:27.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.565+0000 7f98c3974700 1 -- 172.21.15.160:0/3868765978 wait complete. 2024-05-25T06:21:27.568 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:21:27.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.669+0000 7fea3a065700 1 -- 172.21.15.160:0/443921907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 msgr2=0x7fea340fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.669+0000 7fea3a065700 1 --2- 172.21.15.160:0/443921907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea340fb600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea1c004940 tx=0x7fea1c009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:27.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.669+0000 7fea3a065700 1 -- 172.21.15.160:0/443921907 shutdown_connections 2024-05-25T06:21:27.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.669+0000 7fea3a065700 1 --2- 172.21.15.160:0/443921907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea340fb600 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.669+0000 7fea3a065700 1 --2- 172.21.15.160:0/443921907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 0x7fea341020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.669+0000 7fea3a065700 1 -- 172.21.15.160:0/443921907 >> 172.21.15.160:0/443921907 conn(0x7fea34071460 msgr2=0x7fea34071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 -- 172.21.15.160:0/443921907 shutdown_connections 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 -- 172.21.15.160:0/443921907 wait complete. 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 Processor -- start 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 -- start start 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 0x7fea34113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:27.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea341140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:27.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea34114740 con 0x7fea34102670 2024-05-25T06:21:27.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.670+0000 7fea3a065700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3410db30 con 0x7fea34101cd0 2024-05-25T06:21:27.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea341140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:27.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea341140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53026/0 (socket says 172.21.15.160:53026) 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 -- 172.21.15.160:0/762991759 learned_addr learned my addr 172.21.15.160:0/762991759 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea337fe700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 0x7fea34113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 -- 172.21.15.160:0/762991759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 msgr2=0x7fea34113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 0x7fea34113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 -- 172.21.15.160:0/762991759 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c005040 con 0x7fea34102670 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea337fe700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 0x7fea34113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea2bfff700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea341140f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b3b0 tx=0x7fea1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea1c00e040 con 0x7fea34102670 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea1c013420 con 0x7fea34102670 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3410ddb0 con 0x7fea34102670 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea1c01c630 con 0x7fea34102670 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.671+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3410e370 con 0x7fea34102670 2024-05-25T06:21:27.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.672+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea34065200 con 0x7fea34102670 2024-05-25T06:21:27.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.673+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fea1c013590 con 0x7fea34102670 2024-05-25T06:21:27.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.673+0000 7fea30ff9700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea2006e140 0x7fea200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:27.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.673+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fea1c08ed70 con 0x7fea34102670 2024-05-25T06:21:27.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.673+0000 7fea337fe700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea2006e140 0x7fea200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:27.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.674+0000 7fea337fe700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea2006e140 0x7fea200705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea24009fd0 tx=0x7fea24009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:27.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.678+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fea1c05bd00 con 0x7fea34102670 2024-05-25T06:21:27.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.898+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea34109ed0 con 0x7fea2006e140 2024-05-25T06:21:27.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.900+0000 7fea30ff9700 1 -- 172.21.15.160:0/762991759 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fea34109ed0 con 0x7fea2006e140 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:21:27.900 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:21:27.901 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:21:27.901 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:21:27.901 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:21:27.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea2006e140 msgr2=0x7fea200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea2006e140 0x7fea200705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea24009fd0 tx=0x7fea24009ee0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 msgr2=0x7fea341140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:27.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea341140f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b3b0 tx=0x7fea1c009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 shutdown_connections 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea2006e140 0x7fea200705f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea34101cd0 0x7fea34113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 --2- 172.21.15.160:0/762991759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea34102670 0x7fea341140f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 >> 172.21.15.160:0/762991759 conn(0x7fea34071460 msgr2=0x7fea341009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 shutdown_connections 2024-05-25T06:21:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:27.902+0000 7fea3a065700 1 -- 172.21.15.160:0/762991759 wait complete. 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.006+0000 7f4df2f35700 1 -- 172.21.15.160:0/3161027474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec102690 msgr2=0x7f4dec0fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.006+0000 7f4df2f35700 1 --2- 172.21.15.160:0/3161027474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec102690 0x7f4dec0fbd00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8004940 tx=0x7f4dd8009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 -- 172.21.15.160:0/3161027474 shutdown_connections 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 --2- 172.21.15.160:0/3161027474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec102690 0x7f4dec0fbd00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 --2- 172.21.15.160:0/3161027474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec101cf0 0x7f4dec1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 -- 172.21.15.160:0/3161027474 >> 172.21.15.160:0/3161027474 conn(0x7f4dec0f7840 msgr2=0x7f4dec0f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 -- 172.21.15.160:0/3161027474 shutdown_connections 2024-05-25T06:21:28.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 -- 172.21.15.160:0/3161027474 wait complete. 2024-05-25T06:21:28.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 Processor -- start 2024-05-25T06:21:28.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 -- start start 2024-05-25T06:21:28.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 0x7f4dec1127e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:28.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec102690 0x7f4dec112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:28.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.007+0000 7f4df2f35700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec113370 con 0x7f4dec101cf0 2024-05-25T06:21:28.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df2f35700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec10c970 con 0x7f4dec102690 2024-05-25T06:21:28.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 0x7f4dec1127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 0x7f4dec1127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53054/0 (socket says 172.21.15.160:53054) 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 -- 172.21.15.160:0/1342267877 learned_addr learned my addr 172.21.15.160:0/1342267877 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4debfff700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec102690 0x7f4dec112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 -- 172.21.15.160:0/1342267877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec102690 msgr2=0x7f4dec112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec102690 0x7f4dec112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 -- 172.21.15.160:0/1342267877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8005040 con 0x7f4dec101cf0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df0cd1700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 0x7f4dec1127e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4de0010d50 tx=0x7f4de000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de000ed60 con 0x7f4dec101cf0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4de00090d0 con 0x7f4dec101cf0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec10cbf0 con 0x7f4dec101cf0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de0012670 con 0x7f4dec101cf0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.008+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec10d100 con 0x7f4dec101cf0 2024-05-25T06:21:28.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.009+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec0fd440 con 0x7f4dec101cf0 2024-05-25T06:21:28.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.010+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4de0018030 con 0x7f4dec101cf0 2024-05-25T06:21:28.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.010+0000 7f4de97fa700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ddc06e140 0x7f4ddc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:28.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.010+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4de0016070 con 0x7f4dec101cf0 2024-05-25T06:21:28.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.010+0000 7f4debfff700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ddc06e140 0x7f4ddc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:28.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.011+0000 7f4debfff700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ddc06e140 0x7f4ddc0705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8004890 tx=0x7f4dd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:28.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.015+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4de005cf60 con 0x7f4dec101cf0 2024-05-25T06:21:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:28 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2400649811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:28 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3868765978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:21:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:28 smithi160 ceph-mon[112094]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:28.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.298+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4dec108e20 con 0x7f4dec101cf0 2024-05-25T06:21:28.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.298+0000 7f4de97fa700 1 -- 172.21.15.160:0/1342267877 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4de005caf0 con 0x7f4dec101cf0 2024-05-25T06:21:28.299 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:21:28.299 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:21:28.299 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:21:28.299 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:21:28.299 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ddc06e140 msgr2=0x7f4ddc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ddc06e140 0x7f4ddc0705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8004890 tx=0x7f4dd8009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 msgr2=0x7f4dec1127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 0x7f4dec1127e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4de0010d50 tx=0x7f4de000b040 comp rx=0 tx=0).stop 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 shutdown_connections 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dec101cf0 0x7f4dec1127e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ddc06e140 0x7f4ddc0705f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:28.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 --2- 172.21.15.160:0/1342267877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec102690 0x7f4dec112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:28.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 >> 172.21.15.160:0/1342267877 conn(0x7f4dec0f7840 msgr2=0x7f4dec0f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:28.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 shutdown_connections 2024-05-25T06:21:28.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:28.301+0000 7f4df2f35700 1 -- 172.21.15.160:0/1342267877 wait complete. 2024-05-25T06:21:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:28 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2400649811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:21:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:28 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3868765978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:21:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:28 smithi203 ceph-mon[83507]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:21:28.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:29 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1342267877' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:21:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:29 smithi203 ceph-mon[83507]: pgmap v212: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:29 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1342267877' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:21:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:29 smithi160 ceph-mon[112094]: pgmap v212: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:31 smithi203 ceph-mon[83507]: pgmap v213: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:31 smithi160 ceph-mon[112094]: pgmap v213: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:34.272 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:33 smithi160 ceph-mon[112094]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:33 smithi203 ceph-mon[83507]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:35 smithi203 ceph-mon[83507]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:35 smithi160 ceph-mon[112094]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:37 smithi160 ceph-mon[112094]: pgmap v216: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:37 smithi203 ceph-mon[83507]: pgmap v216: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:39 smithi203 ceph-mon[83507]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:39 smithi160 ceph-mon[112094]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:41 smithi203 ceph-mon[83507]: pgmap v218: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:41 smithi160 ceph-mon[112094]: pgmap v218: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:43 smithi203 ceph-mon[83507]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:43 smithi160 ceph-mon[112094]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:45 smithi203 ceph-mon[83507]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:45 smithi160 ceph-mon[112094]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:47 smithi160 ceph-mon[112094]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:47 smithi203 ceph-mon[83507]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:49 smithi203 ceph-mon[83507]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:49 smithi160 ceph-mon[112094]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:51 smithi203 ceph-mon[83507]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:51 smithi160 ceph-mon[112094]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:53 smithi203 ceph-mon[83507]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:53 smithi160 ceph-mon[112094]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:55 smithi203 ceph-mon[83507]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:55 smithi160 ceph-mon[112094]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:57 smithi160 ceph-mon[112094]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:57 smithi203 ceph-mon[83507]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:21:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/815663441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 msgr2=0x7f4eb81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/815663441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb81020c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8004940 tx=0x7f4ea8009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/815663441 shutdown_connections 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/815663441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb8102690 0x7f4eb80fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/815663441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb81020c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/815663441 >> 172.21.15.160:0/815663441 conn(0x7f4eb80f7860 msgr2=0x7f4eb80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.416+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/815663441 shutdown_connections 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/815663441 wait complete. 2024-05-25T06:21:58.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 Processor -- start 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 -- start start 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb8115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb8102690 0x7f4eb81160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8116740 con 0x7f4eb8101cf0 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.417+0000 7f4ebe2ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb810fd20 con 0x7f4eb8102690 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb8115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb8115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38056/0 (socket says 172.21.15.160:38056) 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 -- 172.21.15.160:0/1349272205 learned_addr learned my addr 172.21.15.160:0/1349272205 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:58.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb77fe700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb8102690 0x7f4eb81160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 -- 172.21.15.160:0/1349272205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb8102690 msgr2=0x7f4eb81160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb8102690 0x7f4eb81160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 -- 172.21.15.160:0/1349272205 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8005040 con 0x7f4eb8101cf0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb7fff700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb8115bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800b8b0 tx=0x7f4ea8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ea800e040 con 0x7f4eb8101cf0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb810ffa0 con 0x7f4eb8101cf0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ea8004290 con 0x7f4eb8101cf0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ea801c3f0 con 0x7f4eb8101cf0 2024-05-25T06:21:58.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.418+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb8110560 con 0x7f4eb8101cf0 2024-05-25T06:21:58.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.419+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb80fd440 con 0x7f4eb8101cf0 2024-05-25T06:21:58.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.420+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4ea8013420 con 0x7f4eb8101cf0 2024-05-25T06:21:58.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.420+0000 7f4eb4ff9700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ea006e140 0x7f4ea00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:58.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.421+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4ea808ed90 con 0x7f4eb8101cf0 2024-05-25T06:21:58.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.421+0000 7f4eb77fe700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ea006e140 0x7f4ea00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:58.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.421+0000 7f4eb77fe700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ea006e140 0x7f4ea00705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8111450 tx=0x7f4eac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:58.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.425+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4ea805bc70 con 0x7f4eb8101cf0 2024-05-25T06:21:58.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.637+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eb8111270 con 0x7f4ea006e140 2024-05-25T06:21:58.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.638+0000 7f4eb4ff9700 1 -- 172.21.15.160:0/1349272205 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4eb8111270 con 0x7f4ea006e140 2024-05-25T06:21:58.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ea006e140 msgr2=0x7f4ea00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ea006e140 0x7f4ea00705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8111450 tx=0x7f4eac00b040 comp rx=0 tx=0).stop 2024-05-25T06:21:58.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 msgr2=0x7f4eb8115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb8115bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800b8b0 tx=0x7f4ea8009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 shutdown_connections 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101cf0 0x7f4eb8115bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4ea006e140 0x7f4ea00705f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 --2- 172.21.15.160:0/1349272205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb8102690 0x7f4eb81160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 >> 172.21.15.160:0/1349272205 conn(0x7f4eb80f7860 msgr2=0x7f4eb80f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 shutdown_connections 2024-05-25T06:21:58.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.640+0000 7f4ebe2ae700 1 -- 172.21.15.160:0/1349272205 wait complete. 2024-05-25T06:21:58.652 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:21:58.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 -- 172.21.15.160:0/776585594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87480fbd40 msgr2=0x7f87480fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 --2- 172.21.15.160:0/776585594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87480fbd40 0x7f87480fc1b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8734004940 tx=0x7f8734009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:58.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 -- 172.21.15.160:0/776585594 shutdown_connections 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 --2- 172.21.15.160:0/776585594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87480fbd40 0x7f87480fc1b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 --2- 172.21.15.160:0/776585594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8748101d90 0x7f8748102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 -- 172.21.15.160:0/776585594 >> 172.21.15.160:0/776585594 conn(0x7f8748071d00 msgr2=0x7f8748072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 -- 172.21.15.160:0/776585594 shutdown_connections 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 -- 172.21.15.160:0/776585594 wait complete. 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 Processor -- start 2024-05-25T06:21:58.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.744+0000 7f874f00c700 1 -- start start 2024-05-25T06:21:58.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f874f00c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87480fbd40 0x7f874810ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:58.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f874f00c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 0x7f874810f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f874f00c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874810f770 con 0x7f8748101d90 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f874f00c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874810f8b0 con 0x7f87480fbd40 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 0x7f874810f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 0x7f874810f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38080/0 (socket says 172.21.15.160:38080) 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 -- 172.21.15.160:0/3028677881 learned_addr learned my addr 172.21.15.160:0/3028677881 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f874cda8700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87480fbd40 0x7f874810ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 -- 172.21.15.160:0/3028677881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87480fbd40 msgr2=0x7f874810ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87480fbd40 0x7f874810ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 -- 172.21.15.160:0/3028677881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734005040 con 0x7f8748101d90 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f8747fff700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 0x7f874810f230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f873400b3b0 tx=0x7f8734009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873400e040 con 0x7f8748101d90 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f873400bbc0 con 0x7f8748101d90 2024-05-25T06:21:58.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8748117ba0 con 0x7f8748101d90 2024-05-25T06:21:58.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.745+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873401c3f0 con 0x7f8748101d90 2024-05-25T06:21:58.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.746+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8748118130 con 0x7f8748101d90 2024-05-25T06:21:58.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.747+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87480fd430 con 0x7f8748101d90 2024-05-25T06:21:58.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.747+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f873400bd30 con 0x7f8748101d90 2024-05-25T06:21:58.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.747+0000 7f87457fa700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f873806e140 0x7f87380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:58.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.748+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f873408ee60 con 0x7f8748101d90 2024-05-25T06:21:58.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.748+0000 7f874cda8700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f873806e140 0x7f87380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:58.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.748+0000 7f874cda8700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f873806e140 0x7f87380705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f873c007fd0 tx=0x7f873c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:58.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.752+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8734058190 con 0x7f8748101d90 2024-05-25T06:21:58.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.966+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f874804bed0 con 0x7f873806e140 2024-05-25T06:21:58.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.967+0000 7f87457fa700 1 -- 172.21.15.160:0/3028677881 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f874804bed0 con 0x7f873806e140 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f873806e140 msgr2=0x7f87380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f873806e140 0x7f87380705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f873c007fd0 tx=0x7f873c009040 comp rx=0 tx=0).stop 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 msgr2=0x7f874810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 0x7f874810f230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f873400b3b0 tx=0x7f8734009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 shutdown_connections 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f873806e140 0x7f87380705f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87480fbd40 0x7f874810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 --2- 172.21.15.160:0/3028677881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8748101d90 0x7f874810f230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 >> 172.21.15.160:0/3028677881 conn(0x7f8748071d00 msgr2=0x7f87480fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 shutdown_connections 2024-05-25T06:21:58.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:58.970+0000 7f874f00c700 1 -- 172.21.15.160:0/3028677881 wait complete. 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.073+0000 7f88a3c33700 1 -- 172.21.15.160:0/1321295213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 msgr2=0x7f889c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.073+0000 7f88a3c33700 1 --2- 172.21.15.160:0/1321295213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c0fcdf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f888c004940 tx=0x7f888c009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.073+0000 7f88a3c33700 1 -- 172.21.15.160:0/1321295213 shutdown_connections 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.073+0000 7f88a3c33700 1 --2- 172.21.15.160:0/1321295213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c0fcdf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.073+0000 7f88a3c33700 1 --2- 172.21.15.160:0/1321295213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f889c102de0 0x7f889c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.073+0000 7f88a3c33700 1 -- 172.21.15.160:0/1321295213 >> 172.21.15.160:0/1321295213 conn(0x7f889c0f8950 msgr2=0x7f889c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.074+0000 7f88a3c33700 1 -- 172.21.15.160:0/1321295213 shutdown_connections 2024-05-25T06:21:59.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.074+0000 7f88a3c33700 1 -- 172.21.15.160:0/1321295213 wait complete. 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.074+0000 7f88a3c33700 1 Processor -- start 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.074+0000 7f88a3c33700 1 -- start start 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.074+0000 7f88a3c33700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f889c102de0 0x7f889c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.074+0000 7f88a3c33700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a3c33700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c114730 con 0x7f889c103780 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a3c33700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c10db20 con 0x7f889c102de0 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38108/0 (socket says 172.21.15.160:38108) 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 -- 172.21.15.160:0/667332407 learned_addr learned my addr 172.21.15.160:0/667332407 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:59.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a19cf700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f889c102de0 0x7f889c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 -- 172.21.15.160:0/667332407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f889c102de0 msgr2=0x7f889c113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f889c102de0 0x7f889c113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 -- 172.21.15.160:0/667332407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c005040 con 0x7f889c103780 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a11ce700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c1140e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f888c009100 tx=0x7f888c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f888c00e040 con 0x7f889c103780 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f888c009af0 con 0x7f889c103780 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c10dda0 con 0x7f889c103780 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f888c01c3f0 con 0x7f889c103780 2024-05-25T06:21:59.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.075+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c10e360 con 0x7f889c103780 2024-05-25T06:21:59.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.077+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c0fe530 con 0x7f889c103780 2024-05-25T06:21:59.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.077+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f888c00baf0 con 0x7f889c103780 2024-05-25T06:21:59.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.078+0000 7f88927fc700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888806e140 0x7f88880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.078+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f888c08edb0 con 0x7f889c103780 2024-05-25T06:21:59.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.078+0000 7f88a19cf700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888806e140 0x7f88880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.078+0000 7f88a19cf700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888806e140 0x7f88880705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8898004eb0 tx=0x7f889800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:59.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.083+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f888c05bc90 con 0x7f889c103780 2024-05-25T06:21:59.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.288+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f889c109ec0 con 0x7f888806e140 2024-05-25T06:21:59.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.299+0000 7f88927fc700 1 -- 172.21.15.160:0/667332407 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f889c109ec0 con 0x7f888806e140 2024-05-25T06:21:59.300 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:21:59.300 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (7m) 74s ago 18m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:21:59.300 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (5m) 74s ago 19m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:21:59.300 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (5m) 44s ago 18m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (7m) 74s ago 18m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (98s) 74s ago 13m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (79s) 74s ago 13m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (47s) 44s ago 13m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (62s) 44s ago 13m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (9m) 74s ago 20m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (8m) 44s ago 18m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (6m) 74s ago 20m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (5m) 44s ago 17m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (8m) 74s ago 19m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (8m) 44s ago 18m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (4m) 74s ago 17m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (4m) 74s ago 16m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:21:59.301 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (3m) 74s ago 16m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:21:59.302 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (3m) 44s ago 15m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:21:59.302 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (2m) 44s ago 15m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:21:59.302 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (117s) 44s ago 15m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:21:59.302 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (8m) 74s ago 18m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888806e140 msgr2=0x7f88880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888806e140 0x7f88880705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8898004eb0 tx=0x7f889800b040 comp rx=0 tx=0).stop 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 msgr2=0x7f889c1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c1140e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f888c009100 tx=0x7f888c009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 shutdown_connections 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888806e140 0x7f88880705f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f889c102de0 0x7f889c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 --2- 172.21.15.160:0/667332407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f889c103780 0x7f889c1140e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 >> 172.21.15.160:0/667332407 conn(0x7f889c0f8950 msgr2=0x7f889c0f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 shutdown_connections 2024-05-25T06:21:59.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.301+0000 7f88a3c33700 1 -- 172.21.15.160:0/667332407 wait complete. 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.403+0000 7fca885cb700 1 -- 172.21.15.160:0/4210795866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80102670 msgr2=0x7fca800fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.403+0000 7fca885cb700 1 --2- 172.21.15.160:0/4210795866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80102670 0x7fca800fbce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca74009ce0 tx=0x7fca74004d10 comp rx=0 tx=0).stop 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.403+0000 7fca885cb700 1 -- 172.21.15.160:0/4210795866 shutdown_connections 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.403+0000 7fca885cb700 1 --2- 172.21.15.160:0/4210795866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80102670 0x7fca800fbce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.403+0000 7fca885cb700 1 --2- 172.21.15.160:0/4210795866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca80101cd0 0x7fca801020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.403+0000 7fca885cb700 1 -- 172.21.15.160:0/4210795866 >> 172.21.15.160:0/4210795866 conn(0x7fca800f7860 msgr2=0x7fca800f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 -- 172.21.15.160:0/4210795866 shutdown_connections 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 -- 172.21.15.160:0/4210795866 wait complete. 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 Processor -- start 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 -- start start 2024-05-25T06:21:59.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 0x7fca801129f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca80102670 0x7fca80112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80113530 con 0x7fca80101cd0 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca885cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8010dd10 con 0x7fca80102670 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca86367700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 0x7fca801129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca86367700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 0x7fca801129f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38122/0 (socket says 172.21.15.160:38122) 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.404+0000 7fca86367700 1 -- 172.21.15.160:0/3730622991 learned_addr learned my addr 172.21.15.160:0/3730622991 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca85b66700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca80102670 0x7fca80112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca86367700 1 -- 172.21.15.160:0/3730622991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca80102670 msgr2=0x7fca80112f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca86367700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca80102670 0x7fca80112f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca86367700 1 -- 172.21.15.160:0/3730622991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca74009940 con 0x7fca80101cd0 2024-05-25T06:21:59.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca86367700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 0x7fca801129f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fca70002a50 tx=0x7fca700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca70007cc0 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8010dff0 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca7000f040 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca7000da30 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.405+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8010e580 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.406+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca800664b0 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.407+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fca7000db90 con 0x7fca80101cd0 2024-05-25T06:21:59.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.407+0000 7fca7effd700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca6806e140 0x7fca680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.407+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fca7008d910 con 0x7fca80101cd0 2024-05-25T06:21:59.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.407+0000 7fca85b66700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca6806e140 0x7fca680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.408+0000 7fca85b66700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca6806e140 0x7fca680705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca7400b740 tx=0x7fca74004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:59.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.413+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca70056c40 con 0x7fca80101cd0 2024-05-25T06:21:59.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.691+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca8010f130 con 0x7fca80101cd0 2024-05-25T06:21:59.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.691+0000 7fca7effd700 1 -- 172.21.15.160:0/3730622991 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fca7005a300 con 0x7fca80101cd0 2024-05-25T06:21:59.693 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:21:59.693 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:21:59.693 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:21:59.693 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:59.694 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:21:59.694 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:21:59.694 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:59.694 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:21:59.694 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:21:59.694 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:59.695 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:21:59.695 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:21:59.695 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:21:59.695 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:21:59.696 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:21:59.696 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:21:59.696 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:21:59.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca6806e140 msgr2=0x7fca680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca6806e140 0x7fca680705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca7400b740 tx=0x7fca74004d10 comp rx=0 tx=0).stop 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 msgr2=0x7fca801129f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 0x7fca801129f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fca70002a50 tx=0x7fca700046b0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 shutdown_connections 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca80101cd0 0x7fca801129f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca6806e140 0x7fca680705f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 --2- 172.21.15.160:0/3730622991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca80102670 0x7fca80112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 >> 172.21.15.160:0/3730622991 conn(0x7fca800f7860 msgr2=0x7fca800f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 shutdown_connections 2024-05-25T06:21:59.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.694+0000 7fca885cb700 1 -- 172.21.15.160:0/3730622991 wait complete. 2024-05-25T06:21:59.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 -- 172.21.15.160:0/2300055847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24071f30 msgr2=0x7f1a24070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 --2- 172.21.15.160:0/2300055847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24071f30 0x7f1a24070300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1a18004970 tx=0x7f1a18009e30 comp rx=0 tx=0).stop 2024-05-25T06:21:59.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 -- 172.21.15.160:0/2300055847 shutdown_connections 2024-05-25T06:21:59.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 --2- 172.21.15.160:0/2300055847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a24070840 0x7f1a24070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 --2- 172.21.15.160:0/2300055847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24071f30 0x7f1a24070300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 -- 172.21.15.160:0/2300055847 >> 172.21.15.160:0/2300055847 conn(0x7f1a240f89b0 msgr2=0x7f1a240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:21:59.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 -- 172.21.15.160:0/2300055847 shutdown_connections 2024-05-25T06:21:59.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 -- 172.21.15.160:0/2300055847 wait complete. 2024-05-25T06:21:59.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 Processor -- start 2024-05-25T06:21:59.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.796+0000 7f1a2b067700 1 -- start start 2024-05-25T06:21:59.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a2b067700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 0x7f1a24119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a2b067700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a24071f30 0x7f1a2411a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a2b067700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2411aae0 con 0x7f1a24070840 2024-05-25T06:21:59.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a2b067700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a241140c0 con 0x7f1a24071f30 2024-05-25T06:21:59.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 0x7f1a24119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 0x7f1a24119f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38152/0 (socket says 172.21.15.160:38152) 2024-05-25T06:21:59.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 -- 172.21.15.160:0/1729525606 learned_addr learned my addr 172.21.15.160:0/1729525606 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:21:59.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a23fff700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a24071f30 0x7f1a2411a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 -- 172.21.15.160:0/1729525606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a24071f30 msgr2=0x7f1a2411a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:21:59.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a24071f30 0x7f1a2411a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:21:59.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 -- 172.21.15.160:0/1729525606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18005040 con 0x7f1a24070840 2024-05-25T06:21:59.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a28e03700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 0x7f1a24119f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a18004f50 tx=0x7f1a18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:59.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1800e040 con 0x7f1a24070840 2024-05-25T06:21:59.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a18009400 con 0x7f1a24070840 2024-05-25T06:21:59.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a24114340 con 0x7f1a24070840 2024-05-25T06:21:59.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1801c3f0 con 0x7f1a24070840 2024-05-25T06:21:59.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.797+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a24114880 con 0x7f1a24070840 2024-05-25T06:21:59.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.798+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a24071a40 con 0x7f1a24070840 2024-05-25T06:21:59.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.799+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1a1800bc10 con 0x7f1a24070840 2024-05-25T06:21:59.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.799+0000 7f1a217fa700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a1406e140 0x7f1a140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:21:59.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.800+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1a1808ee00 con 0x7f1a24070840 2024-05-25T06:21:59.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.800+0000 7f1a23fff700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a1406e140 0x7f1a140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:21:59.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.800+0000 7f1a23fff700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a1406e140 0x7f1a140705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a241157f0 tx=0x7f1a1000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:21:59.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:21:59.805+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1a1805bce0 con 0x7f1a24070840 2024-05-25T06:22:00.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.053+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a241155a0 con 0x7f1a24070840 2024-05-25T06:22:00.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.053+0000 7f1a217fa700 1 -- 172.21.15.160:0/1729525606 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f1a1805b870 con 0x7f1a24070840 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:22:00.055 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:22:00.056 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:22:00.057 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:00.058 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:00.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a1406e140 msgr2=0x7f1a140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a1406e140 0x7f1a140705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a241157f0 tx=0x7f1a1000b040 comp rx=0 tx=0).stop 2024-05-25T06:22:00.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 msgr2=0x7f1a24119f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 0x7f1a24119f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a18004f50 tx=0x7f1a18009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:00.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 shutdown_connections 2024-05-25T06:22:00.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a24070840 0x7f1a24119f50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a1406e140 0x7f1a140705f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 --2- 172.21.15.160:0/1729525606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a24071f30 0x7f1a2411a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 >> 172.21.15.160:0/1729525606 conn(0x7f1a240f89b0 msgr2=0x7f1a240f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:00.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 shutdown_connections 2024-05-25T06:22:00.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.055+0000 7f1a2b067700 1 -- 172.21.15.160:0/1729525606 wait complete. 2024-05-25T06:22:00.061 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:22:00.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 -- 172.21.15.160:0/1526666724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 msgr2=0x7f21101020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 --2- 172.21.15.160:0/1526666724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f21101020e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2104004940 tx=0x7f2104009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 -- 172.21.15.160:0/1526666724 shutdown_connections 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 --2- 172.21.15.160:0/1526666724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21101026b0 0x7f21100fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 --2- 172.21.15.160:0/1526666724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f21101020e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 -- 172.21.15.160:0/1526666724 >> 172.21.15.160:0/1526666724 conn(0x7f21100f7860 msgr2=0x7f21100f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 -- 172.21.15.160:0/1526666724 shutdown_connections 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.160+0000 7f21171f9700 1 -- 172.21.15.160:0/1526666724 wait complete. 2024-05-25T06:22:00.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.161+0000 7f21171f9700 1 Processor -- start 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.161+0000 7f21171f9700 1 -- start start 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.161+0000 7f21171f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f2110102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.161+0000 7f21171f9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21101026b0 0x7f21101030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.161+0000 7f21171f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2110104ac0 con 0x7f2110101d10 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.161+0000 7f21171f9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21101035f0 con 0x7f21101026b0 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f2110102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f2110102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38168/0 (socket says 172.21.15.160:38168) 2024-05-25T06:22:00.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 -- 172.21.15.160:0/1156153738 learned_addr learned my addr 172.21.15.160:0/1156153738 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f210ffff700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21101026b0 0x7f21101030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 -- 172.21.15.160:0/1156153738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21101026b0 msgr2=0x7f21101030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21101026b0 0x7f21101030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 -- 172.21.15.160:0/1156153738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2104005040 con 0x7f2110101d10 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f2114f95700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f2110102b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2104009150 tx=0x7f2104009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f210400e040 con 0x7f2110101d10 2024-05-25T06:22:00.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2104013420 con 0x7f2110101d10 2024-05-25T06:22:00.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2110103870 con 0x7f2110101d10 2024-05-25T06:22:00.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f210401c610 con 0x7f2110101d10 2024-05-25T06:22:00.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.162+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211011a130 con 0x7f2110101d10 2024-05-25T06:22:00.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.163+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21100fd460 con 0x7f2110101d10 2024-05-25T06:22:00.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.164+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f210400baf0 con 0x7f2110101d10 2024-05-25T06:22:00.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.164+0000 7f210d7fa700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f210006e140 0x7f21000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:00.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.164+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f210408ed60 con 0x7f2110101d10 2024-05-25T06:22:00.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.164+0000 7f210ffff700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f210006e140 0x7f21000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:00.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.165+0000 7f210ffff700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f210006e140 0x7f21000705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21101042a0 tx=0x7f20fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:00.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.169+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2104058110 con 0x7f2110101d10 2024-05-25T06:22:00.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.382+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21100f8f50 con 0x7f210006e140 2024-05-25T06:22:00.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:59 smithi160 ceph-mon[112094]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:00.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:59 smithi160 ceph-mon[112094]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:00.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:59 smithi160 ceph-mon[112094]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:00.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:21:59 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3730622991' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:22:00.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.384+0000 7f210d7fa700 1 -- 172.21.15.160:0/1156153738 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f21100f8f50 con 0x7f210006e140 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:22:00.384 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:22:00.385 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:22:00.385 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:22:00.385 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:22:00.385 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:22:00.385 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:22:00.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f210006e140 msgr2=0x7f21000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f210006e140 0x7f21000705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21101042a0 tx=0x7f20fc00b040 comp rx=0 tx=0).stop 2024-05-25T06:22:00.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 msgr2=0x7f2110102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f2110102b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2104009150 tx=0x7f2104009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:00.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 shutdown_connections 2024-05-25T06:22:00.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2110101d10 0x7f2110102b70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f210006e140 0x7f21000705f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 --2- 172.21.15.160:0/1156153738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21101026b0 0x7f21101030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 >> 172.21.15.160:0/1156153738 conn(0x7f21100f7860 msgr2=0x7f21100f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:00.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 shutdown_connections 2024-05-25T06:22:00.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.386+0000 7f21171f9700 1 -- 172.21.15.160:0/1156153738 wait complete. 2024-05-25T06:22:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:59 smithi203 ceph-mon[83507]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:59 smithi203 ceph-mon[83507]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:59 smithi203 ceph-mon[83507]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:00.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:21:59 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3730622991' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:22:00.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 -- 172.21.15.160:0/1406155581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 msgr2=0x7f01a01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 --2- 172.21.15.160:0/1406155581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a01031c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0190004940 tx=0x7f0190009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 -- 172.21.15.160:0/1406155581 shutdown_connections 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 --2- 172.21.15.160:0/1406155581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 0x7f01a00fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 --2- 172.21.15.160:0/1406155581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a01031c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 -- 172.21.15.160:0/1406155581 >> 172.21.15.160:0/1406155581 conn(0x7f01a00f8980 msgr2=0x7f01a00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 -- 172.21.15.160:0/1406155581 shutdown_connections 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.489+0000 7f01a787c700 1 -- 172.21.15.160:0/1406155581 wait complete. 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a787c700 1 Processor -- start 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a787c700 1 -- start start 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a787c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a0113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a787c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 0x7f01a0113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a787c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0114630 con 0x7f01a0102df0 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a787c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a010da20 con 0x7f01a0103790 2024-05-25T06:22:00.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a0113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a0113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38194/0 (socket says 172.21.15.160:38194) 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 -- 172.21.15.160:0/701450844 learned_addr learned my addr 172.21.15.160:0/701450844 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a4e17700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 0x7f01a0113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 -- 172.21.15.160:0/701450844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 msgr2=0x7f01a0113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 0x7f01a0113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 -- 172.21.15.160:0/701450844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190005040 con 0x7f01a0102df0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a4e17700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 0x7f01a0113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.490+0000 7f01a5618700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a0113aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0190009b30 tx=0x7f0190009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.491+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019000e070 con 0x7f01a0102df0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.491+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a010dca0 con 0x7f01a0102df0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.491+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0190013cd0 con 0x7f01a0102df0 2024-05-25T06:22:00.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.491+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019000b720 con 0x7f01a0102df0 2024-05-25T06:22:00.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.491+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a010e180 con 0x7f01a0102df0 2024-05-25T06:22:00.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.492+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a0100100 con 0x7f01a0102df0 2024-05-25T06:22:00.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.492+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f019001cab0 con 0x7f01a0102df0 2024-05-25T06:22:00.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.493+0000 7f0195ffb700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f018c06e140 0x7f018c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:00.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.493+0000 7f01a4e17700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f018c06e140 0x7f018c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:00.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.493+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f019008f510 con 0x7f01a0102df0 2024-05-25T06:22:00.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.493+0000 7f01a4e17700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f018c06e140 0x7f018c0705f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f019c004600 tx=0x7f019c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:00.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.498+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f019005c3f0 con 0x7f01a0102df0 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.780+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f01a010f0f0 con 0x7f01a0102df0 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.780+0000 7f0195ffb700 1 -- 172.21.15.160:0/701450844 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f019005bf80 con 0x7f01a0102df0 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:22:00.780 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f018c06e140 msgr2=0x7f018c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f018c06e140 0x7f018c0705f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f019c004600 tx=0x7f019c00b040 comp rx=0 tx=0).stop 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 msgr2=0x7f01a0113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a0113aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0190009b30 tx=0x7f0190009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 shutdown_connections 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01a0102df0 0x7f01a0113aa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f018c06e140 0x7f018c0705f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 --2- 172.21.15.160:0/701450844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a0103790 0x7f01a0113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 >> 172.21.15.160:0/701450844 conn(0x7f01a00f8980 msgr2=0x7f01a01073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 shutdown_connections 2024-05-25T06:22:00.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:00.783+0000 7f01a787c700 1 -- 172.21.15.160:0/701450844 wait complete. 2024-05-25T06:22:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:01 smithi160 ceph-mon[112094]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:01 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1729525606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:22:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:01 smithi160 ceph-mon[112094]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:01.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:01 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/701450844' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:22:01.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:01 smithi160 ceph-mon[112094]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:01.313 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:01 smithi203 ceph-mon[83507]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:01.313 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:01 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1729525606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:22:01.313 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:01 smithi203 ceph-mon[83507]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:01.313 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:01 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/701450844' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:22:01.314 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:01 smithi203 ceph-mon[83507]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:03 smithi203 ceph-mon[83507]: pgmap v229: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:03 smithi160 ceph-mon[112094]: pgmap v229: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:05 smithi203 ceph-mon[83507]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:06.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:05 smithi160 ceph-mon[112094]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:07 smithi203 ceph-mon[83507]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:07 smithi160 ceph-mon[112094]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:09 smithi203 ceph-mon[83507]: pgmap v232: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:09 smithi160 ceph-mon[112094]: pgmap v232: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:11 smithi203 ceph-mon[83507]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:11 smithi160 ceph-mon[112094]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:13 smithi203 ceph-mon[83507]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:13 smithi160 ceph-mon[112094]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:15 smithi203 ceph-mon[83507]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:15 smithi160 ceph-mon[112094]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:17 smithi203 ceph-mon[83507]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:17 smithi160 ceph-mon[112094]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:19 smithi203 ceph-mon[83507]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:19 smithi160 ceph-mon[112094]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:21 smithi203 ceph-mon[83507]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:21 smithi160 ceph-mon[112094]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:24 smithi203 ceph-mon[83507]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:23 smithi160 ceph-mon[112094]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:25 smithi203 ceph-mon[83507]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:25 smithi160 ceph-mon[112094]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:28 smithi160 ceph-mon[112094]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:28 smithi203 ceph-mon[83507]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:29 smithi203 ceph-mon[83507]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:29 smithi160 ceph-mon[112094]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 -- 172.21.15.160:0/2567169951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 msgr2=0x7f23e40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 --2- 172.21.15.160:0/2567169951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e40fce10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f23cc004940 tx=0x7f23cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 -- 172.21.15.160:0/2567169951 shutdown_connections 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 --2- 172.21.15.160:0/2567169951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e40fce10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 --2- 172.21.15.160:0/2567169951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e4102e00 0x7f23e41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 -- 172.21.15.160:0/2567169951 >> 172.21.15.160:0/2567169951 conn(0x7f23e40f89b0 msgr2=0x7f23e40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 -- 172.21.15.160:0/2567169951 shutdown_connections 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.900+0000 7f23e97f4700 1 -- 172.21.15.160:0/2567169951 wait complete. 2024-05-25T06:22:30.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23e97f4700 1 Processor -- start 2024-05-25T06:22:30.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23e97f4700 1 -- start start 2024-05-25T06:22:30.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23e97f4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e4102e00 0x7f23e4113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:30.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23e97f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e4113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:30.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23e97f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41144f0 con 0x7f23e41037a0 2024-05-25T06:22:30.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23e97f4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e410daf0 con 0x7f23e4102e00 2024-05-25T06:22:30.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e4113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:30.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e4113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56082/0 (socket says 172.21.15.160:56082) 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.901+0000 7f23dbfff700 1 -- 172.21.15.160:0/3624837940 learned_addr learned my addr 172.21.15.160:0/3624837940 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23e2ffd700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e4102e00 0x7f23e4113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23dbfff700 1 -- 172.21.15.160:0/3624837940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e4102e00 msgr2=0x7f23e4113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23dbfff700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e4102e00 0x7f23e4113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23dbfff700 1 -- 172.21.15.160:0/3624837940 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc005040 con 0x7f23e41037a0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23dbfff700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e4113ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009100 tx=0x7f23cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23cc00e040 con 0x7f23e41037a0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23cc013420 con 0x7f23e41037a0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e410dd70 con 0x7f23e41037a0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23cc01c610 con 0x7f23e41037a0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.902+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e410e330 con 0x7f23e41037a0 2024-05-25T06:22:30.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.903+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e41052d0 con 0x7f23e41037a0 2024-05-25T06:22:30.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.904+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f23cc013590 con 0x7f23e41037a0 2024-05-25T06:22:30.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.904+0000 7f23db7fe700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f23d006e140 0x7f23d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:30.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.904+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f23cc08ee10 con 0x7f23e41037a0 2024-05-25T06:22:30.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.904+0000 7f23e2ffd700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f23d006e140 0x7f23d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:30.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.905+0000 7f23e2ffd700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f23d006e140 0x7f23d00705f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009fd0 tx=0x7f23d4009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:30.909 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:30.909+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f23cc05bcf0 con 0x7f23e41037a0 2024-05-25T06:22:31.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.129+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23e410efb0 con 0x7f23d006e140 2024-05-25T06:22:31.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.130+0000 7f23db7fe700 1 -- 172.21.15.160:0/3624837940 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f23e410efb0 con 0x7f23d006e140 2024-05-25T06:22:31.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f23d006e140 msgr2=0x7f23d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f23d006e140 0x7f23d00705f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009fd0 tx=0x7f23d4009e50 comp rx=0 tx=0).stop 2024-05-25T06:22:31.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 msgr2=0x7f23e4113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e4113ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009100 tx=0x7f23cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:31.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 shutdown_connections 2024-05-25T06:22:31.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f23d006e140 0x7f23d00705f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e4102e00 0x7f23e4113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 --2- 172.21.15.160:0/3624837940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e41037a0 0x7f23e4113ea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 >> 172.21.15.160:0/3624837940 conn(0x7f23e40f89b0 msgr2=0x7f23e4102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:31.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 shutdown_connections 2024-05-25T06:22:31.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.132+0000 7f23e97f4700 1 -- 172.21.15.160:0/3624837940 wait complete. 2024-05-25T06:22:31.143 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:22:31.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 -- 172.21.15.160:0/2905138264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84101cf0 msgr2=0x7f4f841020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 --2- 172.21.15.160:0/2905138264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84101cf0 0x7f4f841020c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004940 tx=0x7f4f74009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:31.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 -- 172.21.15.160:0/2905138264 shutdown_connections 2024-05-25T06:22:31.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 --2- 172.21.15.160:0/2905138264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84102690 0x7f4f840fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 --2- 172.21.15.160:0/2905138264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84101cf0 0x7f4f841020c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 -- 172.21.15.160:0/2905138264 >> 172.21.15.160:0/2905138264 conn(0x7f4f840f7860 msgr2=0x7f4f840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:31.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 -- 172.21.15.160:0/2905138264 shutdown_connections 2024-05-25T06:22:31.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.236+0000 7f4f89108700 1 -- 172.21.15.160:0/2905138264 wait complete. 2024-05-25T06:22:31.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f89108700 1 Processor -- start 2024-05-25T06:22:31.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f89108700 1 -- start start 2024-05-25T06:22:31.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f89108700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84101cf0 0x7f4f84102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f89108700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 0x7f4f84103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f89108700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84103620 con 0x7f4f84102690 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f89108700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84103760 con 0x7f4f84101cf0 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f8259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 0x7f4f84103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f8259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 0x7f4f84103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56090/0 (socket says 172.21.15.160:56090) 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f8259c700 1 -- 172.21.15.160:0/1983363570 learned_addr learned my addr 172.21.15.160:0/1983363570 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.237+0000 7f4f82d9d700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84101cf0 0x7f4f84102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f8259c700 1 -- 172.21.15.160:0/1983363570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84101cf0 msgr2=0x7f4f84102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f8259c700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84101cf0 0x7f4f84102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f8259c700 1 -- 172.21.15.160:0/1983363570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f74005040 con 0x7f4f84102690 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f82d9d700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84101cf0 0x7f4f84102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f8259c700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 0x7f4f84103090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800eb40 tx=0x7f4f7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f78007cc0 con 0x7f4f84102690 2024-05-25T06:22:31.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f841039b0 con 0x7f4f84102690 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f7800f040 con 0x7f4f84102690 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f780095e0 con 0x7f4f84102690 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.238+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8411a230 con 0x7f4f84102690 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.239+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f840fd440 con 0x7f4f84102690 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.240+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4f78009740 con 0x7f4f84102690 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.240+0000 7f4f6b7fe700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f6c06e140 0x7f4f6c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.240+0000 7f4f82d9d700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f6c06e140 0x7f4f6c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.240+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4f7808d650 con 0x7f4f84102690 2024-05-25T06:22:31.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.241+0000 7f4f82d9d700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f6c06e140 0x7f4f6c0705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b0d0 tx=0x7f4f74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:31.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.245+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4f78056a30 con 0x7f4f84102690 2024-05-25T06:22:31.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.459+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f8404bed0 con 0x7f4f6c06e140 2024-05-25T06:22:31.460 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.460+0000 7f4f6b7fe700 1 -- 172.21.15.160:0/1983363570 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4f8404bed0 con 0x7f4f6c06e140 2024-05-25T06:22:31.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f6c06e140 msgr2=0x7f4f6c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f6c06e140 0x7f4f6c0705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b0d0 tx=0x7f4f74009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 msgr2=0x7f4f84103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 0x7f4f84103090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800eb40 tx=0x7f4f7800b040 comp rx=0 tx=0).stop 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 shutdown_connections 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f6c06e140 0x7f4f6c0705f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84101cf0 0x7f4f84102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 --2- 172.21.15.160:0/1983363570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f84102690 0x7f4f84103090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 >> 172.21.15.160:0/1983363570 conn(0x7f4f840f7860 msgr2=0x7f4f840f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 shutdown_connections 2024-05-25T06:22:31.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.462+0000 7f4f89108700 1 -- 172.21.15.160:0/1983363570 wait complete. 2024-05-25T06:22:31.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 -- 172.21.15.160:0/1114888399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 msgr2=0x7fea900fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 --2- 172.21.15.160:0/1114888399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fb970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea80004940 tx=0x7fea80009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:31.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 -- 172.21.15.160:0/1114888399 shutdown_connections 2024-05-25T06:22:31.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 --2- 172.21.15.160:0/1114888399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea900fbf40 0x7fea90101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 --2- 172.21.15.160:0/1114888399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fb970 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 -- 172.21.15.160:0/1114888399 >> 172.21.15.160:0/1114888399 conn(0x7fea90073350 msgr2=0x7fea90073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:31.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 -- 172.21.15.160:0/1114888399 shutdown_connections 2024-05-25T06:22:31.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.566+0000 7fea95e1c700 1 -- 172.21.15.160:0/1114888399 wait complete. 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea95e1c700 1 Processor -- start 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea95e1c700 1 -- start start 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea95e1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea95e1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea900fbf40 0x7fea900fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea95e1c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900fe480 con 0x7fea900fb5a0 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea95e1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900fcfb0 con 0x7fea900fbf40 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea8f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea8f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56106/0 (socket says 172.21.15.160:56106) 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.567+0000 7fea8f7fe700 1 -- 172.21.15.160:0/247026281 learned_addr learned my addr 172.21.15.160:0/247026281 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea8effd700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea900fbf40 0x7fea900fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea8f7fe700 1 -- 172.21.15.160:0/247026281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea900fbf40 msgr2=0x7fea900fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea8f7fe700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea900fbf40 0x7fea900fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea8f7fe700 1 -- 172.21.15.160:0/247026281 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80005040 con 0x7fea900fb5a0 2024-05-25T06:22:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea8f7fe700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fc530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea80009150 tx=0x7fea80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea80021070 con 0x7fea900fb5a0 2024-05-25T06:22:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900fd230 con 0x7fea900fb5a0 2024-05-25T06:22:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea80013c00 con 0x7fea900fb5a0 2024-05-25T06:22:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea8000e070 con 0x7fea900fb5a0 2024-05-25T06:22:31.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.568+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9011a030 con 0x7fea900fb5a0 2024-05-25T06:22:31.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.569+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea900fe670 con 0x7fea900fb5a0 2024-05-25T06:22:31.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.570+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fea8001c400 con 0x7fea900fb5a0 2024-05-25T06:22:31.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.571+0000 7fea77fff700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea7806e140 0x7fea780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.571+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fea8008eff0 con 0x7fea900fb5a0 2024-05-25T06:22:31.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.571+0000 7fea8effd700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea7806e140 0x7fea780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.571+0000 7fea8effd700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea7806e140 0x7fea780705f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea900fdc60 tx=0x7fea8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:31.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.575+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fea800c3850 con 0x7fea900fb5a0 2024-05-25T06:22:31.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.783+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fea90101120 con 0x7fea7806e140 2024-05-25T06:22:31.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.793+0000 7fea77fff700 1 -- 172.21.15.160:0/247026281 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fea90101120 con 0x7fea7806e140 2024-05-25T06:22:31.793 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:22:31.793 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (8m) 107s ago 19m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (5m) 107s ago 19m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (5m) 76s ago 18m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (8m) 107s ago 19m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (2m) 107s ago 14m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (112s) 107s ago 14m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (79s) 76s ago 13m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (95s) 76s ago 14m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (10m) 107s ago 20m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (9m) 76s ago 18m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (6m) 107s ago 21m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (6m) 76s ago 18m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:22:31.794 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (9m) 107s ago 19m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (8m) 76s ago 18m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (5m) 107s ago 17m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (4m) 107s ago 17m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (4m) 107s ago 16m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (3m) 76s ago 16m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (3m) 76s ago 16m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (2m) 76s ago 15m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:22:31.795 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (8m) 107s ago 19m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.795+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea7806e140 msgr2=0x7fea780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.795+0000 7fea95e1c700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea7806e140 0x7fea780705f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea900fdc60 tx=0x7fea8400b040 comp rx=0 tx=0).stop 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.795+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 msgr2=0x7fea900fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.795+0000 7fea95e1c700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fc530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea80009150 tx=0x7fea80009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 shutdown_connections 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea900fb5a0 0x7fea900fc530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea7806e140 0x7fea780705f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 --2- 172.21.15.160:0/247026281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea900fbf40 0x7fea900fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 >> 172.21.15.160:0/247026281 conn(0x7fea90073350 msgr2=0x7fea90100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 shutdown_connections 2024-05-25T06:22:31.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.796+0000 7fea95e1c700 1 -- 172.21.15.160:0/247026281 wait complete. 2024-05-25T06:22:31.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/1702130672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 msgr2=0x7f1fbc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/1702130672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc103190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1fac004940 tx=0x7f1fac009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/1702130672 shutdown_connections 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/1702130672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fbc103760 0x7f1fbc0fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/1702130672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc103190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/1702130672 >> 172.21.15.160:0/1702130672 conn(0x7f1fbc0f8930 msgr2=0x7f1fbc0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/1702130672 shutdown_connections 2024-05-25T06:22:31.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.899+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/1702130672 wait complete. 2024-05-25T06:22:31.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fc0fc3700 1 Processor -- start 2024-05-25T06:22:31.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fc0fc3700 1 -- start start 2024-05-25T06:22:31.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fc0fc3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fc0fc3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fbc103760 0x7f1fbc113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fc0fc3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc114490 con 0x7f1fbc102dc0 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fc0fc3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc10dad0 con 0x7f1fbc103760 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fba59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fba59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56122/0 (socket says 172.21.15.160:56122) 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.900+0000 7f1fba59c700 1 -- 172.21.15.160:0/550043399 learned_addr learned my addr 172.21.15.160:0/550043399 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fb9d9b700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fbc103760 0x7f1fbc113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fba59c700 1 -- 172.21.15.160:0/550043399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fbc103760 msgr2=0x7f1fbc113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fba59c700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fbc103760 0x7f1fbc113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fba59c700 1 -- 172.21.15.160:0/550043399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac005040 con 0x7f1fbc102dc0 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fba59c700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc113900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b3b0 tx=0x7f1fac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fac00e040 con 0x7f1fbc102dc0 2024-05-25T06:22:31.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc10dd50 con 0x7f1fbc102dc0 2024-05-25T06:22:31.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fac013420 con 0x7f1fbc102dc0 2024-05-25T06:22:31.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fac01c630 con 0x7f1fbc102dc0 2024-05-25T06:22:31.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.901+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc10e310 con 0x7f1fbc102dc0 2024-05-25T06:22:31.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.902+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc0fe510 con 0x7f1fbc102dc0 2024-05-25T06:22:31.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.904+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1fac013590 con 0x7f1fbc102dc0 2024-05-25T06:22:31.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.904+0000 7f1fb2ffd700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa806e140 0x7f1fa80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:31.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.905+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1fac08f1f0 con 0x7f1fbc102dc0 2024-05-25T06:22:31.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.905+0000 7f1fb9d9b700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa806e140 0x7f1fa80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:31.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.905+0000 7f1fb9d9b700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa806e140 0x7f1fa80705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc107000 tx=0x7f1fa4007ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:31.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:31.908+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1fac05c190 con 0x7f1fbc102dc0 2024-05-25T06:22:32.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.189+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1fbc10efb0 con 0x7f1fbc102dc0 2024-05-25T06:22:32.190 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:32 smithi160 ceph-mon[112094]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:32.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.190+0000 7f1fb2ffd700 1 -- 172.21.15.160:0/550043399 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f1fac05bd20 con 0x7f1fbc102dc0 2024-05-25T06:22:32.190 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:22:32.191 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:22:32.192 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:22:32.192 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:22:32.192 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:22:32.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa806e140 msgr2=0x7f1fa80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa806e140 0x7f1fa80705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc107000 tx=0x7f1fa4007ed0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 msgr2=0x7f1fbc113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc113900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b3b0 tx=0x7f1fac009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 shutdown_connections 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fbc102dc0 0x7f1fbc113900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa806e140 0x7f1fa80705f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 --2- 172.21.15.160:0/550043399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fbc103760 0x7f1fbc113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 >> 172.21.15.160:0/550043399 conn(0x7f1fbc0f8930 msgr2=0x7f1fbc0f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 shutdown_connections 2024-05-25T06:22:32.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.192+0000 7f1fc0fc3700 1 -- 172.21.15.160:0/550043399 wait complete. 2024-05-25T06:22:32.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/3980846950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64102de0 msgr2=0x7f1f641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/3980846950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64102de0 0x7f1f641031b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004940 tx=0x7f1f54009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/3980846950 shutdown_connections 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/3980846950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f64103780 0x7f1f640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/3980846950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64102de0 0x7f1f641031b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/3980846950 >> 172.21.15.160:0/3980846950 conn(0x7f1f640f8990 msgr2=0x7f1f640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/3980846950 shutdown_connections 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.295+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/3980846950 wait complete. 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f6a7d4700 1 Processor -- start 2024-05-25T06:22:32.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f6a7d4700 1 -- start start 2024-05-25T06:22:32.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f6a7d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f64102de0 0x7f1f64113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f6a7d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 0x7f1f64113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f6a7d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64114500 con 0x7f1f64103780 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f6a7d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6410db50 con 0x7f1f64102de0 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 0x7f1f64113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 0x7f1f64113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56150/0 (socket says 172.21.15.160:56150) 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f637fe700 1 -- 172.21.15.160:0/4192332154 learned_addr learned my addr 172.21.15.160:0/4192332154 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f63fff700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f64102de0 0x7f1f64113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f637fe700 1 -- 172.21.15.160:0/4192332154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f64102de0 msgr2=0x7f1f64113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.296+0000 7f1f637fe700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f64102de0 0x7f1f64113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f637fe700 1 -- 172.21.15.160:0/4192332154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54005040 con 0x7f1f64103780 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f637fe700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 0x7f1f64113eb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1f58002a50 tx=0x7f1f58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:32.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f58007cc0 con 0x7f1f64103780 2024-05-25T06:22:32.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6410de30 con 0x7f1f64103780 2024-05-25T06:22:32.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f5800f040 con 0x7f1f64103780 2024-05-25T06:22:32.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f5800d9c0 con 0x7f1f64103780 2024-05-25T06:22:32.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.297+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6410e3f0 con 0x7f1f64103780 2024-05-25T06:22:32.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.298+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f641000f0 con 0x7f1f64103780 2024-05-25T06:22:32.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.298+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1f58009730 con 0x7f1f64103780 2024-05-25T06:22:32.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.299+0000 7f1f60ff9700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f4c06e140 0x7f1f4c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.299+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1f5808d8c0 con 0x7f1f64103780 2024-05-25T06:22:32.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.299+0000 7f1f63fff700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f4c06e140 0x7f1f4c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.299+0000 7f1f63fff700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f4c06e140 0x7f1f4c0705f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004f90 tx=0x7f1f54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:32.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.304+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1f5805a720 con 0x7f1f64103780 2024-05-25T06:22:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:32 smithi203 ceph-mon[83507]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.546+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f6410f010 con 0x7f1f64103780 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.547+0000 7f1f60ff9700 1 -- 172.21.15.160:0/4192332154 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f1f5805a2b0 con 0x7f1f64103780 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:22:32.547 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:22:32.548 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:22:32.549 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:32.550 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:22:32.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f4c06e140 msgr2=0x7f1f4c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f4c06e140 0x7f1f4c0705f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004f90 tx=0x7f1f54009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:32.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 msgr2=0x7f1f64113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 0x7f1f64113eb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1f58002a50 tx=0x7f1f58004600 comp rx=0 tx=0).stop 2024-05-25T06:22:32.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 shutdown_connections 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f4c06e140 0x7f1f4c0705f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f64102de0 0x7f1f64113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 --2- 172.21.15.160:0/4192332154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f64103780 0x7f1f64113eb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.549+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 >> 172.21.15.160:0/4192332154 conn(0x7f1f640f8990 msgr2=0x7f1f64107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.550+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 shutdown_connections 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.550+0000 7f1f6a7d4700 1 -- 172.21.15.160:0/4192332154 wait complete. 2024-05-25T06:22:32.554 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/546514844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 msgr2=0x7f3fd4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/546514844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4065ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0004940 tx=0x7f3fd0009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/546514844 shutdown_connections 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/546514844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4065ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/546514844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fd4102e50 0x7f3fd4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/546514844 >> 172.21.15.160:0/546514844 conn(0x7f3fd4071d00 msgr2=0x7f3fd4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/546514844 shutdown_connections 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.653+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/546514844 wait complete. 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fdbe6f700 1 Processor -- start 2024-05-25T06:22:32.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fdbe6f700 1 -- start start 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fdbe6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fdbe6f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fd4102e50 0x7f3fd41140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fdbe6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4114700 con 0x7f3fd4065640 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fdbe6f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd410daf0 con 0x7f3fd4102e50 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fd9c0b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fd9c0b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56174/0 (socket says 172.21.15.160:56174) 2024-05-25T06:22:32.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fd9c0b700 1 -- 172.21.15.160:0/4213618768 learned_addr learned my addr 172.21.15.160:0/4213618768 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.654+0000 7f3fd940a700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fd4102e50 0x7f3fd41140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fd9c0b700 1 -- 172.21.15.160:0/4213618768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fd4102e50 msgr2=0x7f3fd41140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fd9c0b700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fd4102e50 0x7f3fd41140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fd9c0b700 1 -- 172.21.15.160:0/4213618768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0005040 con 0x7f3fd4065640 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fd9c0b700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4113b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400c8c0 tx=0x7f3fc40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc400a760 con 0x7f3fd4065640 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fc400ad90 con 0x7f3fd4065640 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc40113a0 con 0x7f3fd4065640 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd410ddd0 con 0x7f3fd4065640 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.655+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd410e340 con 0x7f3fd4065640 2024-05-25T06:22:32.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.656+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd4100100 con 0x7f3fd4065640 2024-05-25T06:22:32.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.656+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3fc4012400 con 0x7f3fd4065640 2024-05-25T06:22:32.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.657+0000 7f3fca7fc700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3fc006e140 0x7f3fc00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.657+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3fc408dbb0 con 0x7f3fd4065640 2024-05-25T06:22:32.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.657+0000 7f3fd940a700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3fc006e140 0x7f3fc00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.657+0000 7f3fd940a700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3fc006e140 0x7f3fc00705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fd410ecb0 tx=0x7f3fd0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:32.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.661+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3fc405aa90 con 0x7f3fd4065640 2024-05-25T06:22:32.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.877+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fd410f1c0 con 0x7f3fc006e140 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.879+0000 7f3fca7fc700 1 -- 172.21.15.160:0/4213618768 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3fd410f1c0 con 0x7f3fc006e140 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:22:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:22:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:22:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:22:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:22:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:22:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:22:32.880 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:22:32.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3fc006e140 msgr2=0x7f3fc00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3fc006e140 0x7f3fc00705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fd410ecb0 tx=0x7f3fd0009e30 comp rx=0 tx=0).stop 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 msgr2=0x7f3fd4113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4113b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400c8c0 tx=0x7f3fc40044c0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 shutdown_connections 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fd4065640 0x7f3fd4113b70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3fc006e140 0x7f3fc00705f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 --2- 172.21.15.160:0/4213618768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fd4102e50 0x7f3fd41140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.881+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 >> 172.21.15.160:0/4213618768 conn(0x7f3fd4071d00 msgr2=0x7f3fd40fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.882+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 shutdown_connections 2024-05-25T06:22:32.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.882+0000 7f3fdbe6f700 1 -- 172.21.15.160:0/4213618768 wait complete. 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.987+0000 7f7f56254700 1 -- 172.21.15.160:0/717335770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 msgr2=0x7f7f501020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.987+0000 7f7f56254700 1 --2- 172.21.15.160:0/717335770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f501020c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009ce0 tx=0x7f7f48004d10 comp rx=0 tx=0).stop 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.987+0000 7f7f56254700 1 -- 172.21.15.160:0/717335770 shutdown_connections 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.987+0000 7f7f56254700 1 --2- 172.21.15.160:0/717335770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 0x7f7f500fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.987+0000 7f7f56254700 1 --2- 172.21.15.160:0/717335770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f501020c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.987+0000 7f7f56254700 1 -- 172.21.15.160:0/717335770 >> 172.21.15.160:0/717335770 conn(0x7f7f500f7860 msgr2=0x7f7f500f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 -- 172.21.15.160:0/717335770 shutdown_connections 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 -- 172.21.15.160:0/717335770 wait complete. 2024-05-25T06:22:32.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 Processor -- start 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 -- start start 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f50112a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 0x7f7f50112f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f501135c0 con 0x7f7f50101cf0 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.988+0000 7f7f56254700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5010cba0 con 0x7f7f50102690 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f50112a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f50112a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56202/0 (socket says 172.21.15.160:56202) 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 -- 172.21.15.160:0/646851514 learned_addr learned my addr 172.21.15.160:0/646851514 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4effd700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 0x7f7f50112f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 -- 172.21.15.160:0/646851514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 msgr2=0x7f7f50112f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 0x7f7f50112f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 -- 172.21.15.160:0/646851514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f48009940 con 0x7f7f50101cf0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4effd700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 0x7f7f50112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f4f7fe700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f50112a30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005010 tx=0x7f7f48004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4800e040 con 0x7f7f50101cf0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5010ce20 con 0x7f7f50101cf0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f48008e20 con 0x7f7f50101cf0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4801c660 con 0x7f7f50101cf0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.989+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5010d3e0 con 0x7f7f50101cf0 2024-05-25T06:22:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.990+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f500fd440 con 0x7f7f50101cf0 2024-05-25T06:22:32.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.991+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7f48012a70 con 0x7f7f50101cf0 2024-05-25T06:22:32.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.991+0000 7f7f47fff700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f3c06e140 0x7f7f3c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:22:32.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.991+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f4808f760 con 0x7f7f50101cf0 2024-05-25T06:22:32.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.991+0000 7f7f4effd700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f3c06e140 0x7f7f3c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:22:32.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.992+0000 7f7f4effd700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f3c06e140 0x7f7f3c0705f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7f5010e2d0 tx=0x7f7f38009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:22:32.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:32.996+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7f4805c5c0 con 0x7f7f50101cf0 2024-05-25T06:22:33.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/550043399' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:22:33.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4192332154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:22:33.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:33 smithi160 ceph-mon[112094]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:33.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.283+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f50108e80 con 0x7f7f50101cf0 2024-05-25T06:22:33.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.283+0000 7f7f47fff700 1 -- 172.21.15.160:0/646851514 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7f4805c150 con 0x7f7f50101cf0 2024-05-25T06:22:33.284 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:22:33.284 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:22:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:22:33.284 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:22:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f3c06e140 msgr2=0x7f7f3c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f3c06e140 0x7f7f3c0705f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7f5010e2d0 tx=0x7f7f38009040 comp rx=0 tx=0).stop 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 msgr2=0x7f7f50112a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f50112a30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005010 tx=0x7f7f48004d10 comp rx=0 tx=0).stop 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 shutdown_connections 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f50101cf0 0x7f7f50112a30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f3c06e140 0x7f7f3c0705f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 --2- 172.21.15.160:0/646851514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f50102690 0x7f7f50112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 >> 172.21.15.160:0/646851514 conn(0x7f7f500f7860 msgr2=0x7f7f500f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:22:33.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 shutdown_connections 2024-05-25T06:22:33.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:22:33.286+0000 7f7f56254700 1 -- 172.21.15.160:0/646851514 wait complete. 2024-05-25T06:22:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/550043399' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:22:33.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4192332154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:22:33.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:22:33.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:33 smithi203 ceph-mon[83507]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:34 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/646851514' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:22:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:34 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/646851514' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:22:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:35 smithi203 ceph-mon[83507]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:35 smithi160 ceph-mon[112094]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:37 smithi203 ceph-mon[83507]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:37 smithi160 ceph-mon[112094]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:39 smithi203 ceph-mon[83507]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:39 smithi160 ceph-mon[112094]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:41 smithi203 ceph-mon[83507]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:41 smithi160 ceph-mon[112094]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:43 smithi160 ceph-mon[112094]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:44.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:43 smithi203 ceph-mon[83507]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:45 smithi203 ceph-mon[83507]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:46.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:45 smithi160 ceph-mon[112094]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:47 smithi160 ceph-mon[112094]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:47 smithi203 ceph-mon[83507]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:49 smithi203 ceph-mon[83507]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:49 smithi160 ceph-mon[112094]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:51 smithi203 ceph-mon[83507]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:51 smithi160 ceph-mon[112094]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:53 smithi203 ceph-mon[83507]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:54.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:53 smithi160 ceph-mon[112094]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:55 smithi203 ceph-mon[83507]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:55 smithi160 ceph-mon[112094]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:57 smithi160 ceph-mon[112094]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:22:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:57 smithi203 ceph-mon[83507]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:22:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:22:59 smithi203 ceph-mon[83507]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:22:59 smithi160 ceph-mon[112094]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:01 smithi203 ceph-mon[83507]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:01 smithi160 ceph-mon[112094]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:03.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 -- 172.21.15.160:0/360891199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780fbd40 msgr2=0x7fcc780fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 --2- 172.21.15.160:0/360891199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780fbd40 0x7fcc780fc1b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c004940 tx=0x7fcc6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 -- 172.21.15.160:0/360891199 shutdown_connections 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 --2- 172.21.15.160:0/360891199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780fbd40 0x7fcc780fc1b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 --2- 172.21.15.160:0/360891199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc78101d90 0x7fcc78102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 -- 172.21.15.160:0/360891199 >> 172.21.15.160:0/360891199 conn(0x7fcc78071d00 msgr2=0x7fcc78072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.401+0000 7fcc7f728700 1 -- 172.21.15.160:0/360891199 shutdown_connections 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 -- 172.21.15.160:0/360891199 wait complete. 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 Processor -- start 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 -- start start 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc780fbd40 0x7fcc7810ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 0x7fcc7810f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810f770 con 0x7fcc78101d90 2024-05-25T06:23:03.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7f728700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810f8b0 con 0x7fcc780fbd40 2024-05-25T06:23:03.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7ccc3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 0x7fcc7810f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:03.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7ccc3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 0x7fcc7810f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35670/0 (socket says 172.21.15.160:35670) 2024-05-25T06:23:03.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7ccc3700 1 -- 172.21.15.160:0/1975084183 learned_addr learned my addr 172.21.15.160:0/1975084183 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:03.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.402+0000 7fcc7d4c4700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc780fbd40 0x7fcc7810ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:03.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc7ccc3700 1 -- 172.21.15.160:0/1975084183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc780fbd40 msgr2=0x7fcc7810ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc7ccc3700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc780fbd40 0x7fcc7810ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc7ccc3700 1 -- 172.21.15.160:0/1975084183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c005040 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc7ccc3700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 0x7fcc7810f230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00b3b0 tx=0x7fcc6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c00e040 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc78117ba0 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc6c013420 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c01c630 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.403+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc78118130 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.404+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc780fd430 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.405+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcc6c013590 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.405+0000 7fcc69ffb700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcc6406e140 0x7fcc640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.405+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcc6c08eca0 con 0x7fcc78101d90 2024-05-25T06:23:03.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.405+0000 7fcc7d4c4700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcc6406e140 0x7fcc640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:03.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.406+0000 7fcc7d4c4700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcc6406e140 0x7fcc640705f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc70007fd0 tx=0x7fcc70009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:03.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.410+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcc6c093070 con 0x7fcc78101d90 2024-05-25T06:23:03.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.630+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc78108e60 con 0x7fcc6406e140 2024-05-25T06:23:03.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.631+0000 7fcc69ffb700 1 -- 172.21.15.160:0/1975084183 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcc78108e60 con 0x7fcc6406e140 2024-05-25T06:23:03.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcc6406e140 msgr2=0x7fcc640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcc6406e140 0x7fcc640705f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc70007fd0 tx=0x7fcc70009040 comp rx=0 tx=0).stop 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 msgr2=0x7fcc7810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 0x7fcc7810f230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00b3b0 tx=0x7fcc6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 shutdown_connections 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcc6406e140 0x7fcc640705f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc780fbd40 0x7fcc7810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 --2- 172.21.15.160:0/1975084183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc78101d90 0x7fcc7810f230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 >> 172.21.15.160:0/1975084183 conn(0x7fcc78071d00 msgr2=0x7fcc780fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.633+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 shutdown_connections 2024-05-25T06:23:03.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.634+0000 7fcc7f728700 1 -- 172.21.15.160:0/1975084183 wait complete. 2024-05-25T06:23:03.645 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 -- 172.21.15.160:0/1092648378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fb5a0 msgr2=0x7f55440fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 --2- 172.21.15.160:0/1092648378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fb5a0 0x7f55440fb970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5534004940 tx=0x7f5534009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 -- 172.21.15.160:0/1092648378 shutdown_connections 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 --2- 172.21.15.160:0/1092648378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fbf40 0x7f5544101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 --2- 172.21.15.160:0/1092648378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fb5a0 0x7f55440fb970 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 -- 172.21.15.160:0/1092648378 >> 172.21.15.160:0/1092648378 conn(0x7f5544073350 msgr2=0x7f5544073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 -- 172.21.15.160:0/1092648378 shutdown_connections 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.737+0000 7f554a363700 1 -- 172.21.15.160:0/1092648378 wait complete. 2024-05-25T06:23:03.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f554a363700 1 Processor -- start 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f554a363700 1 -- start start 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f554a363700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fb5a0 0x7f5544113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f554a363700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 0x7f5544113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f554a363700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5544114510 con 0x7f55440fbf40 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f554a363700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554410db30 con 0x7f55440fb5a0 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f553bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 0x7f5544113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f553bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 0x7f5544113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35694/0 (socket says 172.21.15.160:35694) 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f553bfff700 1 -- 172.21.15.160:0/415237047 learned_addr learned my addr 172.21.15.160:0/415237047 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f5543fff700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fb5a0 0x7f5544113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f553bfff700 1 -- 172.21.15.160:0/415237047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fb5a0 msgr2=0x7f5544113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f553bfff700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fb5a0 0x7f5544113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f553bfff700 1 -- 172.21.15.160:0/415237047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534005040 con 0x7f55440fbf40 2024-05-25T06:23:03.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.738+0000 7f5543fff700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fb5a0 0x7f5544113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:23:03.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.739+0000 7f553bfff700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 0x7f5544113ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f552c009b80 tx=0x7f552c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:03.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.739+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f552c007cc0 con 0x7f55440fbf40 2024-05-25T06:23:03.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.739+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554410de10 con 0x7f55440fbf40 2024-05-25T06:23:03.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.739+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f552c00f040 con 0x7f55440fbf40 2024-05-25T06:23:03.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.739+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f552c00dab0 con 0x7f55440fbf40 2024-05-25T06:23:03.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.739+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554410e350 con 0x7f55440fbf40 2024-05-25T06:23:03.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.740+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440fe670 con 0x7f55440fbf40 2024-05-25T06:23:03.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.740+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f552c00dd30 con 0x7f55440fbf40 2024-05-25T06:23:03.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.741+0000 7f55417fa700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f553006e140 0x7f55300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:03.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.741+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f552c08d9f0 con 0x7f55440fbf40 2024-05-25T06:23:03.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.741+0000 7f5543fff700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f553006e140 0x7f55300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:03.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.741+0000 7f5543fff700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f553006e140 0x7f55300705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5534004560 tx=0x7f5534009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.745+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f552c05a430 con 0x7f55440fbf40 2024-05-25T06:23:03.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.965+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5544109f40 con 0x7f553006e140 2024-05-25T06:23:03.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.966+0000 7f55417fa700 1 -- 172.21.15.160:0/415237047 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5544109f40 con 0x7f553006e140 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f553006e140 msgr2=0x7f55300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f553006e140 0x7f55300705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5534004560 tx=0x7f5534009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 msgr2=0x7f5544113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 0x7f5544113ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f552c009b80 tx=0x7f552c004600 comp rx=0 tx=0).stop 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 shutdown_connections 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f553006e140 0x7f55300705f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55440fb5a0 0x7f5544113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 --2- 172.21.15.160:0/415237047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55440fbf40 0x7f5544113ec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:03.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.968+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 >> 172.21.15.160:0/415237047 conn(0x7f5544073350 msgr2=0x7f5544100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:03.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.969+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 shutdown_connections 2024-05-25T06:23:03.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:03.969+0000 7f554a363700 1 -- 172.21.15.160:0/415237047 wait complete. 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 -- 172.21.15.160:0/2125622968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 msgr2=0x7f069c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 --2- 172.21.15.160:0/2125622968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c0fcdf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0698004940 tx=0x7f0698009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 -- 172.21.15.160:0/2125622968 shutdown_connections 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 --2- 172.21.15.160:0/2125622968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c0fcdf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 --2- 172.21.15.160:0/2125622968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f069c102de0 0x7f069c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 -- 172.21.15.160:0/2125622968 >> 172.21.15.160:0/2125622968 conn(0x7f069c0f8990 msgr2=0x7f069c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 -- 172.21.15.160:0/2125622968 shutdown_connections 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.071+0000 7f06a391e700 1 -- 172.21.15.160:0/2125622968 wait complete. 2024-05-25T06:23:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a391e700 1 Processor -- start 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a391e700 1 -- start start 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a391e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f069c102de0 0x7f069c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a391e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a391e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c1145a0 con 0x7f069c103780 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a391e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c10db60 con 0x7f069c102de0 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a0eb9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a0eb9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35710/0 (socket says 172.21.15.160:35710) 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.072+0000 7f06a0eb9700 1 -- 172.21.15.160:0/188188933 learned_addr learned my addr 172.21.15.160:0/188188933 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f06a0eb9700 1 -- 172.21.15.160:0/188188933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f069c102de0 msgr2=0x7f069c113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f06a0eb9700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f069c102de0 0x7f069c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f06a0eb9700 1 -- 172.21.15.160:0/188188933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0698005040 con 0x7f069c103780 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f06a0eb9700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c113ec0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0698009180 tx=0x7f0698009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069800e040 con 0x7f069c103780 2024-05-25T06:23:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f069800bbc0 con 0x7f069c103780 2024-05-25T06:23:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c10dde0 con 0x7f069c103780 2024-05-25T06:23:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069801c3f0 con 0x7f069c103780 2024-05-25T06:23:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.073+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c10e3a0 con 0x7f069c103780 2024-05-25T06:23:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.074+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c1000f0 con 0x7f069c103780 2024-05-25T06:23:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.075+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f069800bd30 con 0x7f069c103780 2024-05-25T06:23:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.075+0000 7f0691ffb700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f068806e140 0x7f06880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.075+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f069808eeb0 con 0x7f069c103780 2024-05-25T06:23:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.075+0000 7f06a16ba700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f068806e140 0x7f06880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.076+0000 7f06a16ba700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f068806e140 0x7f06880705f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f068c006fd0 tx=0x7f068c006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.080+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f069805bd90 con 0x7f069c103780 2024-05-25T06:23:04.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.286+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f069c10f070 con 0x7f068806e140 2024-05-25T06:23:04.287 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:03 smithi160 ceph-mon[112094]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:04.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.296+0000 7f0691ffb700 1 -- 172.21.15.160:0/188188933 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f069c10f070 con 0x7f068806e140 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (8m) 2m ago 19m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (6m) 2m ago 20m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (6m) 109s ago 19m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (8m) 2m ago 19m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (2m) 2m ago 14m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (2m) 2m ago 14m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (112s) 109s ago 14m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (2m) 109s ago 14m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (10m) 2m ago 21m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:23:04.298 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (9m) 109s ago 19m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (7m) 2m ago 21m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (7m) 109s ago 19m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (9m) 2m ago 20m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (9m) 109s ago 19m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (5m) 2m ago 18m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (5m) 2m ago 17m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (4m) 2m ago 17m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (4m) 109s ago 16m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (3m) 109s ago 16m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (3m) 109s ago 16m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:23:04.299 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (9m) 2m ago 19m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:23:04.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f068806e140 msgr2=0x7f06880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f068806e140 0x7f06880705f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f068c006fd0 tx=0x7f068c006db0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 msgr2=0x7f069c113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c113ec0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0698009180 tx=0x7f0698009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:04.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 shutdown_connections 2024-05-25T06:23:04.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f068806e140 0x7f06880705f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f069c102de0 0x7f069c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 --2- 172.21.15.160:0/188188933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f069c103780 0x7f069c113ec0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 >> 172.21.15.160:0/188188933 conn(0x7f069c0f8990 msgr2=0x7f069c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:04.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 shutdown_connections 2024-05-25T06:23:04.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.299+0000 7f06a391e700 1 -- 172.21.15.160:0/188188933 wait complete. 2024-05-25T06:23:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:03 smithi203 ceph-mon[83507]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.400+0000 7fc553ba5700 1 -- 172.21.15.160:0/2708424828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c103780 msgr2=0x7fc54c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.400+0000 7fc553ba5700 1 --2- 172.21.15.160:0/2708424828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c103780 0x7fc54c0fcdf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc540004940 tx=0x7fc540009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.400+0000 7fc553ba5700 1 -- 172.21.15.160:0/2708424828 shutdown_connections 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.400+0000 7fc553ba5700 1 --2- 172.21.15.160:0/2708424828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c103780 0x7fc54c0fcdf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.400+0000 7fc553ba5700 1 --2- 172.21.15.160:0/2708424828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc54c102de0 0x7fc54c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.400+0000 7fc553ba5700 1 -- 172.21.15.160:0/2708424828 >> 172.21.15.160:0/2708424828 conn(0x7fc54c0f8990 msgr2=0x7fc54c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 -- 172.21.15.160:0/2708424828 shutdown_connections 2024-05-25T06:23:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 -- 172.21.15.160:0/2708424828 wait complete. 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 Processor -- start 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 -- start start 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 0x7fc54c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc54c103780 0x7fc54c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c114520 con 0x7fc54c102de0 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc553ba5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c10db20 con 0x7fc54c103780 2024-05-25T06:23:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc551941700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 0x7fc54c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc551941700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 0x7fc54c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35740/0 (socket says 172.21.15.160:35740) 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc551941700 1 -- 172.21.15.160:0/1055810294 learned_addr learned my addr 172.21.15.160:0/1055810294 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.401+0000 7fc551140700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc54c103780 0x7fc54c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc551941700 1 -- 172.21.15.160:0/1055810294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc54c103780 msgr2=0x7fc54c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc551941700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc54c103780 0x7fc54c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc551941700 1 -- 172.21.15.160:0/1055810294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc540005040 con 0x7fc54c102de0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc551941700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 0x7fc54c113990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc54400cac0 tx=0x7fc544007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc54400a980 con 0x7fc54c102de0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c10de00 con 0x7fc54c102de0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc544012440 con 0x7fc54c102de0 2024-05-25T06:23:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5440115e0 con 0x7fc54c102de0 2024-05-25T06:23:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.402+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c10e390 con 0x7fc54c102de0 2024-05-25T06:23:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.403+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c0fe530 con 0x7fc54c102de0 2024-05-25T06:23:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.404+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc5440125b0 con 0x7fc54c102de0 2024-05-25T06:23:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.404+0000 7fc54a7fc700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc53406e140 0x7fc5340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.404+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc54408e260 con 0x7fc54c102de0 2024-05-25T06:23:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.404+0000 7fc551140700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc53406e140 0x7fc5340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.405+0000 7fc551140700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc53406e140 0x7fc5340705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc540004890 tx=0x7fc540009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:04.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.409+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc54405b0c0 con 0x7fc54c102de0 2024-05-25T06:23:04.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.688+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc54c109ec0 con 0x7fc54c102de0 2024-05-25T06:23:04.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.688+0000 7fc54a7fc700 1 -- 172.21.15.160:0/1055810294 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fc544055e60 con 0x7fc54c102de0 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:04.690 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:23:04.691 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:23:04.691 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:04.691 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:23:04.691 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:23:04.691 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:23:04.691 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:23:04.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc53406e140 msgr2=0x7fc5340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc53406e140 0x7fc5340705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc540004890 tx=0x7fc540009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:04.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 msgr2=0x7fc54c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 0x7fc54c113990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc54400cac0 tx=0x7fc544007ee0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 shutdown_connections 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c102de0 0x7fc54c113990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc53406e140 0x7fc5340705f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 --2- 172.21.15.160:0/1055810294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc54c103780 0x7fc54c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 >> 172.21.15.160:0/1055810294 conn(0x7fc54c0f8990 msgr2=0x7fc54c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 shutdown_connections 2024-05-25T06:23:04.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.691+0000 7fc553ba5700 1 -- 172.21.15.160:0/1055810294 wait complete. 2024-05-25T06:23:04.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 -- 172.21.15.160:0/2149040164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 msgr2=0x7fe13c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2149040164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c1031b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe124004940 tx=0x7fe124009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:04.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 -- 172.21.15.160:0/2149040164 shutdown_connections 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2149040164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 0x7fe13c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2149040164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c1031b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 -- 172.21.15.160:0/2149040164 >> 172.21.15.160:0/2149040164 conn(0x7fe13c0f8990 msgr2=0x7fe13c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 -- 172.21.15.160:0/2149040164 shutdown_connections 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 -- 172.21.15.160:0/2149040164 wait complete. 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 Processor -- start 2024-05-25T06:23:04.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.793+0000 7fe142ef4700 1 -- start start 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe142ef4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe142ef4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 0x7fe13c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe142ef4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c114500 con 0x7fe13c102de0 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe142ef4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c10db50 con 0x7fe13c103780 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35766/0 (socket says 172.21.15.160:35766) 2024-05-25T06:23:04.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 -- 172.21.15.160:0/2614837307 learned_addr learned my addr 172.21.15.160:0/2614837307 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 -- 172.21.15.160:0/2614837307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 msgr2=0x7fe13c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe13bfff700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 0x7fe13c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 0x7fe13c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 -- 172.21.15.160:0/2614837307 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124005040 con 0x7fe13c102de0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe13bfff700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 0x7fe13c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe140c90700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c113970 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b3b0 tx=0x7fe124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.794+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12400e040 con 0x7fe13c102de0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.795+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe12400bbc0 con 0x7fe13c102de0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.795+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c10ddd0 con 0x7fe13c102de0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.795+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12401c3f0 con 0x7fe13c102de0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.795+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c10e390 con 0x7fe13c102de0 2024-05-25T06:23:04.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.795+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c1000f0 con 0x7fe13c102de0 2024-05-25T06:23:04.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.796+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe12400bd30 con 0x7fe13c102de0 2024-05-25T06:23:04.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.796+0000 7fe1397fa700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe12806e140 0x7fe1280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:04.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.797+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe12408ee00 con 0x7fe13c102de0 2024-05-25T06:23:04.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.797+0000 7fe13bfff700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe12806e140 0x7fe1280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:04.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.797+0000 7fe13bfff700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe12806e140 0x7fe1280705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe13c10ecc0 tx=0x7fe12c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:04.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:04.801+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe12405bc60 con 0x7fe13c102de0 2024-05-25T06:23:05.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.048+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe13c10efc0 con 0x7fe13c102de0 2024-05-25T06:23:05.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.048+0000 7fe1397fa700 1 -- 172.21.15.160:0/2614837307 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fe12405b7f0 con 0x7fe13c102de0 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:23:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:23:05.051 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:05.052 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:05.053 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:23:05.053 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:05.053 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:05.053 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:05.053 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:05.053 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe12806e140 msgr2=0x7fe1280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe12806e140 0x7fe1280705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe13c10ecc0 tx=0x7fe12c009040 comp rx=0 tx=0).stop 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 msgr2=0x7fe13c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c113970 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b3b0 tx=0x7fe124009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 shutdown_connections 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe13c102de0 0x7fe13c113970 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe12806e140 0x7fe1280705f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 --2- 172.21.15.160:0/2614837307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe13c103780 0x7fe13c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 >> 172.21.15.160:0/2614837307 conn(0x7fe13c0f8990 msgr2=0x7fe13c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 shutdown_connections 2024-05-25T06:23:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.051+0000 7fe142ef4700 1 -- 172.21.15.160:0/2614837307 wait complete. 2024-05-25T06:23:05.055 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:23:05.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 -- 172.21.15.160:0/1272496106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8103780 msgr2=0x7f6ab80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 --2- 172.21.15.160:0/1272496106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8103780 0x7f6ab80fcdf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0004940 tx=0x7f6aa0009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:05.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 -- 172.21.15.160:0/1272496106 shutdown_connections 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 --2- 172.21.15.160:0/1272496106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8103780 0x7f6ab80fcdf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 --2- 172.21.15.160:0/1272496106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab8102de0 0x7f6ab81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 -- 172.21.15.160:0/1272496106 >> 172.21.15.160:0/1272496106 conn(0x7f6ab80f8950 msgr2=0x7f6ab80fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 -- 172.21.15.160:0/1272496106 shutdown_connections 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.155+0000 7f6abd487700 1 -- 172.21.15.160:0/1272496106 wait complete. 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6abd487700 1 Processor -- start 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6abd487700 1 -- start start 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6abd487700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 0x7f6ab8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6abd487700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab8103780 0x7f6ab80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6abd487700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab8070d30 con 0x7f6ab8102de0 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6abd487700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab8070e70 con 0x7f6ab8103780 2024-05-25T06:23:05.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6ab6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 0x7f6ab8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6ab6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 0x7f6ab8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35784/0 (socket says 172.21.15.160:35784) 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.156+0000 7f6ab6ffd700 1 -- 172.21.15.160:0/3057388113 learned_addr learned my addr 172.21.15.160:0/3057388113 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6ab67fc700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab8103780 0x7f6ab80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6ab6ffd700 1 -- 172.21.15.160:0/3057388113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab8103780 msgr2=0x7f6ab80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6ab6ffd700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab8103780 0x7f6ab80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6ab6ffd700 1 -- 172.21.15.160:0/3057388113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0005040 con 0x7f6ab8102de0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6ab6ffd700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 0x7f6ab8070260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800c830 tx=0x7f6aa8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aa800a760 con 0x7f6ab8102de0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aa800ad90 con 0x7f6ab8102de0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab8119e70 con 0x7f6ab8102de0 2024-05-25T06:23:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aa80113a0 con 0x7f6ab8102de0 2024-05-25T06:23:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.157+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab811a2f0 con 0x7f6ab8102de0 2024-05-25T06:23:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.158+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab80fe530 con 0x7f6ab8102de0 2024-05-25T06:23:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.158+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6aa8012400 con 0x7f6ab8102de0 2024-05-25T06:23:05.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.159+0000 7f6aaf7fe700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6aa406e140 0x7f6aa40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:05.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.159+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6aa808db90 con 0x7f6ab8102de0 2024-05-25T06:23:05.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.159+0000 7f6ab67fc700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6aa406e140 0x7f6aa40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:05.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.159+0000 7f6ab67fc700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6aa406e140 0x7f6aa40705f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0000c00 tx=0x7f6aa0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.164+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6aa8056f40 con 0x7f6ab8102de0 2024-05-25T06:23:05.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.381+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab8071950 con 0x7f6aa406e140 2024-05-25T06:23:05.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:05 smithi160 ceph-mon[112094]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:05.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:05 smithi160 ceph-mon[112094]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:05.383 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:05 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1055810294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:23:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.382+0000 7f6aaf7fe700 1 -- 172.21.15.160:0/3057388113 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6ab8071950 con 0x7f6aa406e140 2024-05-25T06:23:05.383 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:23:05.384 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:23:05.385 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6aa406e140 msgr2=0x7f6aa40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6aa406e140 0x7f6aa40705f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0000c00 tx=0x7f6aa0009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 msgr2=0x7f6ab8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 0x7f6ab8070260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800c830 tx=0x7f6aa8004470 comp rx=0 tx=0).stop 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 shutdown_connections 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ab8102de0 0x7f6ab8070260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6aa406e140 0x7f6aa40705f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 --2- 172.21.15.160:0/3057388113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab8103780 0x7f6ab80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 >> 172.21.15.160:0/3057388113 conn(0x7f6ab80f8950 msgr2=0x7f6ab8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:05.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 shutdown_connections 2024-05-25T06:23:05.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.385+0000 7f6abd487700 1 -- 172.21.15.160:0/3057388113 wait complete. 2024-05-25T06:23:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:05 smithi203 ceph-mon[83507]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:05 smithi203 ceph-mon[83507]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:05 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1055810294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:23:05.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.487+0000 7fc072ccf700 1 -- 172.21.15.160:0/1625691691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 msgr2=0x7fc06c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.487+0000 7fc072ccf700 1 --2- 172.21.15.160:0/1625691691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c102160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc060004940 tx=0x7fc060009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:05.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 -- 172.21.15.160:0/1625691691 shutdown_connections 2024-05-25T06:23:05.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 --2- 172.21.15.160:0/1625691691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 0x7fc06c0fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 --2- 172.21.15.160:0/1625691691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c102160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 -- 172.21.15.160:0/1625691691 >> 172.21.15.160:0/1625691691 conn(0x7fc06c0f7940 msgr2=0x7fc06c0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:05.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 -- 172.21.15.160:0/1625691691 shutdown_connections 2024-05-25T06:23:05.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 -- 172.21.15.160:0/1625691691 wait complete. 2024-05-25T06:23:05.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 Processor -- start 2024-05-25T06:23:05.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.488+0000 7fc072ccf700 1 -- start start 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc072ccf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc072ccf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 0x7fc06c112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc072ccf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c113480 con 0x7fc06c101d90 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc072ccf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c10c9f0 con 0x7fc06c102730 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c112860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56250/0 (socket says 172.21.15.160:56250) 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 -- 172.21.15.160:0/932462032 learned_addr learned my addr 172.21.15.160:0/932462032 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc06bfff700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 0x7fc06c112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 -- 172.21.15.160:0/932462032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 msgr2=0x7fc06c112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 0x7fc06c112da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 -- 172.21.15.160:0/932462032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060005040 con 0x7fc06c101d90 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc06bfff700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 0x7fc06c112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc070a6b700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c112860 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc06000b3b0 tx=0x7fc060009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.489+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc06000e040 con 0x7fc06c101d90 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.490+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c10cc70 con 0x7fc06c101d90 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.490+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c10d230 con 0x7fc06c101d90 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.490+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc06000bbc0 con 0x7fc06c101d90 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.490+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc06001c3f0 con 0x7fc06c101d90 2024-05-25T06:23:05.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.491+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c0ff0a0 con 0x7fc06c101d90 2024-05-25T06:23:05.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.491+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc06000bd30 con 0x7fc06c101d90 2024-05-25T06:23:05.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.492+0000 7fc0697fa700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc05c06e140 0x7fc05c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:05.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.492+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc06008ed30 con 0x7fc06c101d90 2024-05-25T06:23:05.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.492+0000 7fc06bfff700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc05c06e140 0x7fc05c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:05.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.492+0000 7fc06bfff700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc05c06e140 0x7fc05c0705f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc058006fd0 tx=0x7fc058008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:05.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.496+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc06005bc10 con 0x7fc06c101d90 2024-05-25T06:23:05.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.772+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc06c0ff0a0 con 0x7fc06c101d90 2024-05-25T06:23:05.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.773+0000 7fc0697fa700 1 -- 172.21.15.160:0/932462032 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc06005b7a0 con 0x7fc06c101d90 2024-05-25T06:23:05.774 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:23:05.774 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:23:05.774 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:23:05.774 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:23:05.774 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc05c06e140 msgr2=0x7fc05c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc05c06e140 0x7fc05c0705f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc058006fd0 tx=0x7fc058008040 comp rx=0 tx=0).stop 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 msgr2=0x7fc06c112860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c112860 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc06000b3b0 tx=0x7fc060009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 shutdown_connections 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc06c101d90 0x7fc06c112860 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc05c06e140 0x7fc05c0705f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 --2- 172.21.15.160:0/932462032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06c102730 0x7fc06c112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:05.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 >> 172.21.15.160:0/932462032 conn(0x7fc06c0f7940 msgr2=0x7fc06c0f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:05.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 shutdown_connections 2024-05-25T06:23:05.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:05.775+0000 7fc072ccf700 1 -- 172.21.15.160:0/932462032 wait complete. 2024-05-25T06:23:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:06 smithi160 ceph-mon[112094]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:06 smithi160 ceph-mon[112094]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:06 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2614837307' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:23:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:06 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/932462032' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:23:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:06 smithi203 ceph-mon[83507]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:06 smithi203 ceph-mon[83507]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:06.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:06 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2614837307' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:23:06.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:06 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/932462032' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:23:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:07 smithi203 ceph-mon[83507]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:07 smithi160 ceph-mon[112094]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:08 smithi160 ceph-mon[112094]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:08 smithi203 ceph-mon[83507]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:09 smithi203 ceph-mon[83507]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:09 smithi160 ceph-mon[112094]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:11 smithi203 ceph-mon[83507]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:11 smithi160 ceph-mon[112094]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:13.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:14.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:13 smithi160 ceph-mon[112094]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:13 smithi203 ceph-mon[83507]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:15 smithi203 ceph-mon[83507]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:15 smithi160 ceph-mon[112094]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:17 smithi160 ceph-mon[112094]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:17 smithi203 ceph-mon[83507]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:19 smithi203 ceph-mon[83507]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:19 smithi160 ceph-mon[112094]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:21 smithi203 ceph-mon[83507]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:21 smithi160 ceph-mon[112094]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:23 smithi203 ceph-mon[83507]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:23 smithi160 ceph-mon[112094]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:25 smithi203 ceph-mon[83507]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:25 smithi160 ceph-mon[112094]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:27 smithi160 ceph-mon[112094]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:27 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:28.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:27 smithi203 ceph-mon[83507]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:28.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:27 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:29 smithi203 ceph-mon[83507]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:29 smithi160 ceph-mon[112094]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:31 smithi203 ceph-mon[83507]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:31 smithi160 ceph-mon[112094]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:34.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:33 smithi160 ceph-mon[112094]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:33 smithi203 ceph-mon[83507]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:35 smithi203 ceph-mon[83507]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:35 smithi160 ceph-mon[112094]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:35.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 -- 172.21.15.160:0/3183070047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 msgr2=0x7f954c0fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 --2- 172.21.15.160:0/3183070047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0fcdd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f953c004940 tx=0x7f953c009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 -- 172.21.15.160:0/3183070047 shutdown_connections 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 --2- 172.21.15.160:0/3183070047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0fcdd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 --2- 172.21.15.160:0/3183070047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f954c102dc0 0x7f954c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 -- 172.21.15.160:0/3183070047 >> 172.21.15.160:0/3183070047 conn(0x7f954c0f8930 msgr2=0x7f954c0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 -- 172.21.15.160:0/3183070047 shutdown_connections 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.895+0000 7f9550fe8700 1 -- 172.21.15.160:0/3183070047 wait complete. 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9550fe8700 1 Processor -- start 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9550fe8700 1 -- start start 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9550fe8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f954c102dc0 0x7f954c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9550fe8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:35.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9550fe8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0721b0 con 0x7f954c103760 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9550fe8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c070ce0 con 0x7f954c102dc0 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9549d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9549d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33150/0 (socket says 172.21.15.160:33150) 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.896+0000 7f9549d9b700 1 -- 172.21.15.160:0/1834638345 learned_addr learned my addr 172.21.15.160:0/1834638345 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f954a59c700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f954c102dc0 0x7f954c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9549d9b700 1 -- 172.21.15.160:0/1834638345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f954c102dc0 msgr2=0x7f954c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9549d9b700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f954c102dc0 0x7f954c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9549d9b700 1 -- 172.21.15.160:0/1834638345 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c005040 con 0x7f954c103760 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9549d9b700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0707a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f953c009100 tx=0x7f953c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953c00e040 con 0x7f954c103760 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f953c004290 con 0x7f954c103760 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c070f60 con 0x7f954c103760 2024-05-25T06:23:35.899 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953c01c3f0 con 0x7f954c103760 2024-05-25T06:23:35.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.897+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c11a0a0 con 0x7f954c103760 2024-05-25T06:23:35.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.898+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c0fe510 con 0x7f954c103760 2024-05-25T06:23:35.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.899+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f953c013690 con 0x7f954c103760 2024-05-25T06:23:35.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.899+0000 7f9542ffd700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f953806e140 0x7f95380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:35.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.899+0000 7f954a59c700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f953806e140 0x7f95380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:35.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.900+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f953c08eb70 con 0x7f954c103760 2024-05-25T06:23:35.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.900+0000 7f954a59c700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f953806e140 0x7f95380705f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9534007fd0 tx=0x7f9534009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:35.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:35.904+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f953c057fe0 con 0x7f954c103760 2024-05-25T06:23:36.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.119+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f954c109f10 con 0x7f953806e140 2024-05-25T06:23:36.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.120+0000 7f9542ffd700 1 -- 172.21.15.160:0/1834638345 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f954c109f10 con 0x7f953806e140 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.122+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f953806e140 msgr2=0x7f95380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.122+0000 7f9550fe8700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f953806e140 0x7f95380705f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9534007fd0 tx=0x7f9534009040 comp rx=0 tx=0).stop 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.122+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 msgr2=0x7f954c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.122+0000 7f9550fe8700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0707a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f953c009100 tx=0x7f953c009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 shutdown_connections 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f953806e140 0x7f95380705f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f954c102dc0 0x7f954c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 --2- 172.21.15.160:0/1834638345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f954c103760 0x7f954c0707a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 >> 172.21.15.160:0/1834638345 conn(0x7f954c0f8930 msgr2=0x7f954c0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:36.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 shutdown_connections 2024-05-25T06:23:36.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.123+0000 7f9550fe8700 1 -- 172.21.15.160:0/1834638345 wait complete. 2024-05-25T06:23:36.133 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:23:36.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 -- 172.21.15.160:0/4244660432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 msgr2=0x7f22c01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 --2- 172.21.15.160:0/4244660432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c01021f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22b0004940 tx=0x7f22b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:36.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 -- 172.21.15.160:0/4244660432 shutdown_connections 2024-05-25T06:23:36.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 --2- 172.21.15.160:0/4244660432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c00fc710 0x7f22c00fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 --2- 172.21.15.160:0/4244660432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c01021f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 -- 172.21.15.160:0/4244660432 >> 172.21.15.160:0/4244660432 conn(0x7f22c0071460 msgr2=0x7f22c0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:36.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 -- 172.21.15.160:0/4244660432 shutdown_connections 2024-05-25T06:23:36.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 -- 172.21.15.160:0/4244660432 wait complete. 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 Processor -- start 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.227+0000 7f22c5463700 1 -- start start 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22c5463700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c00fc710 0x7f22c00fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22c5463700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c00fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22c5463700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00fe440 con 0x7f22c0101e20 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22c5463700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00fe580 con 0x7f22c00fc710 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c00fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c00fdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33168/0 (socket says 172.21.15.160:33168) 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 -- 172.21.15.160:0/4285924456 learned_addr learned my addr 172.21.15.160:0/4285924456 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22beffd700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c00fc710 0x7f22c00fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 -- 172.21.15.160:0/4285924456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c00fc710 msgr2=0x7f22c00fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c00fc710 0x7f22c00fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 -- 172.21.15.160:0/4285924456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0005040 con 0x7f22c0101e20 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22be7fc700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c00fdeb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22b4002a50 tx=0x7f22b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.228+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b4007cc0 con 0x7f22c0101e20 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.229+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c0119e20 con 0x7f22c0101e20 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.229+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22b400f040 con 0x7f22c0101e20 2024-05-25T06:23:36.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.229+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b400da30 con 0x7f22c0101e20 2024-05-25T06:23:36.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.229+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c011a2a0 con 0x7f22c0101e20 2024-05-25T06:23:36.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.230+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c0105270 con 0x7f22c0101e20 2024-05-25T06:23:36.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.230+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f22b400db90 con 0x7f22c0101e20 2024-05-25T06:23:36.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.230+0000 7f22a77fe700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f22a806e140 0x7f22a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.230+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f22b408d920 con 0x7f22c0101e20 2024-05-25T06:23:36.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.231+0000 7f22beffd700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f22a806e140 0x7f22a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.231+0000 7f22beffd700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f22a806e140 0x7f22a80705f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22b0004f90 tx=0x7f22b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:36.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.235+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f22b4056c50 con 0x7f22c0101e20 2024-05-25T06:23:36.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.454+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22c0108eb0 con 0x7f22a806e140 2024-05-25T06:23:36.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.455+0000 7f22a77fe700 1 -- 172.21.15.160:0/4285924456 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f22c0108eb0 con 0x7f22a806e140 2024-05-25T06:23:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f22a806e140 msgr2=0x7f22a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f22a806e140 0x7f22a80705f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22b0004f90 tx=0x7f22b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 msgr2=0x7f22c00fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c00fdeb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22b4002a50 tx=0x7f22b40046b0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 shutdown_connections 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f22a806e140 0x7f22a80705f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c00fc710 0x7f22c00fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 --2- 172.21.15.160:0/4285924456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22c0101e20 0x7f22c00fdeb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 >> 172.21.15.160:0/4285924456 conn(0x7f22c0071460 msgr2=0x7f22c0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 shutdown_connections 2024-05-25T06:23:36.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.457+0000 7f22c5463700 1 -- 172.21.15.160:0/4285924456 wait complete. 2024-05-25T06:23:36.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 -- 172.21.15.160:0/825453967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d4102e20 msgr2=0x7fb4d41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/825453967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d4102e20 0x7fb4d41031f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4004940 tx=0x7fb4c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 -- 172.21.15.160:0/825453967 shutdown_connections 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/825453967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/825453967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d4102e20 0x7fb4d41031f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 -- 172.21.15.160:0/825453967 >> 172.21.15.160:0/825453967 conn(0x7fb4d4072380 msgr2=0x7fb4d4072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 -- 172.21.15.160:0/825453967 shutdown_connections 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.560+0000 7fb4dbad6700 1 -- 172.21.15.160:0/825453967 wait complete. 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4dbad6700 1 Processor -- start 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4dbad6700 1 -- start start 2024-05-25T06:23:36.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4dbad6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4d4102e20 0x7fb4d40fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4dbad6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4dbad6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40fe4f0 con 0x7fb4d41037c0 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4dbad6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40fd020 con 0x7fb4d4102e20 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4d9071700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4d9071700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33188/0 (socket says 172.21.15.160:33188) 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.561+0000 7fb4d9071700 1 -- 172.21.15.160:0/2964697344 learned_addr learned my addr 172.21.15.160:0/2964697344 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4d9872700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4d4102e20 0x7fb4d40fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4d9071700 1 -- 172.21.15.160:0/2964697344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4d4102e20 msgr2=0x7fb4d40fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4d9071700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4d4102e20 0x7fb4d40fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4d9071700 1 -- 172.21.15.160:0/2964697344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4005040 con 0x7fb4d41037c0 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4d9071700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fcae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000eb40 tx=0x7fb4d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:36.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4d0007cc0 con 0x7fb4d41037c0 2024-05-25T06:23:36.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d40fd2a0 con 0x7fb4d41037c0 2024-05-25T06:23:36.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4d000f040 con 0x7fb4d41037c0 2024-05-25T06:23:36.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4d000c9d0 con 0x7fb4d41037c0 2024-05-25T06:23:36.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.562+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d4119fe0 con 0x7fb4d41037c0 2024-05-25T06:23:36.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.563+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d4100160 con 0x7fb4d41037c0 2024-05-25T06:23:36.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.564+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb4d000cb30 con 0x7fb4d41037c0 2024-05-25T06:23:36.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.564+0000 7fb4ca7fc700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb4c006e140 0x7fb4c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.564+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb4d008d4b0 con 0x7fb4d41037c0 2024-05-25T06:23:36.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.564+0000 7fb4d9872700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb4c006e140 0x7fb4c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.565+0000 7fb4d9872700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb4c006e140 0x7fb4c00705f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4004890 tx=0x7fb4c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:36.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.569+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb4d0056910 con 0x7fb4d41037c0 2024-05-25T06:23:36.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.773+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4d4109f50 con 0x7fb4c006e140 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.784+0000 7fb4ca7fc700 1 -- 172.21.15.160:0/2964697344 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb4d4109f50 con 0x7fb4c006e140 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (9m) 2m ago 20m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (6m) 2m ago 20m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (6m) 2m ago 19m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (9m) 2m ago 20m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (3m) 2m ago 15m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:23:36.784 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (2m) 2m ago 15m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (2m) 2m ago 15m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (2m) 2m ago 15m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (11m) 2m ago 22m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (10m) 2m ago 19m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (8m) 2m ago 22m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (7m) 2m ago 19m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (10m) 2m ago 20m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (10m) 2m ago 19m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (6m) 2m ago 18m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (5m) 2m ago 18m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (5m) 2m ago 17m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (4m) 2m ago 17m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:23:36.785 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (4m) 2m ago 17m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:23:36.786 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (3m) 2m ago 16m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:23:36.786 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (9m) 2m ago 20m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:23:36.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb4c006e140 msgr2=0x7fb4c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb4c006e140 0x7fb4c00705f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4004890 tx=0x7fb4c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 msgr2=0x7fb4d40fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fcae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000eb40 tx=0x7fb4d000b040 comp rx=0 tx=0).stop 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 shutdown_connections 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb4c006e140 0x7fb4c00705f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4d4102e20 0x7fb4d40fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 --2- 172.21.15.160:0/2964697344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d41037c0 0x7fb4d40fcae0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 >> 172.21.15.160:0/2964697344 conn(0x7fb4d4072380 msgr2=0x7fb4d40fac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 shutdown_connections 2024-05-25T06:23:36.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.786+0000 7fb4dbad6700 1 -- 172.21.15.160:0/2964697344 wait complete. 2024-05-25T06:23:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 -- 172.21.15.160:0/1317128749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 msgr2=0x7fd2e00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 --2- 172.21.15.160:0/1317128749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e00fc1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009ce0 tx=0x7fd2d0004d10 comp rx=0 tx=0).stop 2024-05-25T06:23:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 -- 172.21.15.160:0/1317128749 shutdown_connections 2024-05-25T06:23:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 --2- 172.21.15.160:0/1317128749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e00fc1b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 --2- 172.21.15.160:0/1317128749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e0101d90 0x7fd2e0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 -- 172.21.15.160:0/1317128749 >> 172.21.15.160:0/1317128749 conn(0x7fd2e0071d00 msgr2=0x7fd2e0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 -- 172.21.15.160:0/1317128749 shutdown_connections 2024-05-25T06:23:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.888+0000 7fd2e716a700 1 -- 172.21.15.160:0/1317128749 wait complete. 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e716a700 1 Processor -- start 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e716a700 1 -- start start 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e716a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e010ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e716a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e0101d90 0x7fd2e010f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e716a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e010f870 con 0x7fd2e00fbd40 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e716a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e010f9b0 con 0x7fd2e0101d90 2024-05-25T06:23:36.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e4f06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e010ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e4f06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e010ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33208/0 (socket says 172.21.15.160:33208) 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.889+0000 7fd2e4f06700 1 -- 172.21.15.160:0/129787117 learned_addr learned my addr 172.21.15.160:0/129787117 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2dffff700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e0101d90 0x7fd2e010f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2e4f06700 1 -- 172.21.15.160:0/129787117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e0101d90 msgr2=0x7fd2e010f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2e4f06700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e0101d90 0x7fd2e010f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2e4f06700 1 -- 172.21.15.160:0/129787117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0009940 con 0x7fd2e00fbd40 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2e4f06700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e010ece0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8002a50 tx=0x7fd2d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2d8007cc0 con 0x7fd2e00fbd40 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2d800f040 con 0x7fd2e00fbd40 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e0117d50 con 0x7fd2e00fbd40 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2d800da30 con 0x7fd2e00fbd40 2024-05-25T06:23:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.890+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e0118280 con 0x7fd2e00fbd40 2024-05-25T06:23:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.892+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e00fd430 con 0x7fd2e00fbd40 2024-05-25T06:23:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.892+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd2d800db90 con 0x7fd2e00fbd40 2024-05-25T06:23:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.892+0000 7fd2dd7fa700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2cc06e150 0x7fd2cc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.892+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd2d808d920 con 0x7fd2e00fbd40 2024-05-25T06:23:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.892+0000 7fd2dffff700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2cc06e150 0x7fd2cc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.893+0000 7fd2dffff700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2cc06e150 0x7fd2cc070600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000b740 tx=0x7fd2d0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:36.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:36.897+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2d8056c50 con 0x7fd2e00fbd40 2024-05-25T06:23:37.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.181+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd2e0065d90 con 0x7fd2e00fbd40 2024-05-25T06:23:37.182 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:37 smithi160 ceph-mon[112094]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:37.182 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:37 smithi160 ceph-mon[112094]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:37.182 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:37 smithi160 ceph-mon[112094]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:37.182 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:37 smithi160 ceph-mon[112094]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:37.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.182+0000 7fd2dd7fa700 1 -- 172.21.15.160:0/129787117 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fd2d805a310 con 0x7fd2e00fbd40 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:23:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:23:37.184 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2cc06e150 msgr2=0x7fd2cc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2cc06e150 0x7fd2cc070600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000b740 tx=0x7fd2d0004d10 comp rx=0 tx=0).stop 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 msgr2=0x7fd2e010ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e010ece0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8002a50 tx=0x7fd2d80046b0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 shutdown_connections 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e00fbd40 0x7fd2e010ece0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2cc06e150 0x7fd2cc070600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 --2- 172.21.15.160:0/129787117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e0101d90 0x7fd2e010f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 >> 172.21.15.160:0/129787117 conn(0x7fd2e0071d00 msgr2=0x7fd2e0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 shutdown_connections 2024-05-25T06:23:37.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.184+0000 7fd2e716a700 1 -- 172.21.15.160:0/129787117 wait complete. 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.287+0000 7f9746e3d700 1 -- 172.21.15.160:0/2685756107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97401026a0 msgr2=0x7f97400fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.287+0000 7f9746e3d700 1 --2- 172.21.15.160:0/2685756107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97401026a0 0x7f97400fbd10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f972c004940 tx=0x7f972c009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.287+0000 7f9746e3d700 1 -- 172.21.15.160:0/2685756107 shutdown_connections 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.287+0000 7f9746e3d700 1 --2- 172.21.15.160:0/2685756107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97401026a0 0x7f97400fbd10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.287+0000 7f9746e3d700 1 --2- 172.21.15.160:0/2685756107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9740101d00 0x7f97401020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.287+0000 7f9746e3d700 1 -- 172.21.15.160:0/2685756107 >> 172.21.15.160:0/2685756107 conn(0x7f97400f7870 msgr2=0x7f97400f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 -- 172.21.15.160:0/2685756107 shutdown_connections 2024-05-25T06:23:37.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 -- 172.21.15.160:0/2685756107 wait complete. 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 Processor -- start 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 -- start start 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 0x7f974006a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401026a0 0x7f974006abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974006b200 con 0x7f9740101d00 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.288+0000 7f9746e3d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974011ef30 con 0x7f97401026a0 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 0x7f974006a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 0x7f974006a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33220/0 (socket says 172.21.15.160:33220) 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 -- 172.21.15.160:0/4231004550 learned_addr learned my addr 172.21.15.160:0/4231004550 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f973ffff700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401026a0 0x7f974006abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 -- 172.21.15.160:0/4231004550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401026a0 msgr2=0x7f974006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401026a0 0x7f974006abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 -- 172.21.15.160:0/4231004550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c005040 con 0x7f9740101d00 2024-05-25T06:23:37.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9744bd9700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 0x7f974006a650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9734002a50 tx=0x7f97340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9734007cc0 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f973400f040 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974011f130 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f973400da30 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.289+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974011f5f0 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.290+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97400fd450 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.291+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f973400db90 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.291+0000 7f973d7fa700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f973006e140 0x7f97300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.291+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f973408d900 con 0x7f9740101d00 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.291+0000 7f973ffff700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f973006e140 0x7f97300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.292+0000 7f973ffff700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f973006e140 0x7f97300705f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f97400fa340 tx=0x7f972c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:37.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.296+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f973405a760 con 0x7f9740101d00 2024-05-25T06:23:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:37 smithi203 ceph-mon[83507]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:37 smithi203 ceph-mon[83507]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:37 smithi203 ceph-mon[83507]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:37 smithi203 ceph-mon[83507]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:37.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.536+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f974006b9a0 con 0x7f9740101d00 2024-05-25T06:23:37.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.537+0000 7f973d7fa700 1 -- 172.21.15.160:0/4231004550 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f973405a2f0 con 0x7f9740101d00 2024-05-25T06:23:37.537 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:23:37.537 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:23:37.537 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:23:37.537 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:23:37.538 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:23:37.539 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:37.541 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:23:37.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f973006e140 msgr2=0x7f97300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f973006e140 0x7f97300705f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f97400fa340 tx=0x7f972c009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:37.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 msgr2=0x7f974006a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 0x7f974006a650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9734002a50 tx=0x7f97340046b0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 shutdown_connections 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9740101d00 0x7f974006a650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f973006e140 0x7f97300705f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 --2- 172.21.15.160:0/4231004550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401026a0 0x7f974006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 >> 172.21.15.160:0/4231004550 conn(0x7f97400f7870 msgr2=0x7f97400f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 shutdown_connections 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.539+0000 7f9746e3d700 1 -- 172.21.15.160:0/4231004550 wait complete. 2024-05-25T06:23:37.542 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 -- 172.21.15.160:0/3975407962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e8101d10 msgr2=0x7f58e81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 --2- 172.21.15.160:0/3975407962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e8101d10 0x7f58e81020e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f58e4004940 tx=0x7f58e4009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 -- 172.21.15.160:0/3975407962 shutdown_connections 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 --2- 172.21.15.160:0/3975407962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e81026b0 0x7f58e80fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 --2- 172.21.15.160:0/3975407962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e8101d10 0x7f58e81020e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 -- 172.21.15.160:0/3975407962 >> 172.21.15.160:0/3975407962 conn(0x7f58e80f7860 msgr2=0x7f58e80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 -- 172.21.15.160:0/3975407962 shutdown_connections 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.644+0000 7f58ef75b700 1 -- 172.21.15.160:0/3975407962 wait complete. 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ef75b700 1 Processor -- start 2024-05-25T06:23:37.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ef75b700 1 -- start start 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ef75b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e8101d10 0x7f58e810eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ef75b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 0x7f58e810f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ef75b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e810f5f0 con 0x7f58e81026b0 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ef75b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e810f730 con 0x7f58e8101d10 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 0x7f58e810f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 0x7f58e810f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33236/0 (socket says 172.21.15.160:33236) 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 -- 172.21.15.160:0/1491369741 learned_addr learned my addr 172.21.15.160:0/1491369741 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58ed4f7700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e8101d10 0x7f58e810eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 -- 172.21.15.160:0/1491369741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e8101d10 msgr2=0x7f58e810eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e8101d10 0x7f58e810eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 -- 172.21.15.160:0/1491369741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e4005040 con 0x7f58e81026b0 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.645+0000 7f58eccf6700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 0x7f58e810f0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c890 tx=0x7f58d80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:37.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.646+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d800a7d0 con 0x7f58e81026b0 2024-05-25T06:23:37.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.646+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58d800ae00 con 0x7f58e81026b0 2024-05-25T06:23:37.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.646+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e8117be0 con 0x7f58e81026b0 2024-05-25T06:23:37.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.646+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d80113a0 con 0x7f58e81026b0 2024-05-25T06:23:37.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.646+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8117fb0 con 0x7f58e81026b0 2024-05-25T06:23:37.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.647+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e80fd460 con 0x7f58e81026b0 2024-05-25T06:23:37.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.647+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f58d8012400 con 0x7f58e81026b0 2024-05-25T06:23:37.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.648+0000 7f58ddffb700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58d406e140 0x7f58d40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.648+0000 7f58ed4f7700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58d406e140 0x7f58d40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.648+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f58d808db20 con 0x7f58e81026b0 2024-05-25T06:23:37.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.648+0000 7f58ed4f7700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58d406e140 0x7f58d40705f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58e4004560 tx=0x7f58e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:37.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.652+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f58d8056f90 con 0x7f58e81026b0 2024-05-25T06:23:37.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.868+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58e8108e10 con 0x7f58d406e140 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.870+0000 7f58ddffb700 1 -- 172.21.15.160:0/1491369741 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f58e8108e10 con 0x7f58d406e140 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:23:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:23:37.871 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:23:37.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58d406e140 msgr2=0x7f58d40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58d406e140 0x7f58d40705f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58e4004560 tx=0x7f58e4009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:37.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 msgr2=0x7f58e810f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 0x7f58e810f0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c890 tx=0x7f58d80044c0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 shutdown_connections 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58d406e140 0x7f58d40705f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e8101d10 0x7f58e810eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 --2- 172.21.15.160:0/1491369741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58e81026b0 0x7f58e810f0b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 >> 172.21.15.160:0/1491369741 conn(0x7f58e80f7860 msgr2=0x7f58e80f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 shutdown_connections 2024-05-25T06:23:37.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.872+0000 7f58ef75b700 1 -- 172.21.15.160:0/1491369741 wait complete. 2024-05-25T06:23:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.977+0000 7f4fb4e46700 1 -- 172.21.15.160:0/791181301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 msgr2=0x7f4fb0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.977+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/791181301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb0101a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4004940 tx=0x7f4fa4009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.977+0000 7f4fb4e46700 1 -- 172.21.15.160:0/791181301 shutdown_connections 2024-05-25T06:23:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.977+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/791181301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb0101a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.977+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/791181301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb00fb7f0 0x7f4fb00fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.977+0000 7f4fb4e46700 1 -- 172.21.15.160:0/791181301 >> 172.21.15.160:0/791181301 conn(0x7f4fb0071460 msgr2=0x7f4fb0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 -- 172.21.15.160:0/791181301 shutdown_connections 2024-05-25T06:23:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 -- 172.21.15.160:0/791181301 wait complete. 2024-05-25T06:23:37.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 Processor -- start 2024-05-25T06:23:37.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 -- start start 2024-05-25T06:23:37.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb00fb7f0 0x7f4fb00fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb00fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb00fe6d0 con 0x7f4fb00fc190 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.978+0000 7f4fb4e46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb00fd200 con 0x7f4fb00fb7f0 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb00fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb00fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33258/0 (socket says 172.21.15.160:33258) 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 -- 172.21.15.160:0/1499509543 learned_addr learned my addr 172.21.15.160:0/1499509543 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fae59c700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb00fb7f0 0x7f4fb00fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 -- 172.21.15.160:0/1499509543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb00fb7f0 msgr2=0x7f4fb00fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:37.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb00fb7f0 0x7f4fb00fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 -- 172.21.15.160:0/1499509543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4005040 con 0x7f4fb00fc190 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fadd9b700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb00fccc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000c00 tx=0x7f4fa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa400e040 con 0x7f4fb00fc190 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fa4009400 con 0x7f4fb00fc190 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa401c3f0 con 0x7f4fb00fc190 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb00fd480 con 0x7f4fb00fc190 2024-05-25T06:23:37.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.979+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb011a0b0 con 0x7f4fb00fc190 2024-05-25T06:23:37.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.980+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb0065f30 con 0x7f4fb00fc190 2024-05-25T06:23:37.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.981+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4fa400baf0 con 0x7f4fb00fc190 2024-05-25T06:23:37.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.982+0000 7f4f9effd700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f9806e140 0x7f4f980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:23:37.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.982+0000 7f4fae59c700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f9806e140 0x7f4f980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:23:37.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.982+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4fa408ed60 con 0x7f4fb00fc190 2024-05-25T06:23:37.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.982+0000 7f4fae59c700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f9806e140 0x7f4f980705f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007fd0 tx=0x7f4fa0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:23:37.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:37.986+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4fa4058110 con 0x7f4fb00fc190 2024-05-25T06:23:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:38 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/129787117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:23:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:38 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4231004550' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:23:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:38 smithi160 ceph-mon[112094]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.266+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4fb0109f20 con 0x7f4fb00fc190 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.267+0000 7f4f9effd700 1 -- 172.21.15.160:0/1499509543 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4fa405b7d0 con 0x7f4fb00fc190 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:23:38.267 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:23:38.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f9806e140 msgr2=0x7f4f980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:38.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f9806e140 0x7f4f980705f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007fd0 tx=0x7f4fa0009040 comp rx=0 tx=0).stop 2024-05-25T06:23:38.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 msgr2=0x7f4fb00fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:23:38.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb00fccc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000c00 tx=0x7f4fa4009e30 comp rx=0 tx=0).stop 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 shutdown_connections 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4f9806e140 0x7f4f980705f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb00fb7f0 0x7f4fb00fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 --2- 172.21.15.160:0/1499509543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb00fc190 0x7f4fb00fccc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 >> 172.21.15.160:0/1499509543 conn(0x7f4fb0071460 msgr2=0x7f4fb0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 shutdown_connections 2024-05-25T06:23:38.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:23:38.269+0000 7f4fb4e46700 1 -- 172.21.15.160:0/1499509543 wait complete. 2024-05-25T06:23:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:38 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/129787117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:23:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:38 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4231004550' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:23:38.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:38 smithi203 ceph-mon[83507]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:23:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:39 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1499509543' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:23:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:39 smithi203 ceph-mon[83507]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:39 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1499509543' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:23:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:39 smithi160 ceph-mon[112094]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:41 smithi203 ceph-mon[83507]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:41 smithi160 ceph-mon[112094]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:44.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:43 smithi160 ceph-mon[112094]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:43 smithi203 ceph-mon[83507]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:46.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:45 smithi203 ceph-mon[83507]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:46.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:45 smithi160 ceph-mon[112094]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:47 smithi160 ceph-mon[112094]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:47 smithi203 ceph-mon[83507]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:50.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:49 smithi203 ceph-mon[83507]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:49 smithi160 ceph-mon[112094]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:51 smithi203 ceph-mon[83507]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:51 smithi160 ceph-mon[112094]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:53 smithi203 ceph-mon[83507]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:54.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:53 smithi160 ceph-mon[112094]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:55 smithi203 ceph-mon[83507]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:55 smithi160 ceph-mon[112094]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:57 smithi160 ceph-mon[112094]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:57 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:23:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:57 smithi203 ceph-mon[83507]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:23:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:57 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:23:59 smithi203 ceph-mon[83507]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:23:59 smithi160 ceph-mon[112094]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:01 smithi203 ceph-mon[83507]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:01 smithi160 ceph-mon[112094]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:04.271 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:04 smithi160 ceph-mon[112094]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:04 smithi203 ceph-mon[83507]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:06.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:06 smithi203 ceph-mon[83507]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:06.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:06 smithi160 ceph-mon[112094]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:08.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:08 smithi160 ceph-mon[112094]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:08.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 -- 172.21.15.160:0/1956886610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 msgr2=0x7f54381020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1956886610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f54381020e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5434009ce0 tx=0x7f5434004d10 comp rx=0 tx=0).stop 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 -- 172.21.15.160:0/1956886610 shutdown_connections 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1956886610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54381026b0 0x7f54380fc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1956886610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f54381020e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 -- 172.21.15.160:0/1956886610 >> 172.21.15.160:0/1956886610 conn(0x7f5438073350 msgr2=0x7f5438073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 -- 172.21.15.160:0/1956886610 shutdown_connections 2024-05-25T06:24:08.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.386+0000 7f543f5f0700 1 -- 172.21.15.160:0/1956886610 wait complete. 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543f5f0700 1 Processor -- start 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543f5f0700 1 -- start start 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543f5f0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f543810ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543f5f0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54381026b0 0x7f543810f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543f5f0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543810f780 con 0x7f5438101d10 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543f5f0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543810f8c0 con 0x7f54381026b0 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543d38c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f543810ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543d38c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f543810ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54572/0 (socket says 172.21.15.160:54572) 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543d38c700 1 -- 172.21.15.160:0/1915862747 learned_addr learned my addr 172.21.15.160:0/1915862747 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.387+0000 7f543cb8b700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54381026b0 0x7f543810f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f543d38c700 1 -- 172.21.15.160:0/1915862747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54381026b0 msgr2=0x7f543810f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f543d38c700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54381026b0 0x7f543810f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f543d38c700 1 -- 172.21.15.160:0/1915862747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5434009940 con 0x7f5438101d10 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f543d38c700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f543810ed00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f543400be90 tx=0x7f5434004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f543400e040 con 0x7f5438101d10 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543810fb40 con 0x7f5438101d10 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5434004610 con 0x7f5438101d10 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f543401c6d0 con 0x7f5438101d10 2024-05-25T06:24:08.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.388+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5438118040 con 0x7f5438101d10 2024-05-25T06:24:08.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.389+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54380fd510 con 0x7f5438101d10 2024-05-25T06:24:08.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.390+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5434012b00 con 0x7f5438101d10 2024-05-25T06:24:08.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.391+0000 7f542dffb700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542406e140 0x7f54240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:08.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.391+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f543408f670 con 0x7f5438101d10 2024-05-25T06:24:08.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.391+0000 7f543cb8b700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542406e140 0x7f54240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:08.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.391+0000 7f543cb8b700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542406e140 0x7f54240705f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54380fbc00 tx=0x7f5428009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:08.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.395+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f54340589a0 con 0x7f5438101d10 2024-05-25T06:24:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:08 smithi203 ceph-mon[83507]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:08.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.610+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54380fba20 con 0x7f542406e140 2024-05-25T06:24:08.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.611+0000 7f542dffb700 1 -- 172.21.15.160:0/1915862747 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f54380fba20 con 0x7f542406e140 2024-05-25T06:24:08.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542406e140 msgr2=0x7f54240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542406e140 0x7f54240705f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54380fbc00 tx=0x7f5428009040 comp rx=0 tx=0).stop 2024-05-25T06:24:08.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 msgr2=0x7f543810ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f543810ed00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f543400be90 tx=0x7f5434004d10 comp rx=0 tx=0).stop 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 shutdown_connections 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5438101d10 0x7f543810ed00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542406e140 0x7f54240705f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 --2- 172.21.15.160:0/1915862747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54381026b0 0x7f543810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 >> 172.21.15.160:0/1915862747 conn(0x7f5438073350 msgr2=0x7f5438104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 shutdown_connections 2024-05-25T06:24:08.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.613+0000 7f543f5f0700 1 -- 172.21.15.160:0/1915862747 wait complete. 2024-05-25T06:24:08.624 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:24:08.718 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 -- 172.21.15.160:0/1658180304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 msgr2=0x7feae01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.718 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 --2- 172.21.15.160:0/1658180304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae01031b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fead0004940 tx=0x7fead0009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 -- 172.21.15.160:0/1658180304 shutdown_connections 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 --2- 172.21.15.160:0/1658180304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 0x7feae00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 --2- 172.21.15.160:0/1658180304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae01031b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 -- 172.21.15.160:0/1658180304 >> 172.21.15.160:0/1658180304 conn(0x7feae00f8990 msgr2=0x7feae00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 -- 172.21.15.160:0/1658180304 shutdown_connections 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 -- 172.21.15.160:0/1658180304 wait complete. 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.718+0000 7feae5e36700 1 Processor -- start 2024-05-25T06:24:08.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feae5e36700 1 -- start start 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feae5e36700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feae5e36700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 0x7feae0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feae5e36700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae0114500 con 0x7feae0102de0 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feae5e36700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae010db50 con 0x7feae0103780 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54584/0 (socket says 172.21.15.160:54584) 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadf7fe700 1 -- 172.21.15.160:0/2278330353 learned_addr learned my addr 172.21.15.160:0/2278330353 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadeffd700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 0x7feae0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadf7fe700 1 -- 172.21.15.160:0/2278330353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 msgr2=0x7feae0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadf7fe700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 0x7feae0113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadf7fe700 1 -- 172.21.15.160:0/2278330353 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0005040 con 0x7feae0102de0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.719+0000 7feadeffd700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 0x7feae0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.720+0000 7feadf7fe700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae0113970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fead0009150 tx=0x7fead0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.720+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead000e040 con 0x7feae0102de0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.720+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fead0013420 con 0x7feae0102de0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.720+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae010ddd0 con 0x7feae0102de0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.720+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead001c630 con 0x7feae0102de0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.720+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae010e390 con 0x7feae0102de0 2024-05-25T06:24:08.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.721+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae01000f0 con 0x7feae0102de0 2024-05-25T06:24:08.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.721+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fead001c860 con 0x7feae0102de0 2024-05-25T06:24:08.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.722+0000 7feac7fff700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feac806e140 0x7feac80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:08.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.722+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fead008ecd0 con 0x7feae0102de0 2024-05-25T06:24:08.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.722+0000 7feadeffd700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feac806e140 0x7feac80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:08.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.722+0000 7feadeffd700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feac806e140 0x7feac80705f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feae010ecc0 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:08.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.727+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fead005bc70 con 0x7feae0102de0 2024-05-25T06:24:08.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.948+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feae010f030 con 0x7feac806e140 2024-05-25T06:24:08.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.950+0000 7feac7fff700 1 -- 172.21.15.160:0/2278330353 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feae010f030 con 0x7feac806e140 2024-05-25T06:24:08.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feac806e140 msgr2=0x7feac80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feac806e140 0x7feac80705f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feae010ecc0 tx=0x7fead4009040 comp rx=0 tx=0).stop 2024-05-25T06:24:08.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 msgr2=0x7feae0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:08.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae0113970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fead0009150 tx=0x7fead0009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:08.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 shutdown_connections 2024-05-25T06:24:08.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0102de0 0x7feae0113970 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feac806e140 0x7feac80705f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 --2- 172.21.15.160:0/2278330353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feae0103780 0x7feae0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:08.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 >> 172.21.15.160:0/2278330353 conn(0x7feae00f8990 msgr2=0x7feae0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:08.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 shutdown_connections 2024-05-25T06:24:08.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:08.952+0000 7feae5e36700 1 -- 172.21.15.160:0/2278330353 wait complete. 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 -- 172.21.15.160:0/170114868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 msgr2=0x7f62b80fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 --2- 172.21.15.160:0/170114868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b80fe240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f62a8004940 tx=0x7f62a8009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 -- 172.21.15.160:0/170114868 shutdown_connections 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 --2- 172.21.15.160:0/170114868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80fe810 0x7f62b8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 --2- 172.21.15.160:0/170114868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b80fe240 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 -- 172.21.15.160:0/170114868 >> 172.21.15.160:0/170114868 conn(0x7f62b80f7860 msgr2=0x7f62b80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 -- 172.21.15.160:0/170114868 shutdown_connections 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 -- 172.21.15.160:0/170114868 wait complete. 2024-05-25T06:24:09.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.057+0000 7f62be946700 1 Processor -- start 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62be946700 1 -- start start 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62be946700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b8113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62be946700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80fe810 0x7f62b8113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62be946700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b81144c0 con 0x7f62b80fde70 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62be946700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b810dac0 con 0x7f62b80fe810 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b8113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b8113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54612/0 (socket says 172.21.15.160:54612) 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 -- 172.21.15.160:0/732553374 learned_addr learned my addr 172.21.15.160:0/732553374 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62affff700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80fe810 0x7f62b8113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 -- 172.21.15.160:0/732553374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80fe810 msgr2=0x7f62b8113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80fe810 0x7f62b8113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 -- 172.21.15.160:0/732553374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8005040 con 0x7f62b80fde70 2024-05-25T06:24:09.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b7fff700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b8113930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009380 tx=0x7f62a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:09.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a800e040 con 0x7f62b80fde70 2024-05-25T06:24:09.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62a8009af0 con 0x7f62b80fde70 2024-05-25T06:24:09.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b810dda0 con 0x7f62b80fde70 2024-05-25T06:24:09.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.059+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b810e2e0 con 0x7f62b80fde70 2024-05-25T06:24:09.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.058+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a801c630 con 0x7f62b80fde70 2024-05-25T06:24:09.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.060+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b8066af0 con 0x7f62b80fde70 2024-05-25T06:24:09.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.060+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f62a800bbc0 con 0x7f62b80fde70 2024-05-25T06:24:09.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.061+0000 7f62b57fa700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62a406e140 0x7f62a40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.061+0000 7f62affff700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62a406e140 0x7f62a40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.061+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f62a808ee70 con 0x7f62b80fde70 2024-05-25T06:24:09.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.061+0000 7f62affff700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62a406e140 0x7f62a40705f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62b810ec80 tx=0x7f62a000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:09.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.065+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f62a805bd50 con 0x7f62b80fde70 2024-05-25T06:24:09.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.270+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b8109e30 con 0x7f62a406e140 2024-05-25T06:24:09.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.280+0000 7f62b57fa700 1 -- 172.21.15.160:0/732553374 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f62b8109e30 con 0x7f62a406e140 2024-05-25T06:24:09.280 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (10m) 3m ago 21m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (7m) 3m ago 21m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (7m) 2m ago 20m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (9m) 3m ago 20m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (3m) 3m ago 15m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (3m) 3m ago 15m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (2m) 2m ago 15m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (3m) 2m ago 15m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (11m) 3m ago 22m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (10m) 2m ago 20m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:24:09.281 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (8m) 3m ago 22m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (8m) 2m ago 20m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (10m) 3m ago 21m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (10m) 2m ago 20m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (7m) 3m ago 19m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (6m) 3m ago 18m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (5m) 3m ago 18m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (5m) 2m ago 18m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (4m) 2m ago 17m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (4m) 2m ago 17m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:24:09.282 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (10m) 3m ago 20m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:24:09.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62a406e140 msgr2=0x7f62a40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62a406e140 0x7f62a40705f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62b810ec80 tx=0x7f62a000b040 comp rx=0 tx=0).stop 2024-05-25T06:24:09.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 msgr2=0x7f62b8113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b8113930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009380 tx=0x7f62a8009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:09.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 shutdown_connections 2024-05-25T06:24:09.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b80fde70 0x7f62b8113930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f62a406e140 0x7f62a40705f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 --2- 172.21.15.160:0/732553374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80fe810 0x7f62b8113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 >> 172.21.15.160:0/732553374 conn(0x7f62b80f7860 msgr2=0x7f62b80f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:09.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 shutdown_connections 2024-05-25T06:24:09.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.283+0000 7f62be946700 1 -- 172.21.15.160:0/732553374 wait complete. 2024-05-25T06:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 -- 172.21.15.160:0/606217870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 msgr2=0x7f4cec0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 --2- 172.21.15.160:0/606217870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec0fc1b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0004940 tx=0x7f4ce0009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 -- 172.21.15.160:0/606217870 shutdown_connections 2024-05-25T06:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 --2- 172.21.15.160:0/606217870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec0fc1b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 --2- 172.21.15.160:0/606217870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cec101d90 0x7f4cec102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 -- 172.21.15.160:0/606217870 >> 172.21.15.160:0/606217870 conn(0x7f4cec071d00 msgr2=0x7f4cec072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 -- 172.21.15.160:0/606217870 shutdown_connections 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.385+0000 7f4cf3339700 1 -- 172.21.15.160:0/606217870 wait complete. 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf3339700 1 Processor -- start 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf3339700 1 -- start start 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf3339700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf3339700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cec101d90 0x7f4cec116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf3339700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec116970 con 0x7f4cec0fbd40 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf3339700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec10fd40 con 0x7f4cec101d90 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf10d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf10d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54640/0 (socket says 172.21.15.160:54640) 2024-05-25T06:24:09.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf10d5700 1 -- 172.21.15.160:0/865999115 learned_addr learned my addr 172.21.15.160:0/865999115 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:09.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf08d4700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cec101d90 0x7f4cec116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf10d5700 1 -- 172.21.15.160:0/865999115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cec101d90 msgr2=0x7f4cec116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf10d5700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cec101d90 0x7f4cec116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.386+0000 7f4cf10d5700 1 -- 172.21.15.160:0/865999115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0005040 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.387+0000 7f4cf10d5700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec115de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400cac0 tx=0x7f4ce4007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.387+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce400aa50 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.387+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ce400abb0 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.387+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec110020 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.387+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce40115e0 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.387+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec1105e0 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.388+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec0fd430 con 0x7f4cec0fbd40 2024-05-25T06:24:09.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.389+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4ce400ad20 con 0x7f4cec0fbd40 2024-05-25T06:24:09.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.389+0000 7f4ce9ffb700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4cd406e150 0x7f4cd4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.389+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4ce408e2c0 con 0x7f4cec0fbd40 2024-05-25T06:24:09.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.389+0000 7f4cf08d4700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4cd406e150 0x7f4cd4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.390+0000 7f4cf08d4700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4cd406e150 0x7f4cd4070600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b3b0 tx=0x7f4ce0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:09.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.394+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4ce405b120 con 0x7f4cec0fbd40 2024-05-25T06:24:09.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.669+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4cec05f5c0 con 0x7f4cec0fbd40 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.670+0000 7f4ce9ffb700 1 -- 172.21.15.160:0/865999115 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f4ce405acb0 con 0x7f4cec0fbd40 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:24:09.670 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:24:09.671 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4cd406e150 msgr2=0x7f4cd4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4cd406e150 0x7f4cd4070600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b3b0 tx=0x7f4ce0009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 msgr2=0x7f4cec115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec115de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400cac0 tx=0x7f4ce4007ee0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 shutdown_connections 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cec0fbd40 0x7f4cec115de0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4cd406e150 0x7f4cd4070600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 --2- 172.21.15.160:0/865999115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cec101d90 0x7f4cec116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 >> 172.21.15.160:0/865999115 conn(0x7f4cec071d00 msgr2=0x7f4cec0655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:09.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 shutdown_connections 2024-05-25T06:24:09.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.672+0000 7f4cf3339700 1 -- 172.21.15.160:0/865999115 wait complete. 2024-05-25T06:24:09.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 -- 172.21.15.160:0/4257794911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 msgr2=0x7fcf3c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 --2- 172.21.15.160:0/4257794911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c102160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c004940 tx=0x7fcf2c009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:09.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 -- 172.21.15.160:0/4257794911 shutdown_connections 2024-05-25T06:24:09.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 --2- 172.21.15.160:0/4257794911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3c0fbd40 0x7fcf3c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 --2- 172.21.15.160:0/4257794911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c102160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 -- 172.21.15.160:0/4257794911 >> 172.21.15.160:0/4257794911 conn(0x7fcf3c071d00 msgr2=0x7fcf3c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:09.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 -- 172.21.15.160:0/4257794911 shutdown_connections 2024-05-25T06:24:09.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 -- 172.21.15.160:0/4257794911 wait complete. 2024-05-25T06:24:09.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.774+0000 7fcf42a27700 1 Processor -- start 2024-05-25T06:24:09.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf42a27700 1 -- start start 2024-05-25T06:24:09.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf42a27700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3c0fbd40 0x7fcf3c10c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf42a27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c10ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf42a27700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c10d3b0 con 0x7fcf3c101d90 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf42a27700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c10d520 con 0x7fcf3c0fbd40 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c10ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c10ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54662/0 (socket says 172.21.15.160:54662) 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 -- 172.21.15.160:0/2312073616 learned_addr learned my addr 172.21.15.160:0/2312073616 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3bfff700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3c0fbd40 0x7fcf3c10c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 -- 172.21.15.160:0/2312073616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3c0fbd40 msgr2=0x7fcf3c10c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3c0fbd40 0x7fcf3c10c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 -- 172.21.15.160:0/2312073616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c005040 con 0x7fcf3c101d90 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.775+0000 7fcf3b7fe700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c10ce70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c8c0 tx=0x7fcf300044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.776+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf3000a980 con 0x7fcf3c101d90 2024-05-25T06:24:09.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.776+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c119fa0 con 0x7fcf3c101d90 2024-05-25T06:24:09.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.776+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf3000ce60 con 0x7fcf3c101d90 2024-05-25T06:24:09.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.776+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf300115e0 con 0x7fcf3c101d90 2024-05-25T06:24:09.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.776+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c11a370 con 0x7fcf3c101d90 2024-05-25T06:24:09.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.777+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c0fd430 con 0x7fcf3c101d90 2024-05-25T06:24:09.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.777+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcf30012400 con 0x7fcf3c101d90 2024-05-25T06:24:09.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.778+0000 7fcf38ff9700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf2406e140 0x7fcf240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:09.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.778+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcf3008dc10 con 0x7fcf3c101d90 2024-05-25T06:24:09.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.778+0000 7fcf3bfff700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf2406e140 0x7fcf240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:09.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.778+0000 7fcf3bfff700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf2406e140 0x7fcf240705f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b3b0 tx=0x7fcf2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:09.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:09.783+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcf3005aaf0 con 0x7fcf3c101d90 2024-05-25T06:24:10.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.030+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcf3c10d6e0 con 0x7fcf3c101d90 2024-05-25T06:24:10.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.031+0000 7fcf38ff9700 1 -- 172.21.15.160:0/2312073616 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fcf300126b0 con 0x7fcf3c101d90 2024-05-25T06:24:10.031 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:24:10.031 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:24:10.031 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:24:10.032 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:24:10.033 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:10.034 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf2406e140 msgr2=0x7fcf240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf2406e140 0x7fcf240705f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b3b0 tx=0x7fcf2c009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 msgr2=0x7fcf3c10ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c10ce70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c8c0 tx=0x7fcf300044c0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 shutdown_connections 2024-05-25T06:24:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf2406e140 0x7fcf240705f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3c0fbd40 0x7fcf3c10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 --2- 172.21.15.160:0/2312073616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf3c101d90 0x7fcf3c10ce70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.033+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 >> 172.21.15.160:0/2312073616 conn(0x7fcf3c071d00 msgr2=0x7fcf3c0fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:10.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.034+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 shutdown_connections 2024-05-25T06:24:10.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.034+0000 7fcf42a27700 1 -- 172.21.15.160:0/2312073616 wait complete. 2024-05-25T06:24:10.036 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:24:10.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- 172.21.15.160:0/1703687891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c071f30 msgr2=0x7fac7c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 --2- 172.21.15.160:0/1703687891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c071f30 0x7fac7c070300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fac6c004970 tx=0x7fac6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:10.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- 172.21.15.160:0/1703687891 shutdown_connections 2024-05-25T06:24:10.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 --2- 172.21.15.160:0/1703687891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c070840 0x7fac7c070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 --2- 172.21.15.160:0/1703687891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c071f30 0x7fac7c070300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- 172.21.15.160:0/1703687891 >> 172.21.15.160:0/1703687891 conn(0x7fac7c0f8930 msgr2=0x7fac7c0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- 172.21.15.160:0/1703687891 shutdown_connections 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- 172.21.15.160:0/1703687891 wait complete. 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 Processor -- start 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- start start 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 0x7fac7c119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c071f30 0x7fac7c11a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c11aac0 con 0x7fac7c070840 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.138+0000 7fac80dda700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1140a0 con 0x7fac7c071f30 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 0x7fac7c119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 0x7fac7c119f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54686/0 (socket says 172.21.15.160:54686) 2024-05-25T06:24:10.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 -- 172.21.15.160:0/2663299066 learned_addr learned my addr 172.21.15.160:0/2663299066 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac79d9b700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c071f30 0x7fac7c11a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 -- 172.21.15.160:0/2663299066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c071f30 msgr2=0x7fac7c11a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c071f30 0x7fac7c11a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 -- 172.21.15.160:0/2663299066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c005040 con 0x7fac7c070840 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac79d9b700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c071f30 0x7fac7c11a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac7a59c700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 0x7fac7c119f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6c00e040 con 0x7fac7c070840 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac6c00bce0 con 0x7fac7c070840 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6c01c630 con 0x7fac7c070840 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c114320 con 0x7fac7c070840 2024-05-25T06:24:10.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.139+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c114860 con 0x7fac7c070840 2024-05-25T06:24:10.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.140+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7c071a40 con 0x7fac7c070840 2024-05-25T06:24:10.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.141+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fac6c013980 con 0x7fac7c070840 2024-05-25T06:24:10.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.141+0000 7fac6affd700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac6406e140 0x7fac640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:10.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.141+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fac6c08ef20 con 0x7fac7c070840 2024-05-25T06:24:10.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.142+0000 7fac79d9b700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac6406e140 0x7fac640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:10.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.142+0000 7fac79d9b700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac6406e140 0x7fac640705f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac70009fd0 tx=0x7fac70009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:10.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.146+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fac6c05be00 con 0x7fac7c070840 2024-05-25T06:24:10.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.363+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac7c1155f0 con 0x7fac6406e140 2024-05-25T06:24:10.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:10 smithi160 ceph-mon[112094]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:10.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:10 smithi160 ceph-mon[112094]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:10.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:10 smithi160 ceph-mon[112094]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:10.363 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:10 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/865999115' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.364+0000 7fac6affd700 1 -- 172.21.15.160:0/2663299066 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fac7c1155f0 con 0x7fac6406e140 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:24:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:24:10.365 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:24:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac6406e140 msgr2=0x7fac640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac6406e140 0x7fac640705f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac70009fd0 tx=0x7fac70009ee0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 msgr2=0x7fac7c119f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 0x7fac7c119f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 shutdown_connections 2024-05-25T06:24:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac7c070840 0x7fac7c119f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac6406e140 0x7fac640705f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 --2- 172.21.15.160:0/2663299066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac7c071f30 0x7fac7c11a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 >> 172.21.15.160:0/2663299066 conn(0x7fac7c0f8930 msgr2=0x7fac7c0f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:10.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 shutdown_connections 2024-05-25T06:24:10.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.366+0000 7fac80dda700 1 -- 172.21.15.160:0/2663299066 wait complete. 2024-05-25T06:24:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:10 smithi203 ceph-mon[83507]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:10 smithi203 ceph-mon[83507]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:10 smithi203 ceph-mon[83507]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:10.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:10 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/865999115' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:24:10.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 -- 172.21.15.160:0/3817195877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 msgr2=0x7fef541020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/3817195877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef541020b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fef44004940 tx=0x7fef44009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:10.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 -- 172.21.15.160:0/3817195877 shutdown_connections 2024-05-25T06:24:10.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/3817195877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 0x7fef540fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/3817195877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef541020b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 -- 172.21.15.160:0/3817195877 >> 172.21.15.160:0/3817195877 conn(0x7fef540f7850 msgr2=0x7fef540f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:10.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 -- 172.21.15.160:0/3817195877 shutdown_connections 2024-05-25T06:24:10.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.469+0000 7fef5c3a3700 1 -- 172.21.15.160:0/3817195877 wait complete. 2024-05-25T06:24:10.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5c3a3700 1 Processor -- start 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5c3a3700 1 -- start start 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5c3a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef54102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5c3a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 0x7fef54103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5c3a3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef541036a0 con 0x7fef54101ce0 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5c3a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef541037e0 con 0x7fef54102680 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef54102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef54102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54704/0 (socket says 172.21.15.160:54704) 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 -- 172.21.15.160:0/2122799389 learned_addr learned my addr 172.21.15.160:0/2122799389 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5993e700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 0x7fef54103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 -- 172.21.15.160:0/2122799389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 msgr2=0x7fef54103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 0x7fef54103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 -- 172.21.15.160:0/2122799389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef44005040 con 0x7fef54101ce0 2024-05-25T06:24:10.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5993e700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 0x7fef54103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.470+0000 7fef5a13f700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef54102b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b8b0 tx=0x7fef44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.471+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef4400e040 con 0x7fef54101ce0 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.471+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef54119e80 con 0x7fef54101ce0 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.471+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef44004290 con 0x7fef54101ce0 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.471+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef4401c3f0 con 0x7fef54101ce0 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.471+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5411a360 con 0x7fef54101ce0 2024-05-25T06:24:10.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.472+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef540fd430 con 0x7fef54101ce0 2024-05-25T06:24:10.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.472+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fef440137b0 con 0x7fef54101ce0 2024-05-25T06:24:10.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.473+0000 7fef4affd700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fef4006e140 0x7fef400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:10.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.473+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fef4408eea0 con 0x7fef54101ce0 2024-05-25T06:24:10.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.473+0000 7fef5993e700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fef4006e140 0x7fef400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:10.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.473+0000 7fef5993e700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fef4006e140 0x7fef400705f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef50008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:10.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.477+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fef44058250 con 0x7fef54101ce0 2024-05-25T06:24:10.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.755+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fef5404bed0 con 0x7fef54101ce0 2024-05-25T06:24:10.756 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.756+0000 7fef4affd700 1 -- 172.21.15.160:0/2122799389 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fef4405b910 con 0x7fef54101ce0 2024-05-25T06:24:10.756 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:24:10.756 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:24:10.756 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:24:10.756 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:24:10.756 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fef4006e140 msgr2=0x7fef400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fef4006e140 0x7fef400705f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef50008040 comp rx=0 tx=0).stop 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 msgr2=0x7fef54102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef54102b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b8b0 tx=0x7fef44009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 shutdown_connections 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef54101ce0 0x7fef54102b40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fef4006e140 0x7fef400705f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 --2- 172.21.15.160:0/2122799389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54102680 0x7fef54103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 >> 172.21.15.160:0/2122799389 conn(0x7fef540f7850 msgr2=0x7fef540f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 shutdown_connections 2024-05-25T06:24:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:10.758+0000 7fef5c3a3700 1 -- 172.21.15.160:0/2122799389 wait complete. 2024-05-25T06:24:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:11 smithi203 ceph-mon[83507]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:11 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2312073616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:24:11.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:11 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2122799389' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:24:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:11 smithi160 ceph-mon[112094]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:11 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2312073616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:24:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:11 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2122799389' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:24:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:12 smithi203 ceph-mon[83507]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:12 smithi203 ceph-mon[83507]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:12 smithi160 ceph-mon[112094]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:12 smithi160 ceph-mon[112094]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:13 smithi203 ceph-mon[83507]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:13 smithi160 ceph-mon[112094]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:15 smithi203 ceph-mon[83507]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:15 smithi160 ceph-mon[112094]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:17 smithi203 ceph-mon[83507]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:17 smithi160 ceph-mon[112094]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:19 smithi203 ceph-mon[83507]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:19 smithi160 ceph-mon[112094]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:21 smithi203 ceph-mon[83507]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:21 smithi160 ceph-mon[112094]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:23 smithi203 ceph-mon[83507]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:23 smithi160 ceph-mon[112094]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:25 smithi203 ceph-mon[83507]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:25 smithi160 ceph-mon[112094]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:27 smithi203 ceph-mon[83507]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:27 smithi160 ceph-mon[112094]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:29 smithi203 ceph-mon[83507]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:29 smithi160 ceph-mon[112094]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:31 smithi203 ceph-mon[83507]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:31 smithi160 ceph-mon[112094]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:33 smithi203 ceph-mon[83507]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:33 smithi160 ceph-mon[112094]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:35 smithi203 ceph-mon[83507]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:35 smithi160 ceph-mon[112094]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:37 smithi203 ceph-mon[83507]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:37 smithi160 ceph-mon[112094]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:39 smithi203 ceph-mon[83507]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:39 smithi160 ceph-mon[112094]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:40.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.874+0000 7fb9ee291700 1 -- 172.21.15.160:0/1241337956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 msgr2=0x7fb9e80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:40.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.874+0000 7fb9ee291700 1 --2- 172.21.15.160:0/1241337956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e80fcdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc004940 tx=0x7fb9dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:40.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.874+0000 7fb9ee291700 1 -- 172.21.15.160:0/1241337956 shutdown_connections 2024-05-25T06:24:40.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.874+0000 7fb9ee291700 1 --2- 172.21.15.160:0/1241337956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e80fcdf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:40.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.874+0000 7fb9ee291700 1 --2- 172.21.15.160:0/1241337956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9e8102de0 0x7fb9e81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.874+0000 7fb9ee291700 1 -- 172.21.15.160:0/1241337956 >> 172.21.15.160:0/1241337956 conn(0x7fb9e80f8990 msgr2=0x7fb9e80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.875+0000 7fb9ee291700 1 -- 172.21.15.160:0/1241337956 shutdown_connections 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.875+0000 7fb9ee291700 1 -- 172.21.15.160:0/1241337956 wait complete. 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.875+0000 7fb9ee291700 1 Processor -- start 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.875+0000 7fb9ee291700 1 -- start start 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.875+0000 7fb9ee291700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9e8102de0 0x7fb9e8113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9ee291700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e8113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9ee291700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81144f0 con 0x7fb9e8103780 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9ee291700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e810daf0 con 0x7fb9e8102de0 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e8113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:40.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e8113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35002/0 (socket says 172.21.15.160:35002) 2024-05-25T06:24:40.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 -- 172.21.15.160:0/3093876822 learned_addr learned my addr 172.21.15.160:0/3093876822 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:40.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e7fff700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9e8102de0 0x7fb9e8113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:40.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 -- 172.21.15.160:0/3093876822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9e8102de0 msgr2=0x7fb9e8113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:40.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9e8102de0 0x7fb9e8113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:40.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 -- 172.21.15.160:0/3093876822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc005040 con 0x7fb9e8103780 2024-05-25T06:24:40.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e77fe700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e8113ea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009100 tx=0x7fb9dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:40.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9dc00e040 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9dc009af0 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e810dd70 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9dc01c3f0 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.876+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e810e330 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.877+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e81052b0 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.878+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb9dc00baf0 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.879+0000 7fb9e4ff9700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb9d006e140 0x7fb9d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.879+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb9dc08ecc0 con 0x7fb9e8103780 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.879+0000 7fb9e7fff700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb9d006e140 0x7fb9d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:40.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.879+0000 7fb9e7fff700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb9d006e140 0x7fb9d00705f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80041d0 tx=0x7fb9d80043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:40.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:40.883+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb9dc05bc60 con 0x7fb9e8103780 2024-05-25T06:24:41.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.103+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9e810f020 con 0x7fb9d006e140 2024-05-25T06:24:41.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.105+0000 7fb9e4ff9700 1 -- 172.21.15.160:0/3093876822 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb9e810f020 con 0x7fb9d006e140 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb9d006e140 msgr2=0x7fb9d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb9d006e140 0x7fb9d00705f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80041d0 tx=0x7fb9d80043c0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 msgr2=0x7fb9e8113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e8113ea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009100 tx=0x7fb9dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 shutdown_connections 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb9d006e140 0x7fb9d00705f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9e8102de0 0x7fb9e8113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 --2- 172.21.15.160:0/3093876822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9e8103780 0x7fb9e8113ea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 >> 172.21.15.160:0/3093876822 conn(0x7fb9e80f8990 msgr2=0x7fb9e81021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:41.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 shutdown_connections 2024-05-25T06:24:41.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.107+0000 7fb9ee291700 1 -- 172.21.15.160:0/3093876822 wait complete. 2024-05-25T06:24:41.117 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:24:41.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.209+0000 7f437f613700 1 -- 172.21.15.160:0/2453616511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378103780 msgr2=0x7f43780fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.209+0000 7f437f613700 1 --2- 172.21.15.160:0/2453616511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378103780 0x7f43780fcdf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4374004940 tx=0x7f4374009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:41.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.209+0000 7f437f613700 1 -- 172.21.15.160:0/2453616511 shutdown_connections 2024-05-25T06:24:41.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.209+0000 7f437f613700 1 --2- 172.21.15.160:0/2453616511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378103780 0x7f43780fcdf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.209+0000 7f437f613700 1 --2- 172.21.15.160:0/2453616511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4378102de0 0x7f43781031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 -- 172.21.15.160:0/2453616511 >> 172.21.15.160:0/2453616511 conn(0x7f43780f8990 msgr2=0x7f43780fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:41.211 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:41 smithi160 ceph-mon[112094]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:41.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 -- 172.21.15.160:0/2453616511 shutdown_connections 2024-05-25T06:24:41.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 -- 172.21.15.160:0/2453616511 wait complete. 2024-05-25T06:24:41.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 Processor -- start 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 -- start start 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 0x7f4378115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4378103780 0x7f4378116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43781166c0 con 0x7f4378102de0 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.210+0000 7f437f613700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437810fce0 con 0x7f4378103780 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 0x7f4378115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 0x7f4378115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35020/0 (socket says 172.21.15.160:35020) 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 -- 172.21.15.160:0/2298650022 learned_addr learned my addr 172.21.15.160:0/2298650022 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437cbae700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4378103780 0x7f4378116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 -- 172.21.15.160:0/2298650022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4378103780 msgr2=0x7f4378116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4378103780 0x7f4378116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 -- 172.21.15.160:0/2298650022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4374005040 con 0x7f4378102de0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437d3af700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 0x7f4378115b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f436c002a50 tx=0x7f436c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f436c007cc0 con 0x7f4378102de0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437810ffc0 con 0x7f4378102de0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f436c00f040 con 0x7f4378102de0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.211+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f436c00da30 con 0x7f4378102de0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.212+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4378110550 con 0x7f4378102de0 2024-05-25T06:24:41.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.213+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43780fe530 con 0x7f4378102de0 2024-05-25T06:24:41.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.213+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f436c00db90 con 0x7f4378102de0 2024-05-25T06:24:41.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.213+0000 7f4369ffb700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f436406e140 0x7f43640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.214+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f436c08d840 con 0x7f4378102de0 2024-05-25T06:24:41.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.214+0000 7f437cbae700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f436406e140 0x7f43640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.214+0000 7f437cbae700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f436406e140 0x7f43640705f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4374004890 tx=0x7f4374009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:41.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.219+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f436c056bf0 con 0x7f4378102de0 2024-05-25T06:24:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:41 smithi203 ceph-mon[83507]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:41.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.437+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4378111230 con 0x7f436406e140 2024-05-25T06:24:41.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.438+0000 7f4369ffb700 1 -- 172.21.15.160:0/2298650022 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4378111230 con 0x7f436406e140 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.440+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f436406e140 msgr2=0x7f43640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.440+0000 7f437f613700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f436406e140 0x7f43640705f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4374004890 tx=0x7f4374009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.440+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 msgr2=0x7f4378115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.440+0000 7f437f613700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 0x7f4378115b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f436c002a50 tx=0x7f436c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 shutdown_connections 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4378102de0 0x7f4378115b30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f436406e140 0x7f43640705f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 --2- 172.21.15.160:0/2298650022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4378103780 0x7f4378116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 >> 172.21.15.160:0/2298650022 conn(0x7f43780f8990 msgr2=0x7f4378071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 shutdown_connections 2024-05-25T06:24:41.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.441+0000 7f437f613700 1 -- 172.21.15.160:0/2298650022 wait complete. 2024-05-25T06:24:41.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 -- 172.21.15.160:0/3511466732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 msgr2=0x7f932c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 --2- 172.21.15.160:0/3511466732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c102160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9328009ce0 tx=0x7f9328004d10 comp rx=0 tx=0).stop 2024-05-25T06:24:41.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 -- 172.21.15.160:0/3511466732 shutdown_connections 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 --2- 172.21.15.160:0/3511466732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f932c0fbd40 0x7f932c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 --2- 172.21.15.160:0/3511466732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c102160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 -- 172.21.15.160:0/3511466732 >> 172.21.15.160:0/3511466732 conn(0x7f932c071d00 msgr2=0x7f932c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 -- 172.21.15.160:0/3511466732 shutdown_connections 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 -- 172.21.15.160:0/3511466732 wait complete. 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.545+0000 7f9333dea700 1 Processor -- start 2024-05-25T06:24:41.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9333dea700 1 -- start start 2024-05-25T06:24:41.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9333dea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f932c0fbd40 0x7f932c113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9333dea700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c114260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9333dea700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c1148b0 con 0x7f932c101d90 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9333dea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c10dc80 con 0x7f932c0fbd40 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c114260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c114260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35046/0 (socket says 172.21.15.160:35046) 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 -- 172.21.15.160:0/163038167 learned_addr learned my addr 172.21.15.160:0/163038167 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331b86700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f932c0fbd40 0x7f932c113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 -- 172.21.15.160:0/163038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f932c0fbd40 msgr2=0x7f932c113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f932c0fbd40 0x7f932c113d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 -- 172.21.15.160:0/163038167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9328009940 con 0x7f932c101d90 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f9331385700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c114260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f931c002a50 tx=0x7f931c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.546+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931c007cc0 con 0x7f932c101d90 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.547+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c10df60 con 0x7f932c101d90 2024-05-25T06:24:41.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.547+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f931c00f040 con 0x7f932c101d90 2024-05-25T06:24:41.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.547+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931c00da30 con 0x7f932c101d90 2024-05-25T06:24:41.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.547+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c10e520 con 0x7f932c101d90 2024-05-25T06:24:41.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.548+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c0fd430 con 0x7f932c101d90 2024-05-25T06:24:41.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.548+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f931c00db90 con 0x7f932c101d90 2024-05-25T06:24:41.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.548+0000 7f93227fc700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f931806e140 0x7f93180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.549+0000 7f9331b86700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f931806e140 0x7f93180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.549+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f931c08d910 con 0x7f932c101d90 2024-05-25T06:24:41.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.549+0000 7f9331b86700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f931806e140 0x7f93180705f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f932800b740 tx=0x7f9328004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:41.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.553+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f931c05a770 con 0x7f932c101d90 2024-05-25T06:24:41.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.757+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f932c108e60 con 0x7f931806e140 2024-05-25T06:24:41.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.767+0000 7f93227fc700 1 -- 172.21.15.160:0/163038167 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f932c108e60 con 0x7f931806e140 2024-05-25T06:24:41.767 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:24:41.767 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (10m) 3m ago 21m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:24:41.767 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (8m) 3m ago 21m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (7m) 3m ago 20m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (10m) 3m ago 21m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (4m) 3m ago 16m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (4m) 3m ago 16m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (3m) 3m ago 16m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (3m) 3m ago 16m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (12m) 3m ago 23m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (11m) 3m ago 20m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (9m) 3m ago 23m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (8m) 3m ago 20m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (11m) 3m ago 21m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (11m) 3m ago 20m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (7m) 3m ago 19m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:24:41.768 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (6m) 3m ago 19m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:24:41.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (6m) 3m ago 18m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:24:41.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (5m) 3m ago 18m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:24:41.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (5m) 3m ago 18m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:24:41.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (4m) 3m ago 17m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:24:41.769 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (10m) 3m ago 21m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f931806e140 msgr2=0x7f93180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f931806e140 0x7f93180705f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f932800b740 tx=0x7f9328004d10 comp rx=0 tx=0).stop 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 msgr2=0x7f932c114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c114260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f931c002a50 tx=0x7f931c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 shutdown_connections 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f931806e140 0x7f93180705f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f932c0fbd40 0x7f932c113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 --2- 172.21.15.160:0/163038167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c101d90 0x7f932c114260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.769+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 >> 172.21.15.160:0/163038167 conn(0x7f932c071d00 msgr2=0x7f932c0fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.770+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 shutdown_connections 2024-05-25T06:24:41.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.770+0000 7f9333dea700 1 -- 172.21.15.160:0/163038167 wait complete. 2024-05-25T06:24:41.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 -- 172.21.15.160:0/3769090470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c101d00 msgr2=0x7fd24c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 --2- 172.21.15.160:0/3769090470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c101d00 0x7fd24c1020d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd240004940 tx=0x7fd240009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 -- 172.21.15.160:0/3769090470 shutdown_connections 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 --2- 172.21.15.160:0/3769090470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd24c1026a0 0x7fd24c0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 --2- 172.21.15.160:0/3769090470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c101d00 0x7fd24c1020d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 -- 172.21.15.160:0/3769090470 >> 172.21.15.160:0/3769090470 conn(0x7fd24c0f7870 msgr2=0x7fd24c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 -- 172.21.15.160:0/3769090470 shutdown_connections 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 -- 172.21.15.160:0/3769090470 wait complete. 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.875+0000 7fd252bf7700 1 Processor -- start 2024-05-25T06:24:41.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd252bf7700 1 -- start start 2024-05-25T06:24:41.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd252bf7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd24c101d00 0x7fd24c10eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd252bf7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 0x7fd24c10f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd252bf7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c10f5a0 con 0x7fd24c1026a0 2024-05-25T06:24:41.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd252bf7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c10f6e0 con 0x7fd24c101d00 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd24bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 0x7fd24c10f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd24bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 0x7fd24c10f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35066/0 (socket says 172.21.15.160:35066) 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd24bfff700 1 -- 172.21.15.160:0/4030599323 learned_addr learned my addr 172.21.15.160:0/4030599323 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd250993700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd24c101d00 0x7fd24c10eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd24bfff700 1 -- 172.21.15.160:0/4030599323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd24c101d00 msgr2=0x7fd24c10eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd24bfff700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd24c101d00 0x7fd24c10eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.876+0000 7fd24bfff700 1 -- 172.21.15.160:0/4030599323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240005040 con 0x7fd24c1026a0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.877+0000 7fd24bfff700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 0x7fd24c10f060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd23800b790 tx=0x7fd238009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.877+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23800f6f0 con 0x7fd24c1026a0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.877+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c10f970 con 0x7fd24c1026a0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.877+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd23800fd20 con 0x7fd24c1026a0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.877+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2380103a0 con 0x7fd24c1026a0 2024-05-25T06:24:41.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.877+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c117f90 con 0x7fd24c1026a0 2024-05-25T06:24:41.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.878+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c0fd450 con 0x7fd24c1026a0 2024-05-25T06:24:41.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.878+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd23800f850 con 0x7fd24c1026a0 2024-05-25T06:24:41.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.879+0000 7fd2497fa700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd23c06e140 0x7fd23c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:41.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.879+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd23808d8e0 con 0x7fd24c1026a0 2024-05-25T06:24:41.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.879+0000 7fd250993700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd23c06e140 0x7fd23c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:41.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.880+0000 7fd250993700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd23c06e140 0x7fd23c0705f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd240004f90 tx=0x7fd240009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:41.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:41.884+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd238056c90 con 0x7fd24c1026a0 2024-05-25T06:24:42.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.168+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd24c108e70 con 0x7fd24c1026a0 2024-05-25T06:24:42.168 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:42 smithi160 ceph-mon[112094]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:42.168 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:42 smithi160 ceph-mon[112094]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:42.168 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:42 smithi160 ceph-mon[112094]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.168+0000 7fd2497fa700 1 -- 172.21.15.160:0/4030599323 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fd23805a350 con 0x7fd24c1026a0 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:42.169 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:24:42.170 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd23c06e140 msgr2=0x7fd23c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd23c06e140 0x7fd23c0705f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd240004f90 tx=0x7fd240009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 msgr2=0x7fd24c10f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 0x7fd24c10f060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd23800b790 tx=0x7fd238009220 comp rx=0 tx=0).stop 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 shutdown_connections 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd23c06e140 0x7fd23c0705f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd24c101d00 0x7fd24c10eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 --2- 172.21.15.160:0/4030599323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd24c1026a0 0x7fd24c10f060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.170+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 >> 172.21.15.160:0/4030599323 conn(0x7fd24c0f7870 msgr2=0x7fd24c0f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.171+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 shutdown_connections 2024-05-25T06:24:42.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.171+0000 7fd252bf7700 1 -- 172.21.15.160:0/4030599323 wait complete. 2024-05-25T06:24:42.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.273+0000 7f7fdd05c700 1 -- 172.21.15.160:0/4277217383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 msgr2=0x7f7fd80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.273+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/4277217383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd80fcdf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc004940 tx=0x7f7fcc009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:42.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 -- 172.21.15.160:0/4277217383 shutdown_connections 2024-05-25T06:24:42.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/4277217383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd80fcdf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/4277217383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd8102de0 0x7f7fd81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 -- 172.21.15.160:0/4277217383 >> 172.21.15.160:0/4277217383 conn(0x7f7fd80f8990 msgr2=0x7f7fd80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:42.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 -- 172.21.15.160:0/4277217383 shutdown_connections 2024-05-25T06:24:42.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 -- 172.21.15.160:0/4277217383 wait complete. 2024-05-25T06:24:42.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 Processor -- start 2024-05-25T06:24:42.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 -- start start 2024-05-25T06:24:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.274+0000 7f7fdd05c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd8102de0 0x7f7fd8113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fdd05c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd8113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fdd05c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8114520 con 0x7f7fd8103780 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fdd05c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd810db20 con 0x7f7fd8102de0 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd8113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd8113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35084/0 (socket says 172.21.15.160:35084) 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 -- 172.21.15.160:0/3910888078 learned_addr learned my addr 172.21.15.160:0/3910888078 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd6d9d700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd8102de0 0x7f7fd8113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 -- 172.21.15.160:0/3910888078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd8102de0 msgr2=0x7f7fd8113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd8102de0 0x7f7fd8113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 -- 172.21.15.160:0/3910888078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc005040 con 0x7f7fd8103780 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fd659c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd8113ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009100 tx=0x7f7fcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fcc00e040 con 0x7f7fd8103780 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd810dda0 con 0x7f7fd8103780 2024-05-25T06:24:42.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fcc009af0 con 0x7f7fd8103780 2024-05-25T06:24:42.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fcc01c3f0 con 0x7f7fd8103780 2024-05-25T06:24:42.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.275+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd810e360 con 0x7f7fd8103780 2024-05-25T06:24:42.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.276+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd80fe530 con 0x7f7fd8103780 2024-05-25T06:24:42.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.277+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7fcc00baf0 con 0x7f7fd8103780 2024-05-25T06:24:42.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.277+0000 7f7fbf7fe700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7fc006e140 0x7f7fc00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.277+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7fcc08edb0 con 0x7f7fd8103780 2024-05-25T06:24:42.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.277+0000 7f7fd6d9d700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7fc006e140 0x7f7fc00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.278+0000 7f7fd6d9d700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7fc006e140 0x7f7fc00705f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8004eb0 tx=0x7f7fc800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:42.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.282+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7fcc05bc90 con 0x7f7fd8103780 2024-05-25T06:24:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:42 smithi203 ceph-mon[83507]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:42 smithi203 ceph-mon[83507]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:42.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:42 smithi203 ceph-mon[83507]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:42.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.522+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7fd804c710 con 0x7f7fd8103780 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.523+0000 7f7fbf7fe700 1 -- 172.21.15.160:0/3910888078 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f7fcc05b820 con 0x7f7fd8103780 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:42.523 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:24:42.524 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:24:42.525 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:42.526 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7fc006e140 msgr2=0x7f7fc00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7fc006e140 0x7f7fc00705f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8004eb0 tx=0x7f7fc800b040 comp rx=0 tx=0).stop 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 msgr2=0x7f7fd8113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd8113ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009100 tx=0x7f7fcc009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 shutdown_connections 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7fc006e140 0x7f7fc00705f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd8102de0 0x7f7fd8113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 --2- 172.21.15.160:0/3910888078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fd8103780 0x7f7fd8113ed0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 >> 172.21.15.160:0/3910888078 conn(0x7f7fd80f8990 msgr2=0x7f7fd80f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:42.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 shutdown_connections 2024-05-25T06:24:42.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.525+0000 7f7fdd05c700 1 -- 172.21.15.160:0/3910888078 wait complete. 2024-05-25T06:24:42.528 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:24:42.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 -- 172.21.15.160:0/3592461449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328103780 msgr2=0x7f03280fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 --2- 172.21.15.160:0/3592461449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328103780 0x7f03280fcdf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f031c004940 tx=0x7f031c009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 -- 172.21.15.160:0/3592461449 shutdown_connections 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 --2- 172.21.15.160:0/3592461449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328103780 0x7f03280fcdf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 --2- 172.21.15.160:0/3592461449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328102de0 0x7f03281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 -- 172.21.15.160:0/3592461449 >> 172.21.15.160:0/3592461449 conn(0x7f03280f8990 msgr2=0x7f03280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 -- 172.21.15.160:0/3592461449 shutdown_connections 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 -- 172.21.15.160:0/3592461449 wait complete. 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.628+0000 7f032e5a5700 1 Processor -- start 2024-05-25T06:24:42.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f032e5a5700 1 -- start start 2024-05-25T06:24:42.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f032e5a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 0x7f0328113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f032e5a5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328103780 0x7f0328113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f032e5a5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03281144e0 con 0x7f0328102de0 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f032e5a5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032810dae0 con 0x7f0328103780 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 0x7f0328113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 0x7f0328113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35100/0 (socket says 172.21.15.160:35100) 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 -- 172.21.15.160:0/2814923930 learned_addr learned my addr 172.21.15.160:0/2814923930 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f03277fe700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328103780 0x7f0328113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 -- 172.21.15.160:0/2814923930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328103780 msgr2=0x7f0328113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328103780 0x7f0328113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 -- 172.21.15.160:0/2814923930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c005040 con 0x7f0328102de0 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f03277fe700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328103780 0x7f0328113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.629+0000 7f0327fff700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 0x7f0328113950 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0318002a50 tx=0x7f03180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.630+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0318007cc0 con 0x7f0328102de0 2024-05-25T06:24:42.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.630+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f031800f040 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.630+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0318009640 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.630+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032810ddc0 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.630+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032810e330 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.631+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03280fe530 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.631+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f03180097c0 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.632+0000 7f0324ff9700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f031006e140 0x7f03100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.632+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f031808d910 con 0x7f0328102de0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.632+0000 7f03277fe700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f031006e140 0x7f03100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.632+0000 7f03277fe700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f031006e140 0x7f03100705f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f032810eca0 tx=0x7f031c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.636+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f031805a770 con 0x7f0328102de0 2024-05-25T06:24:42.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.852+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0328109ec0 con 0x7f031006e140 2024-05-25T06:24:42.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.853+0000 7f0324ff9700 1 -- 172.21.15.160:0/2814923930 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0328109ec0 con 0x7f031006e140 2024-05-25T06:24:42.853 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:24:42.853 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:24:42.853 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:24:42.854 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f031006e140 msgr2=0x7f03100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f031006e140 0x7f03100705f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f032810eca0 tx=0x7f031c009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 msgr2=0x7f0328113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 0x7f0328113950 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0318002a50 tx=0x7f03180046b0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 shutdown_connections 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0328102de0 0x7f0328113950 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f031006e140 0x7f03100705f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 --2- 172.21.15.160:0/2814923930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328103780 0x7f0328113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 >> 172.21.15.160:0/2814923930 conn(0x7f03280f8990 msgr2=0x7f03280f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 shutdown_connections 2024-05-25T06:24:42.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.856+0000 7f032e5a5700 1 -- 172.21.15.160:0/2814923930 wait complete. 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.962+0000 7f330e166700 1 -- 172.21.15.160:0/3867326156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308102680 msgr2=0x7f33080fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.962+0000 7f330e166700 1 --2- 172.21.15.160:0/3867326156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308102680 0x7f33080fbcf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32fc004940 tx=0x7f32fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.962+0000 7f330e166700 1 -- 172.21.15.160:0/3867326156 shutdown_connections 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.962+0000 7f330e166700 1 --2- 172.21.15.160:0/3867326156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308102680 0x7f33080fbcf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.962+0000 7f330e166700 1 --2- 172.21.15.160:0/3867326156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308101ce0 0x7f33081020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.962+0000 7f330e166700 1 -- 172.21.15.160:0/3867326156 >> 172.21.15.160:0/3867326156 conn(0x7f33080f7850 msgr2=0x7f33080f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 -- 172.21.15.160:0/3867326156 shutdown_connections 2024-05-25T06:24:42.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 -- 172.21.15.160:0/3867326156 wait complete. 2024-05-25T06:24:42.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 Processor -- start 2024-05-25T06:24:42.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 -- start start 2024-05-25T06:24:42.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 0x7f330810ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308102680 0x7f330810f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330810f6b0 con 0x7f3308101ce0 2024-05-25T06:24:42.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.963+0000 7f330e166700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330810f7f0 con 0x7f3308102680 2024-05-25T06:24:42.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 0x7f330810ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 0x7f330810ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35122/0 (socket says 172.21.15.160:35122) 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 -- 172.21.15.160:0/247471825 learned_addr learned my addr 172.21.15.160:0/247471825 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f3306ffd700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308102680 0x7f330810f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 -- 172.21.15.160:0/247471825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308102680 msgr2=0x7f330810f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308102680 0x7f330810f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 -- 172.21.15.160:0/247471825 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc005040 con 0x7f3308101ce0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f3306ffd700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308102680 0x7f330810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f33077fe700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 0x7f330810ec30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32f800eb40 tx=0x7f32f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f8007cc0 con 0x7f3308101ce0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3308117ae0 con 0x7f3308101ce0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32f800f040 con 0x7f3308101ce0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f8009620 con 0x7f3308101ce0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.964+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3308117ff0 con 0x7f3308101ce0 2024-05-25T06:24:42.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.965+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33080fd430 con 0x7f3308101ce0 2024-05-25T06:24:42.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.966+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f32f8009780 con 0x7f3308101ce0 2024-05-25T06:24:42.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.966+0000 7f32effff700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f32f006e140 0x7f32f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:24:42.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.966+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f32f808d830 con 0x7f3308101ce0 2024-05-25T06:24:42.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.966+0000 7f3306ffd700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f32f006e140 0x7f32f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:24:42.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.967+0000 7f3306ffd700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f32f006e140 0x7f32f00705f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b0d0 tx=0x7f32fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:24:42.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:42.971+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f32f8056c10 con 0x7f3308101ce0 2024-05-25T06:24:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:43 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4030599323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:24:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:43 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3910888078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:24:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:43 smithi160 ceph-mon[112094]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:43.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:43.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:43 smithi160 ceph-mon[112094]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:43.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.255+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f330805f5c0 con 0x7f3308101ce0 2024-05-25T06:24:43.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.255+0000 7f32effff700 1 -- 172.21.15.160:0/247471825 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f32f805a2d0 con 0x7f3308101ce0 2024-05-25T06:24:43.255 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:24:43.256 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:24:43.256 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:24:43.256 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:24:43.256 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f32f006e140 msgr2=0x7f32f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f32f006e140 0x7f32f00705f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b0d0 tx=0x7f32fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 msgr2=0x7f330810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 0x7f330810ec30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32f800eb40 tx=0x7f32f800b040 comp rx=0 tx=0).stop 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 shutdown_connections 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3308101ce0 0x7f330810ec30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f32f006e140 0x7f32f00705f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 --2- 172.21.15.160:0/247471825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3308102680 0x7f330810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 >> 172.21.15.160:0/247471825 conn(0x7f33080f7850 msgr2=0x7f33080f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:24:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 shutdown_connections 2024-05-25T06:24:43.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:24:43.258+0000 7f330e166700 1 -- 172.21.15.160:0/247471825 wait complete. 2024-05-25T06:24:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:43 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4030599323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:24:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:43 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3910888078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:24:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:43 smithi203 ceph-mon[83507]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:24:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:43.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:43 smithi203 ceph-mon[83507]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:44 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/247471825' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:24:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:44 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/247471825' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:24:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:45 smithi203 ceph-mon[83507]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:45 smithi160 ceph-mon[112094]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:47 smithi203 ceph-mon[83507]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:47 smithi160 ceph-mon[112094]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:49 smithi203 ceph-mon[83507]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:49 smithi160 ceph-mon[112094]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:51 smithi203 ceph-mon[83507]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:51 smithi160 ceph-mon[112094]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:53 smithi203 ceph-mon[83507]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:53 smithi160 ceph-mon[112094]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:55 smithi203 ceph-mon[83507]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:55 smithi160 ceph-mon[112094]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:57 smithi203 ceph-mon[83507]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:57 smithi160 ceph-mon[112094]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:24:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:24:59 smithi203 ceph-mon[83507]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:24:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:24:59 smithi160 ceph-mon[112094]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:01 smithi203 ceph-mon[83507]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:01 smithi160 ceph-mon[112094]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:03 smithi203 ceph-mon[83507]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:03 smithi160 ceph-mon[112094]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:05 smithi203 ceph-mon[83507]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:05 smithi160 ceph-mon[112094]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:07 smithi203 ceph-mon[83507]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:07 smithi160 ceph-mon[112094]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:09 smithi203 ceph-mon[83507]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:09 smithi160 ceph-mon[112094]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:11 smithi203 ceph-mon[83507]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:11 smithi160 ceph-mon[112094]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:13.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 -- 172.21.15.160:0/2754935473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4103780 msgr2=0x7f0dc40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 --2- 172.21.15.160:0/2754935473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4103780 0x7f0dc40fcdf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0db8004940 tx=0x7f0db8009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:13.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 -- 172.21.15.160:0/2754935473 shutdown_connections 2024-05-25T06:25:13.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 --2- 172.21.15.160:0/2754935473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4103780 0x7f0dc40fcdf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 --2- 172.21.15.160:0/2754935473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4102de0 0x7f0dc41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 -- 172.21.15.160:0/2754935473 >> 172.21.15.160:0/2754935473 conn(0x7f0dc40f8990 msgr2=0x7f0dc40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:13.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 -- 172.21.15.160:0/2754935473 shutdown_connections 2024-05-25T06:25:13.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 -- 172.21.15.160:0/2754935473 wait complete. 2024-05-25T06:25:13.376 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:13.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.375+0000 7f0dca130700 1 Processor -- start 2024-05-25T06:25:13.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dca130700 1 -- start start 2024-05-25T06:25:13.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dca130700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 0x7f0dc4115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dca130700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4103780 0x7f0dc41160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dca130700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc41166f0 con 0x7f0dc4102de0 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dca130700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc410fcd0 con 0x7f0dc4103780 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 0x7f0dc4115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 0x7f0dc4115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57552/0 (socket says 172.21.15.160:57552) 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc37fe700 1 -- 172.21.15.160:0/286610111 learned_addr learned my addr 172.21.15.160:0/286610111 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc2ffd700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4103780 0x7f0dc41160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc37fe700 1 -- 172.21.15.160:0/286610111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4103780 msgr2=0x7f0dc41160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc37fe700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4103780 0x7f0dc41160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.376+0000 7f0dc37fe700 1 -- 172.21.15.160:0/286610111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8005040 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.377+0000 7f0dc37fe700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 0x7f0dc4115b60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0db4001d80 tx=0x7f0db4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.377+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db4007d10 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.377+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0db4001e80 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.377+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db4014640 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.377+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc410ffb0 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.377+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc4110540 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.378+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc40fe530 con 0x7f0dc4102de0 2024-05-25T06:25:13.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.378+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0db400e960 con 0x7f0dc4102de0 2024-05-25T06:25:13.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.379+0000 7f0dabfff700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:13.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.379+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0db408d830 con 0x7f0dc4102de0 2024-05-25T06:25:13.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.379+0000 7f0dc2ffd700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:13.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.379+0000 7f0dc2ffd700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0db8004890 tx=0x7f0db8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:13.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.384+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0db405a7c0 con 0x7f0dc4102de0 2024-05-25T06:25:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:13.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.599+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dc4111220 con 0x7f0dac06e140 2024-05-25T06:25:13.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.601+0000 7f0dabfff700 1 -- 172.21.15.160:0/286610111 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0dc4111220 con 0x7f0dac06e140 2024-05-25T06:25:13.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 msgr2=0x7f0dac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0db8004890 tx=0x7f0db8009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:13.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 msgr2=0x7f0dc4115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 0x7f0dc4115b60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0db4001d80 tx=0x7f0db4009ec0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 shutdown_connections 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dc4102de0 0x7f0dc4115b60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0dac06e140 0x7f0dac0705f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 --2- 172.21.15.160:0/286610111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dc4103780 0x7f0dc41160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 >> 172.21.15.160:0/286610111 conn(0x7f0dc40f8990 msgr2=0x7f0dc4071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 shutdown_connections 2024-05-25T06:25:13.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.603+0000 7f0dca130700 1 -- 172.21.15.160:0/286610111 wait complete. 2024-05-25T06:25:13.614 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.706+0000 7f11c1f98700 1 -- 172.21.15.160:0/362278917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 msgr2=0x7f11bc0fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.706+0000 7f11c1f98700 1 --2- 172.21.15.160:0/362278917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc0fe1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f11ac004940 tx=0x7f11ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.706+0000 7f11c1f98700 1 -- 172.21.15.160:0/362278917 shutdown_connections 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.706+0000 7f11c1f98700 1 --2- 172.21.15.160:0/362278917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11bc0fe770 0x7f11bc106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.706+0000 7f11c1f98700 1 --2- 172.21.15.160:0/362278917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc0fe1a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.706+0000 7f11c1f98700 1 -- 172.21.15.160:0/362278917 >> 172.21.15.160:0/362278917 conn(0x7f11bc0f7860 msgr2=0x7f11bc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.707+0000 7f11c1f98700 1 -- 172.21.15.160:0/362278917 shutdown_connections 2024-05-25T06:25:13.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.707+0000 7f11c1f98700 1 -- 172.21.15.160:0/362278917 wait complete. 2024-05-25T06:25:13.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.707+0000 7f11c1f98700 1 Processor -- start 2024-05-25T06:25:13.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.707+0000 7f11c1f98700 1 -- start start 2024-05-25T06:25:13.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11c1f98700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc1138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:13.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11c1f98700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11bc0fe770 0x7f11bc113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:13.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11c1f98700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc114430 con 0x7f11bc0fddd0 2024-05-25T06:25:13.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11c1f98700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc10da30 con 0x7f11bc0fe770 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc1138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc1138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57566/0 (socket says 172.21.15.160:57566) 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 -- 172.21.15.160:0/2251953103 learned_addr learned my addr 172.21.15.160:0/2251953103 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11b2ffd700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11bc0fe770 0x7f11bc113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 -- 172.21.15.160:0/2251953103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11bc0fe770 msgr2=0x7f11bc113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11bc0fe770 0x7f11bc113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 -- 172.21.15.160:0/2251953103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac005040 con 0x7f11bc0fddd0 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11bb7fe700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc1138a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009180 tx=0x7f11ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11ac00e040 con 0x7f11bc0fddd0 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11ac013420 con 0x7f11bc0fddd0 2024-05-25T06:25:13.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.708+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc10dcb0 con 0x7f11bc0fddd0 2024-05-25T06:25:13.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.709+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11ac01c630 con 0x7f11bc0fddd0 2024-05-25T06:25:13.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.709+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc10e1f0 con 0x7f11bc0fddd0 2024-05-25T06:25:13.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.710+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11bc066b70 con 0x7f11bc0fddd0 2024-05-25T06:25:13.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.710+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f11ac013590 con 0x7f11bc0fddd0 2024-05-25T06:25:13.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.711+0000 7f11b8ff9700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f11a806e140 0x7f11a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:13.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.711+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f11ac08ef00 con 0x7f11bc0fddd0 2024-05-25T06:25:13.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.711+0000 7f11b2ffd700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f11a806e140 0x7f11a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:13.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.711+0000 7f11b2ffd700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f11a806e140 0x7f11a80705f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009fd0 tx=0x7f11a4009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:13.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.715+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f11ac05bde0 con 0x7f11bc0fddd0 2024-05-25T06:25:13.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.930+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11bc109e60 con 0x7f11a806e140 2024-05-25T06:25:13.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.931+0000 7f11b8ff9700 1 -- 172.21.15.160:0/2251953103 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f11bc109e60 con 0x7f11a806e140 2024-05-25T06:25:13.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f11a806e140 msgr2=0x7f11a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f11a806e140 0x7f11a80705f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009fd0 tx=0x7f11a4009f60 comp rx=0 tx=0).stop 2024-05-25T06:25:13.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 msgr2=0x7f11bc1138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc1138a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009180 tx=0x7f11ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 shutdown_connections 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0fddd0 0x7f11bc1138a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f11a806e140 0x7f11a80705f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 --2- 172.21.15.160:0/2251953103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11bc0fe770 0x7f11bc113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 >> 172.21.15.160:0/2251953103 conn(0x7f11bc0f7860 msgr2=0x7f11bc0f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 shutdown_connections 2024-05-25T06:25:13.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:13.933+0000 7f11c1f98700 1 -- 172.21.15.160:0/2251953103 wait complete. 2024-05-25T06:25:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 -- 172.21.15.160:0/1133573593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964071f30 msgr2=0x7fd964070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 --2- 172.21.15.160:0/1133573593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964071f30 0x7fd964070300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd960009d10 tx=0x7fd960004d10 comp rx=0 tx=0).stop 2024-05-25T06:25:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 -- 172.21.15.160:0/1133573593 shutdown_connections 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 --2- 172.21.15.160:0/1133573593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964070840 0x7fd964070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 --2- 172.21.15.160:0/1133573593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964071f30 0x7fd964070300 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 -- 172.21.15.160:0/1133573593 >> 172.21.15.160:0/1133573593 conn(0x7fd9640f89b0 msgr2=0x7fd9640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 -- 172.21.15.160:0/1133573593 shutdown_connections 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 -- 172.21.15.160:0/1133573593 wait complete. 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.039+0000 7fd96b1da700 1 Processor -- start 2024-05-25T06:25:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd96b1da700 1 -- start start 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd96b1da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 0x7fd96411a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd96b1da700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964071f30 0x7fd96411a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd96b1da700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96411af20 con 0x7fd964070840 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd96b1da700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641142f0 con 0x7fd964071f30 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd968f76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 0x7fd96411a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd968f76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 0x7fd96411a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57590/0 (socket says 172.21.15.160:57590) 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd968f76700 1 -- 172.21.15.160:0/986416160 learned_addr learned my addr 172.21.15.160:0/986416160 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd95bfff700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964071f30 0x7fd96411a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd968f76700 1 -- 172.21.15.160:0/986416160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964071f30 msgr2=0x7fd96411a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd968f76700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964071f30 0x7fd96411a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.040+0000 7fd968f76700 1 -- 172.21.15.160:0/986416160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960009940 con 0x7fd964070840 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.041+0000 7fd968f76700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 0x7fd96411a390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd960008df0 tx=0x7fd960004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.041+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96000e040 con 0x7fd964070840 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.041+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9600128e0 con 0x7fd964070840 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.041+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd964114570 con 0x7fd964070840 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.041+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96001c8a0 con 0x7fd964070840 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.041+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd964114ab0 con 0x7fd964070840 2024-05-25T06:25:14.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.042+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964071a40 con 0x7fd964070840 2024-05-25T06:25:14.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.042+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd960012a50 con 0x7fd964070840 2024-05-25T06:25:14.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.043+0000 7fd9597fa700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd94c06e140 0x7fd94c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.043+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd96008f860 con 0x7fd964070840 2024-05-25T06:25:14.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.043+0000 7fd95bfff700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd94c06e140 0x7fd94c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.043+0000 7fd95bfff700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd94c06e140 0x7fd94c0705f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd964115c30 tx=0x7fd95400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:14.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.047+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd96005c6c0 con 0x7fd964070840 2024-05-25T06:25:14.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.254+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd96404bed0 con 0x7fd94c06e140 2024-05-25T06:25:14.254 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:13 smithi160 ceph-mon[112094]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.264+0000 7fd9597fa700 1 -- 172.21.15.160:0/986416160 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd96404bed0 con 0x7fd94c06e140 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (11m) 4m ago 22m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (8m) 4m ago 22m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (8m) 3m ago 21m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (10m) 4m ago 21m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:25:14.264 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (4m) 4m ago 16m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (4m) 4m ago 16m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (4m) 3m ago 16m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (4m) 3m ago 16m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (12m) 4m ago 23m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (11m) 3m ago 21m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (9m) 4m ago 23m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (9m) 3m ago 21m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (11m) 4m ago 22m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (11m) 3m ago 21m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (8m) 4m ago 20m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (7m) 4m ago 20m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (6m) 4m ago 19m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (6m) 3m ago 19m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:25:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (5m) 3m ago 18m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:25:14.266 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (5m) 3m ago 18m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:25:14.266 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (11m) 4m ago 21m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:25:14.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd94c06e140 msgr2=0x7fd94c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd94c06e140 0x7fd94c0705f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd964115c30 tx=0x7fd95400b040 comp rx=0 tx=0).stop 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 msgr2=0x7fd96411a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 0x7fd96411a390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd960008df0 tx=0x7fd960004d10 comp rx=0 tx=0).stop 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 shutdown_connections 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd964070840 0x7fd96411a390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.266+0000 7fd96b1da700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd94c06e140 0x7fd94c0705f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.267+0000 7fd96b1da700 1 --2- 172.21.15.160:0/986416160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964071f30 0x7fd96411a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.267+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 >> 172.21.15.160:0/986416160 conn(0x7fd9640f89b0 msgr2=0x7fd9640f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.267+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 shutdown_connections 2024-05-25T06:25:14.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.267+0000 7fd96b1da700 1 -- 172.21.15.160:0/986416160 wait complete. 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 -- 172.21.15.160:0/4065693994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 msgr2=0x7f54640fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 --2- 172.21.15.160:0/4065693994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640fcdf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f544c004940 tx=0x7f544c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 -- 172.21.15.160:0/4065693994 shutdown_connections 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 --2- 172.21.15.160:0/4065693994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640fcdf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 --2- 172.21.15.160:0/4065693994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5464102de0 0x7f54641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 -- 172.21.15.160:0/4065693994 >> 172.21.15.160:0/4065693994 conn(0x7f54640f8990 msgr2=0x7f54640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 -- 172.21.15.160:0/4065693994 shutdown_connections 2024-05-25T06:25:14.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 -- 172.21.15.160:0/4065693994 wait complete. 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 Processor -- start 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.369+0000 7f5468dc7700 1 -- start start 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5468dc7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5464102de0 0x7f5464070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5468dc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5468dc7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54640721b0 con 0x7f5464103780 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5468dc7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464070ce0 con 0x7f5464102de0 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57598/0 (socket says 172.21.15.160:57598) 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 -- 172.21.15.160:0/2706602932 learned_addr learned my addr 172.21.15.160:0/2706602932 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f546259c700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5464102de0 0x7f5464070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 -- 172.21.15.160:0/2706602932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5464102de0 msgr2=0x7f5464070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5464102de0 0x7f5464070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 -- 172.21.15.160:0/2706602932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c005040 con 0x7f5464103780 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5459d9b700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640707a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f544c000c00 tx=0x7f544c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f544c021070 con 0x7f5464103780 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f544c013cd0 con 0x7f5464103780 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5464070f60 con 0x7f5464103780 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f544c00e070 con 0x7f5464103780 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.370+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5464119fd0 con 0x7f5464103780 2024-05-25T06:25:14.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.371+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54640fe530 con 0x7f5464103780 2024-05-25T06:25:14.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.372+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f544c01caf0 con 0x7f5464103780 2024-05-25T06:25:14.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.372+0000 7f545b7fe700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f545006e140 0x7f54500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.372+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f544c08f070 con 0x7f5464103780 2024-05-25T06:25:14.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.372+0000 7f546259c700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f545006e140 0x7f54500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.373+0000 7f546259c700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f545006e140 0x7f54500705f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5454004600 tx=0x7f545400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:14.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.377+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f544c058420 con 0x7f5464103780 2024-05-25T06:25:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:13 smithi203 ceph-mon[83507]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:14.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.655+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5464109ec0 con 0x7f5464103780 2024-05-25T06:25:14.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.656+0000 7f545b7fe700 1 -- 172.21.15.160:0/2706602932 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f544c05bae0 con 0x7f5464103780 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:14.656 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:25:14.657 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:25:14.657 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:14.657 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:25:14.657 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:25:14.657 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:25:14.657 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f545006e140 msgr2=0x7f54500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f545006e140 0x7f54500705f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5454004600 tx=0x7f545400b040 comp rx=0 tx=0).stop 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 msgr2=0x7f54640707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640707a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f544c000c00 tx=0x7f544c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 shutdown_connections 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f545006e140 0x7f54500705f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5464102de0 0x7f5464070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 --2- 172.21.15.160:0/2706602932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5464103780 0x7f54640707a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 >> 172.21.15.160:0/2706602932 conn(0x7f54640f8990 msgr2=0x7f54640f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:14.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 shutdown_connections 2024-05-25T06:25:14.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.658+0000 7f5468dc7700 1 -- 172.21.15.160:0/2706602932 wait complete. 2024-05-25T06:25:14.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.760+0000 7f153151e700 1 -- 172.21.15.160:0/2589086460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c102de0 msgr2=0x7f152c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.760+0000 7f153151e700 1 --2- 172.21.15.160:0/2589086460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c102de0 0x7f152c1031b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f151c004940 tx=0x7f151c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- 172.21.15.160:0/2589086460 shutdown_connections 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 --2- 172.21.15.160:0/2589086460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c103780 0x7f152c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 --2- 172.21.15.160:0/2589086460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c102de0 0x7f152c1031b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- 172.21.15.160:0/2589086460 >> 172.21.15.160:0/2589086460 conn(0x7f152c0f8990 msgr2=0x7f152c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- 172.21.15.160:0/2589086460 shutdown_connections 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- 172.21.15.160:0/2589086460 wait complete. 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 Processor -- start 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- start start 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102de0 0x7f152c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 0x7f152c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c114500 con 0x7f152c103780 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.761+0000 7f153151e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c10db50 con 0x7f152c102de0 2024-05-25T06:25:14.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 0x7f152c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 0x7f152c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57612/0 (socket says 172.21.15.160:57612) 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 -- 172.21.15.160:0/3146102579 learned_addr learned my addr 172.21.15.160:0/3146102579 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f152affd700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102de0 0x7f152c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 -- 172.21.15.160:0/3146102579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102de0 msgr2=0x7f152c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102de0 0x7f152c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 -- 172.21.15.160:0/3146102579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c005040 con 0x7f152c103780 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f152affd700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102de0 0x7f152c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f1523fff700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 0x7f152c113eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1514002a50 tx=0x7f1514004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1514007cc0 con 0x7f152c103780 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c10de30 con 0x7f152c103780 2024-05-25T06:25:14.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f151400f040 con 0x7f152c103780 2024-05-25T06:25:14.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f151400daf0 con 0x7f152c103780 2024-05-25T06:25:14.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.762+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c10e3f0 con 0x7f152c103780 2024-05-25T06:25:14.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.763+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c1000f0 con 0x7f152c103780 2024-05-25T06:25:14.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.764+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1514009810 con 0x7f152c103780 2024-05-25T06:25:14.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.764+0000 7f15237fe700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f151806e140 0x7f15180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:14.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.764+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f151408d9a0 con 0x7f152c103780 2024-05-25T06:25:14.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.764+0000 7f152affd700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f151806e140 0x7f15180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:14.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.765+0000 7f152affd700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f151806e140 0x7f15180705f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f151c004560 tx=0x7f151c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:14.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:14.769+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f151405a780 con 0x7f152c103780 2024-05-25T06:25:15.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.012+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f152c10efc0 con 0x7f152c103780 2024-05-25T06:25:15.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.012+0000 7f15237fe700 1 -- 172.21.15.160:0/3146102579 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f151405a310 con 0x7f152c103780 2024-05-25T06:25:15.012 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:25:15.012 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:25:15.012 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:25:15.013 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:25:15.014 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:15.015 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f151806e140 msgr2=0x7f15180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f151806e140 0x7f15180705f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f151c004560 tx=0x7f151c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 msgr2=0x7f152c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 0x7f152c113eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1514002a50 tx=0x7f1514004600 comp rx=0 tx=0).stop 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 shutdown_connections 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f151806e140 0x7f15180705f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102de0 0x7f152c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 --2- 172.21.15.160:0/3146102579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152c103780 0x7f152c113eb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.014+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 >> 172.21.15.160:0/3146102579 conn(0x7f152c0f8990 msgr2=0x7f152c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:15.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.015+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 shutdown_connections 2024-05-25T06:25:15.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.015+0000 7f153151e700 1 -- 172.21.15.160:0/3146102579 wait complete. 2024-05-25T06:25:15.017 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 -- 172.21.15.160:0/4244957079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 msgr2=0x7f5a380fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 --2- 172.21.15.160:0/4244957079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a380fc1b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5a20004940 tx=0x7f5a20009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 -- 172.21.15.160:0/4244957079 shutdown_connections 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 --2- 172.21.15.160:0/4244957079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a380fc1b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 --2- 172.21.15.160:0/4244957079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a38101d90 0x7f5a38102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 -- 172.21.15.160:0/4244957079 >> 172.21.15.160:0/4244957079 conn(0x7f5a38071d00 msgr2=0x7f5a38072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:15.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 -- 172.21.15.160:0/4244957079 shutdown_connections 2024-05-25T06:25:15.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.118+0000 7f5a3d349700 1 -- 172.21.15.160:0/4244957079 wait complete. 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a3d349700 1 Processor -- start 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a3d349700 1 -- start start 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a3d349700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a38115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a3d349700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a38101d90 0x7f5a381162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a3d349700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a38116930 con 0x7f5a380fbd40 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a3d349700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810fd00 con 0x7f5a38101d90 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a38115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a38115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57636/0 (socket says 172.21.15.160:57636) 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 -- 172.21.15.160:0/2980246838 learned_addr learned my addr 172.21.15.160:0/2980246838 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:15.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 -- 172.21.15.160:0/2980246838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a38101d90 msgr2=0x7f5a381162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a38101d90 0x7f5a381162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 -- 172.21.15.160:0/2980246838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a20005040 con 0x7f5a380fbd40 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a36ffd700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a38115da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5a28002a50 tx=0x7f5a280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.119+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a28007cc0 con 0x7f5a380fbd40 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.120+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a2800f040 con 0x7f5a380fbd40 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.120+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3810ffe0 con 0x7f5a380fbd40 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.120+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a28009640 con 0x7f5a380fbd40 2024-05-25T06:25:15.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.120+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a381105a0 con 0x7f5a380fbd40 2024-05-25T06:25:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.121+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a380fd430 con 0x7f5a380fbd40 2024-05-25T06:25:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.121+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5a280097c0 con 0x7f5a380fbd40 2024-05-25T06:25:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.122+0000 7f5a2f7fe700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5a2406e140 0x7f5a240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.122+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5a2808d910 con 0x7f5a380fbd40 2024-05-25T06:25:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.122+0000 7f5a367fc700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5a2406e140 0x7f5a240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:15.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.122+0000 7f5a367fc700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5a2406e140 0x7f5a240705f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a20004540 tx=0x7f5a20009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:15.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.126+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5a2805a770 con 0x7f5a380fbd40 2024-05-25T06:25:15.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.340+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a38108df0 con 0x7f5a2406e140 2024-05-25T06:25:15.341 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:14 smithi160 ceph-mon[112094]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:15.341 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:14 smithi160 ceph-mon[112094]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:15.341 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:14 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2706602932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:25:15.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.342+0000 7f5a2f7fe700 1 -- 172.21.15.160:0/2980246838 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a38108df0 con 0x7f5a2406e140 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:25:15.342 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:25:15.343 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:25:15.343 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:25:15.343 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:25:15.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5a2406e140 msgr2=0x7f5a240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5a2406e140 0x7f5a240705f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a20004540 tx=0x7f5a20009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:15.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 msgr2=0x7f5a38115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a38115da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5a28002a50 tx=0x7f5a280046b0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 shutdown_connections 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a380fbd40 0x7f5a38115da0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5a2406e140 0x7f5a240705f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 --2- 172.21.15.160:0/2980246838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a38101d90 0x7f5a381162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 >> 172.21.15.160:0/2980246838 conn(0x7f5a38071d00 msgr2=0x7f5a380f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 shutdown_connections 2024-05-25T06:25:15.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.344+0000 7f5a3d349700 1 -- 172.21.15.160:0/2980246838 wait complete. 2024-05-25T06:25:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:14 smithi203 ceph-mon[83507]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:14 smithi203 ceph-mon[83507]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:14 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2706602932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1713582183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc1026a0 msgr2=0x7f5fdc0fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1713582183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc1026a0 0x7f5fdc0fbd10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4004940 tx=0x7f5fc4009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1713582183 shutdown_connections 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1713582183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc1026a0 0x7f5fdc0fbd10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1713582183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fdc101d00 0x7f5fdc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1713582183 >> 172.21.15.160:0/1713582183 conn(0x7f5fdc0f7870 msgr2=0x7f5fdc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1713582183 shutdown_connections 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.447+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1713582183 wait complete. 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fe25c7700 1 Processor -- start 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fe25c7700 1 -- start start 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fe25c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 0x7f5fdc10eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:15.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fe25c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fdc1026a0 0x7f5fdc10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fe25c7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc10f600 con 0x7f5fdc101d00 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fe25c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc10f770 con 0x7f5fdc1026a0 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 0x7f5fdc10eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 0x7f5fdc10eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57654/0 (socket says 172.21.15.160:57654) 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 -- 172.21.15.160:0/1114588642 learned_addr learned my addr 172.21.15.160:0/1114588642 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fd3fff700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fdc1026a0 0x7f5fdc10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 -- 172.21.15.160:0/1114588642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fdc1026a0 msgr2=0x7f5fdc10f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fdc1026a0 0x7f5fdc10f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 -- 172.21.15.160:0/1114588642 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc4005040 con 0x7f5fdc101d00 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.448+0000 7f5fdbfff700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 0x7f5fdc10eb80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b790 tx=0x7f5fcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:15.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.449+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fcc00f6f0 con 0x7f5fdc101d00 2024-05-25T06:25:15.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.449+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fcc00fd20 con 0x7f5fdc101d00 2024-05-25T06:25:15.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.449+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc117bf0 con 0x7f5fdc101d00 2024-05-25T06:25:15.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.449+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fcc0103a0 con 0x7f5fdc101d00 2024-05-25T06:25:15.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.449+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc117ff0 con 0x7f5fdc101d00 2024-05-25T06:25:15.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.450+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc0fd450 con 0x7f5fdc101d00 2024-05-25T06:25:15.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.450+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5fcc011400 con 0x7f5fdc101d00 2024-05-25T06:25:15.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.451+0000 7f5fd97fa700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5fc806e140 0x7f5fc80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:15.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.451+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5fcc08d880 con 0x7f5fdc101d00 2024-05-25T06:25:15.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.451+0000 7f5fd3fff700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5fc806e140 0x7f5fc80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:15.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.451+0000 7f5fd3fff700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5fc806e140 0x7f5fc80705f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc066a40 tx=0x7f5fc4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:15.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.455+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5fcc056cf0 con 0x7f5fdc101d00 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.735+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5fdc108ee0 con 0x7f5fdc101d00 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.736+0000 7f5fd97fa700 1 -- 172.21.15.160:0/1114588642 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5fcc05a3b0 con 0x7f5fdc101d00 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:25:15.736 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5fc806e140 msgr2=0x7f5fc80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5fc806e140 0x7f5fc80705f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc066a40 tx=0x7f5fc4009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 msgr2=0x7f5fdc10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 0x7f5fdc10eb80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b790 tx=0x7f5fcc009250 comp rx=0 tx=0).stop 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 shutdown_connections 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5fdc101d00 0x7f5fdc10eb80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5fc806e140 0x7f5fc80705f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 --2- 172.21.15.160:0/1114588642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fdc1026a0 0x7f5fdc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 >> 172.21.15.160:0/1114588642 conn(0x7f5fdc0f7870 msgr2=0x7f5fdc0f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:15.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 shutdown_connections 2024-05-25T06:25:15.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:15.738+0000 7f5fe25c7700 1 -- 172.21.15.160:0/1114588642 wait complete. 2024-05-25T06:25:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:15 smithi160 ceph-mon[112094]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:15 smithi160 ceph-mon[112094]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:15 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3146102579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:25:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:15 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1114588642' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:25:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:15 smithi203 ceph-mon[83507]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:15 smithi203 ceph-mon[83507]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:15 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3146102579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:25:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:15 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1114588642' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:25:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:16 smithi203 ceph-mon[83507]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:16 smithi160 ceph-mon[112094]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:17 smithi160 ceph-mon[112094]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:17 smithi203 ceph-mon[83507]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:19 smithi203 ceph-mon[83507]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:19 smithi160 ceph-mon[112094]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:21 smithi203 ceph-mon[83507]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:21 smithi160 ceph-mon[112094]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:23 smithi203 ceph-mon[83507]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:23 smithi160 ceph-mon[112094]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:25 smithi203 ceph-mon[83507]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:25 smithi160 ceph-mon[112094]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:27 smithi203 ceph-mon[83507]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:27 smithi160 ceph-mon[112094]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:29 smithi203 ceph-mon[83507]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:29 smithi160 ceph-mon[112094]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:31 smithi203 ceph-mon[83507]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:31 smithi160 ceph-mon[112094]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:33 smithi203 ceph-mon[83507]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:33 smithi160 ceph-mon[112094]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:35 smithi203 ceph-mon[83507]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:35 smithi160 ceph-mon[112094]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:37.201 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:37 smithi203 ceph-mon[83507]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:37 smithi160 ceph-mon[112094]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:39 smithi203 ceph-mon[83507]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:39 smithi160 ceph-mon[112094]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:41 smithi203 ceph-mon[83507]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:41 smithi160 ceph-mon[112094]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:43 smithi203 ceph-mon[83507]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:43 smithi160 ceph-mon[112094]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:45.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:45 smithi203 ceph-mon[83507]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:45 smithi160 ceph-mon[112094]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:45.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 -- 172.21.15.160:0/3660805902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74103780 msgr2=0x7fbc740fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:45.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3660805902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74103780 0x7fbc740fcdf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc64004940 tx=0x7fbc64009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:45.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 -- 172.21.15.160:0/3660805902 shutdown_connections 2024-05-25T06:25:45.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3660805902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74103780 0x7fbc740fcdf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:45.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3660805902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74102de0 0x7fbc741031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:45.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 -- 172.21.15.160:0/3660805902 >> 172.21.15.160:0/3660805902 conn(0x7fbc740f8990 msgr2=0x7fbc740fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 -- 172.21.15.160:0/3660805902 shutdown_connections 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 -- 172.21.15.160:0/3660805902 wait complete. 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 Processor -- start 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.856+0000 7fbc7b277700 1 -- start start 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc7b277700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 0x7fbc741138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc7b277700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74103780 0x7fbc74113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc7b277700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74114460 con 0x7fbc74102de0 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc7b277700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7410da60 con 0x7fbc74103780 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 0x7fbc741138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 0x7fbc741138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56646/0 (socket says 172.21.15.160:56646) 2024-05-25T06:25:45.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 -- 172.21.15.160:0/3312494031 learned_addr learned my addr 172.21.15.160:0/3312494031 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc78812700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74103780 0x7fbc74113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 -- 172.21.15.160:0/3312494031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74103780 msgr2=0x7fbc74113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74103780 0x7fbc74113e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 -- 172.21.15.160:0/3312494031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc64005040 con 0x7fbc74102de0 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc78812700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74103780 0x7fbc74113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc79013700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 0x7fbc741138d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc70002a80 tx=0x7fbc700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.857+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc70007cc0 con 0x7fbc74102de0 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.858+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7410dd40 con 0x7fbc74102de0 2024-05-25T06:25:45.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.858+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc7000f040 con 0x7fbc74102de0 2024-05-25T06:25:45.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.858+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc700096f0 con 0x7fbc74102de0 2024-05-25T06:25:45.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.858+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7410e300 con 0x7fbc74102de0 2024-05-25T06:25:45.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.859+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc740fe530 con 0x7fbc74102de0 2024-05-25T06:25:45.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.859+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbc7000d690 con 0x7fbc74102de0 2024-05-25T06:25:45.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.859+0000 7fbc69ffb700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbc6006e140 0x7fbc600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:45.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.860+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbc7008d320 con 0x7fbc74102de0 2024-05-25T06:25:45.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.860+0000 7fbc78812700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbc6006e140 0x7fbc600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:45.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.860+0000 7fbc78812700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbc6006e140 0x7fbc600705f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b3b0 tx=0x7fbc64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:45.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:45.864+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbc7005a180 con 0x7fbc74102de0 2024-05-25T06:25:46.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.084+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc74109ec0 con 0x7fbc6006e140 2024-05-25T06:25:46.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.085+0000 7fbc69ffb700 1 -- 172.21.15.160:0/3312494031 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbc74109ec0 con 0x7fbc6006e140 2024-05-25T06:25:46.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbc6006e140 msgr2=0x7fbc600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbc6006e140 0x7fbc600705f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b3b0 tx=0x7fbc64009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 msgr2=0x7fbc741138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 0x7fbc741138d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc70002a80 tx=0x7fbc700046b0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 shutdown_connections 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc74102de0 0x7fbc741138d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbc6006e140 0x7fbc600705f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 --2- 172.21.15.160:0/3312494031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc74103780 0x7fbc74113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 >> 172.21.15.160:0/3312494031 conn(0x7fbc740f8990 msgr2=0x7fbc740f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 shutdown_connections 2024-05-25T06:25:46.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.087+0000 7fbc7b277700 1 -- 172.21.15.160:0/3312494031 wait complete. 2024-05-25T06:25:46.098 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:25:46.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 -- 172.21.15.160:0/3268361830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c103780 msgr2=0x7f829c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 --2- 172.21.15.160:0/3268361830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c103780 0x7f829c0fcdf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8290004940 tx=0x7f8290009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:46.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 -- 172.21.15.160:0/3268361830 shutdown_connections 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 --2- 172.21.15.160:0/3268361830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c103780 0x7f829c0fcdf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 --2- 172.21.15.160:0/3268361830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f829c102de0 0x7f829c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 -- 172.21.15.160:0/3268361830 >> 172.21.15.160:0/3268361830 conn(0x7f829c0f8990 msgr2=0x7f829c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 -- 172.21.15.160:0/3268361830 shutdown_connections 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.191+0000 7f82a3a60700 1 -- 172.21.15.160:0/3268361830 wait complete. 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a3a60700 1 Processor -- start 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a3a60700 1 -- start start 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a3a60700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 0x7f829c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a3a60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f829c103780 0x7f829c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a3a60700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1144e0 con 0x7f829c102de0 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a3a60700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c10dae0 con 0x7f829c103780 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a17fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 0x7f829c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a17fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 0x7f829c113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56656/0 (socket says 172.21.15.160:56656) 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a17fc700 1 -- 172.21.15.160:0/1111136525 learned_addr learned my addr 172.21.15.160:0/1111136525 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.192+0000 7f82a0ffb700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f829c103780 0x7f829c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f82a17fc700 1 -- 172.21.15.160:0/1111136525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f829c103780 msgr2=0x7f829c113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f82a17fc700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f829c103780 0x7f829c113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f82a17fc700 1 -- 172.21.15.160:0/1111136525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290005040 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f82a17fc700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 0x7f829c113950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f828c001d80 tx=0x7f828c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c007d10 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f828c001e80 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c10ddc0 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c014640 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.193+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c10e350 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.194+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c0fe530 con 0x7f829c102de0 2024-05-25T06:25:46.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.195+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f828c00e960 con 0x7f829c102de0 2024-05-25T06:25:46.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.195+0000 7f8299ffb700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f828406e140 0x7f82840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.195+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f828c08d830 con 0x7f829c102de0 2024-05-25T06:25:46.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.195+0000 7f82a0ffb700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f828406e140 0x7f82840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.195+0000 7f82a0ffb700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f828406e140 0x7f82840705f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8290004890 tx=0x7f8290009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:46.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.200+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f828c05a7c0 con 0x7f829c102de0 2024-05-25T06:25:46.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.413+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f829c109f10 con 0x7f828406e140 2024-05-25T06:25:46.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.414+0000 7f8299ffb700 1 -- 172.21.15.160:0/1111136525 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f829c109f10 con 0x7f828406e140 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f828406e140 msgr2=0x7f82840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f828406e140 0x7f82840705f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8290004890 tx=0x7f8290009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 msgr2=0x7f829c113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 0x7f829c113950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f828c001d80 tx=0x7f828c009ec0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 shutdown_connections 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f829c102de0 0x7f829c113950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f828406e140 0x7f82840705f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 --2- 172.21.15.160:0/1111136525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f829c103780 0x7f829c113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 >> 172.21.15.160:0/1111136525 conn(0x7f829c0f8990 msgr2=0x7f829c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 shutdown_connections 2024-05-25T06:25:46.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.417+0000 7f82a3a60700 1 -- 172.21.15.160:0/1111136525 wait complete. 2024-05-25T06:25:46.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 -- 172.21.15.160:0/960196072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f70101d00 msgr2=0x7f1f701020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 --2- 172.21.15.160:0/960196072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f70101d00 0x7f1f701020d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c004940 tx=0x7f1f6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:46.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 -- 172.21.15.160:0/960196072 shutdown_connections 2024-05-25T06:25:46.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 --2- 172.21.15.160:0/960196072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f701026a0 0x7f1f700fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 --2- 172.21.15.160:0/960196072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f70101d00 0x7f1f701020d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 -- 172.21.15.160:0/960196072 >> 172.21.15.160:0/960196072 conn(0x7f1f700f7870 msgr2=0x7f1f700f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:46.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 -- 172.21.15.160:0/960196072 shutdown_connections 2024-05-25T06:25:46.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 -- 172.21.15.160:0/960196072 wait complete. 2024-05-25T06:25:46.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 Processor -- start 2024-05-25T06:25:46.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.520+0000 7f1f771f9700 1 -- start start 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f771f9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f70101d00 0x7f1f70113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f771f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 0x7f1f70113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f771f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f70114510 con 0x7f1f701026a0 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f771f9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7010db10 con 0x7f1f70101d00 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 0x7f1f70113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 0x7f1f70113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56676/0 (socket says 172.21.15.160:56676) 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 -- 172.21.15.160:0/1188669621 learned_addr learned my addr 172.21.15.160:0/1188669621 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f74f95700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f70101d00 0x7f1f70113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 -- 172.21.15.160:0/1188669621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f70101d00 msgr2=0x7f1f70113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f70101d00 0x7f1f70113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 -- 172.21.15.160:0/1188669621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c005040 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f67fff700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 0x7f1f70113ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c890 tx=0x7f1f600044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.521+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6000a700 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.522+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7010ddf0 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.522+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f6000ad30 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.522+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f600113a0 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.522+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7010e3b0 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.523+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f700fd450 con 0x7f1f701026a0 2024-05-25T06:25:46.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.523+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1f60011610 con 0x7f1f701026a0 2024-05-25T06:25:46.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.524+0000 7f1f657fa700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f5806e140 0x7f1f580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.524+0000 7f1f74f95700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f5806e140 0x7f1f580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.524+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1f6008dc10 con 0x7f1f701026a0 2024-05-25T06:25:46.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.524+0000 7f1f74f95700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f5806e140 0x7f1f580705f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c004f90 tx=0x7f1f6c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:46.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.528+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1f6005ab60 con 0x7f1f701026a0 2024-05-25T06:25:46.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.732+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f70108e00 con 0x7f1f5806e140 2024-05-25T06:25:46.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.743+0000 7f1f657fa700 1 -- 172.21.15.160:0/1188669621 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1f70108e00 con 0x7f1f5806e140 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (11m) 5m ago 22m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (9m) 5m ago 22m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (9m) 4m ago 21m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (11m) 5m ago 22m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (5m) 5m ago 17m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (5m) 5m ago 17m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (4m) 4m ago 17m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (4m) 4m ago 17m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (13m) 5m ago 24m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (12m) 4m ago 21m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (10m) 5m ago 24m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (9m) 4m ago 21m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (12m) 5m ago 22m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (12m) 4m ago 21m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (8m) 5m ago 21m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (8m) 5m ago 20m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (7m) 5m ago 20m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (6m) 4m ago 19m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (6m) 4m ago 19m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (5m) 4m ago 18m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (11m) 5m ago 22m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f5806e140 msgr2=0x7f1f580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f5806e140 0x7f1f580705f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c004f90 tx=0x7f1f6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 msgr2=0x7f1f70113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 0x7f1f70113ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c890 tx=0x7f1f600044c0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 shutdown_connections 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1f5806e140 0x7f1f580705f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f70101d00 0x7f1f70113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 --2- 172.21.15.160:0/1188669621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f701026a0 0x7f1f70113ec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 >> 172.21.15.160:0/1188669621 conn(0x7f1f700f7870 msgr2=0x7f1f70066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:46.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 shutdown_connections 2024-05-25T06:25:46.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.745+0000 7f1f771f9700 1 -- 172.21.15.160:0/1188669621 wait complete. 2024-05-25T06:25:46.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/336527938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fe780 msgr2=0x7fc6a4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/336527938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fe780 0x7fc6a4106dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc698004940 tx=0x7fc698009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:46.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/336527938 shutdown_connections 2024-05-25T06:25:46.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/336527938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fe780 0x7fc6a4106dc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/336527938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/336527938 >> 172.21.15.160:0/336527938 conn(0x7fc6a40f7850 msgr2=0x7fc6a40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/336527938 shutdown_connections 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/336527938 wait complete. 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 Processor -- start 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.846+0000 7fc6ac2f4700 1 -- start start 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6ac2f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6ac2f4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a40fe780 0x7fc6a40fc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6ac2f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40fc6e0 con 0x7fc6a40fdde0 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6ac2f4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40fc820 con 0x7fc6a40fe780 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56692/0 (socket says 172.21.15.160:56692) 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 -- 172.21.15.160:0/1382271543 learned_addr learned my addr 172.21.15.160:0/1382271543 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6a988f700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a40fe780 0x7fc6a40fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 -- 172.21.15.160:0/1382271543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a40fe780 msgr2=0x7fc6a40fc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:46.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a40fe780 0x7fc6a40fc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:46.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 -- 172.21.15.160:0/1382271543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698005040 con 0x7fc6a40fdde0 2024-05-25T06:25:46.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.847+0000 7fc6aa090700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fbc10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc69c001d80 tx=0x7fc69c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:46.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.848+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69c007d10 con 0x7fc6a40fdde0 2024-05-25T06:25:46.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.848+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a4119e80 con 0x7fc6a40fdde0 2024-05-25T06:25:46.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.848+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc69c001e80 con 0x7fc6a40fdde0 2024-05-25T06:25:46.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.848+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69c014640 con 0x7fc6a40fdde0 2024-05-25T06:25:46.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.848+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a411a280 con 0x7fc6a40fdde0 2024-05-25T06:25:46.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.849+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a4066750 con 0x7fc6a40fdde0 2024-05-25T06:25:46.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.850+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc69c00e960 con 0x7fc6a40fdde0 2024-05-25T06:25:46.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.850+0000 7fc6a2ffd700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc68c06e140 0x7fc68c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:46.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.850+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc69c08d800 con 0x7fc6a40fdde0 2024-05-25T06:25:46.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.850+0000 7fc6a988f700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc68c06e140 0x7fc68c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:46.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.851+0000 7fc6a988f700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc68c06e140 0x7fc68c0705f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40fd550 tx=0x7fc698009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:46.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:46.855+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc69c056c60 con 0x7fc6a40fdde0 2024-05-25T06:25:47.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.138+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6a40fd300 con 0x7fc6a40fdde0 2024-05-25T06:25:47.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:47 smithi160 ceph-mon[112094]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:47.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:47 smithi160 ceph-mon[112094]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:47.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:47 smithi160 ceph-mon[112094]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:47.139 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:47 smithi160 ceph-mon[112094]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:47.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.138+0000 7fc6a2ffd700 1 -- 172.21.15.160:0/1382271543 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fc69c05a320 con 0x7fc6a40fdde0 2024-05-25T06:25:47.139 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:25:47.139 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:25:47.139 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:25:47.140 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.140+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc68c06e140 msgr2=0x7fc68c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc68c06e140 0x7fc68c0705f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40fd550 tx=0x7fc698009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 msgr2=0x7fc6a40fbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fbc10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc69c001d80 tx=0x7fc69c009ec0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 shutdown_connections 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6a40fdde0 0x7fc6a40fbc10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc68c06e140 0x7fc68c0705f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 --2- 172.21.15.160:0/1382271543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a40fe780 0x7fc6a40fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 >> 172.21.15.160:0/1382271543 conn(0x7fc6a40f7850 msgr2=0x7fc6a40f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:47.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 shutdown_connections 2024-05-25T06:25:47.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.141+0000 7fc6ac2f4700 1 -- 172.21.15.160:0/1382271543 wait complete. 2024-05-25T06:25:47.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.242+0000 7f88a545a700 1 -- 172.21.15.160:0/3340664573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 msgr2=0x7f88a0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.242+0000 7f88a545a700 1 --2- 172.21.15.160:0/3340664573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0101a60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8890004880 tx=0x7f8890009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:47.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 -- 172.21.15.160:0/3340664573 shutdown_connections 2024-05-25T06:25:47.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 --2- 172.21.15.160:0/3340664573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0101a60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 --2- 172.21.15.160:0/3340664573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a00fb0d0 0x7f88a00fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 -- 172.21.15.160:0/3340664573 >> 172.21.15.160:0/3340664573 conn(0x7f88a00f6b70 msgr2=0x7f88a00f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:47.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 -- 172.21.15.160:0/3340664573 shutdown_connections 2024-05-25T06:25:47.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 -- 172.21.15.160:0/3340664573 wait complete. 2024-05-25T06:25:47.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 Processor -- start 2024-05-25T06:25:47.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.243+0000 7f88a545a700 1 -- start start 2024-05-25T06:25:47.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88a545a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a00fb0d0 0x7f88a0113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88a545a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88a545a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a01145c0 con 0x7f88a00fba70 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88a545a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a010dba0 con 0x7f88a00fb0d0 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56712/0 (socket says 172.21.15.160:56712) 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 -- 172.21.15.160:0/1755512699 learned_addr learned my addr 172.21.15.160:0/1755512699 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:47.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889effd700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a00fb0d0 0x7f88a0113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 -- 172.21.15.160:0/1755512699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a00fb0d0 msgr2=0x7f88a0113a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a00fb0d0 0x7f88a0113a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 -- 172.21.15.160:0/1755512699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890005040 con 0x7f88a00fba70 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f889e7fc700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0113f70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8890013450 tx=0x7f8890009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889000e030 con 0x7f88a00fba70 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a010de20 con 0x7f88a00fba70 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8890013d60 con 0x7f88a00fba70 2024-05-25T06:25:47.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889001cdf0 con 0x7f88a00fba70 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.244+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a010e360 con 0x7f88a00fba70 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.245+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a00fc300 con 0x7f88a00fba70 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.246+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8890025400 con 0x7f88a00fba70 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.246+0000 7f88977fe700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888c06e140 0x7f888c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.246+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f889008f340 con 0x7f88a00fba70 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.246+0000 7f889effd700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888c06e140 0x7f888c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.247+0000 7f889effd700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888c06e140 0x7f888c0705f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8888009fd0 tx=0x7f888800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:47.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.251+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f889005c220 con 0x7f88a00fba70 2024-05-25T06:25:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:47 smithi203 ceph-mon[83507]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:47 smithi203 ceph-mon[83507]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:47 smithi203 ceph-mon[83507]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:47 smithi203 ceph-mon[83507]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:47.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.493+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f88a010f0b0 con 0x7f88a00fba70 2024-05-25T06:25:47.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.493+0000 7f88977fe700 1 -- 172.21.15.160:0/1755512699 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f889005bdb0 con 0x7f88a00fba70 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:25:47.494 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:25:47.495 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:25:47.496 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:25:47.497 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:47.497 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:47.497 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:47.497 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:25:47.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888c06e140 msgr2=0x7f888c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888c06e140 0x7f888c0705f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8888009fd0 tx=0x7f888800b040 comp rx=0 tx=0).stop 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 msgr2=0x7f88a0113f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0113f70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8890013450 tx=0x7f8890009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 shutdown_connections 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f888c06e140 0x7f888c0705f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a00fb0d0 0x7f88a0113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 --2- 172.21.15.160:0/1755512699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a00fba70 0x7f88a0113f70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 >> 172.21.15.160:0/1755512699 conn(0x7f88a00f6b70 msgr2=0x7f88a00f70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 shutdown_connections 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.496+0000 7f88a545a700 1 -- 172.21.15.160:0/1755512699 wait complete. 2024-05-25T06:25:47.498 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:25:47.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 -- 172.21.15.160:0/2411150093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 msgr2=0x7fd5380fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 --2- 172.21.15.160:0/2411150093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd5380fc1b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004940 tx=0x7fd52c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:47.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 -- 172.21.15.160:0/2411150093 shutdown_connections 2024-05-25T06:25:47.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 --2- 172.21.15.160:0/2411150093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd5380fc1b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 --2- 172.21.15.160:0/2411150093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd538101d90 0x7fd538102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 -- 172.21.15.160:0/2411150093 >> 172.21.15.160:0/2411150093 conn(0x7fd538071d00 msgr2=0x7fd538072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:47.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 -- 172.21.15.160:0/2411150093 shutdown_connections 2024-05-25T06:25:47.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 -- 172.21.15.160:0/2411150093 wait complete. 2024-05-25T06:25:47.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.599+0000 7fd53f98d700 1 Processor -- start 2024-05-25T06:25:47.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53f98d700 1 -- start start 2024-05-25T06:25:47.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53f98d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd538112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53f98d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd538101d90 0x7fd538112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53f98d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd538113610 con 0x7fd5380fbd40 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53f98d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53810ca00 con 0x7fd538101d90 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd538112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd538112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56730/0 (socket says 172.21.15.160:56730) 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 -- 172.21.15.160:0/3323809699 learned_addr learned my addr 172.21.15.160:0/3323809699 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53cf28700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd538101d90 0x7fd538112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 -- 172.21.15.160:0/3323809699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd538101d90 msgr2=0x7fd538112fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd538101d90 0x7fd538112fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 -- 172.21.15.160:0/3323809699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c005040 con 0x7fd5380fbd40 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53d729700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd538112a80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd530010ea0 tx=0x7fd53000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd530007920 con 0x7fd5380fbd40 2024-05-25T06:25:47.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.600+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53810cc80 con 0x7fd5380fbd40 2024-05-25T06:25:47.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.601+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5300090d0 con 0x7fd5380fbd40 2024-05-25T06:25:47.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.601+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd530012830 con 0x7fd5380fbd40 2024-05-25T06:25:47.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.601+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53810d190 con 0x7fd5380fbd40 2024-05-25T06:25:47.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.602+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5380fd430 con 0x7fd5380fbd40 2024-05-25T06:25:47.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.602+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd530009240 con 0x7fd5380fbd40 2024-05-25T06:25:47.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.602+0000 7fd535ffb700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd52006e140 0x7fd5200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.603+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd530016070 con 0x7fd5380fbd40 2024-05-25T06:25:47.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.603+0000 7fd53cf28700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd52006e140 0x7fd5200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.603+0000 7fd53cf28700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd52006e140 0x7fd5200705f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009b30 tx=0x7fd52c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:47.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.607+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd53005d310 con 0x7fd5380fbd40 2024-05-25T06:25:47.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.827+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd53804bed0 con 0x7fd52006e140 2024-05-25T06:25:47.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.828+0000 7fd535ffb700 1 -- 172.21.15.160:0/3323809699 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd53804bed0 con 0x7fd52006e140 2024-05-25T06:25:47.828 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:25:47.829 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:25:47.830 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:25:47.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd52006e140 msgr2=0x7fd5200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd52006e140 0x7fd5200705f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009b30 tx=0x7fd52c009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:47.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 msgr2=0x7fd538112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd538112a80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd530010ea0 tx=0x7fd53000b040 comp rx=0 tx=0).stop 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 shutdown_connections 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5380fbd40 0x7fd538112a80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd52006e140 0x7fd5200705f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 --2- 172.21.15.160:0/3323809699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd538101d90 0x7fd538112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 >> 172.21.15.160:0/3323809699 conn(0x7fd538071d00 msgr2=0x7fd5380f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 shutdown_connections 2024-05-25T06:25:47.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.831+0000 7fd53f98d700 1 -- 172.21.15.160:0/3323809699 wait complete. 2024-05-25T06:25:47.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 -- 172.21.15.160:0/1347171539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d0101d90 msgr2=0x7ff4d0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 --2- 172.21.15.160:0/1347171539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d0101d90 0x7ff4d0102160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8004940 tx=0x7ff4b8009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 -- 172.21.15.160:0/1347171539 shutdown_connections 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 --2- 172.21.15.160:0/1347171539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d00fbd40 0x7ff4d00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 --2- 172.21.15.160:0/1347171539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d0101d90 0x7ff4d0102160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 -- 172.21.15.160:0/1347171539 >> 172.21.15.160:0/1347171539 conn(0x7ff4d0071d00 msgr2=0x7ff4d0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 -- 172.21.15.160:0/1347171539 shutdown_connections 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.934+0000 7ff4d6180700 1 -- 172.21.15.160:0/1347171539 wait complete. 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4d6180700 1 Processor -- start 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4d6180700 1 -- start start 2024-05-25T06:25:47.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4d6180700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 0x7ff4d010eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4d6180700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d0101d90 0x7ff4d010efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4d6180700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010f630 con 0x7ff4d00fbd40 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4d6180700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010f770 con 0x7ff4d0101d90 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 0x7ff4d010eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 0x7ff4d010eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56748/0 (socket says 172.21.15.160:56748) 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4cf7fe700 1 -- 172.21.15.160:0/3245401429 learned_addr learned my addr 172.21.15.160:0/3245401429 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.935+0000 7ff4ceffd700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d0101d90 0x7ff4d010efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4cf7fe700 1 -- 172.21.15.160:0/3245401429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d0101d90 msgr2=0x7ff4d010efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4cf7fe700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d0101d90 0x7ff4d010efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4cf7fe700 1 -- 172.21.15.160:0/3245401429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8005040 con 0x7ff4d00fbd40 2024-05-25T06:25:47.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4cf7fe700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 0x7ff4d010eaa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b8b0 tx=0x7ff4b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:47.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b800e040 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4b8004290 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b801c3f0 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d0117b10 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.936+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d0117ff0 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.937+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d00fd430 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.938+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff4b8013820 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.938+0000 7ff4c7fff700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4bc06e140 0x7ff4bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.938+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff4b808ede0 con 0x7ff4d00fbd40 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.938+0000 7ff4ceffd700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4bc06e140 0x7ff4bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:25:47.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.939+0000 7ff4ceffd700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4bc06e140 0x7ff4bc0705f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0065fe0 tx=0x7ff4c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:25:47.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:47.943+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff4b8058190 con 0x7ff4d00fbd40 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.226+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff4d0065d90 con 0x7ff4d00fbd40 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.226+0000 7ff4c7fff700 1 -- 172.21.15.160:0/3245401429 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff4b805b850 con 0x7ff4d00fbd40 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:25:48.226 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:25:48.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4bc06e140 msgr2=0x7ff4bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4bc06e140 0x7ff4bc0705f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0065fe0 tx=0x7ff4c000b040 comp rx=0 tx=0).stop 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 msgr2=0x7ff4d010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 0x7ff4d010eaa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b8b0 tx=0x7ff4b8009e30 comp rx=0 tx=0).stop 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 shutdown_connections 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d00fbd40 0x7ff4d010eaa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4bc06e140 0x7ff4bc0705f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 --2- 172.21.15.160:0/3245401429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d0101d90 0x7ff4d010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.228+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 >> 172.21.15.160:0/3245401429 conn(0x7ff4d0071d00 msgr2=0x7ff4d00fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.229+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 shutdown_connections 2024-05-25T06:25:48.229 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:25:48.229+0000 7ff4d6180700 1 -- 172.21.15.160:0/3245401429 wait complete. 2024-05-25T06:25:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:48 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1382271543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:25:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:48 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1755512699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:25:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:48 smithi203 ceph-mon[83507]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:48.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:48 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1382271543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:25:48.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:48 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1755512699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:25:48.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:48 smithi160 ceph-mon[112094]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:25:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:49 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3245401429' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:25:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:49 smithi203 ceph-mon[83507]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:49 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3245401429' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:25:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:49 smithi160 ceph-mon[112094]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:51 smithi203 ceph-mon[83507]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:51 smithi160 ceph-mon[112094]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:53 smithi203 ceph-mon[83507]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:53 smithi160 ceph-mon[112094]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:55 smithi203 ceph-mon[83507]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:55 smithi160 ceph-mon[112094]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:57 smithi203 ceph-mon[83507]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:57 smithi160 ceph-mon[112094]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:25:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:25:59 smithi203 ceph-mon[83507]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:25:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:25:59 smithi160 ceph-mon[112094]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:01 smithi203 ceph-mon[83507]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:01 smithi160 ceph-mon[112094]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:03 smithi203 ceph-mon[83507]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:03 smithi160 ceph-mon[112094]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:05 smithi203 ceph-mon[83507]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:05 smithi160 ceph-mon[112094]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:07 smithi203 ceph-mon[83507]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:07 smithi160 ceph-mon[112094]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:09 smithi203 ceph-mon[83507]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:09 smithi160 ceph-mon[112094]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:11 smithi203 ceph-mon[83507]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:11 smithi160 ceph-mon[112094]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:13 smithi203 ceph-mon[83507]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:13 smithi160 ceph-mon[112094]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:15 smithi203 ceph-mon[83507]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:15 smithi160 ceph-mon[112094]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:17 smithi203 ceph-mon[83507]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:17 smithi160 ceph-mon[112094]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:18.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.345+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1696087263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fe7b0 msgr2=0x7fd7dc106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.345+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1696087263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fe7b0 0x7fd7dc106df0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004940 tx=0x7fd7d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1696087263 shutdown_connections 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1696087263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fe7b0 0x7fd7dc106df0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1696087263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1696087263 >> 172.21.15.160:0/1696087263 conn(0x7fd7dc0f7860 msgr2=0x7fd7dc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1696087263 shutdown_connections 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1696087263 wait complete. 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 Processor -- start 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- start start 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:18.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7dc0fe7b0 0x7fd7dc0fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0fdb90 con 0x7fd7dc0fde10 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.346+0000 7fd7e3b51700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0fc6c0 con 0x7fd7dc0fe7b0 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52006/0 (socket says 172.21.15.160:52006) 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 -- 172.21.15.160:0/1209443671 learned_addr learned my addr 172.21.15.160:0/1209443671 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e10ec700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7dc0fe7b0 0x7fd7dc0fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 -- 172.21.15.160:0/1209443671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7dc0fe7b0 msgr2=0x7fd7dc0fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7dc0fe7b0 0x7fd7dc0fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 -- 172.21.15.160:0/1209443671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d8005040 con 0x7fd7dc0fde10 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e18ed700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fbc40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0002a80 tx=0x7fd7d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d0007cc0 con 0x7fd7dc0fde10 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7d000f040 con 0x7fd7dc0fde10 2024-05-25T06:26:18.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc0fc9a0 con 0x7fd7dc0fde10 2024-05-25T06:26:18.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d000dac0 con 0x7fd7dc0fde10 2024-05-25T06:26:18.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.347+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc11a050 con 0x7fd7dc0fde10 2024-05-25T06:26:18.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.349+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc066b50 con 0x7fd7dc0fde10 2024-05-25T06:26:18.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.349+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd7d000dc20 con 0x7fd7dc0fde10 2024-05-25T06:26:18.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.350+0000 7fd7ce7fc700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd7c806e140 0x7fd7c80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:18.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.350+0000 7fd7e10ec700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd7c806e140 0x7fd7c80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:18.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.350+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd7d008d700 con 0x7fd7dc0fde10 2024-05-25T06:26:18.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.350+0000 7fd7e10ec700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd7c806e140 0x7fd7c80705f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0fd370 tx=0x7fd7d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:18.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.355+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd7d0056ab0 con 0x7fd7dc0fde10 2024-05-25T06:26:18.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.572+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7dc0f9420 con 0x7fd7c806e140 2024-05-25T06:26:18.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.573+0000 7fd7ce7fc700 1 -- 172.21.15.160:0/1209443671 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd7dc0f9420 con 0x7fd7c806e140 2024-05-25T06:26:18.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd7c806e140 msgr2=0x7fd7c80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd7c806e140 0x7fd7c80705f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0fd370 tx=0x7fd7d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 msgr2=0x7fd7dc0fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fbc40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0002a80 tx=0x7fd7d00046b0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 shutdown_connections 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7dc0fde10 0x7fd7dc0fbc40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd7c806e140 0x7fd7c80705f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 --2- 172.21.15.160:0/1209443671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7dc0fe7b0 0x7fd7dc0fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.575+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 >> 172.21.15.160:0/1209443671 conn(0x7fd7dc0f7860 msgr2=0x7fd7dc0f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.576+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 shutdown_connections 2024-05-25T06:26:18.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.576+0000 7fd7e3b51700 1 -- 172.21.15.160:0/1209443671 wait complete. 2024-05-25T06:26:18.586 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:26:18.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 -- 172.21.15.160:0/2477035186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584103790 msgr2=0x7f65840fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 --2- 172.21.15.160:0/2477035186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584103790 0x7f65840fce00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f656c004940 tx=0x7f656c009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 -- 172.21.15.160:0/2477035186 shutdown_connections 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 --2- 172.21.15.160:0/2477035186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584103790 0x7f65840fce00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 --2- 172.21.15.160:0/2477035186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6584102df0 0x7f65841031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 -- 172.21.15.160:0/2477035186 >> 172.21.15.160:0/2477035186 conn(0x7f65840f8980 msgr2=0x7f65840fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 -- 172.21.15.160:0/2477035186 shutdown_connections 2024-05-25T06:26:18.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.679+0000 7f658982e700 1 -- 172.21.15.160:0/2477035186 wait complete. 2024-05-25T06:26:18.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.680+0000 7f658982e700 1 Processor -- start 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.680+0000 7f658982e700 1 -- start start 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.680+0000 7f658982e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 0x7f6584113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f658982e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6584103790 0x7f65841140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f658982e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584114710 con 0x7f6584102df0 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f658982e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658410db00 con 0x7f6584103790 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 0x7f6584113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 0x7f6584113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52020/0 (socket says 172.21.15.160:52020) 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 -- 172.21.15.160:0/1157811716 learned_addr learned my addr 172.21.15.160:0/1157811716 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f65827fc700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6584103790 0x7f65841140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 -- 172.21.15.160:0/1157811716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6584103790 msgr2=0x7f65841140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6584103790 0x7f65841140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 -- 172.21.15.160:0/1157811716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c005040 con 0x7f6584102df0 2024-05-25T06:26:18.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f6582ffd700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 0x7f6584113b80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c830 tx=0x7f657c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:18.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c0106f0 con 0x7f6584102df0 2024-05-25T06:26:18.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f657c010d20 con 0x7f6584102df0 2024-05-25T06:26:18.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658410dde0 con 0x7f6584102df0 2024-05-25T06:26:18.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c0113a0 con 0x7f6584102df0 2024-05-25T06:26:18.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.681+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658410e370 con 0x7f6584102df0 2024-05-25T06:26:18.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.682+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65840fe540 con 0x7f6584102df0 2024-05-25T06:26:18.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.683+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f657c011500 con 0x7f6584102df0 2024-05-25T06:26:18.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.683+0000 7f657b7fe700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f657006e140 0x7f65700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:18.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.684+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f657c08dad0 con 0x7f6584102df0 2024-05-25T06:26:18.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.684+0000 7f65827fc700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f657006e140 0x7f65700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:18.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.684+0000 7f65827fc700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f657006e140 0x7f65700705f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f658410ede0 tx=0x7f656c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:18.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.688+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f657c05a9b0 con 0x7f6584102df0 2024-05-25T06:26:18.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.901+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6584109f20 con 0x7f657006e140 2024-05-25T06:26:18.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.903+0000 7f657b7fe700 1 -- 172.21.15.160:0/1157811716 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6584109f20 con 0x7f657006e140 2024-05-25T06:26:18.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f657006e140 msgr2=0x7f65700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f657006e140 0x7f65700705f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f658410ede0 tx=0x7f656c009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:18.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 msgr2=0x7f6584113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:18.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 0x7f6584113b80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c830 tx=0x7f657c004470 comp rx=0 tx=0).stop 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 shutdown_connections 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6584102df0 0x7f6584113b80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f657006e140 0x7f65700705f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 --2- 172.21.15.160:0/1157811716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6584103790 0x7f65841140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 >> 172.21.15.160:0/1157811716 conn(0x7f65840f8980 msgr2=0x7f65840f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 shutdown_connections 2024-05-25T06:26:18.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:18.905+0000 7f658982e700 1 -- 172.21.15.160:0/1157811716 wait complete. 2024-05-25T06:26:19.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1157018225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 msgr2=0x7fd8b41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1157018225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b41020a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4004940 tx=0x7fd8a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:19.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1157018225 shutdown_connections 2024-05-25T06:26:19.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1157018225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102670 0x7fd8b40fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1157018225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b41020a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1157018225 >> 172.21.15.160:0/1157018225 conn(0x7fd8b4071460 msgr2=0x7fd8b4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:19.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1157018225 shutdown_connections 2024-05-25T06:26:19.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1157018225 wait complete. 2024-05-25T06:26:19.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 Processor -- start 2024-05-25T06:26:19.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.008+0000 7fd8b8ac0700 1 -- start start 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b8ac0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b8ac0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102670 0x7fd8b41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b8ac0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4114720 con 0x7fd8b4101cd0 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b8ac0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b410db10 con 0x7fd8b4102670 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b4113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52034/0 (socket says 172.21.15.160:52034) 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 -- 172.21.15.160:0/1232390384 learned_addr learned my addr 172.21.15.160:0/1232390384 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8abfff700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102670 0x7fd8b41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 -- 172.21.15.160:0/1232390384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102670 msgr2=0x7fd8b41140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102670 0x7fd8b41140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 -- 172.21.15.160:0/1232390384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a4005040 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b259c700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b4113b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009150 tx=0x7fd8a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a400e040 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b410dd90 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8a4013420 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.009+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a401c630 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.010+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b410e350 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.010+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b41052e0 con 0x7fd8b4101cd0 2024-05-25T06:26:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.011+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd8a4013590 con 0x7fd8b4101cd0 2024-05-25T06:26:19.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.011+0000 7fd8aaffd700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd8a006e150 0x7fd8a0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.011+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd8a408edf0 con 0x7fd8b4101cd0 2024-05-25T06:26:19.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.011+0000 7fd8abfff700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd8a006e150 0x7fd8a0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.012+0000 7fd8abfff700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd8a006e150 0x7fd8a0070600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009fd0 tx=0x7fd89c009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:19.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.016+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd8a405bcd0 con 0x7fd8b4101cd0 2024-05-25T06:26:19.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.224+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b410f230 con 0x7fd8a006e150 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.234+0000 7fd8aaffd700 1 -- 172.21.15.160:0/1232390384 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd8b410f230 con 0x7fd8a006e150 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (12m) 5m ago 23m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (9m) 5m ago 23m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (9m) 5m ago 22m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (11m) 5m ago 23m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (5m) 5m ago 17m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (5m) 5m ago 17m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:26:19.234 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (5m) 5m ago 17m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (5m) 5m ago 17m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (13m) 5m ago 24m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (13m) 5m ago 22m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (10m) 5m ago 24m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (10m) 5m ago 22m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (12m) 5m ago 23m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (12m) 5m ago 22m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (9m) 5m ago 21m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (8m) 5m ago 21m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (8m) 5m ago 20m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:26:19.235 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (7m) 5m ago 20m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:26:19.236 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (6m) 5m ago 19m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:26:19.236 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (6m) 5m ago 19m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:26:19.236 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (12m) 5m ago 22m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd8a006e150 msgr2=0x7fd8a0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd8a006e150 0x7fd8a0070600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009fd0 tx=0x7fd89c009f20 comp rx=0 tx=0).stop 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 msgr2=0x7fd8b4113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b4113b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009150 tx=0x7fd8a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 shutdown_connections 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8b4101cd0 0x7fd8b4113b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd8a006e150 0x7fd8a0070600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 --2- 172.21.15.160:0/1232390384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102670 0x7fd8b41140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 >> 172.21.15.160:0/1232390384 conn(0x7fd8b4071460 msgr2=0x7fd8b40fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:19.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 shutdown_connections 2024-05-25T06:26:19.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.236+0000 7fd8b8ac0700 1 -- 172.21.15.160:0/1232390384 wait complete. 2024-05-25T06:26:19.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 -- 172.21.15.160:0/3988457160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 msgr2=0x7faffc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 --2- 172.21.15.160:0/3988457160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc103330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fafec004940 tx=0x7fafec009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:19.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 -- 172.21.15.160:0/3988457160 shutdown_connections 2024-05-25T06:26:19.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 --2- 172.21.15.160:0/3988457160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc0fb520 0x7faffc0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 --2- 172.21.15.160:0/3988457160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc103330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 -- 172.21.15.160:0/3988457160 >> 172.21.15.160:0/3988457160 conn(0x7faffc0714d0 msgr2=0x7faffc0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:19.339 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:19 smithi160 ceph-mon[112094]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:19.339 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:19 smithi160 ceph-mon[112094]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:19.340 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:19 smithi160 ceph-mon[112094]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 -- 172.21.15.160:0/3988457160 shutdown_connections 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.338+0000 7fb001a8d700 1 -- 172.21.15.160:0/3988457160 wait complete. 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7fb001a8d700 1 Processor -- start 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7fb001a8d700 1 -- start start 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7fb001a8d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc0fb520 0x7faffc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7fb001a8d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7fb001a8d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc114530 con 0x7faffc102f60 2024-05-25T06:26:19.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7fb001a8d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc10db30 con 0x7faffc0fb520 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7faff3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7faff3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52054/0 (socket says 172.21.15.160:52054) 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7faff3fff700 1 -- 172.21.15.160:0/233662033 learned_addr learned my addr 172.21.15.160:0/233662033 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.339+0000 7faffb7fe700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc0fb520 0x7faffc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff3fff700 1 -- 172.21.15.160:0/233662033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc0fb520 msgr2=0x7faffc1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff3fff700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc0fb520 0x7faffc1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff3fff700 1 -- 172.21.15.160:0/233662033 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec005040 con 0x7faffc102f60 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff3fff700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc113ee0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c8c0 tx=0x7fafe40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafe400a980 con 0x7faffc102f60 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafe400ce60 con 0x7faffc102f60 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc10de10 con 0x7faffc102f60 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafe40115e0 con 0x7faffc102f60 2024-05-25T06:26:19.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.340+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc10e350 con 0x7faffc102f60 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.341+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc100250 con 0x7faffc102f60 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.342+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fafe4012400 con 0x7faffc102f60 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.342+0000 7faff8ff9700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fafe806e140 0x7fafe80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.342+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fafe408dc40 con 0x7faffc102f60 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.342+0000 7faffb7fe700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fafe806e140 0x7fafe80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.343+0000 7faffb7fe700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fafe806e140 0x7fafe80705f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fafec004f90 tx=0x7fafec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:19.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.347+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fafe4093020 con 0x7faffc102f60 2024-05-25T06:26:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:19 smithi203 ceph-mon[83507]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:19 smithi203 ceph-mon[83507]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:19 smithi203 ceph-mon[83507]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:19.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.623+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faffc10eff0 con 0x7faffc102f60 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.624+0000 7faff8ff9700 1 -- 172.21.15.160:0/233662033 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fafe405ab20 con 0x7faffc102f60 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:26:19.624 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:26:19.625 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:26:19.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fafe806e140 msgr2=0x7fafe80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fafe806e140 0x7fafe80705f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fafec004f90 tx=0x7fafec009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:19.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 msgr2=0x7faffc113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc113ee0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c8c0 tx=0x7fafe40044c0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 shutdown_connections 2024-05-25T06:26:19.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fafe806e140 0x7fafe80705f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc0fb520 0x7faffc1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 --2- 172.21.15.160:0/233662033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faffc102f60 0x7faffc113ee0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 >> 172.21.15.160:0/233662033 conn(0x7faffc0714d0 msgr2=0x7faffc0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:19.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 shutdown_connections 2024-05-25T06:26:19.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.626+0000 7fb001a8d700 1 -- 172.21.15.160:0/233662033 wait complete. 2024-05-25T06:26:19.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2990156201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 msgr2=0x7f4c640fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2990156201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c640fc1b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4c58004940 tx=0x7f4c58009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:19.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2990156201 shutdown_connections 2024-05-25T06:26:19.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2990156201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c640fc1b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2990156201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 0x7f4c64102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2990156201 >> 172.21.15.160:0/2990156201 conn(0x7f4c64071d00 msgr2=0x7f4c64072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:19.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2990156201 shutdown_connections 2024-05-25T06:26:19.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.729+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2990156201 wait complete. 2024-05-25T06:26:19.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c6b5e4700 1 Processor -- start 2024-05-25T06:26:19.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c6b5e4700 1 -- start start 2024-05-25T06:26:19.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c6b5e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c64112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c6b5e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 0x7f4c64112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c6b5e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64113640 con 0x7f4c640fbd40 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c6b5e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6410ca30 con 0x7f4c64101d90 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c69380700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c64112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c69380700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c64112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52070/0 (socket says 172.21.15.160:52070) 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c69380700 1 -- 172.21.15.160:0/2965340523 learned_addr learned my addr 172.21.15.160:0/2965340523 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c68b7f700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 0x7f4c64112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c69380700 1 -- 172.21.15.160:0/2965340523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 msgr2=0x7f4c64112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c69380700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 0x7f4c64112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c69380700 1 -- 172.21.15.160:0/2965340523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58005040 con 0x7f4c640fbd40 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.730+0000 7f4c68b7f700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 0x7f4c64112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:26:19.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.731+0000 7f4c69380700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c64112ab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00cb10 tx=0x7f4c5c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:19.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.731+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c004200 con 0x7f4c640fbd40 2024-05-25T06:26:19.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.731+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6410cd10 con 0x7f4c640fbd40 2024-05-25T06:26:19.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.731+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c5c00acc0 con 0x7f4c640fbd40 2024-05-25T06:26:19.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.731+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c003d90 con 0x7f4c640fbd40 2024-05-25T06:26:19.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.731+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6410d2d0 con 0x7f4c640fbd40 2024-05-25T06:26:19.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.732+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c640fd430 con 0x7f4c640fbd40 2024-05-25T06:26:19.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.733+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4c5c003430 con 0x7f4c640fbd40 2024-05-25T06:26:19.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.733+0000 7f4c61ffb700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c4c06e150 0x7f4c4c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:19.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.733+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4c5c08e150 con 0x7f4c640fbd40 2024-05-25T06:26:19.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.733+0000 7f4c68b7f700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c4c06e150 0x7f4c4c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:19.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.733+0000 7f4c68b7f700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c4c06e150 0x7f4c4c070600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009fd0 tx=0x7f4c58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:19.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.738+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4c5c05b060 con 0x7f4c640fbd40 2024-05-25T06:26:19.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.977+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4c64108e10 con 0x7f4c640fbd40 2024-05-25T06:26:19.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.977+0000 7f4c61ffb700 1 -- 172.21.15.160:0/2965340523 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f4c5c05abf0 con 0x7f4c640fbd40 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:26:19.978 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:26:19.979 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:19.980 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:19.981 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:19.981 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:19.981 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c4c06e150 msgr2=0x7f4c4c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c4c06e150 0x7f4c4c070600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009fd0 tx=0x7f4c58009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 msgr2=0x7f4c64112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c64112ab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00cb10 tx=0x7f4c5c007f70 comp rx=0 tx=0).stop 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 shutdown_connections 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c640fbd40 0x7f4c64112ab0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c4c06e150 0x7f4c4c070600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 --2- 172.21.15.160:0/2965340523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c64101d90 0x7f4c64112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 >> 172.21.15.160:0/2965340523 conn(0x7f4c64071d00 msgr2=0x7f4c64104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 shutdown_connections 2024-05-25T06:26:19.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:19.980+0000 7f4c6b5e4700 1 -- 172.21.15.160:0/2965340523 wait complete. 2024-05-25T06:26:19.983 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:26:20.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 -- 172.21.15.160:0/3378016974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 msgr2=0x7f1fb8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/3378016974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8106e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8004940 tx=0x7f1fa8009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:20.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 -- 172.21.15.160:0/3378016974 shutdown_connections 2024-05-25T06:26:20.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/3378016974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8106e60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/3378016974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb80fddf0 0x7f1fb80fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 -- 172.21.15.160:0/3378016974 >> 172.21.15.160:0/3378016974 conn(0x7f1fb80f7810 msgr2=0x7f1fb80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:20.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 -- 172.21.15.160:0/3378016974 shutdown_connections 2024-05-25T06:26:20.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.084+0000 7f1fbcde5700 1 -- 172.21.15.160:0/3378016974 wait complete. 2024-05-25T06:26:20.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fbcde5700 1 Processor -- start 2024-05-25T06:26:20.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fbcde5700 1 -- start start 2024-05-25T06:26:20.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fbcde5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb80fddf0 0x7f1fb8113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:20.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fbcde5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:20.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fbcde5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb8114560 con 0x7f1fb80fe790 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fbcde5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810dad0 con 0x7f1fb80fddf0 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fb5d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fb5d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52096/0 (socket says 172.21.15.160:52096) 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fb5d9b700 1 -- 172.21.15.160:0/1385383939 learned_addr learned my addr 172.21.15.160:0/1385383939 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.085+0000 7f1fb659c700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb80fddf0 0x7f1fb8113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fb5d9b700 1 -- 172.21.15.160:0/1385383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb80fddf0 msgr2=0x7f1fb8113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fb5d9b700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb80fddf0 0x7f1fb8113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fb5d9b700 1 -- 172.21.15.160:0/1385383939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8005040 con 0x7f1fb80fe790 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fb5d9b700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8113e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009380 tx=0x7f1fa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fa800e040 con 0x7f1fb80fe790 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fa8009af0 con 0x7f1fb80fe790 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fa801c3f0 con 0x7f1fb80fe790 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb810dd50 con 0x7f1fb80fe790 2024-05-25T06:26:20.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.086+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb810e240 con 0x7f1fb80fe790 2024-05-25T06:26:20.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.087+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb81017e0 con 0x7f1fb80fe790 2024-05-25T06:26:20.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.088+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1fa8013420 con 0x7f1fb80fe790 2024-05-25T06:26:20.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.088+0000 7f1fa6ffd700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa006e140 0x7f1fa00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:20.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.088+0000 7f1fb659c700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa006e140 0x7f1fa00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:20.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.088+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1fa808eed0 con 0x7f1fb80fe790 2024-05-25T06:26:20.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.089+0000 7f1fb659c700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa006e140 0x7f1fa00705f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009bf0 tx=0x7f1fac0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:20.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.095+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1fa805bd30 con 0x7f1fb80fe790 2024-05-25T06:26:20.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.311+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fb810f030 con 0x7f1fa006e140 2024-05-25T06:26:20.312 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:20 smithi160 ceph-mon[112094]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:20.312 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:20 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/233662033' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:26:20.312 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:20 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2965340523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:26:20.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.312+0000 7f1fa6ffd700 1 -- 172.21.15.160:0/1385383939 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1fb810f030 con 0x7f1fa006e140 2024-05-25T06:26:20.312 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:26:20.313 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:26:20.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa006e140 msgr2=0x7f1fa00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa006e140 0x7f1fa00705f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009bf0 tx=0x7f1fac0091a0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 msgr2=0x7f1fb8113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8113e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009380 tx=0x7f1fa8009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 shutdown_connections 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1fa006e140 0x7f1fa00705f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb80fddf0 0x7f1fb8113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 --2- 172.21.15.160:0/1385383939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb80fe790 0x7f1fb8113e80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 >> 172.21.15.160:0/1385383939 conn(0x7f1fb80f7810 msgr2=0x7f1fb80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 shutdown_connections 2024-05-25T06:26:20.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.316+0000 7f1fbcde5700 1 -- 172.21.15.160:0/1385383939 wait complete. 2024-05-25T06:26:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:20 smithi203 ceph-mon[83507]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:20 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/233662033' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:26:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:20 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2965340523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:26:20.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 -- 172.21.15.160:0/109303938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 msgr2=0x7fc7240fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 --2- 172.21.15.160:0/109303938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc7240fe1c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc714004940 tx=0x7fc714009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:20.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 -- 172.21.15.160:0/109303938 shutdown_connections 2024-05-25T06:26:20.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 --2- 172.21.15.160:0/109303938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7240fe790 0x7fc724106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 --2- 172.21.15.160:0/109303938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc7240fe1c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 -- 172.21.15.160:0/109303938 >> 172.21.15.160:0/109303938 conn(0x7fc7240f7870 msgr2=0x7fc7240f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:20.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 -- 172.21.15.160:0/109303938 shutdown_connections 2024-05-25T06:26:20.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.419+0000 7fc72a22e700 1 -- 172.21.15.160:0/109303938 wait complete. 2024-05-25T06:26:20.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.420+0000 7fc72a22e700 1 Processor -- start 2024-05-25T06:26:20.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.420+0000 7fc72a22e700 1 -- start start 2024-05-25T06:26:20.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.420+0000 7fc72a22e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc724113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.420+0000 7fc72a22e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7240fe790 0x7fc724113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.420+0000 7fc72a22e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc724114520 con 0x7fc7240fddf0 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.420+0000 7fc72a22e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72410db20 con 0x7fc7240fe790 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc724113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc724113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52112/0 (socket says 172.21.15.160:52112) 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 -- 172.21.15.160:0/3771075795 learned_addr learned my addr 172.21.15.160:0/3771075795 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc722ffd700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7240fe790 0x7fc724113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 -- 172.21.15.160:0/3771075795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7240fe790 msgr2=0x7fc724113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7240fe790 0x7fc724113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 -- 172.21.15.160:0/3771075795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714005040 con 0x7fc7240fddf0 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc7237fe700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc724113990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc714009380 tx=0x7fc714009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71400e040 con 0x7fc7240fddf0 2024-05-25T06:26:20.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72410dda0 con 0x7fc7240fddf0 2024-05-25T06:26:20.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc714009af0 con 0x7fc7240fddf0 2024-05-25T06:26:20.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71401c630 con 0x7fc7240fddf0 2024-05-25T06:26:20.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.421+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72410e2e0 con 0x7fc7240fddf0 2024-05-25T06:26:20.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.422+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc724066ab0 con 0x7fc7240fddf0 2024-05-25T06:26:20.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.423+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc71400bbc0 con 0x7fc7240fddf0 2024-05-25T06:26:20.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.423+0000 7fc71bfff700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc71006e140 0x7fc7100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:20.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.423+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc71408ee40 con 0x7fc7240fddf0 2024-05-25T06:26:20.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.423+0000 7fc722ffd700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc71006e140 0x7fc7100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:20.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.424+0000 7fc722ffd700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc71006e140 0x7fc7100705f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc70c007fd0 tx=0x7fc70c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:20.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.428+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc71405bd20 con 0x7fc7240fddf0 2024-05-25T06:26:20.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.705+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc72410f040 con 0x7fc7240fddf0 2024-05-25T06:26:20.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.705+0000 7fc71bfff700 1 -- 172.21.15.160:0/3771075795 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc71405b8b0 con 0x7fc7240fddf0 2024-05-25T06:26:20.705 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:26:20.706 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:26:20.706 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:26:20.706 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:26:20.706 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc71006e140 msgr2=0x7fc7100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc71006e140 0x7fc7100705f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc70c007fd0 tx=0x7fc70c009040 comp rx=0 tx=0).stop 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 msgr2=0x7fc724113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc724113990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc714009380 tx=0x7fc714009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 shutdown_connections 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7240fddf0 0x7fc724113990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc71006e140 0x7fc7100705f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 --2- 172.21.15.160:0/3771075795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7240fe790 0x7fc724113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 >> 172.21.15.160:0/3771075795 conn(0x7fc7240f7870 msgr2=0x7fc7240f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 shutdown_connections 2024-05-25T06:26:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:20.708+0000 7fc72a22e700 1 -- 172.21.15.160:0/3771075795 wait complete. 2024-05-25T06:26:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:21 smithi203 ceph-mon[83507]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:21 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3771075795' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:26:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:21 smithi203 ceph-mon[83507]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:21 smithi160 ceph-mon[112094]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:21 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3771075795' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:26:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:21 smithi160 ceph-mon[112094]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:23 smithi203 ceph-mon[83507]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:23 smithi160 ceph-mon[112094]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:25 smithi203 ceph-mon[83507]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:25 smithi160 ceph-mon[112094]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:27 smithi203 ceph-mon[83507]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:27 smithi160 ceph-mon[112094]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:29 smithi203 ceph-mon[83507]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:29 smithi160 ceph-mon[112094]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:31 smithi203 ceph-mon[83507]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:31 smithi160 ceph-mon[112094]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:33 smithi203 ceph-mon[83507]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:33 smithi160 ceph-mon[112094]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:35 smithi203 ceph-mon[83507]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:35 smithi160 ceph-mon[112094]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:37 smithi203 ceph-mon[83507]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:37 smithi160 ceph-mon[112094]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:39 smithi203 ceph-mon[83507]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:39 smithi160 ceph-mon[112094]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:41 smithi203 ceph-mon[83507]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:41 smithi160 ceph-mon[112094]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:43 smithi203 ceph-mon[83507]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:43 smithi160 ceph-mon[112094]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:45 smithi203 ceph-mon[83507]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:45 smithi160 ceph-mon[112094]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:47 smithi203 ceph-mon[83507]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:47 smithi160 ceph-mon[112094]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:49 smithi203 ceph-mon[83507]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:49 smithi160 ceph-mon[112094]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 -- 172.21.15.160:0/460632544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 msgr2=0x7f0720106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 --2- 172.21.15.160:0/460632544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f0720106e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0708004940 tx=0x7f0708009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 -- 172.21.15.160:0/460632544 shutdown_connections 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 --2- 172.21.15.160:0/460632544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f0720106e50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 --2- 172.21.15.160:0/460632544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 0x7f07200fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 -- 172.21.15.160:0/460632544 >> 172.21.15.160:0/460632544 conn(0x7f07200f7860 msgr2=0x7f07200f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 -- 172.21.15.160:0/460632544 shutdown_connections 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.825+0000 7f0726e06700 1 -- 172.21.15.160:0/460632544 wait complete. 2024-05-25T06:26:50.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f0726e06700 1 Processor -- start 2024-05-25T06:26:50.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f0726e06700 1 -- start start 2024-05-25T06:26:50.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f0726e06700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 0x7f0720113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:50.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f0726e06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f07201140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:50.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f0726e06700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720114720 con 0x7f07200fe780 2024-05-25T06:26:50.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f0726e06700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072010db10 con 0x7f07200fdde0 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.826+0000 7f071ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f07201140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f07201140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57072/0 (socket says 172.21.15.160:57072) 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071ffff700 1 -- 172.21.15.160:0/2768119072 learned_addr learned my addr 172.21.15.160:0/2768119072 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f0724ba2700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 0x7f0720113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071ffff700 1 -- 172.21.15.160:0/2768119072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 msgr2=0x7f0720113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071ffff700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 0x7f0720113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071ffff700 1 -- 172.21.15.160:0/2768119072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708005040 con 0x7f07200fe780 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f0724ba2700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 0x7f0720113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071ffff700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f07201140d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0708009380 tx=0x7f0708009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070800e040 con 0x7f07200fe780 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072010dd90 con 0x7f07200fe780 2024-05-25T06:26:50.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0708009af0 con 0x7f07200fe780 2024-05-25T06:26:50.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070801c630 con 0x7f07200fe780 2024-05-25T06:26:50.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.827+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072010e2d0 con 0x7f07200fe780 2024-05-25T06:26:50.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.828+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07200666d0 con 0x7f07200fe780 2024-05-25T06:26:50.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.829+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f070800bbc0 con 0x7f07200fe780 2024-05-25T06:26:50.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.829+0000 7f071d7fa700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f070c06e140 0x7f070c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:50.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.829+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f070808eeb0 con 0x7f07200fe780 2024-05-25T06:26:50.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.829+0000 7f0724ba2700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f070c06e140 0x7f070c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:50.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.830+0000 7f0724ba2700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f070c06e140 0x7f070c0705f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0710001250 tx=0x7f071000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:50.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:50.834+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f070805bd90 con 0x7f07200fe780 2024-05-25T06:26:51.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.047+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0720109e70 con 0x7f070c06e140 2024-05-25T06:26:51.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.049+0000 7f071d7fa700 1 -- 172.21.15.160:0/2768119072 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0720109e70 con 0x7f070c06e140 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f070c06e140 msgr2=0x7f070c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f070c06e140 0x7f070c0705f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0710001250 tx=0x7f071000b040 comp rx=0 tx=0).stop 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 msgr2=0x7f07201140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f07201140d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0708009380 tx=0x7f0708009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 shutdown_connections 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f070c06e140 0x7f070c0705f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07200fdde0 0x7f0720113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 --2- 172.21.15.160:0/2768119072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07200fe780 0x7f07201140d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 >> 172.21.15.160:0/2768119072 conn(0x7f07200f7860 msgr2=0x7f07200f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:51.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 shutdown_connections 2024-05-25T06:26:51.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.051+0000 7f0726e06700 1 -- 172.21.15.160:0/2768119072 wait complete. 2024-05-25T06:26:51.062 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3568300885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac101d90 msgr2=0x7f50ac102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3568300885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac101d90 0x7f50ac102160 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f50a4004940 tx=0x7f50a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3568300885 shutdown_connections 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3568300885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac0fbd40 0x7f50ac0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3568300885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac101d90 0x7f50ac102160 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3568300885 >> 172.21.15.160:0/3568300885 conn(0x7f50ac071d00 msgr2=0x7f50ac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3568300885 shutdown_connections 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.155+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3568300885 wait complete. 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b2d9a700 1 Processor -- start 2024-05-25T06:26:51.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b2d9a700 1 -- start start 2024-05-25T06:26:51.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b2d9a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 0x7f50ac10ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b2d9a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac101d90 0x7f50ac10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b2d9a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac10f6a0 con 0x7f50ac0fbd40 2024-05-25T06:26:51.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b2d9a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac10f7e0 con 0x7f50ac101d90 2024-05-25T06:26:51.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b0b36700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 0x7f50ac10ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b0b36700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 0x7f50ac10ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57104/0 (socket says 172.21.15.160:57104) 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50b0b36700 1 -- 172.21.15.160:0/3209866672 learned_addr learned my addr 172.21.15.160:0/3209866672 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.156+0000 7f50abfff700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac101d90 0x7f50ac10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50b0b36700 1 -- 172.21.15.160:0/3209866672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac101d90 msgr2=0x7f50ac10f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50b0b36700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac101d90 0x7f50ac10f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50b0b36700 1 -- 172.21.15.160:0/3209866672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a4005040 con 0x7f50ac0fbd40 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50b0b36700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 0x7f50ac10ec20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009b30 tx=0x7f50a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a4021070 con 0x7f50ac0fbd40 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a4013c00 con 0x7f50ac0fbd40 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a400e070 con 0x7f50ac0fbd40 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac117ad0 con 0x7f50ac0fbd40 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.157+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac117f30 con 0x7f50ac0fbd40 2024-05-25T06:26:51.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.158+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac0fd430 con 0x7f50ac0fbd40 2024-05-25T06:26:51.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.158+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f50a401cb60 con 0x7f50ac0fbd40 2024-05-25T06:26:51.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.159+0000 7f50a97fa700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f509806e140 0x7f50980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.159+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f50a408f040 con 0x7f50ac0fbd40 2024-05-25T06:26:51.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.159+0000 7f50abfff700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f509806e140 0x7f50980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.159+0000 7f50abfff700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f509806e140 0x7f50980705f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50ac066000 tx=0x7f509c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:51.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.164+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f50a40583f0 con 0x7f50ac0fbd40 2024-05-25T06:26:51.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.376+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50ac065db0 con 0x7f509806e140 2024-05-25T06:26:51.377 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:51 smithi160 ceph-mon[112094]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:51.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.378+0000 7f50a97fa700 1 -- 172.21.15.160:0/3209866672 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f50ac065db0 con 0x7f509806e140 2024-05-25T06:26:51.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f509806e140 msgr2=0x7f50980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f509806e140 0x7f50980705f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50ac066000 tx=0x7f509c00b040 comp rx=0 tx=0).stop 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 msgr2=0x7f50ac10ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 0x7f50ac10ec20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009b30 tx=0x7f50a4009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 shutdown_connections 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50ac0fbd40 0x7f50ac10ec20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f509806e140 0x7f50980705f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 --2- 172.21.15.160:0/3209866672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac101d90 0x7f50ac10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 >> 172.21.15.160:0/3209866672 conn(0x7f50ac071d00 msgr2=0x7f50ac0fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 shutdown_connections 2024-05-25T06:26:51.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.380+0000 7f50b2d9a700 1 -- 172.21.15.160:0/3209866672 wait complete. 2024-05-25T06:26:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:51 smithi203 ceph-mon[83507]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:51.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.482+0000 7f0eb11bb700 1 -- 172.21.15.160:0/1035413009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac1037c0 msgr2=0x7f0eac0fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.482+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/1035413009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac1037c0 0x7f0eac0fb3c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0004940 tx=0x7f0ea0009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.482+0000 7f0eb11bb700 1 -- 172.21.15.160:0/1035413009 shutdown_connections 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.482+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/1035413009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac1037c0 0x7f0eac0fb3c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.482+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/1035413009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac102e20 0x7f0eac1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.482+0000 7f0eb11bb700 1 -- 172.21.15.160:0/1035413009 >> 172.21.15.160:0/1035413009 conn(0x7f0eac072380 msgr2=0x7f0eac072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.483+0000 7f0eb11bb700 1 -- 172.21.15.160:0/1035413009 shutdown_connections 2024-05-25T06:26:51.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.483+0000 7f0eb11bb700 1 -- 172.21.15.160:0/1035413009 wait complete. 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.483+0000 7f0eb11bb700 1 Processor -- start 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.483+0000 7f0eb11bb700 1 -- start start 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eb11bb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 0x7f0eac113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eb11bb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac1037c0 0x7f0eac113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eb11bb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1144d0 con 0x7f0eac102e20 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eb11bb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac10dad0 con 0x7f0eac1037c0 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 0x7f0eac113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 0x7f0eac113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57120/0 (socket says 172.21.15.160:57120) 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 -- 172.21.15.160:0/2271778147 learned_addr learned my addr 172.21.15.160:0/2271778147 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaa59c700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac1037c0 0x7f0eac113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 -- 172.21.15.160:0/2271778147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac1037c0 msgr2=0x7f0eac113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac1037c0 0x7f0eac113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 -- 172.21.15.160:0/2271778147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0005040 con 0x7f0eac102e20 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eaad9d700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 0x7f0eac113940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c002a80 tx=0x7f0e9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9c007cc0 con 0x7f0eac102e20 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e9c00f040 con 0x7f0eac102e20 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac10ddb0 con 0x7f0eac102e20 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9c009630 con 0x7f0eac102e20 2024-05-25T06:26:51.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.484+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac10e2f0 con 0x7f0eac102e20 2024-05-25T06:26:51.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.485+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac100160 con 0x7f0eac102e20 2024-05-25T06:26:51.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.486+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0e9c00d690 con 0x7f0eac102e20 2024-05-25T06:26:51.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.486+0000 7f0e937fe700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e9406e140 0x7f0e940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.486+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0e9c08d420 con 0x7f0eac102e20 2024-05-25T06:26:51.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.486+0000 7f0eaa59c700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e9406e140 0x7f0e940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.487+0000 7f0eaa59c700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e9406e140 0x7f0e940705f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0eac10ec90 tx=0x7f0ea0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:51.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.491+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0e9c05a280 con 0x7f0eac102e20 2024-05-25T06:26:51.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.694+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eac10ef90 con 0x7f0e9406e140 2024-05-25T06:26:51.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.705+0000 7f0e937fe700 1 -- 172.21.15.160:0/2271778147 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0eac10ef90 con 0x7f0e9406e140 2024-05-25T06:26:51.705 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:26:51.705 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (12m) 6m ago 23m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:26:51.705 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (10m) 6m ago 24m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:26:51.705 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (10m) 5m ago 23m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:26:51.705 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (12m) 6m ago 23m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (6m) 6m ago 18m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (6m) 6m ago 18m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (5m) 5m ago 18m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (5m) 5m ago 18m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (14m) 6m ago 25m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (13m) 5m ago 22m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (11m) 6m ago 25m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (10m) 5m ago 22m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (13m) 6m ago 23m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (13m) 5m ago 22m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (9m) 6m ago 22m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (9m) 6m ago 21m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:26:51.706 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (8m) 6m ago 21m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:26:51.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (7m) 5m ago 20m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:26:51.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (7m) 5m ago 20m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:26:51.707 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (6m) 5m ago 19m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:26:51.707 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (13m) 6m ago 23m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:26:51.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e9406e140 msgr2=0x7f0e940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e9406e140 0x7f0e940705f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0eac10ec90 tx=0x7f0ea0009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 msgr2=0x7f0eac113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 0x7f0eac113940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c002a80 tx=0x7f0e9c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 shutdown_connections 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0eac102e20 0x7f0eac113940 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e9406e140 0x7f0e940705f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 --2- 172.21.15.160:0/2271778147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac1037c0 0x7f0eac113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.707+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 >> 172.21.15.160:0/2271778147 conn(0x7f0eac072380 msgr2=0x7f0eac0fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.708+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 shutdown_connections 2024-05-25T06:26:51.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.708+0000 7f0eb11bb700 1 -- 172.21.15.160:0/2271778147 wait complete. 2024-05-25T06:26:51.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 -- 172.21.15.160:0/1738355025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 msgr2=0x7f627c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 --2- 172.21.15.160:0/1738355025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c0fcdf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f626c009ce0 tx=0x7f626c004d10 comp rx=0 tx=0).stop 2024-05-25T06:26:51.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 -- 172.21.15.160:0/1738355025 shutdown_connections 2024-05-25T06:26:51.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 --2- 172.21.15.160:0/1738355025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c0fcdf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 --2- 172.21.15.160:0/1738355025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f627c102de0 0x7f627c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 -- 172.21.15.160:0/1738355025 >> 172.21.15.160:0/1738355025 conn(0x7f627c0f8990 msgr2=0x7f627c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:51.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 -- 172.21.15.160:0/1738355025 shutdown_connections 2024-05-25T06:26:51.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.809+0000 7f6282b9d700 1 -- 172.21.15.160:0/1738355025 wait complete. 2024-05-25T06:26:51.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6282b9d700 1 Processor -- start 2024-05-25T06:26:51.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6282b9d700 1 -- start start 2024-05-25T06:26:51.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6282b9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f627c102de0 0x7f627c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6282b9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6282b9d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c114520 con 0x7f627c103780 2024-05-25T06:26:51.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6282b9d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c10db20 con 0x7f627c102de0 2024-05-25T06:26:51.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f627bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f627bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57130/0 (socket says 172.21.15.160:57130) 2024-05-25T06:26:51.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f627bfff700 1 -- 172.21.15.160:0/902802585 learned_addr learned my addr 172.21.15.160:0/902802585 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:51.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.810+0000 7f6280939700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f627c102de0 0x7f627c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f627bfff700 1 -- 172.21.15.160:0/902802585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f627c102de0 msgr2=0x7f627c113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:51.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f627bfff700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f627c102de0 0x7f627c113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:51.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f627bfff700 1 -- 172.21.15.160:0/902802585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c009940 con 0x7f627c103780 2024-05-25T06:26:51.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f627bfff700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c113ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f626c000f80 tx=0x7f626c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:51.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c00e040 con 0x7f627c103780 2024-05-25T06:26:51.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f626c01cb20 con 0x7f627c103780 2024-05-25T06:26:51.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c10dda0 con 0x7f627c103780 2024-05-25T06:26:51.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c012990 con 0x7f627c103780 2024-05-25T06:26:51.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.811+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c10e360 con 0x7f627c103780 2024-05-25T06:26:51.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.812+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c0fe530 con 0x7f627c103780 2024-05-25T06:26:51.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.813+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f626c01cc90 con 0x7f627c103780 2024-05-25T06:26:51.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.813+0000 7f62797fa700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f626806e140 0x7f62680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:51.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.813+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f626c08fe30 con 0x7f627c103780 2024-05-25T06:26:51.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.813+0000 7f6280939700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f626806e140 0x7f62680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:51.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.814+0000 7f6280939700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f626806e140 0x7f62680705f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6274009b80 tx=0x7f627400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:51.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:51.818+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f626c05cc90 con 0x7f627c103780 2024-05-25T06:26:52.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.098+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f627c04c710 con 0x7f627c103780 2024-05-25T06:26:52.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.098+0000 7f62797fa700 1 -- 172.21.15.160:0/902802585 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f626c05c820 con 0x7f627c103780 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:26:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:26:52.100 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:26:52.100 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:26:52.100 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:26:52.100 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:26:52.100 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f626806e140 msgr2=0x7f62680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f626806e140 0x7f62680705f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6274009b80 tx=0x7f627400b040 comp rx=0 tx=0).stop 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 msgr2=0x7f627c113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c113ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f626c000f80 tx=0x7f626c004d10 comp rx=0 tx=0).stop 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 shutdown_connections 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f626806e140 0x7f62680705f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f627c102de0 0x7f627c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 --2- 172.21.15.160:0/902802585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c103780 0x7f627c113ed0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 >> 172.21.15.160:0/902802585 conn(0x7f627c0f8990 msgr2=0x7f627c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 shutdown_connections 2024-05-25T06:26:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.101+0000 7f6282b9d700 1 -- 172.21.15.160:0/902802585 wait complete. 2024-05-25T06:26:52.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 -- 172.21.15.160:0/2639254496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304102de0 msgr2=0x7fc3041031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 --2- 172.21.15.160:0/2639254496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304102de0 0x7fc3041031b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec004940 tx=0x7fc2ec009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:52.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 -- 172.21.15.160:0/2639254496 shutdown_connections 2024-05-25T06:26:52.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 --2- 172.21.15.160:0/2639254496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304103780 0x7fc3040fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 --2- 172.21.15.160:0/2639254496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304102de0 0x7fc3041031b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 -- 172.21.15.160:0/2639254496 >> 172.21.15.160:0/2639254496 conn(0x7fc3040f8990 msgr2=0x7fc3040fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 -- 172.21.15.160:0/2639254496 shutdown_connections 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.203+0000 7fc309933700 1 -- 172.21.15.160:0/2639254496 wait complete. 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc309933700 1 Processor -- start 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc309933700 1 -- start start 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc309933700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304102de0 0x7fc304070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc309933700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 0x7fc3040707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc309933700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304072160 con 0x7fc304103780 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc309933700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304070ce0 con 0x7fc304102de0 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc3027fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 0x7fc3040707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc3027fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 0x7fc3040707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57146/0 (socket says 172.21.15.160:57146) 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc3027fc700 1 -- 172.21.15.160:0/534115300 learned_addr learned my addr 172.21.15.160:0/534115300 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc302ffd700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304102de0 0x7fc304070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc3027fc700 1 -- 172.21.15.160:0/534115300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304102de0 msgr2=0x7fc304070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc3027fc700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304102de0 0x7fc304070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc3027fc700 1 -- 172.21.15.160:0/534115300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec005040 con 0x7fc304103780 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.204+0000 7fc302ffd700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304102de0 0x7fc304070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.205+0000 7fc3027fc700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 0x7fc3040707a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4002a50 tx=0x7fc2f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.205+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f4007cc0 con 0x7fc304103780 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.205+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc304070fc0 con 0x7fc304103780 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.205+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2f400f040 con 0x7fc304103780 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.205+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f400da40 con 0x7fc304103780 2024-05-25T06:26:52.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.205+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30411a160 con 0x7fc304103780 2024-05-25T06:26:52.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.206+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3041000f0 con 0x7fc304103780 2024-05-25T06:26:52.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.206+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc2f40097c0 con 0x7fc304103780 2024-05-25T06:26:52.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.207+0000 7fc2fb7fe700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc2f006e140 0x7fc2f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.207+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc2f408d9a0 con 0x7fc304103780 2024-05-25T06:26:52.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.207+0000 7fc302ffd700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc2f006e140 0x7fc2f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.207+0000 7fc302ffd700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc2f006e140 0x7fc2f00705f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec004560 tx=0x7fc2ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:52.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.211+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc2f4056cd0 con 0x7fc304103780 2024-05-25T06:26:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:52 smithi203 ceph-mon[83507]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:52 smithi203 ceph-mon[83507]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:52.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:52 smithi203 ceph-mon[83507]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:52.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.450+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc304071720 con 0x7fc304103780 2024-05-25T06:26:52.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:52 smithi160 ceph-mon[112094]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:52.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:52 smithi160 ceph-mon[112094]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:52.451 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:52 smithi160 ceph-mon[112094]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:52.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.451+0000 7fc2fb7fe700 1 -- 172.21.15.160:0/534115300 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fc2f405a390 con 0x7fc304103780 2024-05-25T06:26:52.451 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:26:52.451 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:26:52.451 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:26:52.451 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:26:52.452 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:26:52.453 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:52.454 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc2f006e140 msgr2=0x7fc2f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc2f006e140 0x7fc2f00705f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec004560 tx=0x7fc2ec009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 msgr2=0x7fc3040707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 0x7fc3040707a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4002a50 tx=0x7fc2f4004600 comp rx=0 tx=0).stop 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 shutdown_connections 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc2f006e140 0x7fc2f00705f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc304102de0 0x7fc304070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 --2- 172.21.15.160:0/534115300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc304103780 0x7fc3040707a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 >> 172.21.15.160:0/534115300 conn(0x7fc3040f8990 msgr2=0x7fc304107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:52.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 shutdown_connections 2024-05-25T06:26:52.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.453+0000 7fc309933700 1 -- 172.21.15.160:0/534115300 wait complete. 2024-05-25T06:26:52.456 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 -- 172.21.15.160:0/1456231190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 msgr2=0x7faf141020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 --2- 172.21.15.160:0/1456231190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf141020d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faf04004940 tx=0x7faf04009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 -- 172.21.15.160:0/1456231190 shutdown_connections 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 --2- 172.21.15.160:0/1456231190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf141026a0 0x7faf140fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 --2- 172.21.15.160:0/1456231190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf141020d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 -- 172.21.15.160:0/1456231190 >> 172.21.15.160:0/1456231190 conn(0x7faf140f7870 msgr2=0x7faf140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 -- 172.21.15.160:0/1456231190 shutdown_connections 2024-05-25T06:26:52.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 -- 172.21.15.160:0/1456231190 wait complete. 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 Processor -- start 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.557+0000 7faf1aa28700 1 -- start start 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf1aa28700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf14113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf1aa28700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf141026a0 0x7faf14113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf1aa28700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14114500 con 0x7faf14101d00 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf1aa28700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1410db20 con 0x7faf141026a0 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf14113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf14113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57154/0 (socket says 172.21.15.160:57154) 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 -- 172.21.15.160:0/4245091378 learned_addr learned my addr 172.21.15.160:0/4245091378 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf137fe700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf141026a0 0x7faf14113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 -- 172.21.15.160:0/4245091378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf141026a0 msgr2=0x7faf14113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf141026a0 0x7faf14113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 -- 172.21.15.160:0/4245091378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04005040 con 0x7faf14101d00 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf13fff700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf14113970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b3b0 tx=0x7faf04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf0400e040 con 0x7faf14101d00 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1410dda0 con 0x7faf14101d00 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf04013420 con 0x7faf14101d00 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf0401c630 con 0x7faf14101d00 2024-05-25T06:26:52.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.558+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1410e360 con 0x7faf14101d00 2024-05-25T06:26:52.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.559+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf140fd450 con 0x7faf14101d00 2024-05-25T06:26:52.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.560+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faf04013590 con 0x7faf14101d00 2024-05-25T06:26:52.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.560+0000 7faf10ff9700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faefc06e140 0x7faefc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.561+0000 7faf137fe700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faefc06e140 0x7faefc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.561+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faf0408ed70 con 0x7faf14101d00 2024-05-25T06:26:52.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.561+0000 7faf137fe700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faefc06e140 0x7faefc0705f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faf1410f210 tx=0x7faf0800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:52.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.565+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faf04058120 con 0x7faf14101d00 2024-05-25T06:26:52.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.779+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf1410f030 con 0x7faefc06e140 2024-05-25T06:26:52.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.780+0000 7faf10ff9700 1 -- 172.21.15.160:0/4245091378 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faf1410f030 con 0x7faefc06e140 2024-05-25T06:26:52.780 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:26:52.780 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:26:52.781 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faefc06e140 msgr2=0x7faefc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faefc06e140 0x7faefc0705f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faf1410f210 tx=0x7faf0800b040 comp rx=0 tx=0).stop 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 msgr2=0x7faf14113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf14113970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b3b0 tx=0x7faf04009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 shutdown_connections 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf14101d00 0x7faf14113970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faefc06e140 0x7faefc0705f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 --2- 172.21.15.160:0/4245091378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf141026a0 0x7faf14113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 >> 172.21.15.160:0/4245091378 conn(0x7faf140f7870 msgr2=0x7faf140f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 shutdown_connections 2024-05-25T06:26:52.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.783+0000 7faf1aa28700 1 -- 172.21.15.160:0/4245091378 wait complete. 2024-05-25T06:26:52.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 -- 172.21.15.160:0/761227767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fe7b0 msgr2=0x7f8284106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 --2- 172.21.15.160:0/761227767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fe7b0 0x7f8284106e80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8274004940 tx=0x7f8274009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:52.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 -- 172.21.15.160:0/761227767 shutdown_connections 2024-05-25T06:26:52.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 --2- 172.21.15.160:0/761227767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fe7b0 0x7f8284106e80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 --2- 172.21.15.160:0/761227767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82840fde10 0x7f82840fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 -- 172.21.15.160:0/761227767 >> 172.21.15.160:0/761227767 conn(0x7f82840f7850 msgr2=0x7f82840f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:52.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 -- 172.21.15.160:0/761227767 shutdown_connections 2024-05-25T06:26:52.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.885+0000 7f8289128700 1 -- 172.21.15.160:0/761227767 wait complete. 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8289128700 1 Processor -- start 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8289128700 1 -- start start 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8289128700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 0x7f8284113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8289128700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82840fe7b0 0x7f8284114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8289128700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82841146a0 con 0x7f82840fde10 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8289128700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828410da90 con 0x7f82840fe7b0 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8282d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 0x7f8284113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8282d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 0x7f8284113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57174/0 (socket says 172.21.15.160:57174) 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f8282d9d700 1 -- 172.21.15.160:0/1270853908 learned_addr learned my addr 172.21.15.160:0/1270853908 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.886+0000 7f828259c700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82840fe7b0 0x7f8284114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f8282d9d700 1 -- 172.21.15.160:0/1270853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82840fe7b0 msgr2=0x7f8284114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f8282d9d700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82840fe7b0 0x7f8284114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f8282d9d700 1 -- 172.21.15.160:0/1270853908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274005040 con 0x7f82840fde10 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f8282d9d700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 0x7f8284113b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f827800eb40 tx=0x7f827800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8278007cc0 con 0x7f82840fde10 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f827800f040 con 0x7f82840fde10 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828410dd10 con 0x7f82840fde10 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f827800ca10 con 0x7f82840fde10 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.887+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828410e250 con 0x7f82840fde10 2024-05-25T06:26:52.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.888+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8284103cb0 con 0x7f82840fde10 2024-05-25T06:26:52.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.888+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f827800cb70 con 0x7f82840fde10 2024-05-25T06:26:52.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.889+0000 7f826b7fe700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f826c06e140 0x7f826c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:26:52.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.889+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f827808d660 con 0x7f82840fde10 2024-05-25T06:26:52.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.889+0000 7f828259c700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f826c06e140 0x7f826c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:26:52.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.889+0000 7f828259c700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f826c06e140 0x7f826c0705f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f828410ec50 tx=0x7f8274009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:26:52.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:52.893+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f827805a5f0 con 0x7f82840fde10 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.176+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f828410f160 con 0x7f82840fde10 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.177+0000 7f826b7fe700 1 -- 172.21.15.160:0/1270853908 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f827805a180 con 0x7f82840fde10 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:26:53.177 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:26:53.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f826c06e140 msgr2=0x7f826c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:53.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f826c06e140 0x7f826c0705f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f828410ec50 tx=0x7f8274009e30 comp rx=0 tx=0).stop 2024-05-25T06:26:53.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 msgr2=0x7f8284113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:26:53.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 0x7f8284113b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f827800eb40 tx=0x7f827800b040 comp rx=0 tx=0).stop 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 shutdown_connections 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82840fde10 0x7f8284113b10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f826c06e140 0x7f826c0705f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 --2- 172.21.15.160:0/1270853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82840fe7b0 0x7f8284114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 >> 172.21.15.160:0/1270853908 conn(0x7f82840f7850 msgr2=0x7f8284100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 shutdown_connections 2024-05-25T06:26:53.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:26:53.179+0000 7f8289128700 1 -- 172.21.15.160:0/1270853908 wait complete. 2024-05-25T06:26:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:53 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/902802585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:26:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:53 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/534115300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:26:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:53 smithi203 ceph-mon[83507]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:53 smithi203 ceph-mon[83507]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:53 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/902802585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:26:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:53 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/534115300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:26:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:53 smithi160 ceph-mon[112094]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:26:53.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:53 smithi160 ceph-mon[112094]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:54.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:54 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1270853908' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:26:54.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:54 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1270853908' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:26:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:55 smithi203 ceph-mon[83507]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:55 smithi160 ceph-mon[112094]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:57 smithi203 ceph-mon[83507]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:57 smithi160 ceph-mon[112094]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:26:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:26:59 smithi203 ceph-mon[83507]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:26:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:26:59 smithi160 ceph-mon[112094]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:01 smithi203 ceph-mon[83507]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:01 smithi160 ceph-mon[112094]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:03 smithi203 ceph-mon[83507]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:03 smithi160 ceph-mon[112094]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:05.370 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:05 smithi160 ceph-mon[112094]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:05 smithi203 ceph-mon[83507]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:07 smithi203 ceph-mon[83507]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:07 smithi160 ceph-mon[112094]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:09 smithi203 ceph-mon[83507]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:09 smithi160 ceph-mon[112094]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:11 smithi203 ceph-mon[83507]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:11 smithi160 ceph-mon[112094]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:13 smithi203 ceph-mon[83507]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:13 smithi160 ceph-mon[112094]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:15 smithi203 ceph-mon[83507]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:15 smithi160 ceph-mon[112094]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:17 smithi203 ceph-mon[83507]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:17 smithi160 ceph-mon[112094]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:19 smithi203 ceph-mon[83507]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:19 smithi160 ceph-mon[112094]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:21 smithi203 ceph-mon[83507]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:21 smithi160 ceph-mon[112094]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:23.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 -- 172.21.15.160:0/2967056640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc101d90 msgr2=0x7f64fc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 --2- 172.21.15.160:0/2967056640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc101d90 0x7f64fc102160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ce0 tx=0x7f64f4004d10 comp rx=0 tx=0).stop 2024-05-25T06:27:23.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 -- 172.21.15.160:0/2967056640 shutdown_connections 2024-05-25T06:27:23.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 --2- 172.21.15.160:0/2967056640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64fc0fbd40 0x7f64fc0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 --2- 172.21.15.160:0/2967056640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc101d90 0x7f64fc102160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 -- 172.21.15.160:0/2967056640 >> 172.21.15.160:0/2967056640 conn(0x7f64fc071d00 msgr2=0x7f64fc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:23.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 -- 172.21.15.160:0/2967056640 shutdown_connections 2024-05-25T06:27:23.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.296+0000 7f65011d0700 1 -- 172.21.15.160:0/2967056640 wait complete. 2024-05-25T06:27:23.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f65011d0700 1 Processor -- start 2024-05-25T06:27:23.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f65011d0700 1 -- start start 2024-05-25T06:27:23.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f65011d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 0x7f64fc10ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f65011d0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64fc101d90 0x7f64fc10f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f65011d0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc10f850 con 0x7f64fc0fbd40 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f65011d0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc10f9c0 con 0x7f64fc101d90 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f64fad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 0x7f64fc10ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f64fad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 0x7f64fc10ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60394/0 (socket says 172.21.15.160:60394) 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f64fad9d700 1 -- 172.21.15.160:0/2467191261 learned_addr learned my addr 172.21.15.160:0/2467191261 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.297+0000 7f64fa59c700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64fc101d90 0x7f64fc10f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64fad9d700 1 -- 172.21.15.160:0/2467191261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64fc101d90 msgr2=0x7f64fc10f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64fad9d700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64fc101d90 0x7f64fc10f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64fad9d700 1 -- 172.21.15.160:0/2467191261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4009940 con 0x7f64fc0fbd40 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64fad9d700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 0x7f64fc10ecc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f64f4003400 tx=0x7f64f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64f400e040 con 0x7f64fc0fbd40 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64f4008e20 con 0x7f64fc0fbd40 2024-05-25T06:27:23.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64f401c760 con 0x7f64fc0fbd40 2024-05-25T06:27:23.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc117d30 con 0x7f64fc0fbd40 2024-05-25T06:27:23.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.298+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc118210 con 0x7f64fc0fbd40 2024-05-25T06:27:23.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.299+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0fd430 con 0x7f64fc0fbd40 2024-05-25T06:27:23.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.300+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f64f4012b70 con 0x7f64fc0fbd40 2024-05-25T06:27:23.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.300+0000 7f64f37fe700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f64e806e140 0x7f64e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.300+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f64f408f640 con 0x7f64fc0fbd40 2024-05-25T06:27:23.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.300+0000 7f64fa59c700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f64e806e140 0x7f64e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.301+0000 7f64fa59c700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f64e806e140 0x7f64e80705f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64fc065f90 tx=0x7f64e4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:23.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.305+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f64f4058970 con 0x7f64fc0fbd40 2024-05-25T06:27:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:23 smithi203 ceph-mon[83507]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:23 smithi160 ceph-mon[112094]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:23.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.523+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64fc0fac20 con 0x7f64e806e140 2024-05-25T06:27:23.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.525+0000 7f64f37fe700 1 -- 172.21.15.160:0/2467191261 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f64fc0fac20 con 0x7f64e806e140 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f64e806e140 msgr2=0x7f64e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f64e806e140 0x7f64e80705f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64fc065f90 tx=0x7f64e4009ee0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 msgr2=0x7f64fc10ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 0x7f64fc10ecc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f64f4003400 tx=0x7f64f4004d10 comp rx=0 tx=0).stop 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 shutdown_connections 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64fc0fbd40 0x7f64fc10ecc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f64e806e140 0x7f64e80705f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 --2- 172.21.15.160:0/2467191261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64fc101d90 0x7f64fc10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 >> 172.21.15.160:0/2467191261 conn(0x7f64fc071d00 msgr2=0x7f64fc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:23.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 shutdown_connections 2024-05-25T06:27:23.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.527+0000 7f65011d0700 1 -- 172.21.15.160:0/2467191261 wait complete. 2024-05-25T06:27:23.538 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 -- 172.21.15.160:0/3368688406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 msgr2=0x7f7958103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 --2- 172.21.15.160:0/3368688406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958103330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7948004940 tx=0x7f7948009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 -- 172.21.15.160:0/3368688406 shutdown_connections 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 --2- 172.21.15.160:0/3368688406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79580fb520 0x7f79580fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 --2- 172.21.15.160:0/3368688406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958103330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 -- 172.21.15.160:0/3368688406 >> 172.21.15.160:0/3368688406 conn(0x7f79580714d0 msgr2=0x7f79580718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:23.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 -- 172.21.15.160:0/3368688406 shutdown_connections 2024-05-25T06:27:23.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 -- 172.21.15.160:0/3368688406 wait complete. 2024-05-25T06:27:23.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.631+0000 7f795d4ef700 1 Processor -- start 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f795d4ef700 1 -- start start 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f795d4ef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79580fb520 0x7f7958113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f795d4ef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f795d4ef700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7958114510 con 0x7f7958102f60 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f795d4ef700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795810db30 con 0x7f79580fb520 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60416/0 (socket says 172.21.15.160:60416) 2024-05-25T06:27:23.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 -- 172.21.15.160:0/790019765 learned_addr learned my addr 172.21.15.160:0/790019765 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f7956ffd700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79580fb520 0x7f7958113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 -- 172.21.15.160:0/790019765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79580fb520 msgr2=0x7f7958113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79580fb520 0x7f7958113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 -- 172.21.15.160:0/790019765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7948005040 con 0x7f7958102f60 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794e7fc700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958113ec0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7940002a50 tx=0x7f79400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.632+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7940007cc0 con 0x7f7958102f60 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.633+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795810de10 con 0x7f7958102f60 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.633+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f794000f040 con 0x7f7958102f60 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.633+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f794000da30 con 0x7f7958102f60 2024-05-25T06:27:23.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.633+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795810e350 con 0x7f7958102f60 2024-05-25T06:27:23.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.634+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7958100250 con 0x7f7958102f60 2024-05-25T06:27:23.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.634+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f794000db90 con 0x7f7958102f60 2024-05-25T06:27:23.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.635+0000 7f794ffff700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f794406e140 0x7f79440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.635+0000 7f7956ffd700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f794406e140 0x7f79440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.635+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f794008d910 con 0x7f7958102f60 2024-05-25T06:27:23.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.635+0000 7f7956ffd700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f794406e140 0x7f79440705f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7948004f90 tx=0x7f7948009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:23.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.639+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7940056c40 con 0x7f7958102f60 2024-05-25T06:27:23.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.861+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f795810efd0 con 0x7f794406e140 2024-05-25T06:27:23.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.862+0000 7f794ffff700 1 -- 172.21.15.160:0/790019765 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f795810efd0 con 0x7f794406e140 2024-05-25T06:27:23.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f794406e140 msgr2=0x7f79440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f794406e140 0x7f79440705f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7948004f90 tx=0x7f7948009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:23.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 msgr2=0x7f7958113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958113ec0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7940002a50 tx=0x7f79400046b0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 shutdown_connections 2024-05-25T06:27:23.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f794406e140 0x7f79440705f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79580fb520 0x7f7958113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 --2- 172.21.15.160:0/790019765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7958102f60 0x7f7958113ec0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 >> 172.21.15.160:0/790019765 conn(0x7f79580714d0 msgr2=0x7f79580fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:23.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 shutdown_connections 2024-05-25T06:27:23.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.865+0000 7f795d4ef700 1 -- 172.21.15.160:0/790019765 wait complete. 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 -- 172.21.15.160:0/4134259268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf240fbd40 msgr2=0x7faf240fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4134259268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf240fbd40 0x7faf240fc1b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7faf14004940 tx=0x7faf14009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 -- 172.21.15.160:0/4134259268 shutdown_connections 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4134259268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf240fbd40 0x7faf240fc1b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4134259268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf24101d90 0x7faf24102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 -- 172.21.15.160:0/4134259268 >> 172.21.15.160:0/4134259268 conn(0x7faf24071d00 msgr2=0x7faf24072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 -- 172.21.15.160:0/4134259268 shutdown_connections 2024-05-25T06:27:23.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 -- 172.21.15.160:0/4134259268 wait complete. 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 Processor -- start 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.968+0000 7faf2c58f700 1 -- start start 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2c58f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf240fbd40 0x7faf2410eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2c58f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 0x7faf2410f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2c58f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2410f6c0 con 0x7faf24101d90 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2c58f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2410f800 con 0x7faf240fbd40 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 0x7faf2410f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 0x7faf2410f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60434/0 (socket says 172.21.15.160:60434) 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 -- 172.21.15.160:0/4178369615 learned_addr learned my addr 172.21.15.160:0/4178369615 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2a32b700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf240fbd40 0x7faf2410eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 -- 172.21.15.160:0/4178369615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf240fbd40 msgr2=0x7faf2410eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf240fbd40 0x7faf2410eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:23.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 -- 172.21.15.160:0/4178369615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf14005040 con 0x7faf24101d90 2024-05-25T06:27:23.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf29b2a700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 0x7faf2410f070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf14009180 tx=0x7faf14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:23.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf1400e040 con 0x7faf24101d90 2024-05-25T06:27:23.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf1400bbc0 con 0x7faf24101d90 2024-05-25T06:27:23.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf1401c3f0 con 0x7faf24101d90 2024-05-25T06:27:23.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf24117ba0 con 0x7faf24101d90 2024-05-25T06:27:23.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.969+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf241180b0 con 0x7faf24101d90 2024-05-25T06:27:23.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.970+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf240fd430 con 0x7faf24101d90 2024-05-25T06:27:23.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.971+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faf1400bd30 con 0x7faf24101d90 2024-05-25T06:27:23.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.971+0000 7faf1affd700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faf1006e150 0x7faf10070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:23.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.972+0000 7faf2a32b700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faf1006e150 0x7faf10070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:23.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.972+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faf1408ec70 con 0x7faf24101d90 2024-05-25T06:27:23.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.972+0000 7faf2a32b700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faf1006e150 0x7faf10070600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf20004f40 tx=0x7faf2000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:23.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:23.976+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faf140580e0 con 0x7faf24101d90 2024-05-25T06:27:24.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.186+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faf2404bed0 con 0x7faf1006e150 2024-05-25T06:27:24.187 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:24 smithi160 ceph-mon[112094]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:24.187 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:24 smithi160 ceph-mon[112094]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:24.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.199+0000 7faf1affd700 1 -- 172.21.15.160:0/4178369615 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faf2404bed0 con 0x7faf1006e150 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (13m) 6m ago 24m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (10m) 6m ago 24m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (10m) 6m ago 23m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (12m) 6m ago 24m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (7m) 6m ago 18m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (6m) 6m ago 18m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (6m) 6m ago 18m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (6m) 6m ago 18m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (14m) 6m ago 25m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:27:24.199 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (14m) 6m ago 23m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (11m) 6m ago 25m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (11m) 6m ago 23m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (13m) 6m ago 24m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (13m) 6m ago 23m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (10m) 6m ago 22m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (9m) 6m ago 22m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (9m) 6m ago 21m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (8m) 6m ago 21m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (7m) 6m ago 20m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (7m) 6m ago 20m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:27:24.200 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (13m) 6m ago 23m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faf1006e150 msgr2=0x7faf10070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faf1006e150 0x7faf10070600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf20004f40 tx=0x7faf2000b040 comp rx=0 tx=0).stop 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 msgr2=0x7faf2410f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 0x7faf2410f070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf14009180 tx=0x7faf14009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 shutdown_connections 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7faf1006e150 0x7faf10070600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf240fbd40 0x7faf2410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 --2- 172.21.15.160:0/4178369615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf24101d90 0x7faf2410f070 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 >> 172.21.15.160:0/4178369615 conn(0x7faf24071d00 msgr2=0x7faf24104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:24.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 shutdown_connections 2024-05-25T06:27:24.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.201+0000 7faf2c58f700 1 -- 172.21.15.160:0/4178369615 wait complete. 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 -- 172.21.15.160:0/826455418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 msgr2=0x7f30e80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 --2- 172.21.15.160:0/826455418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e80fc1b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f30d8004940 tx=0x7f30d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 -- 172.21.15.160:0/826455418 shutdown_connections 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 --2- 172.21.15.160:0/826455418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e80fc1b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 --2- 172.21.15.160:0/826455418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e8101d90 0x7f30e8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 -- 172.21.15.160:0/826455418 >> 172.21.15.160:0/826455418 conn(0x7f30e8071d00 msgr2=0x7f30e8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 -- 172.21.15.160:0/826455418 shutdown_connections 2024-05-25T06:27:24.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 -- 172.21.15.160:0/826455418 wait complete. 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.303+0000 7f30ee592700 1 Processor -- start 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30ee592700 1 -- start start 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30ee592700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e8113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30ee592700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e8101d90 0x7f30e81140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30ee592700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8114720 con 0x7f30e80fbd40 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30ee592700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e810db30 con 0x7f30e8101d90 2024-05-25T06:27:24.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e8113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e8113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60450/0 (socket says 172.21.15.160:60450) 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e7fff700 1 -- 172.21.15.160:0/1493738364 learned_addr learned my addr 172.21.15.160:0/1493738364 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e77fe700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e8101d90 0x7f30e81140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e7fff700 1 -- 172.21.15.160:0/1493738364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e8101d90 msgr2=0x7f30e81140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e7fff700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e8101d90 0x7f30e81140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.304+0000 7f30e7fff700 1 -- 172.21.15.160:0/1493738364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8005040 con 0x7f30e80fbd40 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.305+0000 7f30e7fff700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e8113b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c810 tx=0x7f30d0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.305+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30d0012810 con 0x7f30e80fbd40 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.305+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e810de10 con 0x7f30e80fbd40 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.305+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30d0012e40 con 0x7f30e80fbd40 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.305+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30d000a370 con 0x7f30e80fbd40 2024-05-25T06:27:24.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.305+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e810e3d0 con 0x7f30e80fbd40 2024-05-25T06:27:24.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.306+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e80fd430 con 0x7f30e80fbd40 2024-05-25T06:27:24.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.306+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f30d0011810 con 0x7f30e80fbd40 2024-05-25T06:27:24.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.307+0000 7f30e4ff9700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30d406e140 0x7f30d40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:24.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.307+0000 7f30e77fe700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30d406e140 0x7f30d40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:24.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.307+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f30d008dff0 con 0x7f30e80fbd40 2024-05-25T06:27:24.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.307+0000 7f30e77fe700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30d406e140 0x7f30d40705f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f30d8000c00 tx=0x7f30d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:24.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.312+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30d0057320 con 0x7f30e80fbd40 2024-05-25T06:27:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:24 smithi203 ceph-mon[83507]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:24 smithi203 ceph-mon[83507]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:24.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.591+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30e804bed0 con 0x7f30e80fbd40 2024-05-25T06:27:24.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.592+0000 7f30e4ff9700 1 -- 172.21.15.160:0/1493738364 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f30d005a9e0 con 0x7f30e80fbd40 2024-05-25T06:27:24.592 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:27:24.592 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:27:24.592 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:27:24.592 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:27:24.593 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:27:24.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30d406e140 msgr2=0x7f30d40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30d406e140 0x7f30d40705f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f30d8000c00 tx=0x7f30d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:24.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 msgr2=0x7f30e8113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e8113b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c810 tx=0x7f30d0004470 comp rx=0 tx=0).stop 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 shutdown_connections 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e80fbd40 0x7f30e8113b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f30d406e140 0x7f30d40705f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 --2- 172.21.15.160:0/1493738364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e8101d90 0x7f30e81140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.594+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 >> 172.21.15.160:0/1493738364 conn(0x7f30e8071d00 msgr2=0x7f30e80655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.595+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 shutdown_connections 2024-05-25T06:27:24.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.595+0000 7f30ee592700 1 -- 172.21.15.160:0/1493738364 wait complete. 2024-05-25T06:27:24.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 -- 172.21.15.160:0/3363173993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 msgr2=0x7ff6840fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 --2- 172.21.15.160:0/3363173993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff6840fcdf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff674004940 tx=0x7ff674009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:24.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 -- 172.21.15.160:0/3363173993 shutdown_connections 2024-05-25T06:27:24.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 --2- 172.21.15.160:0/3363173993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff6840fcdf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 --2- 172.21.15.160:0/3363173993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 0x7ff6841031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 -- 172.21.15.160:0/3363173993 >> 172.21.15.160:0/3363173993 conn(0x7ff6840f8990 msgr2=0x7ff6840fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:24.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 -- 172.21.15.160:0/3363173993 shutdown_connections 2024-05-25T06:27:24.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 -- 172.21.15.160:0/3363173993 wait complete. 2024-05-25T06:27:24.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 Processor -- start 2024-05-25T06:27:24.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.696+0000 7ff688e54700 1 -- start start 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff688e54700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 0x7ff684113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff688e54700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff684113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff688e54700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684114490 con 0x7ff684103780 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff688e54700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410da90 con 0x7ff684102de0 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff684113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff684113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60470/0 (socket says 172.21.15.160:60470) 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 -- 172.21.15.160:0/525363817 learned_addr learned my addr 172.21.15.160:0/525363817 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:24.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff68259c700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 0x7ff684113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 -- 172.21.15.160:0/525363817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 msgr2=0x7ff684113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 0x7ff684113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 -- 172.21.15.160:0/525363817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674005040 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff68259c700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 0x7ff684113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff681d9b700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff684113e40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff674009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff674021070 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.697+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff674013cd0 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.698+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68410dd10 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.698+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff67400e070 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.698+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68410e1f0 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.699+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6840fe530 con 0x7ff684103780 2024-05-25T06:27:24.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.699+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff67401caf0 con 0x7ff684103780 2024-05-25T06:27:24.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.700+0000 7ff672ffd700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff66c06e140 0x7ff66c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:24.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.700+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff67408f020 con 0x7ff684103780 2024-05-25T06:27:24.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.700+0000 7ff68259c700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff66c06e140 0x7ff66c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:24.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.700+0000 7ff68259c700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff66c06e140 0x7ff66c0705f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff678004600 tx=0x7ff67800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:24.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.704+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff67405bf00 con 0x7ff684103780 2024-05-25T06:27:24.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.946+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff684109ec0 con 0x7ff684103780 2024-05-25T06:27:24.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.946+0000 7ff672ffd700 1 -- 172.21.15.160:0/525363817 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ff67405ba90 con 0x7ff684103780 2024-05-25T06:27:24.946 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:27:24.946 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:27:24.946 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:27:24.946 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:27:24.947 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:27:24.948 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:24.949 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff66c06e140 msgr2=0x7ff66c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff66c06e140 0x7ff66c0705f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff678004600 tx=0x7ff67800b040 comp rx=0 tx=0).stop 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 msgr2=0x7ff684113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff684113e40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff674009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 shutdown_connections 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff66c06e140 0x7ff66c0705f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff684102de0 0x7ff684113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 --2- 172.21.15.160:0/525363817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff684103780 0x7ff684113e40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:24.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 >> 172.21.15.160:0/525363817 conn(0x7ff6840f8990 msgr2=0x7ff6840f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:24.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 shutdown_connections 2024-05-25T06:27:24.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:24.948+0000 7ff688e54700 1 -- 172.21.15.160:0/525363817 wait complete. 2024-05-25T06:27:24.951 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:27:25.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 -- 172.21.15.160:0/3026224038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b680fbd40 msgr2=0x7f7b680fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/3026224038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b680fbd40 0x7f7b680fc1b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c004940 tx=0x7f7b5c009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:25.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 -- 172.21.15.160:0/3026224038 shutdown_connections 2024-05-25T06:27:25.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/3026224038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b680fbd40 0x7f7b680fc1b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/3026224038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b68101d90 0x7f7b68102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 -- 172.21.15.160:0/3026224038 >> 172.21.15.160:0/3026224038 conn(0x7f7b68071d00 msgr2=0x7f7b68072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:25.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 -- 172.21.15.160:0/3026224038 shutdown_connections 2024-05-25T06:27:25.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.052+0000 7f7b6d87e700 1 -- 172.21.15.160:0/3026224038 wait complete. 2024-05-25T06:27:25.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b6d87e700 1 Processor -- start 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b6d87e700 1 -- start start 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b6d87e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b680fbd40 0x7f7b68113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b6d87e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 0x7f7b681140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b6d87e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68114740 con 0x7f7b68101d90 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b6d87e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6810db30 con 0x7f7b680fbd40 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b667fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 0x7f7b681140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b667fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 0x7f7b681140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60480/0 (socket says 172.21.15.160:60480) 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b667fc700 1 -- 172.21.15.160:0/1575635694 learned_addr learned my addr 172.21.15.160:0/1575635694 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b66ffd700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b680fbd40 0x7f7b68113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b667fc700 1 -- 172.21.15.160:0/1575635694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b680fbd40 msgr2=0x7f7b68113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b667fc700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b680fbd40 0x7f7b68113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.053+0000 7f7b667fc700 1 -- 172.21.15.160:0/1575635694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c005040 con 0x7f7b68101d90 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.054+0000 7f7b667fc700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 0x7f7b681140f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b3b0 tx=0x7f7b5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:25.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.054+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b5c00e040 con 0x7f7b68101d90 2024-05-25T06:27:25.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.054+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6810ddb0 con 0x7f7b68101d90 2024-05-25T06:27:25.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.054+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b5c00bbc0 con 0x7f7b68101d90 2024-05-25T06:27:25.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.054+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b5c01c3f0 con 0x7f7b68101d90 2024-05-25T06:27:25.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.054+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6810e370 con 0x7f7b68101d90 2024-05-25T06:27:25.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.055+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b680fd430 con 0x7f7b68101d90 2024-05-25T06:27:25.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.055+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7b5c00bd30 con 0x7f7b68101d90 2024-05-25T06:27:25.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.056+0000 7f7b4f7fe700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5006e140 0x7f7b500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:25.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.056+0000 7f7b66ffd700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5006e140 0x7f7b500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:25.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.056+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7b5c08edb0 con 0x7f7b68101d90 2024-05-25T06:27:25.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.056+0000 7f7b66ffd700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5006e140 0x7f7b500705f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b58007fd0 tx=0x7f7b58009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:25.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.060+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7b5c05bc90 con 0x7f7b68101d90 2024-05-25T06:27:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:25 smithi160 ceph-mon[112094]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:25 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1493738364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:27:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:25 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/525363817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:27:25.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:25 smithi160 ceph-mon[112094]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:25.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.279+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b68108df0 con 0x7f7b5006e140 2024-05-25T06:27:25.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.280+0000 7f7b4f7fe700 1 -- 172.21.15.160:0/1575635694 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7b68108df0 con 0x7f7b5006e140 2024-05-25T06:27:25.280 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:27:25.281 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:27:25.282 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5006e140 msgr2=0x7f7b500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5006e140 0x7f7b500705f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b58007fd0 tx=0x7f7b58009040 comp rx=0 tx=0).stop 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 msgr2=0x7f7b681140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 0x7f7b681140f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b3b0 tx=0x7f7b5c009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 shutdown_connections 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7b5006e140 0x7f7b500705f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b680fbd40 0x7f7b68113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 --2- 172.21.15.160:0/1575635694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b68101d90 0x7f7b681140f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.282+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 >> 172.21.15.160:0/1575635694 conn(0x7f7b68071d00 msgr2=0x7f7b680655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:25.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.283+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 shutdown_connections 2024-05-25T06:27:25.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.283+0000 7f7b6d87e700 1 -- 172.21.15.160:0/1575635694 wait complete. 2024-05-25T06:27:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 -- 172.21.15.160:0/2758557534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 msgr2=0x7f91cc0fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 --2- 172.21.15.160:0/2758557534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc0fbcf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f91b8004940 tx=0x7f91b8009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 -- 172.21.15.160:0/2758557534 shutdown_connections 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 --2- 172.21.15.160:0/2758557534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc0fbcf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 --2- 172.21.15.160:0/2758557534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91cc101ce0 0x7f91cc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 -- 172.21.15.160:0/2758557534 >> 172.21.15.160:0/2758557534 conn(0x7f91cc0f7870 msgr2=0x7f91cc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 -- 172.21.15.160:0/2758557534 shutdown_connections 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 -- 172.21.15.160:0/2758557534 wait complete. 2024-05-25T06:27:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 Processor -- start 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.386+0000 7f91d2ede700 1 -- start start 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91d2ede700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91cc101ce0 0x7f91cc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91d2ede700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91d2ede700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc114500 con 0x7f91cc102680 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91d2ede700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc10db20 con 0x7f91cc101ce0 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60494/0 (socket says 172.21.15.160:60494) 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 -- 172.21.15.160:0/3780420950 learned_addr learned my addr 172.21.15.160:0/3780420950 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91d0c7a700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91cc101ce0 0x7f91cc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 -- 172.21.15.160:0/3780420950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91cc101ce0 msgr2=0x7f91cc113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91cc101ce0 0x7f91cc113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 -- 172.21.15.160:0/3780420950 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8005040 con 0x7f91cc102680 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91cbfff700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc113eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009100 tx=0x7f91b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91b800e040 con 0x7f91cc102680 2024-05-25T06:27:25.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91b8009af0 con 0x7f91cc102680 2024-05-25T06:27:25.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.387+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc10dda0 con 0x7f91cc102680 2024-05-25T06:27:25.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.388+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91b801c3f0 con 0x7f91cc102680 2024-05-25T06:27:25.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.388+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc10e360 con 0x7f91cc102680 2024-05-25T06:27:25.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.389+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc0fd430 con 0x7f91cc102680 2024-05-25T06:27:25.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.389+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f91b800baf0 con 0x7f91cc102680 2024-05-25T06:27:25.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.389+0000 7f91c97fa700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f91bc06e140 0x7f91bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:25.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.389+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f91b808ee00 con 0x7f91cc102680 2024-05-25T06:27:25.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.389+0000 7f91d0c7a700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f91bc06e140 0x7f91bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:25.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.390+0000 7f91d0c7a700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f91bc06e140 0x7f91bc0705f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f91c0004eb0 tx=0x7f91c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:25.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.394+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f91b8058170 con 0x7f91cc102680 2024-05-25T06:27:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:25 smithi203 ceph-mon[83507]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:25 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1493738364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:27:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:25 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/525363817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:27:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:25 smithi203 ceph-mon[83507]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:25.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.672+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f91cc10efc0 con 0x7f91cc102680 2024-05-25T06:27:25.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.672+0000 7f91c97fa700 1 -- 172.21.15.160:0/3780420950 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f91b805b830 con 0x7f91cc102680 2024-05-25T06:27:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:27:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:27:25.673 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:27:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:27:25.673 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f91bc06e140 msgr2=0x7f91bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f91bc06e140 0x7f91bc0705f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f91c0004eb0 tx=0x7f91c000b040 comp rx=0 tx=0).stop 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 msgr2=0x7f91cc113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc113eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009100 tx=0x7f91b8009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 shutdown_connections 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f91bc06e140 0x7f91bc0705f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91cc101ce0 0x7f91cc113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 --2- 172.21.15.160:0/3780420950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91cc102680 0x7f91cc113eb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:25.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 >> 172.21.15.160:0/3780420950 conn(0x7f91cc0f7870 msgr2=0x7f91cc0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:25.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 shutdown_connections 2024-05-25T06:27:25.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:25.675+0000 7f91d2ede700 1 -- 172.21.15.160:0/3780420950 wait complete. 2024-05-25T06:27:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:26 smithi203 ceph-mon[83507]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:26 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3780420950' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:27:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:26 smithi160 ceph-mon[112094]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:26 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3780420950' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:27:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:27 smithi203 ceph-mon[83507]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:27 smithi160 ceph-mon[112094]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:29 smithi203 ceph-mon[83507]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:29 smithi160 ceph-mon[112094]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:31 smithi203 ceph-mon[83507]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:31 smithi160 ceph-mon[112094]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:33 smithi203 ceph-mon[83507]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:33 smithi160 ceph-mon[112094]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:35 smithi203 ceph-mon[83507]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:35 smithi160 ceph-mon[112094]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:37 smithi203 ceph-mon[83507]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:37 smithi160 ceph-mon[112094]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:39.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:39 smithi203 ceph-mon[83507]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:39.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:39 smithi160 ceph-mon[112094]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:41 smithi203 ceph-mon[83507]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:41 smithi160 ceph-mon[112094]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:44.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:43 smithi203 ceph-mon[83507]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:43 smithi160 ceph-mon[112094]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:45 smithi203 ceph-mon[83507]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:45 smithi160 ceph-mon[112094]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:47 smithi203 ceph-mon[83507]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:47 smithi160 ceph-mon[112094]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:49 smithi203 ceph-mon[83507]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:49 smithi160 ceph-mon[112094]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:51 smithi203 ceph-mon[83507]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:51 smithi160 ceph-mon[112094]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:53 smithi203 ceph-mon[83507]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:53 smithi160 ceph-mon[112094]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:55 smithi203 ceph-mon[83507]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:55 smithi160 ceph-mon[112094]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:55.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 -- 172.21.15.160:0/2996968221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070c10 msgr2=0x7f8ff01090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:55.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 --2- 172.21.15.160:0/2996968221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070c10 0x7f8ff01090f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0004970 tx=0x7f8fe0009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:55.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 -- 172.21.15.160:0/2996968221 shutdown_connections 2024-05-25T06:27:55.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 --2- 172.21.15.160:0/2996968221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070c10 0x7f8ff01090f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:55.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 --2- 172.21.15.160:0/2996968221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff0070300 0x7f8ff00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 -- 172.21.15.160:0/2996968221 >> 172.21.15.160:0/2996968221 conn(0x7f8ff00f8990 msgr2=0x7f8ff00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 -- 172.21.15.160:0/2996968221 shutdown_connections 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.793+0000 7f8ff8404700 1 -- 172.21.15.160:0/2996968221 wait complete. 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff8404700 1 Processor -- start 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff8404700 1 -- start start 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff8404700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 0x7f8ff0115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff8404700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff0070c10 0x7f8ff0116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff8404700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01166d0 con 0x7f8ff0070300 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff8404700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff010fcb0 con 0x7f8ff0070c10 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff61a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 0x7f8ff0115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff61a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 0x7f8ff0115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60928/0 (socket says 172.21.15.160:60928) 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff61a0700 1 -- 172.21.15.160:0/364463803 learned_addr learned my addr 172.21.15.160:0/364463803 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff599f700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff0070c10 0x7f8ff0116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:55.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff61a0700 1 -- 172.21.15.160:0/364463803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff0070c10 msgr2=0x7f8ff0116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:55.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff61a0700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff0070c10 0x7f8ff0116080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:55.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.794+0000 7f8ff61a0700 1 -- 172.21.15.160:0/364463803 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0005040 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.795+0000 7f8ff61a0700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 0x7f8ff0115b40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8fec001d80 tx=0x7f8fec009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.795+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fec007d10 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.795+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fec007e70 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.795+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff010ff90 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.795+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fec014640 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.795+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01104a0 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.796+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff0071a40 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.796+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8fec00e960 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.797+0000 7f8fe6ffd700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8fdc06e140 0x7f8fdc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.797+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8fec08d8a0 con 0x7f8ff0070300 2024-05-25T06:27:55.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.797+0000 7f8ff599f700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8fdc06e140 0x7f8fdc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:55.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.797+0000 7f8ff599f700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8fdc06e140 0x7f8fdc0705f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0004f50 tx=0x7f8fe0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:55.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:55.802+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8fec05a830 con 0x7f8ff0070300 2024-05-25T06:27:56.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.015+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ff0111190 con 0x7f8fdc06e140 2024-05-25T06:27:56.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.016+0000 7f8fe6ffd700 1 -- 172.21.15.160:0/364463803 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ff0111190 con 0x7f8fdc06e140 2024-05-25T06:27:56.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8fdc06e140 msgr2=0x7f8fdc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8fdc06e140 0x7f8fdc0705f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0004f50 tx=0x7f8fe0009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:56.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 msgr2=0x7f8ff0115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 0x7f8ff0115b40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8fec001d80 tx=0x7f8fec009ec0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 shutdown_connections 2024-05-25T06:27:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ff0070300 0x7f8ff0115b40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8fdc06e140 0x7f8fdc0705f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 --2- 172.21.15.160:0/364463803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff0070c10 0x7f8ff0116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 >> 172.21.15.160:0/364463803 conn(0x7f8ff00f8990 msgr2=0x7f8ff00f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 shutdown_connections 2024-05-25T06:27:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.018+0000 7f8ff8404700 1 -- 172.21.15.160:0/364463803 wait complete. 2024-05-25T06:27:56.029 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:27:56.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 -- 172.21.15.160:0/2025253490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9081026b0 msgr2=0x7fe9080fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 --2- 172.21.15.160:0/2025253490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9081026b0 0x7fe9080fbd20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8004940 tx=0x7fe8f8009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 -- 172.21.15.160:0/2025253490 shutdown_connections 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 --2- 172.21.15.160:0/2025253490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9081026b0 0x7fe9080fbd20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 --2- 172.21.15.160:0/2025253490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe908101d10 0x7fe9081020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 -- 172.21.15.160:0/2025253490 >> 172.21.15.160:0/2025253490 conn(0x7fe9080f7860 msgr2=0x7fe9080f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 -- 172.21.15.160:0/2025253490 shutdown_connections 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 -- 172.21.15.160:0/2025253490 wait complete. 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.123+0000 7fe90f506700 1 Processor -- start 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90f506700 1 -- start start 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90f506700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 0x7fe9081128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90f506700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9081026b0 0x7fe908112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90f506700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe908113430 con 0x7fe908101d10 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90f506700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90810ca30 con 0x7fe9081026b0 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 0x7fe9081128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 0x7fe9081128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60940/0 (socket says 172.21.15.160:60940) 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 -- 172.21.15.160:0/557467418 learned_addr learned my addr 172.21.15.160:0/557467418 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90caa1700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9081026b0 0x7fe908112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 -- 172.21.15.160:0/557467418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9081026b0 msgr2=0x7fe908112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9081026b0 0x7fe908112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 -- 172.21.15.160:0/557467418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8005040 con 0x7fe908101d10 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe90d2a2700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 0x7fe9081128a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe904001d80 tx=0x7fe904009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.124+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe904007d10 con 0x7fe908101d10 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.125+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90810cd10 con 0x7fe908101d10 2024-05-25T06:27:56.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.125+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe904007e70 con 0x7fe908101d10 2024-05-25T06:27:56.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.125+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe904014640 con 0x7fe908101d10 2024-05-25T06:27:56.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.125+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90810d2d0 con 0x7fe908101d10 2024-05-25T06:27:56.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.126+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9080fd460 con 0x7fe908101d10 2024-05-25T06:27:56.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.126+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe90400e960 con 0x7fe908101d10 2024-05-25T06:27:56.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.126+0000 7fe8fdffb700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe8f406e140 0x7fe8f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.126+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe90408d960 con 0x7fe908101d10 2024-05-25T06:27:56.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.127+0000 7fe90caa1700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe8f406e140 0x7fe8f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.127+0000 7fe90caa1700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe8f406e140 0x7fe8f40705f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009fd0 tx=0x7fe8f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:56.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.131+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe90405a870 con 0x7fe908101d10 2024-05-25T06:27:56.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.347+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe908108df0 con 0x7fe8f406e140 2024-05-25T06:27:56.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.348+0000 7fe8fdffb700 1 -- 172.21.15.160:0/557467418 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe908108df0 con 0x7fe8f406e140 2024-05-25T06:27:56.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe8f406e140 msgr2=0x7fe8f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe8f406e140 0x7fe8f40705f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009fd0 tx=0x7fe8f8009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:56.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 msgr2=0x7fe9081128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 0x7fe9081128a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe904001d80 tx=0x7fe904009ec0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 shutdown_connections 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe908101d10 0x7fe9081128a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe8f406e140 0x7fe8f40705f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 --2- 172.21.15.160:0/557467418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9081026b0 0x7fe908112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 >> 172.21.15.160:0/557467418 conn(0x7fe9080f7860 msgr2=0x7fe9080f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 shutdown_connections 2024-05-25T06:27:56.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.350+0000 7fe90f506700 1 -- 172.21.15.160:0/557467418 wait complete. 2024-05-25T06:27:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:56 smithi203 ceph-mon[83507]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:56.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.452+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/847338488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 msgr2=0x7f1a340fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.452+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/847338488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fe1c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a24004940 tx=0x7f1a24009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:56.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.452+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/847338488 shutdown_connections 2024-05-25T06:27:56.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.452+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/847338488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a340fe790 0x7f1a34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.452+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/847338488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fe1c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.452+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/847338488 >> 172.21.15.160:0/847338488 conn(0x7f1a340f7850 msgr2=0x7f1a340f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:56.453 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:56 smithi160 ceph-mon[112094]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/847338488 shutdown_connections 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/847338488 wait complete. 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 Processor -- start 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 -- start start 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a340fe790 0x7f1a340fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340fdc00 con 0x7f1a340fddf0 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.453+0000 7f1a3bfc7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340fc730 con 0x7f1a340fe790 2024-05-25T06:27:56.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60954/0 (socket says 172.21.15.160:60954) 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 -- 172.21.15.160:0/3554123636 learned_addr learned my addr 172.21.15.160:0/3554123636 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39562700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a340fe790 0x7f1a340fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 -- 172.21.15.160:0/3554123636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a340fe790 msgr2=0x7f1a340fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a340fe790 0x7f1a340fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 -- 172.21.15.160:0/3554123636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24005040 con 0x7f1a340fddf0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a39d63700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fbcb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a24004890 tx=0x7f1a24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2400e040 con 0x7f1a340fddf0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a340fc9b0 con 0x7f1a340fddf0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a24009af0 con 0x7f1a340fddf0 2024-05-25T06:27:56.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2401c630 con 0x7f1a340fddf0 2024-05-25T06:27:56.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.454+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a34119fe0 con 0x7f1a340fddf0 2024-05-25T06:27:56.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.456+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a34066a90 con 0x7f1a340fddf0 2024-05-25T06:27:56.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.456+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1a2400bbc0 con 0x7f1a340fddf0 2024-05-25T06:27:56.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.456+0000 7f1a2a7fc700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a2006e140 0x7f1a200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.456+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1a2408ee60 con 0x7f1a340fddf0 2024-05-25T06:27:56.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.456+0000 7f1a39562700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a2006e140 0x7f1a200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.457+0000 7f1a39562700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a2006e140 0x7f1a200705f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009fd0 tx=0x7f1a30009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:56.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.461+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1a24058210 con 0x7f1a340fddf0 2024-05-25T06:27:56.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.664+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a340fd230 con 0x7f1a2006e140 2024-05-25T06:27:56.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.675+0000 7f1a2a7fc700 1 -- 172.21.15.160:0/3554123636 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1a340fd230 con 0x7f1a2006e140 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (13m) 7m ago 24m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (11m) 7m ago 25m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (11m) 6m ago 24m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (13m) 7m ago 24m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (7m) 7m ago 19m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (7m) 7m ago 19m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (6m) 6m ago 19m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (7m) 6m ago 19m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (15m) 7m ago 26m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (14m) 6m ago 24m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:27:56.675 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (12m) 7m ago 26m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (11m) 6m ago 23m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (14m) 7m ago 25m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (14m) 6m ago 24m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (10m) 7m ago 23m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (10m) 7m ago 22m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (9m) 7m ago 22m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (9m) 6m ago 21m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (8m) 6m ago 21m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (7m) 6m ago 21m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:27:56.676 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (14m) 7m ago 24m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:27:56.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a2006e140 msgr2=0x7f1a200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a2006e140 0x7f1a200705f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009fd0 tx=0x7f1a30009f60 comp rx=0 tx=0).stop 2024-05-25T06:27:56.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 msgr2=0x7f1a340fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fbcb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a24004890 tx=0x7f1a24009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:56.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 shutdown_connections 2024-05-25T06:27:56.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340fddf0 0x7f1a340fbcb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1a2006e140 0x7f1a200705f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 --2- 172.21.15.160:0/3554123636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a340fe790 0x7f1a340fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 >> 172.21.15.160:0/3554123636 conn(0x7f1a340f7850 msgr2=0x7f1a340f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:56.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 shutdown_connections 2024-05-25T06:27:56.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.677+0000 7f1a3bfc7700 1 -- 172.21.15.160:0/3554123636 wait complete. 2024-05-25T06:27:56.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 -- 172.21.15.160:0/3303012313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fb610 msgr2=0x7f8d400fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 --2- 172.21.15.160:0/3303012313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fb610 0x7f8d400fb9e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d34004970 tx=0x7f8d34009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 -- 172.21.15.160:0/3303012313 shutdown_connections 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 --2- 172.21.15.160:0/3303012313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400fbf20 0x7f8d4010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 --2- 172.21.15.160:0/3303012313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fb610 0x7f8d400fb9e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 -- 172.21.15.160:0/3303012313 >> 172.21.15.160:0/3303012313 conn(0x7f8d40072380 msgr2=0x7f8d40072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 -- 172.21.15.160:0/3303012313 shutdown_connections 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 -- 172.21.15.160:0/3303012313 wait complete. 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 Processor -- start 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.780+0000 7f8d46aea700 1 -- start start 2024-05-25T06:27:56.782 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d46aea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400fb610 0x7f8d40119e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d46aea700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 0x7f8d4011a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d46aea700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4011aa20 con 0x7f8d400fbf20 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d46aea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40114000 con 0x7f8d400fb610 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 0x7f8d4011a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 0x7f8d4011a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60966/0 (socket says 172.21.15.160:60966) 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 -- 172.21.15.160:0/1021500587 learned_addr learned my addr 172.21.15.160:0/1021500587 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d44886700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400fb610 0x7f8d40119e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 -- 172.21.15.160:0/1021500587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400fb610 msgr2=0x7f8d40119e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400fb610 0x7f8d40119e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 -- 172.21.15.160:0/1021500587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34005040 con 0x7f8d400fbf20 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.781+0000 7f8d3ffff700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 0x7f8d4011a3d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c002a50 tx=0x7f8d2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:56.783 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.782+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d2c007cc0 con 0x7f8d400fbf20 2024-05-25T06:27:56.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.782+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d2c00f040 con 0x7f8d400fbf20 2024-05-25T06:27:56.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.782+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d401142e0 con 0x7f8d400fbf20 2024-05-25T06:27:56.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.782+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d2c00db70 con 0x7f8d400fbf20 2024-05-25T06:27:56.784 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.782+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d401148a0 con 0x7f8d400fbf20 2024-05-25T06:27:56.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.783+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d400fc3d0 con 0x7f8d400fbf20 2024-05-25T06:27:56.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.783+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8d2c00dd10 con 0x7f8d400fbf20 2024-05-25T06:27:56.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.784+0000 7f8d3d7fa700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8d3006e140 0x7f8d300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:56.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.784+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8d2c08d290 con 0x7f8d400fbf20 2024-05-25T06:27:56.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.784+0000 7f8d44886700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8d3006e140 0x7f8d300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:56.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.784+0000 7f8d44886700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8d3006e140 0x7f8d300705f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400b370 tx=0x7f8d34009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:56.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:56.788+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8d2c05a0f0 con 0x7f8d400fbf20 2024-05-25T06:27:57.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.081+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d40110510 con 0x7f8d400fbf20 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.082+0000 7f8d3d7fa700 1 -- 172.21.15.160:0/1021500587 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f8d2c059c80 con 0x7f8d400fbf20 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:27:57.082 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:27:57.083 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:27:57.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8d3006e140 msgr2=0x7f8d300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8d3006e140 0x7f8d300705f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400b370 tx=0x7f8d34009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:57.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 msgr2=0x7f8d4011a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 0x7f8d4011a3d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c002a50 tx=0x7f8d2c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 shutdown_connections 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8d3006e140 0x7f8d300705f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400fb610 0x7f8d40119e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 --2- 172.21.15.160:0/1021500587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d400fbf20 0x7f8d4011a3d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 >> 172.21.15.160:0/1021500587 conn(0x7f8d40072380 msgr2=0x7f8d400fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 shutdown_connections 2024-05-25T06:27:57.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.084+0000 7f8d46aea700 1 -- 172.21.15.160:0/1021500587 wait complete. 2024-05-25T06:27:57.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 -- 172.21.15.160:0/447654703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 msgr2=0x7f0e880fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 --2- 172.21.15.160:0/447654703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e880fb3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0e78004940 tx=0x7f0e78009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:57.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 -- 172.21.15.160:0/447654703 shutdown_connections 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 --2- 172.21.15.160:0/447654703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e880fb3c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 --2- 172.21.15.160:0/447654703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e88102e20 0x7f0e881031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 -- 172.21.15.160:0/447654703 >> 172.21.15.160:0/447654703 conn(0x7f0e88072380 msgr2=0x7f0e88072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 -- 172.21.15.160:0/447654703 shutdown_connections 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.187+0000 7f0e8d134700 1 -- 172.21.15.160:0/447654703 wait complete. 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8d134700 1 Processor -- start 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8d134700 1 -- start start 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8d134700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e88102e20 0x7f0e88113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8d134700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e88113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8d134700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88114660 con 0x7f0e881037c0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8d134700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8810dbb0 con 0x7f0e88102e20 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e88113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e88113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60992/0 (socket says 172.21.15.160:60992) 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8659c700 1 -- 172.21.15.160:0/1151875207 learned_addr learned my addr 172.21.15.160:0/1151875207 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e86d9d700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e88102e20 0x7f0e88113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8659c700 1 -- 172.21.15.160:0/1151875207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e88102e20 msgr2=0x7f0e88113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8659c700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e88102e20 0x7f0e88113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.188+0000 7f0e8659c700 1 -- 172.21.15.160:0/1151875207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e78005040 con 0x7f0e881037c0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.189+0000 7f0e8659c700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e88113f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009100 tx=0x7f0e78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.189+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7800e040 con 0x7f0e881037c0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.189+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e78013420 con 0x7f0e881037c0 2024-05-25T06:27:57.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.189+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7801c630 con 0x7f0e881037c0 2024-05-25T06:27:57.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.189+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8810ddd0 con 0x7f0e881037c0 2024-05-25T06:27:57.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.189+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8810e310 con 0x7f0e881037c0 2024-05-25T06:27:57.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.190+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e880fe6e0 con 0x7f0e881037c0 2024-05-25T06:27:57.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.190+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0e78013590 con 0x7f0e881037c0 2024-05-25T06:27:57.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.191+0000 7f0e7f7fe700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e7406e140 0x7f0e740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.191+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0e7808ed10 con 0x7f0e881037c0 2024-05-25T06:27:57.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.191+0000 7f0e86d9d700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e7406e140 0x7f0e740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.191+0000 7f0e86d9d700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e7406e140 0x7f0e740705f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e70006fd0 tx=0x7f0e70008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:57.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.195+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0e7805bc30 con 0x7f0e881037c0 2024-05-25T06:27:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:57 smithi203 ceph-mon[83507]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:57 smithi203 ceph-mon[83507]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:57 smithi203 ceph-mon[83507]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:57 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1021500587' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:27:57.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.434+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e880fe6e0 con 0x7f0e881037c0 2024-05-25T06:27:57.435 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:57 smithi160 ceph-mon[112094]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:57.435 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:57 smithi160 ceph-mon[112094]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:57.435 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:57 smithi160 ceph-mon[112094]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:57.435 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:57 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1021500587' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:27:57.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.434+0000 7f0e7f7fe700 1 -- 172.21.15.160:0/1151875207 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f0e7805b7c0 con 0x7f0e881037c0 2024-05-25T06:27:57.435 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:27:57.435 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:27:57.435 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:27:57.436 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:27:57.437 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:57.438 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:57.439 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:27:57.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e7406e140 msgr2=0x7f0e740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e7406e140 0x7f0e740705f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e70006fd0 tx=0x7f0e70008040 comp rx=0 tx=0).stop 2024-05-25T06:27:57.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 msgr2=0x7f0e88113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e88113f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009100 tx=0x7f0e78009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 shutdown_connections 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0e7406e140 0x7f0e740705f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e88102e20 0x7f0e88113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 --2- 172.21.15.160:0/1151875207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e881037c0 0x7f0e88113f80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 >> 172.21.15.160:0/1151875207 conn(0x7f0e88072380 msgr2=0x7f0e880f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 shutdown_connections 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.437+0000 7f0e8d134700 1 -- 172.21.15.160:0/1151875207 wait complete. 2024-05-25T06:27:57.440 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 -- 172.21.15.160:0/4252821837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0103780 msgr2=0x7f68c00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/4252821837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0103780 0x7f68c00fcdf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004940 tx=0x7f68b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 -- 172.21.15.160:0/4252821837 shutdown_connections 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/4252821837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0103780 0x7f68c00fcdf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/4252821837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c0102de0 0x7f68c01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 -- 172.21.15.160:0/4252821837 >> 172.21.15.160:0/4252821837 conn(0x7f68c00f8990 msgr2=0x7f68c00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 -- 172.21.15.160:0/4252821837 shutdown_connections 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.540+0000 7f68c7ddd700 1 -- 172.21.15.160:0/4252821837 wait complete. 2024-05-25T06:27:57.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c7ddd700 1 Processor -- start 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c7ddd700 1 -- start start 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c7ddd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 0x7f68c0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c7ddd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c0103780 0x7f68c00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c7ddd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00721b0 con 0x7f68c0102de0 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c7ddd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0070ce0 con 0x7f68c0103780 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 0x7f68c0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 0x7f68c0070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:32786/0 (socket says 172.21.15.160:32786) 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 -- 172.21.15.160:0/3823295776 learned_addr learned my addr 172.21.15.160:0/3823295776 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5378700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c0103780 0x7f68c00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 -- 172.21.15.160:0/3823295776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c0103780 msgr2=0x7f68c00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c0103780 0x7f68c00707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 -- 172.21.15.160:0/3823295776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0005040 con 0x7f68c0102de0 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.541+0000 7f68c5b79700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 0x7f68c0070260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f68bc002a50 tx=0x7f68bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:57.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.542+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68bc007cc0 con 0x7f68c0102de0 2024-05-25T06:27:57.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.542+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68bc00f040 con 0x7f68c0102de0 2024-05-25T06:27:57.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.542+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c0070fc0 con 0x7f68c0102de0 2024-05-25T06:27:57.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.542+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68bc00da30 con 0x7f68c0102de0 2024-05-25T06:27:57.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.542+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c011a100 con 0x7f68c0102de0 2024-05-25T06:27:57.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.543+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c00fe530 con 0x7f68c0102de0 2024-05-25T06:27:57.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.543+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f68bc00db90 con 0x7f68c0102de0 2024-05-25T06:27:57.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.544+0000 7f68b67fc700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68ac06e140 0x7f68ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.544+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f68bc08d920 con 0x7f68c0102de0 2024-05-25T06:27:57.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.544+0000 7f68c5378700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68ac06e140 0x7f68ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.544+0000 7f68c5378700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68ac06e140 0x7f68ac0705f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68c0071990 tx=0x7f68b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:57.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.548+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f68bc056c50 con 0x7f68c0102de0 2024-05-25T06:27:57.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.763+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68c00fa0a0 con 0x7f68ac06e140 2024-05-25T06:27:57.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.764+0000 7f68b67fc700 1 -- 172.21.15.160:0/3823295776 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f68c00fa0a0 con 0x7f68ac06e140 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:27:57.765 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:27:57.766 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:27:57.766 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:27:57.766 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68ac06e140 msgr2=0x7f68ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68ac06e140 0x7f68ac0705f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68c0071990 tx=0x7f68b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 msgr2=0x7f68c0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 0x7f68c0070260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f68bc002a50 tx=0x7f68bc0046b0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 shutdown_connections 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68c0102de0 0x7f68c0070260 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f68ac06e140 0x7f68ac0705f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 --2- 172.21.15.160:0/3823295776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c0103780 0x7f68c00707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 >> 172.21.15.160:0/3823295776 conn(0x7f68c00f8990 msgr2=0x7f68c00f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:57.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 shutdown_connections 2024-05-25T06:27:57.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.767+0000 7f68c7ddd700 1 -- 172.21.15.160:0/3823295776 wait complete. 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 -- 172.21.15.160:0/579105620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc0fb520 msgr2=0x7f28bc0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 --2- 172.21.15.160:0/579105620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc0fb520 0x7f28bc0fb990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f28ac004940 tx=0x7f28ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 -- 172.21.15.160:0/579105620 shutdown_connections 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 --2- 172.21.15.160:0/579105620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc0fb520 0x7f28bc0fb990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 --2- 172.21.15.160:0/579105620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28bc102f60 0x7f28bc103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 -- 172.21.15.160:0/579105620 >> 172.21.15.160:0/579105620 conn(0x7f28bc0714d0 msgr2=0x7f28bc0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 -- 172.21.15.160:0/579105620 shutdown_connections 2024-05-25T06:27:57.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 -- 172.21.15.160:0/579105620 wait complete. 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 Processor -- start 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.868+0000 7f28c40eb700 1 -- start start 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c40eb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28bc0fb520 0x7f28bc113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c40eb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 0x7f28bc113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c40eb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc114510 con 0x7f28bc102f60 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c40eb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc10db10 con 0x7f28bc0fb520 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 0x7f28bc113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 0x7f28bc113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:32810/0 (socket says 172.21.15.160:32810) 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 -- 172.21.15.160:0/2613810787 learned_addr learned my addr 172.21.15.160:0/2613810787 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1e87700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28bc0fb520 0x7f28bc113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 -- 172.21.15.160:0/2613810787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28bc0fb520 msgr2=0x7f28bc113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28bc0fb520 0x7f28bc113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 -- 172.21.15.160:0/2613810787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac005040 con 0x7f28bc102f60 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c1686700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 0x7f28bc113ec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b3b0 tx=0x7f28ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ac00e040 con 0x7f28bc102f60 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ac00bbc0 con 0x7f28bc102f60 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc10dd90 con 0x7f28bc102f60 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ac01c3f0 con 0x7f28bc102f60 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.869+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc10e2d0 con 0x7f28bc102f60 2024-05-25T06:27:57.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.870+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc100250 con 0x7f28bc102f60 2024-05-25T06:27:57.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.871+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f28ac00bd30 con 0x7f28bc102f60 2024-05-25T06:27:57.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.871+0000 7f28b27fc700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f28a806e140 0x7f28a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:27:57.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.871+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f28ac08ee00 con 0x7f28bc102f60 2024-05-25T06:27:57.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.871+0000 7f28c1e87700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f28a806e140 0x7f28a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:27:57.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.872+0000 7f28c1e87700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f28a806e140 0x7f28a80705f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28b8004fa0 tx=0x7f28b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:27:57.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:57.876+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f28ac05bc60 con 0x7f28bc102f60 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.155+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f28bc10efd0 con 0x7f28bc102f60 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.156+0000 7f28b27fc700 1 -- 172.21.15.160:0/2613810787 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f28ac05b7f0 con 0x7f28bc102f60 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:27:58.156 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:27:58.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f28a806e140 msgr2=0x7f28a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:58.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f28a806e140 0x7f28a80705f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28b8004fa0 tx=0x7f28b800b040 comp rx=0 tx=0).stop 2024-05-25T06:27:58.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 msgr2=0x7f28bc113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:27:58.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 0x7f28bc113ec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b3b0 tx=0x7f28ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 shutdown_connections 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f28a806e140 0x7f28a80705f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28bc0fb520 0x7f28bc113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 --2- 172.21.15.160:0/2613810787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28bc102f60 0x7f28bc113ec0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 >> 172.21.15.160:0/2613810787 conn(0x7f28bc0714d0 msgr2=0x7f28bc0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 shutdown_connections 2024-05-25T06:27:58.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:27:58.158+0000 7f28c40eb700 1 -- 172.21.15.160:0/2613810787 wait complete. 2024-05-25T06:27:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:58 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1151875207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:27:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:58 smithi203 ceph-mon[83507]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:58 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1151875207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:27:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:58 smithi160 ceph-mon[112094]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:27:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:27:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:59 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2613810787' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:27:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:27:59 smithi203 ceph-mon[83507]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:27:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:59 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2613810787' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:27:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:27:59 smithi160 ceph-mon[112094]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:01 smithi203 ceph-mon[83507]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:01 smithi160 ceph-mon[112094]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:03 smithi203 ceph-mon[83507]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:03 smithi160 ceph-mon[112094]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:05 smithi203 ceph-mon[83507]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:05 smithi160 ceph-mon[112094]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:07 smithi203 ceph-mon[83507]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:07 smithi160 ceph-mon[112094]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:09 smithi203 ceph-mon[83507]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:09 smithi160 ceph-mon[112094]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:11 smithi203 ceph-mon[83507]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:11 smithi160 ceph-mon[112094]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:13 smithi203 ceph-mon[83507]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:13 smithi160 ceph-mon[112094]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:15 smithi203 ceph-mon[83507]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:15 smithi160 ceph-mon[112094]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:17 smithi203 ceph-mon[83507]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:17 smithi160 ceph-mon[112094]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:19 smithi203 ceph-mon[83507]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:19 smithi160 ceph-mon[112094]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:21 smithi203 ceph-mon[83507]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:21 smithi160 ceph-mon[112094]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:23 smithi203 ceph-mon[83507]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:23 smithi160 ceph-mon[112094]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:25 smithi203 ceph-mon[83507]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:25 smithi160 ceph-mon[112094]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:27 smithi203 ceph-mon[83507]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:27.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:27 smithi160 ceph-mon[112094]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 -- 172.21.15.160:0/3384044601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 msgr2=0x7f81cc101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 --2- 172.21.15.160:0/3384044601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc101a30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f81bc004940 tx=0x7f81bc009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 -- 172.21.15.160:0/3384044601 shutdown_connections 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 --2- 172.21.15.160:0/3384044601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc101a30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 --2- 172.21.15.160:0/3384044601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 0x7f81cc0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 -- 172.21.15.160:0/3384044601 >> 172.21.15.160:0/3384044601 conn(0x7f81cc073350 msgr2=0x7f81cc073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 -- 172.21.15.160:0/3384044601 shutdown_connections 2024-05-25T06:28:28.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 -- 172.21.15.160:0/3384044601 wait complete. 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 Processor -- start 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.275+0000 7f81d3be8700 1 -- start start 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d3be8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 0x7f81cc0fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d3be8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc0fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d3be8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0fe480 con 0x7f81cc0fbf40 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d3be8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0fcfb0 con 0x7f81cc0fb5a0 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc0fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc0fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47892/0 (socket says 172.21.15.160:47892) 2024-05-25T06:28:28.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 -- 172.21.15.160:0/1609129081 learned_addr learned my addr 172.21.15.160:0/1609129081 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1984700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 0x7f81cc0fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 -- 172.21.15.160:0/1609129081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 msgr2=0x7f81cc0fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 0x7f81cc0fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 -- 172.21.15.160:0/1609129081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc005040 con 0x7f81cc0fbf40 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1984700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 0x7f81cc0fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d1183700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc0fca70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f81bc000c00 tx=0x7f81bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81bc00e040 con 0x7f81cc0fbf40 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81bc013420 con 0x7f81cc0fbf40 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc0fd230 con 0x7f81cc0fbf40 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.276+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81bc01c630 con 0x7f81cc0fbf40 2024-05-25T06:28:28.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.277+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc11a110 con 0x7f81cc0fbf40 2024-05-25T06:28:28.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.278+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc0fe670 con 0x7f81cc0fbf40 2024-05-25T06:28:28.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.278+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f81bc013590 con 0x7f81cc0fbf40 2024-05-25T06:28:28.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.279+0000 7f81c27fc700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f81b806e140 0x7f81b80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.279+0000 7f81d1984700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f81b806e140 0x7f81b80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.279+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f81bc08edc0 con 0x7f81cc0fbf40 2024-05-25T06:28:28.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.279+0000 7f81d1984700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f81b806e140 0x7f81b80705f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f81c8004c60 tx=0x7f81c8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:28.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.283+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f81bc058170 con 0x7f81cc0fbf40 2024-05-25T06:28:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:28.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.498+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81cc101120 con 0x7f81b806e140 2024-05-25T06:28:28.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.499+0000 7f81c27fc700 1 -- 172.21.15.160:0/1609129081 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f81cc101120 con 0x7f81b806e140 2024-05-25T06:28:28.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f81b806e140 msgr2=0x7f81b80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f81b806e140 0x7f81b80705f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f81c8004c60 tx=0x7f81c8009040 comp rx=0 tx=0).stop 2024-05-25T06:28:28.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 msgr2=0x7f81cc0fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc0fca70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f81bc000c00 tx=0x7f81bc009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 shutdown_connections 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f81b806e140 0x7f81b80705f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81cc0fb5a0 0x7f81cc0fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 --2- 172.21.15.160:0/1609129081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81cc0fbf40 0x7f81cc0fca70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 >> 172.21.15.160:0/1609129081 conn(0x7f81cc073350 msgr2=0x7f81cc100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 shutdown_connections 2024-05-25T06:28:28.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.501+0000 7f81d3be8700 1 -- 172.21.15.160:0/1609129081 wait complete. 2024-05-25T06:28:28.512 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:28:28.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 -- 172.21.15.160:0/1832839996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 msgr2=0x7fabf41018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/1832839996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf41018c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fabf0004940 tx=0x7fabf0009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 -- 172.21.15.160:0/1832839996 shutdown_connections 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/1832839996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 0x7fabf40fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/1832839996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf41018c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 -- 172.21.15.160:0/1832839996 >> 172.21.15.160:0/1832839996 conn(0x7fabf4073350 msgr2=0x7fabf4073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 -- 172.21.15.160:0/1832839996 shutdown_connections 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.605+0000 7fabfb7fa700 1 -- 172.21.15.160:0/1832839996 wait complete. 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabfb7fa700 1 Processor -- start 2024-05-25T06:28:28.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabfb7fa700 1 -- start start 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabfb7fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 0x7fabf410eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabfb7fa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf410f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabfb7fa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf410f5d0 con 0x7fabf41014f0 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabfb7fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf410f710 con 0x7fabf40fb4e0 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabf8d95700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf410f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabf8d95700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf410f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47906/0 (socket says 172.21.15.160:47906) 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabf8d95700 1 -- 172.21.15.160:0/2573357075 learned_addr learned my addr 172.21.15.160:0/2573357075 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:28.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.606+0000 7fabf9596700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 0x7fabf410eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabf8d95700 1 -- 172.21.15.160:0/2573357075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 msgr2=0x7fabf410eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabf8d95700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 0x7fabf410eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabf8d95700 1 -- 172.21.15.160:0/2573357075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0005040 con 0x7fabf41014f0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabf9596700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 0x7fabf410eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabf8d95700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf410f090 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c830 tx=0x7fabe4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe400a760 con 0x7fabf41014f0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf4117bc0 con 0x7fabf41014f0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabe400ad90 con 0x7fabf41014f0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe40113a0 con 0x7fabf41014f0 2024-05-25T06:28:28.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.607+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4117f10 con 0x7fabf41014f0 2024-05-25T06:28:28.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.608+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf41038d0 con 0x7fabf41014f0 2024-05-25T06:28:28.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.609+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fabe4012400 con 0x7fabf41014f0 2024-05-25T06:28:28.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.609+0000 7fabe9ffb700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fabe006e140 0x7fabe00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.609+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fabe408dc20 con 0x7fabf41014f0 2024-05-25T06:28:28.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.609+0000 7fabf9596700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fabe006e140 0x7fabe00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.610+0000 7fabf9596700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fabe006e140 0x7fabe00705f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fabf0004560 tx=0x7fabf0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:28.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.614+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fabe4056f90 con 0x7fabf41014f0 2024-05-25T06:28:28.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.829+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf40fda70 con 0x7fabe006e140 2024-05-25T06:28:28.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.830+0000 7fabe9ffb700 1 -- 172.21.15.160:0/2573357075 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fabf40fda70 con 0x7fabe006e140 2024-05-25T06:28:28.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fabe006e140 msgr2=0x7fabe00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fabe006e140 0x7fabe00705f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fabf0004560 tx=0x7fabf0009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:28.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 msgr2=0x7fabf410f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf410f090 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c830 tx=0x7fabe4004470 comp rx=0 tx=0).stop 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 shutdown_connections 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fabe006e140 0x7fabe00705f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf40fb4e0 0x7fabf410eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 --2- 172.21.15.160:0/2573357075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf41014f0 0x7fabf410f090 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 >> 172.21.15.160:0/2573357075 conn(0x7fabf4073350 msgr2=0x7fabf40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 shutdown_connections 2024-05-25T06:28:28.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.833+0000 7fabfb7fa700 1 -- 172.21.15.160:0/2573357075 wait complete. 2024-05-25T06:28:28.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 -- 172.21.15.160:0/1370111037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214102dc0 msgr2=0x7f9214103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 --2- 172.21.15.160:0/1370111037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214102dc0 0x7f9214103190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9210004940 tx=0x7f9210009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 -- 172.21.15.160:0/1370111037 shutdown_connections 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 --2- 172.21.15.160:0/1370111037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9214103760 0x7f92140fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 --2- 172.21.15.160:0/1370111037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214102dc0 0x7f9214103190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 -- 172.21.15.160:0/1370111037 >> 172.21.15.160:0/1370111037 conn(0x7f92140f8930 msgr2=0x7f92140fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 -- 172.21.15.160:0/1370111037 shutdown_connections 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 -- 172.21.15.160:0/1370111037 wait complete. 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 Processor -- start 2024-05-25T06:28:28.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.936+0000 7f921c208700 1 -- start start 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f921c208700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9214102dc0 0x7f9214113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f921c208700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 0x7f9214113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f921c208700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92141144f0 con 0x7f9214103760 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f921c208700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921410daf0 con 0x7f9214102dc0 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 0x7f9214113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 0x7f9214113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47920/0 (socket says 172.21.15.160:47920) 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 -- 172.21.15.160:0/2622670796 learned_addr learned my addr 172.21.15.160:0/2622670796 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f9219fa4700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9214102dc0 0x7f9214113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 -- 172.21.15.160:0/2622670796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9214102dc0 msgr2=0x7f9214113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9214102dc0 0x7f9214113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 -- 172.21.15.160:0/2622670796 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210005040 con 0x7f9214103760 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f92197a3700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 0x7f9214113ea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9204002a50 tx=0x7f9204004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:28.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.937+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9204007cc0 con 0x7f9214103760 2024-05-25T06:28:28.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.938+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f920400f040 con 0x7f9214103760 2024-05-25T06:28:28.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.938+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921410ddd0 con 0x7f9214103760 2024-05-25T06:28:28.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.938+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920400d9c0 con 0x7f9214103760 2024-05-25T06:28:28.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.938+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921410e390 con 0x7f9214103760 2024-05-25T06:28:28.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.938+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92140fe510 con 0x7f9214103760 2024-05-25T06:28:28.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.939+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9204009730 con 0x7f9214103760 2024-05-25T06:28:28.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.940+0000 7f920a7fc700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f920006e140 0x7f92000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:28.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.940+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f920408d830 con 0x7f9214103760 2024-05-25T06:28:28.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.940+0000 7f9219fa4700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f920006e140 0x7f92000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:28.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.940+0000 7f9219fa4700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f920006e140 0x7f92000705f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9210004f90 tx=0x7f9210009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:28.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:28.944+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f920405a6d0 con 0x7f9214103760 2024-05-25T06:28:29.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.150+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9214109f60 con 0x7f920006e140 2024-05-25T06:28:29.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.161+0000 7f920a7fc700 1 -- 172.21.15.160:0/2622670796 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f921000b5c0 con 0x7f920006e140 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (14m) 7m ago 25m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (11m) 7m ago 25m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (11m) 7m ago 24m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (13m) 7m ago 25m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (8m) 7m ago 20m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (7m) 7m ago 19m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (7m) 7m ago 19m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (7m) 7m ago 20m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (16m) 7m ago 26m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:28:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (15m) 7m ago 24m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (12m) 7m ago 27m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (12m) 7m ago 24m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (15m) 7m ago 25m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (14m) 7m ago 24m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (11m) 7m ago 23m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (10m) 7m ago 23m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (10m) 7m ago 22m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (9m) 7m ago 22m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (9m) 7m ago 21m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (8m) 7m ago 21m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:28:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (14m) 7m ago 24m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:28:29.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f920006e140 msgr2=0x7f92000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f920006e140 0x7f92000705f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9210004f90 tx=0x7f9210009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:29.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 msgr2=0x7f9214113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 0x7f9214113ea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9204002a50 tx=0x7f9204004600 comp rx=0 tx=0).stop 2024-05-25T06:28:29.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 shutdown_connections 2024-05-25T06:28:29.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f920006e140 0x7f92000705f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9214102dc0 0x7f9214113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 --2- 172.21.15.160:0/2622670796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9214103760 0x7f9214113ea0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 >> 172.21.15.160:0/2622670796 conn(0x7f92140f8930 msgr2=0x7f92140f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:29.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 shutdown_connections 2024-05-25T06:28:29.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.163+0000 7f921c208700 1 -- 172.21.15.160:0/2622670796 wait complete. 2024-05-25T06:28:29.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:29 smithi160 ceph-mon[112094]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:29.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:29 smithi160 ceph-mon[112094]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:29.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:29 smithi160 ceph-mon[112094]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 -- 172.21.15.160:0/81991387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fbf40 msgr2=0x7f0be4101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 --2- 172.21.15.160:0/81991387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fbf40 0x7f0be4101a30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4004940 tx=0x7f0bd4009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 -- 172.21.15.160:0/81991387 shutdown_connections 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 --2- 172.21.15.160:0/81991387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fbf40 0x7f0be4101a30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 --2- 172.21.15.160:0/81991387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0be40fb5a0 0x7f0be40fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 -- 172.21.15.160:0/81991387 >> 172.21.15.160:0/81991387 conn(0x7f0be4073350 msgr2=0x7f0be4073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 -- 172.21.15.160:0/81991387 shutdown_connections 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 -- 172.21.15.160:0/81991387 wait complete. 2024-05-25T06:28:29.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 Processor -- start 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.265+0000 7f0bec28a700 1 -- start start 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bec28a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 0x7f0be4115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bec28a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0be40fbf40 0x7f0be4116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bec28a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4116780 con 0x7f0be40fb5a0 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bec28a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be410fda0 con 0x7f0be40fbf40 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 0x7f0be4115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 0x7f0be4115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47948/0 (socket says 172.21.15.160:47948) 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 -- 172.21.15.160:0/3903283372 learned_addr learned my addr 172.21.15.160:0/3903283372 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0be9825700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0be40fbf40 0x7f0be4116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 -- 172.21.15.160:0/3903283372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0be40fbf40 msgr2=0x7f0be4116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0be40fbf40 0x7f0be4116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 -- 172.21.15.160:0/3903283372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4005040 con 0x7f0be40fb5a0 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bea026700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 0x7f0be4115bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c830 tx=0x7f0be0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0be000a760 con 0x7f0be40fb5a0 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0be000ad90 con 0x7f0be40fb5a0 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be4110080 con 0x7f0be40fb5a0 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0be00113a0 con 0x7f0be40fb5a0 2024-05-25T06:28:29.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.266+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be41105c0 con 0x7f0be40fb5a0 2024-05-25T06:28:29.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.267+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be4101170 con 0x7f0be40fb5a0 2024-05-25T06:28:29.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.268+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0be0012400 con 0x7f0be40fb5a0 2024-05-25T06:28:29.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.268+0000 7f0bdaffd700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0bd006e140 0x7f0bd00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:29.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.268+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0be008dbe0 con 0x7f0be40fb5a0 2024-05-25T06:28:29.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.268+0000 7f0be9825700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0bd006e140 0x7f0bd00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:29.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.269+0000 7f0be9825700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0bd006e140 0x7f0bd00705f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0be41114d0 tx=0x7f0bd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:29.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.273+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0be0056f90 con 0x7f0be40fb5a0 2024-05-25T06:28:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:29 smithi203 ceph-mon[83507]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:29 smithi203 ceph-mon[83507]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:29 smithi203 ceph-mon[83507]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:29.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.552+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0be4109f00 con 0x7f0be40fb5a0 2024-05-25T06:28:29.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.552+0000 7f0bdaffd700 1 -- 172.21.15.160:0/3903283372 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f0be005a650 con 0x7f0be40fb5a0 2024-05-25T06:28:29.552 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:28:29.553 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:28:29.554 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:28:29.554 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:28:29.554 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:28:29.554 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:28:29.554 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:28:29.554 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0bd006e140 msgr2=0x7f0bd00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0bd006e140 0x7f0bd00705f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0be41114d0 tx=0x7f0bd4009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 msgr2=0x7f0be4115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 0x7f0be4115bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c830 tx=0x7f0be0004470 comp rx=0 tx=0).stop 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 shutdown_connections 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be40fb5a0 0x7f0be4115bf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0bd006e140 0x7f0bd00705f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 --2- 172.21.15.160:0/3903283372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0be40fbf40 0x7f0be4116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 >> 172.21.15.160:0/3903283372 conn(0x7f0be4073350 msgr2=0x7f0be40f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.554+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 shutdown_connections 2024-05-25T06:28:29.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.555+0000 7f0bec28a700 1 -- 172.21.15.160:0/3903283372 wait complete. 2024-05-25T06:28:29.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 -- 172.21.15.160:0/2340755875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 msgr2=0x7f5274065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 --2- 172.21.15.160:0/2340755875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f5274065ab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5268004940 tx=0x7f5268009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:29.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 -- 172.21.15.160:0/2340755875 shutdown_connections 2024-05-25T06:28:29.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 --2- 172.21.15.160:0/2340755875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f5274065ab0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 --2- 172.21.15.160:0/2340755875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5274102e50 0x7f5274103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 -- 172.21.15.160:0/2340755875 >> 172.21.15.160:0/2340755875 conn(0x7f5274071d00 msgr2=0x7f5274072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:29.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 -- 172.21.15.160:0/2340755875 shutdown_connections 2024-05-25T06:28:29.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.656+0000 7f527917e700 1 -- 172.21.15.160:0/2340755875 wait complete. 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f527917e700 1 Processor -- start 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f527917e700 1 -- start start 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f527917e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f52740fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f527917e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5274102e50 0x7f52740fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f527917e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740fc880 con 0x7f5274065640 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f527917e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740fc9c0 con 0x7f5274102e50 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.657+0000 7f5272d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f52740fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f5272d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f52740fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47958/0 (socket says 172.21.15.160:47958) 2024-05-25T06:28:29.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f5272d9d700 1 -- 172.21.15.160:0/1400980807 learned_addr learned my addr 172.21.15.160:0/1400980807 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f527259c700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5274102e50 0x7f52740fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f5272d9d700 1 -- 172.21.15.160:0/1400980807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5274102e50 msgr2=0x7f52740fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f5272d9d700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5274102e50 0x7f52740fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f5272d9d700 1 -- 172.21.15.160:0/1400980807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268005040 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f5272d9d700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f52740fbdb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5264002a50 tx=0x7f52640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5264007cc0 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f526400f040 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5274119e70 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5264009640 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.658+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527411a2f0 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.659+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5274100100 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.660+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f52640097a0 con 0x7f5274065640 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.660+0000 7f525b7fe700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f525c06e150 0x7f525c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:29.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.660+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f526408d970 con 0x7f5274065640 2024-05-25T06:28:29.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.660+0000 7f527259c700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f525c06e150 0x7f525c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:29.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.661+0000 7f527259c700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f525c06e150 0x7f525c070600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f52740fd6f0 tx=0x7f5268009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:29.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.665+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5264056ca0 con 0x7f5274065640 2024-05-25T06:28:29.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.904+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f527404bed0 con 0x7f5274065640 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.904+0000 7f525b7fe700 1 -- 172.21.15.160:0/1400980807 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f526405a360 con 0x7f5274065640 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:28:29.904 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:28:29.905 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:28:29.906 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:28:29.907 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f525c06e150 msgr2=0x7f525c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f525c06e150 0x7f525c070600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f52740fd6f0 tx=0x7f5268009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 msgr2=0x7f52740fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f52740fbdb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5264002a50 tx=0x7f52640046b0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 shutdown_connections 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5274065640 0x7f52740fbdb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f525c06e150 0x7f525c070600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 --2- 172.21.15.160:0/1400980807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5274102e50 0x7f52740fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.906+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 >> 172.21.15.160:0/1400980807 conn(0x7f5274071d00 msgr2=0x7f52740fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.907+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 shutdown_connections 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:29.907+0000 7f527917e700 1 -- 172.21.15.160:0/1400980807 wait complete. 2024-05-25T06:28:29.908 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 -- 172.21.15.160:0/3073701035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4102df0 msgr2=0x7fd2b41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 --2- 172.21.15.160:0/3073701035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4102df0 0x7fd2b41031c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac004940 tx=0x7fd2ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 -- 172.21.15.160:0/3073701035 shutdown_connections 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 --2- 172.21.15.160:0/3073701035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4103790 0x7fd2b40fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 --2- 172.21.15.160:0/3073701035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4102df0 0x7fd2b41031c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 -- 172.21.15.160:0/3073701035 >> 172.21.15.160:0/3073701035 conn(0x7fd2b40f8980 msgr2=0x7fd2b40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 -- 172.21.15.160:0/3073701035 shutdown_connections 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.010+0000 7fd2b899b700 1 -- 172.21.15.160:0/3073701035 wait complete. 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b899b700 1 Processor -- start 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b899b700 1 -- start start 2024-05-25T06:28:30.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b899b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4102df0 0x7fd2b4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b899b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 0x7fd2b41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b899b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41147b0 con 0x7fd2b4103790 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b899b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b410db10 con 0x7fd2b4102df0 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 0x7fd2b41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 0x7fd2b41140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47968/0 (socket says 172.21.15.160:47968) 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b1d9b700 1 -- 172.21.15.160:0/700785941 learned_addr learned my addr 172.21.15.160:0/700785941 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.011+0000 7fd2b259c700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4102df0 0x7fd2b4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b1d9b700 1 -- 172.21.15.160:0/700785941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4102df0 msgr2=0x7fd2b4113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b1d9b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4102df0 0x7fd2b4113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b1d9b700 1 -- 172.21.15.160:0/700785941 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac005040 con 0x7fd2b4103790 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b259c700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4102df0 0x7fd2b4113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b1d9b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 0x7fd2b41140d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00bae0 tx=0x7fd29c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29c00c8c0 con 0x7fd2b4103790 2024-05-25T06:28:30.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd29c014410 con 0x7fd2b4103790 2024-05-25T06:28:30.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b410ddf0 con 0x7fd2b4103790 2024-05-25T06:28:30.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29c007610 con 0x7fd2b4103790 2024-05-25T06:28:30.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.012+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b410e3b0 con 0x7fd2b4103790 2024-05-25T06:28:30.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.013+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b40fe540 con 0x7fd2b4103790 2024-05-25T06:28:30.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.014+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd29c014580 con 0x7fd2b4103790 2024-05-25T06:28:30.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.014+0000 7fd2aaffd700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2a006e140 0x7fd2a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:30.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.014+0000 7fd2b259c700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2a006e140 0x7fd2a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:30.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.014+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd29c08db80 con 0x7fd2b4103790 2024-05-25T06:28:30.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.015+0000 7fd2b259c700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2a006e140 0x7fd2a00705f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac004f90 tx=0x7fd2ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:30.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.019+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd29c05aa60 con 0x7fd2b4103790 2024-05-25T06:28:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:30 smithi160 ceph-mon[112094]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:30 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3903283372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:28:30.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:30 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1400980807' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:28:30.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.238+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2b4109f60 con 0x7fd2a006e140 2024-05-25T06:28:30.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.240+0000 7fd2aaffd700 1 -- 172.21.15.160:0/700785941 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd2b4109f60 con 0x7fd2a006e140 2024-05-25T06:28:30.239 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:28:30.240 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2a006e140 msgr2=0x7fd2a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2a006e140 0x7fd2a00705f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac004f90 tx=0x7fd2ac009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 msgr2=0x7fd2b41140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 0x7fd2b41140d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00bae0 tx=0x7fd29c009810 comp rx=0 tx=0).stop 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 shutdown_connections 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd2a006e140 0x7fd2a00705f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b4102df0 0x7fd2b4113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 --2- 172.21.15.160:0/700785941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2b4103790 0x7fd2b41140d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 >> 172.21.15.160:0/700785941 conn(0x7fd2b40f8980 msgr2=0x7fd2b40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:30.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 shutdown_connections 2024-05-25T06:28:30.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.242+0000 7fd2b899b700 1 -- 172.21.15.160:0/700785941 wait complete. 2024-05-25T06:28:30.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.344+0000 7fca2f63c700 1 -- 172.21.15.160:0/4009840225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca28101d10 msgr2=0x7fca281020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.344+0000 7fca2f63c700 1 --2- 172.21.15.160:0/4009840225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca28101d10 0x7fca281020e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fca24004940 tx=0x7fca24009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.344+0000 7fca2f63c700 1 -- 172.21.15.160:0/4009840225 shutdown_connections 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 --2- 172.21.15.160:0/4009840225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca281026b0 0x7fca280fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 --2- 172.21.15.160:0/4009840225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca28101d10 0x7fca281020e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 -- 172.21.15.160:0/4009840225 >> 172.21.15.160:0/4009840225 conn(0x7fca280f7860 msgr2=0x7fca280f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 -- 172.21.15.160:0/4009840225 shutdown_connections 2024-05-25T06:28:30.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 -- 172.21.15.160:0/4009840225 wait complete. 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 Processor -- start 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 -- start start 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca28101d10 0x7fca2810eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 0x7fca2810f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2810f5b0 con 0x7fca281026b0 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2f63c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2810f720 con 0x7fca28101d10 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2cbd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 0x7fca2810f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2cbd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 0x7fca2810f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47988/0 (socket says 172.21.15.160:47988) 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.345+0000 7fca2cbd7700 1 -- 172.21.15.160:0/2398496467 learned_addr learned my addr 172.21.15.160:0/2398496467 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2d3d8700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca28101d10 0x7fca2810eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2cbd7700 1 -- 172.21.15.160:0/2398496467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca28101d10 msgr2=0x7fca2810eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2cbd7700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca28101d10 0x7fca2810eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2cbd7700 1 -- 172.21.15.160:0/2398496467 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24005040 con 0x7fca281026b0 2024-05-25T06:28:30.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2cbd7700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 0x7fca2810f070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fca18002a50 tx=0x7fca18004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:30.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca18007cc0 con 0x7fca281026b0 2024-05-25T06:28:30.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2810f9b0 con 0x7fca281026b0 2024-05-25T06:28:30.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca1800f040 con 0x7fca281026b0 2024-05-25T06:28:30.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1800d9c0 con 0x7fca281026b0 2024-05-25T06:28:30.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.346+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca28117f50 con 0x7fca281026b0 2024-05-25T06:28:30.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.347+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca280fd460 con 0x7fca281026b0 2024-05-25T06:28:30.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.348+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fca18009730 con 0x7fca281026b0 2024-05-25T06:28:30.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.348+0000 7fca1dffb700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca1406e140 0x7fca140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:28:30.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.348+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fca1808d8f0 con 0x7fca281026b0 2024-05-25T06:28:30.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.348+0000 7fca2d3d8700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca1406e140 0x7fca140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:28:30.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.349+0000 7fca2d3d8700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca1406e140 0x7fca140705f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca24004f90 tx=0x7fca24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:28:30.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.353+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca18056c20 con 0x7fca281026b0 2024-05-25T06:28:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:30 smithi203 ceph-mon[83507]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:30 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3903283372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:28:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:30 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1400980807' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:28:30.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.637+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca2804c710 con 0x7fca281026b0 2024-05-25T06:28:30.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.637+0000 7fca1dffb700 1 -- 172.21.15.160:0/2398496467 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fca1805a2e0 con 0x7fca281026b0 2024-05-25T06:28:30.638 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:28:30.638 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:28:30.638 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:28:30.638 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:28:30.638 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca1406e140 msgr2=0x7fca140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca1406e140 0x7fca140705f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca24004f90 tx=0x7fca24009e30 comp rx=0 tx=0).stop 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 msgr2=0x7fca2810f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 0x7fca2810f070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fca18002a50 tx=0x7fca18004600 comp rx=0 tx=0).stop 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 shutdown_connections 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fca1406e140 0x7fca140705f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca28101d10 0x7fca2810eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 --2- 172.21.15.160:0/2398496467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca281026b0 0x7fca2810f070 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 >> 172.21.15.160:0/2398496467 conn(0x7fca280f7860 msgr2=0x7fca280f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 shutdown_connections 2024-05-25T06:28:30.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:28:30.640+0000 7fca2f63c700 1 -- 172.21.15.160:0/2398496467 wait complete. 2024-05-25T06:28:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:31 smithi203 ceph-mon[83507]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:31 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2398496467' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:28:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:31 smithi203 ceph-mon[83507]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:31 smithi160 ceph-mon[112094]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:28:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:31 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2398496467' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:28:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:31 smithi160 ceph-mon[112094]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:33 smithi203 ceph-mon[83507]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:33 smithi160 ceph-mon[112094]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:35 smithi203 ceph-mon[83507]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:35 smithi160 ceph-mon[112094]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:37 smithi203 ceph-mon[83507]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:37 smithi160 ceph-mon[112094]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:39 smithi203 ceph-mon[83507]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:39 smithi160 ceph-mon[112094]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:41 smithi203 ceph-mon[83507]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:41 smithi160 ceph-mon[112094]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:44 smithi203 ceph-mon[83507]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:44 smithi160 ceph-mon[112094]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:45 smithi203 ceph-mon[83507]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:45 smithi160 ceph-mon[112094]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:47 smithi203 ceph-mon[83507]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:47 smithi160 ceph-mon[112094]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:49 smithi203 ceph-mon[83507]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:49 smithi160 ceph-mon[112094]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:51 smithi203 ceph-mon[83507]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:51 smithi160 ceph-mon[112094]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:53 smithi203 ceph-mon[83507]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:53 smithi160 ceph-mon[112094]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:55 smithi203 ceph-mon[83507]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:55 smithi160 ceph-mon[112094]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:57 smithi203 ceph-mon[83507]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:57 smithi160 ceph-mon[112094]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:28:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:28:59 smithi203 ceph-mon[83507]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:28:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:28:59 smithi160 ceph-mon[112094]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:00.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 -- 172.21.15.160:0/886877845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fc560 msgr2=0x7f9864100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 --2- 172.21.15.160:0/886877845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fc560 0x7f9864100b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9858004940 tx=0x7f9858009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 -- 172.21.15.160:0/886877845 shutdown_connections 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 --2- 172.21.15.160:0/886877845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fc560 0x7f9864100b80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 --2- 172.21.15.160:0/886877845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640fbbc0 0x7f98640fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 -- 172.21.15.160:0/886877845 >> 172.21.15.160:0/886877845 conn(0x7f98640f7860 msgr2=0x7f98640f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 -- 172.21.15.160:0/886877845 shutdown_connections 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 -- 172.21.15.160:0/886877845 wait complete. 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 Processor -- start 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.757+0000 7f986c13e700 1 -- start start 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f986c13e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 0x7f9864119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f986c13e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640fc560 0x7f986411a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f986c13e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986411ab60 con 0x7f98640fbbc0 2024-05-25T06:29:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f986c13e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9864114140 con 0x7f98640fc560 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 0x7f9864119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 0x7f9864119fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46548/0 (socket says 172.21.15.160:46548) 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 -- 172.21.15.160:0/4286361368 learned_addr learned my addr 172.21.15.160:0/4286361368 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f98696d9700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640fc560 0x7f986411a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 -- 172.21.15.160:0/4286361368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640fc560 msgr2=0x7f986411a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640fc560 0x7f986411a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 -- 172.21.15.160:0/4286361368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858005040 con 0x7f98640fbbc0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f9869eda700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 0x7f9864119fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f985400c8c0 tx=0x7f98540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985400a760 con 0x7f98640fbbc0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f985400ad90 con 0x7f98640fbbc0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9864114420 con 0x7f98640fbbc0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98540113a0 con 0x7f98640fbbc0 2024-05-25T06:29:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.758+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9864114960 con 0x7f98640fbbc0 2024-05-25T06:29:00.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.759+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9864066b70 con 0x7f98640fbbc0 2024-05-25T06:29:00.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.760+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9854012400 con 0x7f98640fbbc0 2024-05-25T06:29:00.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.760+0000 7f98627fc700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f984c06e140 0x7f984c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:00.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.761+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f985408dbb0 con 0x7f98640fbbc0 2024-05-25T06:29:00.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.761+0000 7f98696d9700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f984c06e140 0x7f984c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:00.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.761+0000 7f98696d9700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f984c06e140 0x7f984c0705f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9864115870 tx=0x7f9858009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:00.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.765+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f985405aa90 con 0x7f98640fbbc0 2024-05-25T06:29:00.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.980+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9864115620 con 0x7f984c06e140 2024-05-25T06:29:00.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.982+0000 7f98627fc700 1 -- 172.21.15.160:0/4286361368 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9864115620 con 0x7f984c06e140 2024-05-25T06:29:00.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f984c06e140 msgr2=0x7f984c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:00.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f984c06e140 0x7f984c0705f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9864115870 tx=0x7f9858009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 msgr2=0x7f9864119fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 0x7f9864119fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f985400c8c0 tx=0x7f98540044c0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 shutdown_connections 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98640fbbc0 0x7f9864119fd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f984c06e140 0x7f984c0705f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 --2- 172.21.15.160:0/4286361368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640fc560 0x7f986411a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 >> 172.21.15.160:0/4286361368 conn(0x7f98640f7860 msgr2=0x7f98640f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 shutdown_connections 2024-05-25T06:29:00.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:00.984+0000 7f986c13e700 1 -- 172.21.15.160:0/4286361368 wait complete. 2024-05-25T06:29:00.995 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.088+0000 7f9688623700 1 -- 172.21.15.160:0/3850065640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 msgr2=0x7f9680101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.088+0000 7f9688623700 1 --2- 172.21.15.160:0/3850065640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680101a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9670004940 tx=0x7f9670009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.088+0000 7f9688623700 1 -- 172.21.15.160:0/3850065640 shutdown_connections 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.088+0000 7f9688623700 1 --2- 172.21.15.160:0/3850065640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680101a30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.088+0000 7f9688623700 1 --2- 172.21.15.160:0/3850065640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 0x7f96800fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.088+0000 7f9688623700 1 -- 172.21.15.160:0/3850065640 >> 172.21.15.160:0/3850065640 conn(0x7f9680073350 msgr2=0x7f9680073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.089+0000 7f9688623700 1 -- 172.21.15.160:0/3850065640 shutdown_connections 2024-05-25T06:29:01.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.089+0000 7f9688623700 1 -- 172.21.15.160:0/3850065640 wait complete. 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.089+0000 7f9688623700 1 Processor -- start 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.089+0000 7f9688623700 1 -- start start 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9688623700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 0x7f9680113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9688623700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9688623700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96801145d0 con 0x7f96800fbf40 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9688623700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968010dbb0 con 0x7f96800fb5a0 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46566/0 (socket says 172.21.15.160:46566) 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 -- 172.21.15.160:0/2876305732 learned_addr learned my addr 172.21.15.160:0/2876305732 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f96863bf700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 0x7f9680113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 -- 172.21.15.160:0/2876305732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 msgr2=0x7f9680113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 0x7f9680113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 -- 172.21.15.160:0/2876305732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670005040 con 0x7f96800fbf40 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f96863bf700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 0x7f9680113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9685bbe700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680113f80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f967000b3b0 tx=0x7f9670009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.090+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967000e040 con 0x7f96800fbf40 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.091+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9670013420 con 0x7f96800fbf40 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.091+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968010de30 con 0x7f96800fbf40 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.091+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967001c630 con 0x7f96800fbf40 2024-05-25T06:29:01.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.091+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968010e370 con 0x7f96800fbf40 2024-05-25T06:29:01.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.092+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96800fe670 con 0x7f96800fbf40 2024-05-25T06:29:01.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.092+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9670013590 con 0x7f96800fbf40 2024-05-25T06:29:01.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.093+0000 7f9676ffd700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f966c06e140 0x7f966c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.093+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f967008ec90 con 0x7f96800fbf40 2024-05-25T06:29:01.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.093+0000 7f96863bf700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f966c06e140 0x7f966c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.093+0000 7f96863bf700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f966c06e140 0x7f966c0705f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f967c004fd0 tx=0x7f967c0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:01.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.097+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f967005bc20 con 0x7f96800fbf40 2024-05-25T06:29:01.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.311+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9680109f70 con 0x7f966c06e140 2024-05-25T06:29:01.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.313+0000 7f9676ffd700 1 -- 172.21.15.160:0/2876305732 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9680109f70 con 0x7f966c06e140 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f966c06e140 msgr2=0x7f966c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f966c06e140 0x7f966c0705f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f967c004fd0 tx=0x7f967c0043c0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 msgr2=0x7f9680113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680113f80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f967000b3b0 tx=0x7f9670009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 shutdown_connections 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f966c06e140 0x7f966c0705f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96800fb5a0 0x7f9680113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 --2- 172.21.15.160:0/2876305732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96800fbf40 0x7f9680113f80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 >> 172.21.15.160:0/2876305732 conn(0x7f9680073350 msgr2=0x7f9680100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:01.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 shutdown_connections 2024-05-25T06:29:01.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.315+0000 7f9688623700 1 -- 172.21.15.160:0/2876305732 wait complete. 2024-05-25T06:29:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:01 smithi203 ceph-mon[83507]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:01 smithi203 ceph-mon[83507]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 -- 172.21.15.160:0/817819088 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 msgr2=0x7fb0600fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 --2- 172.21.15.160:0/817819088 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb0600fc1b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb054004940 tx=0x7fb054009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 -- 172.21.15.160:0/817819088 shutdown_connections 2024-05-25T06:29:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 --2- 172.21.15.160:0/817819088 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb0600fc1b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 --2- 172.21.15.160:0/817819088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb060101d90 0x7fb060102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 -- 172.21.15.160:0/817819088 >> 172.21.15.160:0/817819088 conn(0x7fb060071d00 msgr2=0x7fb060072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 -- 172.21.15.160:0/817819088 shutdown_connections 2024-05-25T06:29:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.415+0000 7fb0651b9700 1 -- 172.21.15.160:0/817819088 wait complete. 2024-05-25T06:29:01.416 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:01 smithi160 ceph-mon[112094]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:01.416 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:01 smithi160 ceph-mon[112094]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb0651b9700 1 Processor -- start 2024-05-25T06:29:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb0651b9700 1 -- start start 2024-05-25T06:29:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb0651b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb060102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb0651b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb060101d90 0x7fb060103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb0651b9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb060103680 con 0x7fb0600fbd40 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb0651b9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0601037c0 con 0x7fb060101d90 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb05ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb060102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb05ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb060102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46586/0 (socket says 172.21.15.160:46586) 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb05ed9d700 1 -- 172.21.15.160:0/4239728846 learned_addr learned my addr 172.21.15.160:0/4239728846 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.416+0000 7fb05e59c700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb060101d90 0x7fb060103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb05ed9d700 1 -- 172.21.15.160:0/4239728846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb060101d90 msgr2=0x7fb060103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb05ed9d700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb060101d90 0x7fb060103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb05ed9d700 1 -- 172.21.15.160:0/4239728846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054005040 con 0x7fb0600fbd40 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb05ed9d700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb060102b20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb05000b790 tx=0x7fb050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb05000f6f0 con 0x7fb0600fbd40 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb060119ea0 con 0x7fb0600fbd40 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb05000fd20 con 0x7fb0600fbd40 2024-05-25T06:29:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0500103a0 con 0x7fb0600fbd40 2024-05-25T06:29:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.417+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06011a380 con 0x7fb0600fbd40 2024-05-25T06:29:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.418+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0600fd430 con 0x7fb0600fbd40 2024-05-25T06:29:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.419+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb050011400 con 0x7fb0600fbd40 2024-05-25T06:29:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.419+0000 7fb0477fe700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb04806e140 0x7fb0480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.419+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb05008d940 con 0x7fb0600fbd40 2024-05-25T06:29:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.419+0000 7fb05e59c700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb04806e140 0x7fb0480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.420+0000 7fb05e59c700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb04806e140 0x7fb0480705f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb060104460 tx=0x7fb054009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:01.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.424+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb050056cf0 con 0x7fb0600fbd40 2024-05-25T06:29:01.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.629+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb06004bf40 con 0x7fb04806e140 2024-05-25T06:29:01.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.640+0000 7fb0477fe700 1 -- 172.21.15.160:0/4239728846 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb06004bf40 con 0x7fb04806e140 2024-05-25T06:29:01.639 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (14m) 8m ago 25m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (12m) 8m ago 26m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (12m) 7m ago 25m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (14m) 8m ago 25m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (8m) 8m ago 20m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (8m) 8m ago 20m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (7m) 7m ago 20m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (8m) 7m ago 20m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (16m) 8m ago 27m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (15m) 7m ago 25m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (13m) 8m ago 27m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (13m) 7m ago 24m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (15m) 8m ago 26m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (15m) 7m ago 25m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:29:01.640 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (11m) 8m ago 24m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:29:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (11m) 8m ago 23m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:29:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (10m) 8m ago 23m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:29:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (10m) 7m ago 22m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:29:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (9m) 7m ago 22m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:29:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (9m) 7m ago 22m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:29:01.641 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (15m) 8m ago 25m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb04806e140 msgr2=0x7fb0480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb04806e140 0x7fb0480705f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb060104460 tx=0x7fb054009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 msgr2=0x7fb060102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb060102b20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb05000b790 tx=0x7fb050009250 comp rx=0 tx=0).stop 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 shutdown_connections 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0600fbd40 0x7fb060102b20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb04806e140 0x7fb0480705f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 --2- 172.21.15.160:0/4239728846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb060101d90 0x7fb060103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 >> 172.21.15.160:0/4239728846 conn(0x7fb060071d00 msgr2=0x7fb060065610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:01.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 shutdown_connections 2024-05-25T06:29:01.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.642+0000 7fb0651b9700 1 -- 172.21.15.160:0/4239728846 wait complete. 2024-05-25T06:29:01.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 -- 172.21.15.160:0/4002532755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00101ce0 msgr2=0x7fda001020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 --2- 172.21.15.160:0/4002532755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00101ce0 0x7fda001020b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0004940 tx=0x7fd9f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:01.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 -- 172.21.15.160:0/4002532755 shutdown_connections 2024-05-25T06:29:01.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 --2- 172.21.15.160:0/4002532755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda00102680 0x7fda000fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 --2- 172.21.15.160:0/4002532755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00101ce0 0x7fda001020b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 -- 172.21.15.160:0/4002532755 >> 172.21.15.160:0/4002532755 conn(0x7fda000f7850 msgr2=0x7fda000f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 -- 172.21.15.160:0/4002532755 shutdown_connections 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.744+0000 7fda084b4700 1 -- 172.21.15.160:0/4002532755 wait complete. 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda084b4700 1 Processor -- start 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda084b4700 1 -- start start 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda084b4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda00101ce0 0x7fda00112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda084b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 0x7fda00112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda084b4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001135d0 con 0x7fda00102680 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda084b4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0010c9c0 con 0x7fda00101ce0 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda05a4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 0x7fda00112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda05a4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 0x7fda00112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46606/0 (socket says 172.21.15.160:46606) 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda05a4f700 1 -- 172.21.15.160:0/3310475280 learned_addr learned my addr 172.21.15.160:0/3310475280 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda06250700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda00101ce0 0x7fda00112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda05a4f700 1 -- 172.21.15.160:0/3310475280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda00101ce0 msgr2=0x7fda00112a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda05a4f700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda00101ce0 0x7fda00112a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.745+0000 7fda05a4f700 1 -- 172.21.15.160:0/3310475280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0005040 con 0x7fda00102680 2024-05-25T06:29:01.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.746+0000 7fda05a4f700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 0x7fda00112f80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400cae0 tx=0x7fd9f4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.746+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9f400a980 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.746+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9f4012440 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.746+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9f40115c0 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.746+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0010cca0 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.746+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0010d260 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.747+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda000fd430 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.747+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd9f4012a50 con 0x7fda00102680 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.748+0000 7fd9feffd700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd9e806e150 0x7fd9e8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.748+0000 7fda06250700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd9e806e150 0x7fd9e8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:01.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.748+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd9f408e0e0 con 0x7fda00102680 2024-05-25T06:29:01.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.748+0000 7fda06250700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd9e806e150 0x7fd9e8070600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000b3b0 tx=0x7fd9f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:01.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:01.753+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd9f405afc0 con 0x7fda00102680 2024-05-25T06:29:02.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.031+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda00108de0 con 0x7fda00102680 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.032+0000 7fd9feffd700 1 -- 172.21.15.160:0/3310475280 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fd9f405ab50 con 0x7fda00102680 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:29:02.032 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:29:02.033 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:29:02.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd9e806e150 msgr2=0x7fd9e8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd9e806e150 0x7fd9e8070600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000b3b0 tx=0x7fd9f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 msgr2=0x7fda00112f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 0x7fda00112f80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400cae0 tx=0x7fd9f4007f60 comp rx=0 tx=0).stop 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 shutdown_connections 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd9e806e150 0x7fd9e8070600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda00101ce0 0x7fda00112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 --2- 172.21.15.160:0/3310475280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda00102680 0x7fda00112f80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 >> 172.21.15.160:0/3310475280 conn(0x7fda000f7850 msgr2=0x7fda000f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.034+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 shutdown_connections 2024-05-25T06:29:02.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.035+0000 7fda084b4700 1 -- 172.21.15.160:0/3310475280 wait complete. 2024-05-25T06:29:02.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.138+0000 7ff1813ca700 1 -- 172.21.15.160:0/4180637774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 msgr2=0x7ff17c0fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.138+0000 7ff1813ca700 1 --2- 172.21.15.160:0/4180637774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c0fcdd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff170004940 tx=0x7ff170009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:02.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 -- 172.21.15.160:0/4180637774 shutdown_connections 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 --2- 172.21.15.160:0/4180637774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c0fcdd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 --2- 172.21.15.160:0/4180637774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 0x7ff17c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 -- 172.21.15.160:0/4180637774 >> 172.21.15.160:0/4180637774 conn(0x7ff17c0f8950 msgr2=0x7ff17c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 -- 172.21.15.160:0/4180637774 shutdown_connections 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 -- 172.21.15.160:0/4180637774 wait complete. 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 Processor -- start 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.139+0000 7ff1813ca700 1 -- start start 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1813ca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 0x7ff17c113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1813ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1813ca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c114740 con 0x7ff17c103760 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1813ca700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c10daa0 con 0x7ff17c102dc0 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46620/0 (socket says 172.21.15.160:46620) 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 -- 172.21.15.160:0/293631111 learned_addr learned my addr 172.21.15.160:0/293631111 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17affd700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 0x7ff17c113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 -- 172.21.15.160:0/293631111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 msgr2=0x7ff17c113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 0x7ff17c113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 -- 172.21.15.160:0/293631111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170005040 con 0x7ff17c103760 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17affd700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 0x7ff17c113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff17a7fc700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c114060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff170009100 tx=0x7ff170009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17000e040 con 0x7ff17c103760 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c10dd20 con 0x7ff17c103760 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff170013420 con 0x7ff17c103760 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17001c610 con 0x7ff17c103760 2024-05-25T06:29:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.140+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c10e2e0 con 0x7ff17c103760 2024-05-25T06:29:02.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.141+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c1000d0 con 0x7ff17c103760 2024-05-25T06:29:02.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.142+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff170013590 con 0x7ff17c103760 2024-05-25T06:29:02.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.142+0000 7ff1637fe700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff16406e140 0x7ff1640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.142+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff17008ee10 con 0x7ff17c103760 2024-05-25T06:29:02.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.142+0000 7ff17affd700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff16406e140 0x7ff1640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.143+0000 7ff17affd700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff16406e140 0x7ff1640705f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006fd0 tx=0x7ff16c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:02.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.147+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff17005bcf0 con 0x7ff17c103760 2024-05-25T06:29:02.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.389+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff17c10f170 con 0x7ff17c103760 2024-05-25T06:29:02.389 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:02 smithi160 ceph-mon[112094]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:02.389 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:02 smithi160 ceph-mon[112094]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:02.390 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:02 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3310475280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.389+0000 7ff1637fe700 1 -- 172.21.15.160:0/293631111 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ff17005b880 con 0x7ff17c103760 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:29:02.390 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:29:02.391 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:02.392 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:02.393 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:29:02.393 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:02.393 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:02.393 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:02.393 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:02.393 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff16406e140 msgr2=0x7ff1640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff16406e140 0x7ff1640705f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006fd0 tx=0x7ff16c008040 comp rx=0 tx=0).stop 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 msgr2=0x7ff17c114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c114060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff170009100 tx=0x7ff170009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 shutdown_connections 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff16406e140 0x7ff1640705f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff17c102dc0 0x7ff17c113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 --2- 172.21.15.160:0/293631111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff17c103760 0x7ff17c114060 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 >> 172.21.15.160:0/293631111 conn(0x7ff17c0f8950 msgr2=0x7ff17c107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:02.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 shutdown_connections 2024-05-25T06:29:02.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.392+0000 7ff1813ca700 1 -- 172.21.15.160:0/293631111 wait complete. 2024-05-25T06:29:02.395 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:29:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:02 smithi203 ceph-mon[83507]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:02 smithi203 ceph-mon[83507]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:02 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3310475280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:29:02.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 -- 172.21.15.160:0/1440561825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f40fbd40 msgr2=0x7f09f40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1440561825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f40fbd40 0x7f09f40fc1b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09e4004940 tx=0x7f09e4009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:02.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 -- 172.21.15.160:0/1440561825 shutdown_connections 2024-05-25T06:29:02.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1440561825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f40fbd40 0x7f09f40fc1b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1440561825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f4101d90 0x7f09f4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 -- 172.21.15.160:0/1440561825 >> 172.21.15.160:0/1440561825 conn(0x7f09f4071d00 msgr2=0x7f09f4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:02.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 -- 172.21.15.160:0/1440561825 shutdown_connections 2024-05-25T06:29:02.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.497+0000 7f09fb40f700 1 -- 172.21.15.160:0/1440561825 wait complete. 2024-05-25T06:29:02.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09fb40f700 1 Processor -- start 2024-05-25T06:29:02.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09fb40f700 1 -- start start 2024-05-25T06:29:02.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09fb40f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fbd40 0x7f09f4112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09fb40f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 0x7f09f4112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09fb40f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4113640 con 0x7f09f4101d90 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09fb40f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f410ca30 con 0x7f09f40fbd40 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f89aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 0x7f09f4112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f89aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 0x7f09f4112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46644/0 (socket says 172.21.15.160:46644) 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f89aa700 1 -- 172.21.15.160:0/1071616669 learned_addr learned my addr 172.21.15.160:0/1071616669 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f91ab700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fbd40 0x7f09f4112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f89aa700 1 -- 172.21.15.160:0/1071616669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fbd40 msgr2=0x7f09f4112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f89aa700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fbd40 0x7f09f4112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.498+0000 7f09f89aa700 1 -- 172.21.15.160:0/1071616669 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4005040 con 0x7f09f4101d90 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.499+0000 7f09f89aa700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 0x7f09f4112ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09e400b3b0 tx=0x7f09e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.499+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e400e040 con 0x7f09f4101d90 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.499+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f410ccb0 con 0x7f09f4101d90 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.499+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09e4013420 con 0x7f09f4101d90 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.499+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e401c630 con 0x7f09f4101d90 2024-05-25T06:29:02.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.499+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f410d270 con 0x7f09f4101d90 2024-05-25T06:29:02.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.500+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f40fd430 con 0x7f09f4101d90 2024-05-25T06:29:02.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.500+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f09e4013590 con 0x7f09f4101d90 2024-05-25T06:29:02.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.501+0000 7f09e9ffb700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09e0070350 0x7f09e0072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.501+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f09e408ec40 con 0x7f09f4101d90 2024-05-25T06:29:02.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.501+0000 7f09f91ab700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09e0070350 0x7f09e0072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.501+0000 7f09f91ab700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09e0070350 0x7f09e0072800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09f0007a40 tx=0x7f09f0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:02.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.506+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f09e405bbe0 con 0x7f09f4101d90 2024-05-25T06:29:02.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.721+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09f4108e60 con 0x7f09e0070350 2024-05-25T06:29:02.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.723+0000 7f09e9ffb700 1 -- 172.21.15.160:0/1071616669 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f09f4108e60 con 0x7f09e0070350 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:29:02.723 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:29:02.724 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:29:02.724 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09e0070350 msgr2=0x7f09e0072800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09e0070350 0x7f09e0072800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09f0007a40 tx=0x7f09f0009da0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 msgr2=0x7f09f4112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 0x7f09f4112ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09e400b3b0 tx=0x7f09e4009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 shutdown_connections 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09e0070350 0x7f09e0072800 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fbd40 0x7f09f4112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 --2- 172.21.15.160:0/1071616669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4101d90 0x7f09f4112ff0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 >> 172.21.15.160:0/1071616669 conn(0x7f09f4071d00 msgr2=0x7f09f40fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:02.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 shutdown_connections 2024-05-25T06:29:02.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.725+0000 7f09fb40f700 1 -- 172.21.15.160:0/1071616669 wait complete. 2024-05-25T06:29:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 -- 172.21.15.160:0/4294922167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fde40 msgr2=0x7f52000fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 --2- 172.21.15.160:0/4294922167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fde40 0x7f52000fe210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f51fc004940 tx=0x7f51fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 -- 172.21.15.160:0/4294922167 shutdown_connections 2024-05-25T06:29:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 --2- 172.21.15.160:0/4294922167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52000fe7e0 0x7f5200106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 --2- 172.21.15.160:0/4294922167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fde40 0x7f52000fe210 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 -- 172.21.15.160:0/4294922167 >> 172.21.15.160:0/4294922167 conn(0x7f52000f7850 msgr2=0x7f52000f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 -- 172.21.15.160:0/4294922167 shutdown_connections 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.830+0000 7f5208097700 1 -- 172.21.15.160:0/4294922167 wait complete. 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5208097700 1 Processor -- start 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5208097700 1 -- start start 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5208097700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52000fde40 0x7f5200113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5208097700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 0x7f5200113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5208097700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52001145f0 con 0x7f52000fe7e0 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5208097700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520010d9e0 con 0x7f52000fde40 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205632700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 0x7f5200113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205632700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 0x7f5200113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46658/0 (socket says 172.21.15.160:46658) 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205632700 1 -- 172.21.15.160:0/3193236813 learned_addr learned my addr 172.21.15.160:0/3193236813 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205e33700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52000fde40 0x7f5200113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205632700 1 -- 172.21.15.160:0/3193236813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52000fde40 msgr2=0x7f5200113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:02.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205632700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52000fde40 0x7f5200113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:02.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.831+0000 7f5205632700 1 -- 172.21.15.160:0/3193236813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc005040 con 0x7f52000fe7e0 2024-05-25T06:29:02.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.832+0000 7f5205632700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 0x7f5200113fa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51f000eb40 tx=0x7f51f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.832+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f0007cc0 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.832+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51f000f040 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.832+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520010dc60 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.832+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f0009620 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.832+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520010e1a0 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.833+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52000666f0 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.833+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f51f0009780 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.834+0000 7f51f67fc700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51ec06e150 0x7f51ec070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.834+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f51f008d820 con 0x7f52000fe7e0 2024-05-25T06:29:02.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.834+0000 7f5205e33700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51ec06e150 0x7f51ec070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:02.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.834+0000 7f5205e33700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51ec06e150 0x7f51ec070600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00b0d0 tx=0x7f51fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:02.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:02.838+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f51f005a730 con 0x7f52000fe7e0 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.134+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f520005f5c0 con 0x7f52000fe7e0 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.134+0000 7f51f67fc700 1 -- 172.21.15.160:0/3193236813 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f51f005a2c0 con 0x7f52000fe7e0 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:29:03.134 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.136+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51ec06e150 msgr2=0x7f51ec070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.136+0000 7f5208097700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51ec06e150 0x7f51ec070600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00b0d0 tx=0x7f51fc009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.136+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 msgr2=0x7f5200113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.136+0000 7f5208097700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 0x7f5200113fa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51f000eb40 tx=0x7f51f000b040 comp rx=0 tx=0).stop 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 shutdown_connections 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51ec06e150 0x7f51ec070600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52000fde40 0x7f5200113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 --2- 172.21.15.160:0/3193236813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52000fe7e0 0x7f5200113fa0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 >> 172.21.15.160:0/3193236813 conn(0x7f52000f7850 msgr2=0x7f52000f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 shutdown_connections 2024-05-25T06:29:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:03.137+0000 7f5208097700 1 -- 172.21.15.160:0/3193236813 wait complete. 2024-05-25T06:29:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:03 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/293631111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:29:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:03 smithi203 ceph-mon[83507]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:03 smithi203 ceph-mon[83507]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:03 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/293631111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:29:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:03 smithi160 ceph-mon[112094]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:03 smithi160 ceph-mon[112094]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:04 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3193236813' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:29:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:04 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3193236813' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:29:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:05 smithi203 ceph-mon[83507]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:05 smithi160 ceph-mon[112094]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:07 smithi203 ceph-mon[83507]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:07 smithi160 ceph-mon[112094]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:09 smithi203 ceph-mon[83507]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:09 smithi160 ceph-mon[112094]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:11 smithi203 ceph-mon[83507]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:11 smithi160 ceph-mon[112094]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:13 smithi203 ceph-mon[83507]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:13 smithi160 ceph-mon[112094]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:15 smithi203 ceph-mon[83507]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:15 smithi160 ceph-mon[112094]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:17 smithi203 ceph-mon[83507]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:17 smithi160 ceph-mon[112094]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:19 smithi203 ceph-mon[83507]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:19 smithi160 ceph-mon[112094]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:21 smithi203 ceph-mon[83507]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:21 smithi160 ceph-mon[112094]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:23 smithi203 ceph-mon[83507]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:23 smithi160 ceph-mon[112094]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:25 smithi203 ceph-mon[83507]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:25 smithi160 ceph-mon[112094]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:27 smithi203 ceph-mon[83507]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:27 smithi160 ceph-mon[112094]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:28.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:29 smithi203 ceph-mon[83507]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:29 smithi160 ceph-mon[112094]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:31 smithi203 ceph-mon[83507]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:31 smithi160 ceph-mon[112094]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:33.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 -- 172.21.15.160:0/3083147380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 msgr2=0x7f10480fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 --2- 172.21.15.160:0/3083147380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10480fb600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1030004940 tx=0x7f1030009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:33.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 -- 172.21.15.160:0/3083147380 shutdown_connections 2024-05-25T06:29:33.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 --2- 172.21.15.160:0/3083147380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10480fb600 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 --2- 172.21.15.160:0/3083147380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 0x7f10481020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 -- 172.21.15.160:0/3083147380 >> 172.21.15.160:0/3083147380 conn(0x7f1048071460 msgr2=0x7f1048071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 -- 172.21.15.160:0/3083147380 shutdown_connections 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.250+0000 7f104d645700 1 -- 172.21.15.160:0/3083147380 wait complete. 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f104d645700 1 Processor -- start 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f104d645700 1 -- start start 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f104d645700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 0x7f1048113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f104d645700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10481140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f104d645700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048114740 con 0x7f1048102670 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f104d645700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104810db30 con 0x7f1048101cd0 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f103e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10481140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f103e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10481140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50222/0 (socket says 172.21.15.160:50222) 2024-05-25T06:29:33.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.251+0000 7f103e7fc700 1 -- 172.21.15.160:0/1243104847 learned_addr learned my addr 172.21.15.160:0/1243104847 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f1046ffd700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 0x7f1048113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103e7fc700 1 -- 172.21.15.160:0/1243104847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 msgr2=0x7f1048113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103e7fc700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 0x7f1048113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103e7fc700 1 -- 172.21.15.160:0/1243104847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030005040 con 0x7f1048102670 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f1046ffd700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 0x7f1048113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103e7fc700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10481140f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f103000b3b0 tx=0x7f1030009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103000e040 con 0x7f1048102670 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1030013420 con 0x7f1048102670 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104810ddb0 con 0x7f1048102670 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103001c630 con 0x7f1048102670 2024-05-25T06:29:33.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.252+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104810e370 con 0x7f1048102670 2024-05-25T06:29:33.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.253+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048065200 con 0x7f1048102670 2024-05-25T06:29:33.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.254+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1030013590 con 0x7f1048102670 2024-05-25T06:29:33.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.254+0000 7f103ffff700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f103406e140 0x7f10340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.254+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f103008ed40 con 0x7f1048102670 2024-05-25T06:29:33.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.254+0000 7f1046ffd700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f103406e140 0x7f10340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.255+0000 7f1046ffd700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f103406e140 0x7f10340705f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1038007fd0 tx=0x7f1038009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:33.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.259+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1030092030 con 0x7f1048102670 2024-05-25T06:29:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:33 smithi203 ceph-mon[83507]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:33 smithi160 ceph-mon[112094]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:33.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.474+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1048109ed0 con 0x7f103406e140 2024-05-25T06:29:33.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.475+0000 7f103ffff700 1 -- 172.21.15.160:0/1243104847 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1048109ed0 con 0x7f103406e140 2024-05-25T06:29:33.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.477+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f103406e140 msgr2=0x7f10340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.477+0000 7f104d645700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f103406e140 0x7f10340705f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1038007fd0 tx=0x7f1038009040 comp rx=0 tx=0).stop 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.477+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 msgr2=0x7f10481140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.477+0000 7f104d645700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10481140f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f103000b3b0 tx=0x7f1030009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 shutdown_connections 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f103406e140 0x7f10340705f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1048101cd0 0x7f1048113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 --2- 172.21.15.160:0/1243104847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1048102670 0x7f10481140f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 >> 172.21.15.160:0/1243104847 conn(0x7f1048071460 msgr2=0x7f10481009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 shutdown_connections 2024-05-25T06:29:33.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.478+0000 7f104d645700 1 -- 172.21.15.160:0/1243104847 wait complete. 2024-05-25T06:29:33.488 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:29:33.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/2601573528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f08102e00 msgr2=0x7f7f081031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/2601573528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f08102e00 0x7f7f081031d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004940 tx=0x7f7ef8009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/2601573528 shutdown_connections 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/2601573528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f081037a0 0x7f7f080fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/2601573528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f08102e00 0x7f7f081031d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/2601573528 >> 172.21.15.160:0/2601573528 conn(0x7f7f080f89b0 msgr2=0x7f7f080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/2601573528 shutdown_connections 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.581+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/2601573528 wait complete. 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f0d3d1700 1 Processor -- start 2024-05-25T06:29:33.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f0d3d1700 1 -- start start 2024-05-25T06:29:33.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f0d3d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f08102e00 0x7f7f08070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f0d3d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 0x7f7f080707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f0d3d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08072240 con 0x7f7f081037a0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f0d3d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08070ce0 con 0x7f7f08102e00 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 0x7f7f080707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 0x7f7f080707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50242/0 (socket says 172.21.15.160:50242) 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f067fc700 1 -- 172.21.15.160:0/1227702087 learned_addr learned my addr 172.21.15.160:0/1227702087 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f06ffd700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f08102e00 0x7f7f08070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f067fc700 1 -- 172.21.15.160:0/1227702087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f08102e00 msgr2=0x7f7f08070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f067fc700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f08102e00 0x7f7f08070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.582+0000 7f7f067fc700 1 -- 172.21.15.160:0/1227702087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef8005040 con 0x7f7f081037a0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.583+0000 7f7f067fc700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 0x7f7f080707a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b790 tx=0x7f7efc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.583+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc00f6f0 con 0x7f7f081037a0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.583+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7efc00fd20 con 0x7f7f081037a0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.583+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f08070fc0 con 0x7f7f081037a0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.583+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc0103a0 con 0x7f7f081037a0 2024-05-25T06:29:33.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.583+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0811a1f0 con 0x7f7f081037a0 2024-05-25T06:29:33.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.584+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f080fe550 con 0x7f7f081037a0 2024-05-25T06:29:33.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.584+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7efc011400 con 0x7f7f081037a0 2024-05-25T06:29:33.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.585+0000 7f7eef7fe700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef006e140 0x7f7ef00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.585+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7efc08d900 con 0x7f7f081037a0 2024-05-25T06:29:33.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.585+0000 7f7f06ffd700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef006e140 0x7f7ef00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.585+0000 7f7f06ffd700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef006e140 0x7f7ef00705f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004f90 tx=0x7f7ef8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:33.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.589+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7efc056cb0 con 0x7f7f081037a0 2024-05-25T06:29:33.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.805+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f08109fe0 con 0x7f7ef006e140 2024-05-25T06:29:33.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.806+0000 7f7eef7fe700 1 -- 172.21.15.160:0/1227702087 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f08109fe0 con 0x7f7ef006e140 2024-05-25T06:29:33.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef006e140 msgr2=0x7f7ef00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef006e140 0x7f7ef00705f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004f90 tx=0x7f7ef8009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 msgr2=0x7f7f080707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 0x7f7f080707a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b790 tx=0x7f7efc009220 comp rx=0 tx=0).stop 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 shutdown_connections 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef006e140 0x7f7ef00705f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f08102e00 0x7f7f08070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 --2- 172.21.15.160:0/1227702087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f081037a0 0x7f7f080707a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.808+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 >> 172.21.15.160:0/1227702087 conn(0x7f7f080f89b0 msgr2=0x7f7f080f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.809+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 shutdown_connections 2024-05-25T06:29:33.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.809+0000 7f7f0d3d1700 1 -- 172.21.15.160:0/1227702087 wait complete. 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 -- 172.21.15.160:0/4271872282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380103780 msgr2=0x7f23800fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 --2- 172.21.15.160:0/4271872282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380103780 0x7f23800fcdf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2374004940 tx=0x7f2374009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 -- 172.21.15.160:0/4271872282 shutdown_connections 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 --2- 172.21.15.160:0/4271872282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380103780 0x7f23800fcdf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 --2- 172.21.15.160:0/4271872282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380102de0 0x7f23801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 -- 172.21.15.160:0/4271872282 >> 172.21.15.160:0/4271872282 conn(0x7f23800f8990 msgr2=0x7f23800fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 -- 172.21.15.160:0/4271872282 shutdown_connections 2024-05-25T06:29:33.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 -- 172.21.15.160:0/4271872282 wait complete. 2024-05-25T06:29:33.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 Processor -- start 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.911+0000 7f238645d700 1 -- start start 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f238645d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 0x7f2380113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f238645d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380103780 0x7f2380113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f238645d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2380114520 con 0x7f2380102de0 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f238645d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238010db20 con 0x7f2380103780 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 0x7f2380113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 0x7f2380113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50258/0 (socket says 172.21.15.160:50258) 2024-05-25T06:29:33.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 -- 172.21.15.160:0/1465790760 learned_addr learned my addr 172.21.15.160:0/1465790760 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237f7fe700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380103780 0x7f2380113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 -- 172.21.15.160:0/1465790760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380103780 msgr2=0x7f2380113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380103780 0x7f2380113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 -- 172.21.15.160:0/1465790760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374005040 con 0x7f2380102de0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237ffff700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 0x7f2380113990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f237000c830 tx=0x7f2370004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23700106f0 con 0x7f2380102de0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238010de00 con 0x7f2380102de0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2370010d20 con 0x7f2380102de0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23700113a0 con 0x7f2380102de0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.912+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238010e390 con 0x7f2380102de0 2024-05-25T06:29:33.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.913+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23800fe530 con 0x7f2380102de0 2024-05-25T06:29:33.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.914+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2370012400 con 0x7f2380102de0 2024-05-25T06:29:33.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.914+0000 7f237cff9700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f236806e140 0x7f23680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:33.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.914+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f237008dac0 con 0x7f2380102de0 2024-05-25T06:29:33.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.914+0000 7f237f7fe700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f236806e140 0x7f23680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:33.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.915+0000 7f237f7fe700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f236806e140 0x7f23680705f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2374004890 tx=0x7f2374009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:33.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:33.919+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f237005a9a0 con 0x7f2380102de0 2024-05-25T06:29:34.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.125+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2380109ec0 con 0x7f236806e140 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.135+0000 7f237cff9700 1 -- 172.21.15.160:0/1465790760 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2380109ec0 con 0x7f236806e140 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (15m) 8m ago 26m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (12m) 8m ago 26m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (12m) 8m ago 25m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (15m) 8m ago 26m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (9m) 8m ago 21m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (8m) 8m ago 21m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:29:34.135 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (8m) 8m ago 21m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (8m) 8m ago 21m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (17m) 8m ago 27m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (16m) 8m ago 25m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (14m) 8m ago 28m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (13m) 8m ago 25m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (16m) 8m ago 26m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (16m) 8m ago 25m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (12m) 8m ago 24m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (11m) 8m ago 24m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (11m) 8m ago 23m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (10m) 8m ago 23m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (10m) 8m ago 23m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (9m) 8m ago 22m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:29:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (15m) 8m ago 26m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:29:34.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f236806e140 msgr2=0x7f23680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f236806e140 0x7f23680705f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2374004890 tx=0x7f2374009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:34.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 msgr2=0x7f2380113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 0x7f2380113990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f237000c830 tx=0x7f2370004470 comp rx=0 tx=0).stop 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 shutdown_connections 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380102de0 0x7f2380113990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f236806e140 0x7f23680705f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 --2- 172.21.15.160:0/1465790760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380103780 0x7f2380113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.137+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 >> 172.21.15.160:0/1465790760 conn(0x7f23800f8990 msgr2=0x7f23800f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.138+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 shutdown_connections 2024-05-25T06:29:34.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.138+0000 7f238645d700 1 -- 172.21.15.160:0/1465790760 wait complete. 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 -- 172.21.15.160:0/621843589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4103780 msgr2=0x7fa4e40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 --2- 172.21.15.160:0/621843589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4103780 0x7fa4e40fcdf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8004940 tx=0x7fa4d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 -- 172.21.15.160:0/621843589 shutdown_connections 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 --2- 172.21.15.160:0/621843589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4103780 0x7fa4e40fcdf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 --2- 172.21.15.160:0/621843589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4102de0 0x7fa4e41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 -- 172.21.15.160:0/621843589 >> 172.21.15.160:0/621843589 conn(0x7fa4e40f8990 msgr2=0x7fa4e40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:34.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 -- 172.21.15.160:0/621843589 shutdown_connections 2024-05-25T06:29:34.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 -- 172.21.15.160:0/621843589 wait complete. 2024-05-25T06:29:34.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.239+0000 7fa4ea640700 1 Processor -- start 2024-05-25T06:29:34.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4ea640700 1 -- start start 2024-05-25T06:29:34.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4ea640700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 0x7fa4e4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4ea640700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4103780 0x7fa4e4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4ea640700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4114520 con 0x7fa4e4102de0 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4ea640700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e410db20 con 0x7fa4e4103780 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 0x7fa4e4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 0x7fa4e4113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50282/0 (socket says 172.21.15.160:50282) 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 -- 172.21.15.160:0/1809869824 learned_addr learned my addr 172.21.15.160:0/1809869824 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e37fe700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4103780 0x7fa4e4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 -- 172.21.15.160:0/1809869824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4103780 msgr2=0x7fa4e4113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4103780 0x7fa4e4113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 -- 172.21.15.160:0/1809869824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8005040 con 0x7fa4e4102de0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e37fe700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4103780 0x7fa4e4113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.240+0000 7fa4e3fff700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 0x7fa4e4113990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4002a50 tx=0x7fa4d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.241+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d4007cc0 con 0x7fa4e4102de0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.241+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4d400f040 con 0x7fa4e4102de0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.241+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e410de00 con 0x7fa4e4102de0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.241+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d4009640 con 0x7fa4e4102de0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.241+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e410e3c0 con 0x7fa4e4102de0 2024-05-25T06:29:34.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.242+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e40fe530 con 0x7fa4e4102de0 2024-05-25T06:29:34.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.242+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa4d40097a0 con 0x7fa4e4102de0 2024-05-25T06:29:34.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.242+0000 7fa4e0ff9700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa4cc06e140 0x7fa4cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.243+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa4d408d990 con 0x7fa4e4102de0 2024-05-25T06:29:34.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.243+0000 7fa4e37fe700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa4cc06e140 0x7fa4cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.243+0000 7fa4e37fe700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa4cc06e140 0x7fa4cc0705f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800b3b0 tx=0x7fa4d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:34.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.247+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa4d405a7f0 con 0x7fa4e4102de0 2024-05-25T06:29:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:34 smithi203 ceph-mon[83507]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:34 smithi203 ceph-mon[83507]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:34 smithi160 ceph-mon[112094]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:34 smithi160 ceph-mon[112094]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:34.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.528+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa4e4109ec0 con 0x7fa4e4102de0 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.529+0000 7fa4e0ff9700 1 -- 172.21.15.160:0/1809869824 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fa4d405a380 con 0x7fa4e4102de0 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:29:34.529 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:29:34.530 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:29:34.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa4cc06e140 msgr2=0x7fa4cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa4cc06e140 0x7fa4cc0705f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800b3b0 tx=0x7fa4d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:34.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 msgr2=0x7fa4e4113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 0x7fa4e4113990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4002a50 tx=0x7fa4d40046b0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 shutdown_connections 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4e4102de0 0x7fa4e4113990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa4cc06e140 0x7fa4cc0705f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 --2- 172.21.15.160:0/1809869824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4e4103780 0x7fa4e4113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 >> 172.21.15.160:0/1809869824 conn(0x7fa4e40f8990 msgr2=0x7fa4e40f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 shutdown_connections 2024-05-25T06:29:34.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.531+0000 7fa4ea640700 1 -- 172.21.15.160:0/1809869824 wait complete. 2024-05-25T06:29:34.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 -- 172.21.15.160:0/1265146683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 msgr2=0x7f91a4070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/1265146683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4070cb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f918c004970 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 -- 172.21.15.160:0/1265146683 shutdown_connections 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/1265146683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4070cb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/1265146683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91a4071f30 0x7f91a4070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 -- 172.21.15.160:0/1265146683 >> 172.21.15.160:0/1265146683 conn(0x7f91a40f8990 msgr2=0x7f91a40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 -- 172.21.15.160:0/1265146683 shutdown_connections 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 -- 172.21.15.160:0/1265146683 wait complete. 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.633+0000 7f91a9ba3700 1 Processor -- start 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a9ba3700 1 -- start start 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a9ba3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a9ba3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91a4071f30 0x7f91a411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a9ba3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a411aba0 con 0x7f91a4070840 2024-05-25T06:29:34.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a9ba3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a41140f0 con 0x7f91a4071f30 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50290/0 (socket says 172.21.15.160:50290) 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 -- 172.21.15.160:0/3646669337 learned_addr learned my addr 172.21.15.160:0/3646669337 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f919bfff700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91a4071f30 0x7f91a411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 -- 172.21.15.160:0/3646669337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91a4071f30 msgr2=0x7f91a411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91a4071f30 0x7f91a411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 -- 172.21.15.160:0/3646669337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c005040 con 0x7f91a4070840 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a37fe700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4119f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f919400b790 tx=0x7f9194009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f919400f770 con 0x7f91a4070840 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.634+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f919400fda0 con 0x7f91a4070840 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.635+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a4114370 con 0x7f91a4070840 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.635+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f919400d310 con 0x7f91a4070840 2024-05-25T06:29:34.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.635+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a4114880 con 0x7f91a4070840 2024-05-25T06:29:34.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.635+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a4071a40 con 0x7f91a4070840 2024-05-25T06:29:34.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.636+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9194010420 con 0x7f91a4070840 2024-05-25T06:29:34.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.636+0000 7f91a0ff9700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f919006e140 0x7f91900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.637+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f919408d930 con 0x7f91a4070840 2024-05-25T06:29:34.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.637+0000 7f919bfff700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f919006e140 0x7f91900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.637+0000 7f919bfff700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f919006e140 0x7f91900705f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f918c004f50 tx=0x7f918c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:34.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.641+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f919405a810 con 0x7f91a4070840 2024-05-25T06:29:34.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.880+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f91a4115670 con 0x7f91a4070840 2024-05-25T06:29:34.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.881+0000 7f91a0ff9700 1 -- 172.21.15.160:0/3646669337 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f919405a3a0 con 0x7f91a4070840 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:29:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:29:34.884 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:29:34.885 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:29:34.886 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:34.886 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:34.886 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:34.886 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f919006e140 msgr2=0x7f91900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f919006e140 0x7f91900705f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f918c004f50 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 msgr2=0x7f91a4119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4119f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f919400b790 tx=0x7f9194009250 comp rx=0 tx=0).stop 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 shutdown_connections 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a4070840 0x7f91a4119f80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f919006e140 0x7f91900705f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 --2- 172.21.15.160:0/3646669337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91a4071f30 0x7f91a411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 >> 172.21.15.160:0/3646669337 conn(0x7f91a40f8990 msgr2=0x7f91a40f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 shutdown_connections 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.883+0000 7f91a9ba3700 1 -- 172.21.15.160:0/3646669337 wait complete. 2024-05-25T06:29:34.887 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:29:34.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.987+0000 7f395f32f700 1 -- 172.21.15.160:0/4010006150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3958102e20 msgr2=0x7f39581031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.987+0000 7f395f32f700 1 --2- 172.21.15.160:0/4010006150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3958102e20 0x7f39581031f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3954004940 tx=0x7f3954009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- 172.21.15.160:0/4010006150 shutdown_connections 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 --2- 172.21.15.160:0/4010006150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39581037c0 0x7f39580fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 --2- 172.21.15.160:0/4010006150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3958102e20 0x7f39581031f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- 172.21.15.160:0/4010006150 >> 172.21.15.160:0/4010006150 conn(0x7f3958072380 msgr2=0x7f3958072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- 172.21.15.160:0/4010006150 shutdown_connections 2024-05-25T06:29:34.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- 172.21.15.160:0/4010006150 wait complete. 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 Processor -- start 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- start start 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3958102e20 0x7f39581139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 0x7f3958113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3958114580 con 0x7f39581037c0 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.988+0000 7f395f32f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395810db80 con 0x7f3958102e20 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 0x7f3958113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 0x7f3958113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50298/0 (socket says 172.21.15.160:50298) 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 -- 172.21.15.160:0/4144583663 learned_addr learned my addr 172.21.15.160:0/4144583663 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395d0cb700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3958102e20 0x7f39581139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 -- 172.21.15.160:0/4144583663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3958102e20 msgr2=0x7f39581139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:34.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3958102e20 0x7f39581139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 -- 172.21.15.160:0/4144583663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954005040 con 0x7f39581037c0 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395c8ca700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 0x7f3958113f30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f394800c890 tx=0x7f39480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f394800a700 con 0x7f39581037c0 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f394800ad30 con 0x7f39581037c0 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39480113a0 con 0x7f39581037c0 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.989+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395810de60 con 0x7f39581037c0 2024-05-25T06:29:34.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.990+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395810e3a0 con 0x7f39581037c0 2024-05-25T06:29:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.991+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3958100160 con 0x7f39581037c0 2024-05-25T06:29:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.991+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3948012400 con 0x7f39581037c0 2024-05-25T06:29:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.992+0000 7f394dffb700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f394406e140 0x7f39440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.992+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f394808db00 con 0x7f39581037c0 2024-05-25T06:29:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.992+0000 7f395d0cb700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f394406e140 0x7f39440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:34.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.992+0000 7f395d0cb700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f394406e140 0x7f39440705f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3954004890 tx=0x7f3954009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:34.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:34.996+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f394805aaa0 con 0x7f39581037c0 2024-05-25T06:29:35.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.215+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f395810efd0 con 0x7f394406e140 2024-05-25T06:29:35.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:35 smithi160 ceph-mon[112094]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:35.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:35 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1809869824' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:29:35.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:35 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3646669337' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:29:35.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:35 smithi160 ceph-mon[112094]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.216+0000 7f394dffb700 1 -- 172.21.15.160:0/4144583663 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f395810efd0 con 0x7f394406e140 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:29:35.216 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:29:35.217 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:29:35.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f394406e140 msgr2=0x7f39440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f394406e140 0x7f39440705f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3954004890 tx=0x7f3954009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 msgr2=0x7f3958113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 0x7f3958113f30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f394800c890 tx=0x7f39480044c0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 shutdown_connections 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f394406e140 0x7f39440705f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3958102e20 0x7f39581139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 --2- 172.21.15.160:0/4144583663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39581037c0 0x7f3958113f30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 >> 172.21.15.160:0/4144583663 conn(0x7f3958072380 msgr2=0x7f39580facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 shutdown_connections 2024-05-25T06:29:35.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.219+0000 7f395f32f700 1 -- 172.21.15.160:0/4144583663 wait complete. 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 -- 172.21.15.160:0/2167439138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 msgr2=0x7f59800fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 --2- 172.21.15.160:0/2167439138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f59800fc1b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5974004940 tx=0x7f5974009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 -- 172.21.15.160:0/2167439138 shutdown_connections 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 --2- 172.21.15.160:0/2167439138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f59800fc1b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 --2- 172.21.15.160:0/2167439138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5980101d90 0x7f5980102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 -- 172.21.15.160:0/2167439138 >> 172.21.15.160:0/2167439138 conn(0x7f5980071d00 msgr2=0x7f5980072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 -- 172.21.15.160:0/2167439138 shutdown_connections 2024-05-25T06:29:35.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 -- 172.21.15.160:0/2167439138 wait complete. 2024-05-25T06:29:35.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 Processor -- start 2024-05-25T06:29:35.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.322+0000 7f5987f23700 1 -- start start 2024-05-25T06:29:35.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5987f23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f5980112a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:35.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5987f23700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5980101d90 0x7f5980112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:35.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5987f23700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980113620 con 0x7f59800fbd40 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5987f23700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598010ca10 con 0x7f5980101d90 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f5980112a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f5980112a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:50316/0 (socket says 172.21.15.160:50316) 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 -- 172.21.15.160:0/4176899667 learned_addr learned my addr 172.21.15.160:0/4176899667 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f59854be700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5980101d90 0x7f5980112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 -- 172.21.15.160:0/4176899667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5980101d90 msgr2=0x7f5980112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5980101d90 0x7f5980112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 -- 172.21.15.160:0/4176899667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974005040 con 0x7f59800fbd40 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5985cbf700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f5980112a90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5978001d80 tx=0x7f5978009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5978007d10 con 0x7f59800fbd40 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5978007e70 con 0x7f59800fbd40 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598010ccf0 con 0x7f59800fbd40 2024-05-25T06:29:35.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5978014640 con 0x7f59800fbd40 2024-05-25T06:29:35.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.323+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598010d2b0 con 0x7f59800fbd40 2024-05-25T06:29:35.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.325+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59800fd430 con 0x7f59800fbd40 2024-05-25T06:29:35.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.325+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f597800e960 con 0x7f59800fbd40 2024-05-25T06:29:35.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.325+0000 7f597e7fc700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f596806e140 0x7f59680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:29:35.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.325+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f597808d960 con 0x7f59800fbd40 2024-05-25T06:29:35.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.325+0000 7f59854be700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f596806e140 0x7f59680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:29:35.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.326+0000 7f59854be700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f596806e140 0x7f59680705f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f597400b3b0 tx=0x7f5974009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:29:35.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.330+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f597805a870 con 0x7f59800fbd40 2024-05-25T06:29:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:35 smithi203 ceph-mon[83507]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:35 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1809869824' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:29:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:35 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3646669337' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:29:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:35 smithi203 ceph-mon[83507]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:35.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.609+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5980108df0 con 0x7f59800fbd40 2024-05-25T06:29:35.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.610+0000 7f597e7fc700 1 -- 172.21.15.160:0/4176899667 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f597805a400 con 0x7f59800fbd40 2024-05-25T06:29:35.610 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:29:35.610 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:29:35.610 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:29:35.610 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:29:35.610 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:29:35.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f596806e140 msgr2=0x7f59680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:35.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f596806e140 0x7f59680705f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f597400b3b0 tx=0x7f5974009e30 comp rx=0 tx=0).stop 2024-05-25T06:29:35.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 msgr2=0x7f5980112a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:29:35.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f5980112a90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5978001d80 tx=0x7f5978009ec0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 shutdown_connections 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59800fbd40 0x7f5980112a90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f596806e140 0x7f59680705f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 --2- 172.21.15.160:0/4176899667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5980101d90 0x7f5980112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 >> 172.21.15.160:0/4176899667 conn(0x7f5980071d00 msgr2=0x7f5980104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 shutdown_connections 2024-05-25T06:29:35.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:29:35.612+0000 7f5987f23700 1 -- 172.21.15.160:0/4176899667 wait complete. 2024-05-25T06:29:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:36 smithi203 ceph-mon[83507]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:36 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4176899667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:29:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:36 smithi160 ceph-mon[112094]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:29:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:36 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4176899667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:29:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:37 smithi203 ceph-mon[83507]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:37 smithi160 ceph-mon[112094]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:39 smithi203 ceph-mon[83507]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:39 smithi160 ceph-mon[112094]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:41 smithi203 ceph-mon[83507]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:41 smithi160 ceph-mon[112094]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:43 smithi203 ceph-mon[83507]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:43 smithi160 ceph-mon[112094]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:45 smithi203 ceph-mon[83507]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:45 smithi160 ceph-mon[112094]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:47 smithi203 ceph-mon[83507]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:47 smithi160 ceph-mon[112094]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:49 smithi203 ceph-mon[83507]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:49 smithi160 ceph-mon[112094]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:51 smithi203 ceph-mon[83507]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:51 smithi160 ceph-mon[112094]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:53 smithi203 ceph-mon[83507]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:53 smithi160 ceph-mon[112094]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:55 smithi203 ceph-mon[83507]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:55 smithi160 ceph-mon[112094]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:57 smithi203 ceph-mon[83507]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:57 smithi160 ceph-mon[112094]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:58.221 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:29:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:29:59 smithi203 ceph-mon[83507]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:29:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:29:59 smithi160 ceph-mon[112094]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:00 smithi203 ceph-mon[83507]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:30:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:00 smithi203 ceph-mon[83507]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:30:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:00 smithi203 ceph-mon[83507]: fs cephfs is degraded 2024-05-25T06:30:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:00 smithi203 ceph-mon[83507]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:30:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:00 smithi203 ceph-mon[83507]: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:29:59.999+0000 7fd61c060700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:29:59.999+0000 7fd61c060700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:29:59.999+0000 7fd61c060700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:29:59.999+0000 7fd61c060700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:29:59.999+0000 7fd61c060700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-mon[112094]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-mon[112094]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:30:00.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-mon[112094]: fs cephfs is degraded 2024-05-25T06:30:00.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-mon[112094]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:30:00.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:00 smithi160 ceph-mon[112094]: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:30:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:01 smithi203 ceph-mon[83507]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:01 smithi160 ceph-mon[112094]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:03 smithi203 ceph-mon[83507]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:03 smithi160 ceph-mon[112094]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:05 smithi203 ceph-mon[83507]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:05 smithi160 ceph-mon[112094]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.727+0000 7f57264aa700 1 -- 172.21.15.160:0/750361234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 msgr2=0x7f57201031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.727+0000 7f57264aa700 1 --2- 172.21.15.160:0/750361234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f57201031c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5710004940 tx=0x7f5710009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.727+0000 7f57264aa700 1 -- 172.21.15.160:0/750361234 shutdown_connections 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.727+0000 7f57264aa700 1 --2- 172.21.15.160:0/750361234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5720103790 0x7f57200fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.727+0000 7f57264aa700 1 --2- 172.21.15.160:0/750361234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f57201031c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.727+0000 7f57264aa700 1 -- 172.21.15.160:0/750361234 >> 172.21.15.160:0/750361234 conn(0x7f57200f8980 msgr2=0x7f57200fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.728+0000 7f57264aa700 1 -- 172.21.15.160:0/750361234 shutdown_connections 2024-05-25T06:30:05.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.728+0000 7f57264aa700 1 -- 172.21.15.160:0/750361234 wait complete. 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.728+0000 7f57264aa700 1 Processor -- start 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f57264aa700 1 -- start start 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f57264aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f5720113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f57264aa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5720103790 0x7f57201140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f57264aa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57201147a0 con 0x7f5720102df0 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f57264aa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572010db00 con 0x7f5720103790 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f5720113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f5720113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54354/0 (socket says 172.21.15.160:54354) 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 -- 172.21.15.160:0/860621710 learned_addr learned my addr 172.21.15.160:0/860621710 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:05.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571f7fe700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5720103790 0x7f57201140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 -- 172.21.15.160:0/860621710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5720103790 msgr2=0x7f57201140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5720103790 0x7f57201140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 -- 172.21.15.160:0/860621710 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5710005040 con 0x7f5720102df0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.729+0000 7f571ffff700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f5720113b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f571000b3b0 tx=0x7f5710009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.730+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571000e040 con 0x7f5720102df0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.730+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5710013420 con 0x7f5720102df0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.730+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572010dd80 con 0x7f5720102df0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.730+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571001c630 con 0x7f5720102df0 2024-05-25T06:30:05.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.730+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572010e340 con 0x7f5720102df0 2024-05-25T06:30:05.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.731+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57200fe540 con 0x7f5720102df0 2024-05-25T06:30:05.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.731+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5710013590 con 0x7f5720102df0 2024-05-25T06:30:05.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.732+0000 7f571cff9700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570c06e140 0x7f570c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:05.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.732+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f571008ec70 con 0x7f5720102df0 2024-05-25T06:30:05.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.732+0000 7f571f7fe700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570c06e140 0x7f570c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:05.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.732+0000 7f571f7fe700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570c06e140 0x7f570c0705f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f572010ecc0 tx=0x7f57080091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:05.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.737+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f571005bb50 con 0x7f5720102df0 2024-05-25T06:30:05.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.956+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5720109fd0 con 0x7f570c06e140 2024-05-25T06:30:05.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.957+0000 7f571cff9700 1 -- 172.21.15.160:0/860621710 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5720109fd0 con 0x7f570c06e140 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570c06e140 msgr2=0x7f570c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570c06e140 0x7f570c0705f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f572010ecc0 tx=0x7f57080091a0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 msgr2=0x7f5720113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f5720113b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f571000b3b0 tx=0x7f5710009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 shutdown_connections 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5720102df0 0x7f5720113b80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570c06e140 0x7f570c0705f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 --2- 172.21.15.160:0/860621710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5720103790 0x7f57201140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 >> 172.21.15.160:0/860621710 conn(0x7f57200f8980 msgr2=0x7f57200f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 shutdown_connections 2024-05-25T06:30:05.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:05.960+0000 7f57264aa700 1 -- 172.21.15.160:0/860621710 wait complete. 2024-05-25T06:30:05.971 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 -- 172.21.15.160:0/3684983247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20103760 msgr2=0x7f7d200fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/3684983247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20103760 0x7f7d200fcdd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d08004940 tx=0x7f7d08009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 -- 172.21.15.160:0/3684983247 shutdown_connections 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/3684983247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20103760 0x7f7d200fcdd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/3684983247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20102dc0 0x7f7d20103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 -- 172.21.15.160:0/3684983247 >> 172.21.15.160:0/3684983247 conn(0x7f7d200f8930 msgr2=0x7f7d200fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.064+0000 7f7d25ca4700 1 -- 172.21.15.160:0/3684983247 shutdown_connections 2024-05-25T06:30:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 -- 172.21.15.160:0/3684983247 wait complete. 2024-05-25T06:30:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 Processor -- start 2024-05-25T06:30:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 -- start start 2024-05-25T06:30:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 0x7f7d20113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20103760 0x7f7d20113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d201144e0 con 0x7f7d20102dc0 2024-05-25T06:30:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.065+0000 7f7d25ca4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2010dae0 con 0x7f7d20103760 2024-05-25T06:30:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 0x7f7d20113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 0x7f7d20113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54386/0 (socket says 172.21.15.160:54386) 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 -- 172.21.15.160:0/1166190051 learned_addr learned my addr 172.21.15.160:0/1166190051 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d17fff700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20103760 0x7f7d20113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 -- 172.21.15.160:0/1166190051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20103760 msgr2=0x7f7d20113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20103760 0x7f7d20113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 -- 172.21.15.160:0/1166190051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08005040 con 0x7f7d20102dc0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d17fff700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20103760 0x7f7d20113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1f7fe700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 0x7f7d20113950 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d10002a50 tx=0x7f7d100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d10007cc0 con 0x7f7d20102dc0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2010ddc0 con 0x7f7d20102dc0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.066+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d1000f040 con 0x7f7d20102dc0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.067+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d10009640 con 0x7f7d20102dc0 2024-05-25T06:30:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.067+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2010e380 con 0x7f7d20102dc0 2024-05-25T06:30:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.068+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d200fe510 con 0x7f7d20102dc0 2024-05-25T06:30:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.068+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7d100097a0 con 0x7f7d20102dc0 2024-05-25T06:30:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.068+0000 7f7d1cff9700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d0c06df30 0x7f7d0c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.069+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7d1008d700 con 0x7f7d20102dc0 2024-05-25T06:30:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.069+0000 7f7d17fff700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d0c06df30 0x7f7d0c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.069+0000 7f7d17fff700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d0c06df30 0x7f7d0c0703e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d2010eca0 tx=0x7f7d08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:06.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.073+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7d1005a560 con 0x7f7d20102dc0 2024-05-25T06:30:06.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.290+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d2010efa0 con 0x7f7d0c06df30 2024-05-25T06:30:06.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.291+0000 7f7d1cff9700 1 -- 172.21.15.160:0/1166190051 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7d2010efa0 con 0x7f7d0c06df30 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.293+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d0c06df30 msgr2=0x7f7d0c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.293+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d0c06df30 0x7f7d0c0703e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d2010eca0 tx=0x7f7d08009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 msgr2=0x7f7d20113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 0x7f7d20113950 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d10002a50 tx=0x7f7d100046b0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 shutdown_connections 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20102dc0 0x7f7d20113950 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d0c06df30 0x7f7d0c0703e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 --2- 172.21.15.160:0/1166190051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d20103760 0x7f7d20113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 >> 172.21.15.160:0/1166190051 conn(0x7f7d200f8930 msgr2=0x7f7d200f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:06.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 shutdown_connections 2024-05-25T06:30:06.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.294+0000 7f7d25ca4700 1 -- 172.21.15.160:0/1166190051 wait complete. 2024-05-25T06:30:06.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 -- 172.21.15.160:0/751991032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 msgr2=0x7f72dc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 --2- 172.21.15.160:0/751991032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc103330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f72d0004940 tx=0x7f72d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:06.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 -- 172.21.15.160:0/751991032 shutdown_connections 2024-05-25T06:30:06.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 --2- 172.21.15.160:0/751991032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0fb520 0x7f72dc0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 --2- 172.21.15.160:0/751991032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc103330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 -- 172.21.15.160:0/751991032 >> 172.21.15.160:0/751991032 conn(0x7f72dc0714d0 msgr2=0x7f72dc0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:06.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.396+0000 7f72e2afa700 1 -- 172.21.15.160:0/751991032 shutdown_connections 2024-05-25T06:30:06.398 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:06 smithi160 ceph-mon[112094]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:06.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.397+0000 7f72e2afa700 1 -- 172.21.15.160:0/751991032 wait complete. 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.397+0000 7f72e2afa700 1 Processor -- start 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.397+0000 7f72e2afa700 1 -- start start 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e2afa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0fb520 0x7f72dc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e2afa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e2afa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc116720 con 0x7f72dc102f60 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e2afa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc10fd00 con 0x7f72dc0fb520 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54410/0 (socket says 172.21.15.160:54410) 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 -- 172.21.15.160:0/427921244 learned_addr learned my addr 172.21.15.160:0/427921244 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e0896700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0fb520 0x7f72dc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 -- 172.21.15.160:0/427921244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0fb520 msgr2=0x7f72dc115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0fb520 0x7f72dc115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 -- 172.21.15.160:0/427921244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0005040 con 0x7f72dc102f60 2024-05-25T06:30:06.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72dbfff700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc1160d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c8c0 tx=0x7f72c80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:06.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c800a760 con 0x7f72dc102f60 2024-05-25T06:30:06.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc10ffe0 con 0x7f72dc102f60 2024-05-25T06:30:06.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72c800ad90 con 0x7f72dc102f60 2024-05-25T06:30:06.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c80113a0 con 0x7f72dc102f60 2024-05-25T06:30:06.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.398+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc110520 con 0x7f72dc102f60 2024-05-25T06:30:06.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:06 smithi203 ceph-mon[83507]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:06.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.399+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc107410 con 0x7f72dc102f60 2024-05-25T06:30:06.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.400+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f72c8012400 con 0x7f72dc102f60 2024-05-25T06:30:06.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.400+0000 7f72d97fa700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f72cc06e140 0x7f72cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.400+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f72c808db60 con 0x7f72dc102f60 2024-05-25T06:30:06.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.400+0000 7f72e0896700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f72cc06e140 0x7f72cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.401+0000 7f72e0896700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f72cc06e140 0x7f72cc0705f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f72d0004f90 tx=0x7f72d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:06.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.406+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f72c805aa40 con 0x7f72dc102f60 2024-05-25T06:30:06.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.612+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f72dc1111e0 con 0x7f72cc06e140 2024-05-25T06:30:06.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.622+0000 7f72d97fa700 1 -- 172.21.15.160:0/427921244 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f72dc1111e0 con 0x7f72cc06e140 2024-05-25T06:30:06.623 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:30:06.623 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (15m) 9m ago 27m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:30:06.623 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (13m) 9m ago 27m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:30:06.623 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (13m) 8m ago 26m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:30:06.623 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (15m) 9m ago 26m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (9m) 9m ago 21m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (9m) 9m ago 21m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (8m) 8m ago 21m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (9m) 8m ago 21m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (17m) 9m ago 28m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (16m) 8m ago 26m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (14m) 9m ago 28m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (14m) 8m ago 26m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (16m) 9m ago 27m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (16m) 8m ago 26m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (12m) 9m ago 25m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:30:06.624 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (12m) 9m ago 24m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:30:06.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (11m) 9m ago 24m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:30:06.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (11m) 8m ago 24m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:30:06.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (10m) 8m ago 23m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:30:06.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (10m) 8m ago 23m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:30:06.625 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (16m) 9m ago 26m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f72cc06e140 msgr2=0x7f72cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f72cc06e140 0x7f72cc0705f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f72d0004f90 tx=0x7f72d0009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 msgr2=0x7f72dc1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc1160d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c8c0 tx=0x7f72c80044c0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 shutdown_connections 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f72cc06e140 0x7f72cc0705f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0fb520 0x7f72dc115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 --2- 172.21.15.160:0/427921244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72dc102f60 0x7f72dc1160d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 >> 172.21.15.160:0/427921244 conn(0x7f72dc0714d0 msgr2=0x7f72dc0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:06.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 shutdown_connections 2024-05-25T06:30:06.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.625+0000 7f72e2afa700 1 -- 172.21.15.160:0/427921244 wait complete. 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 -- 172.21.15.160:0/1380570064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 msgr2=0x7f1734106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 --2- 172.21.15.160:0/1380570064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734106ea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1724004940 tx=0x7f1724009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 -- 172.21.15.160:0/1380570064 shutdown_connections 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 --2- 172.21.15.160:0/1380570064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734106ea0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 --2- 172.21.15.160:0/1380570064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17340fde30 0x7f17340fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 -- 172.21.15.160:0/1380570064 >> 172.21.15.160:0/1380570064 conn(0x7f17340f7870 msgr2=0x7f17340f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 -- 172.21.15.160:0/1380570064 shutdown_connections 2024-05-25T06:30:06.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 -- 172.21.15.160:0/1380570064 wait complete. 2024-05-25T06:30:06.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 Processor -- start 2024-05-25T06:30:06.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.726+0000 7f173c052700 1 -- start start 2024-05-25T06:30:06.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f173c052700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17340fde30 0x7f17341139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f173c052700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f173c052700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1734114580 con 0x7f17340fe7d0 2024-05-25T06:30:06.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f173c052700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173410db80 con 0x7f17340fde30 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54424/0 (socket says 172.21.15.160:54424) 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 -- 172.21.15.160:0/3243402546 learned_addr learned my addr 172.21.15.160:0/3243402546 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f1739dee700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17340fde30 0x7f17341139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 -- 172.21.15.160:0/3243402546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17340fde30 msgr2=0x7f17341139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17340fde30 0x7f17341139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 -- 172.21.15.160:0/3243402546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724005040 con 0x7f17340fe7d0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f17395ed700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734113f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f172400b3b0 tx=0x7f1724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172400e040 con 0x7f17340fe7d0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1724013420 con 0x7f17340fe7d0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.727+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172401c5f0 con 0x7f17340fe7d0 2024-05-25T06:30:06.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.728+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173410de00 con 0x7f17340fe7d0 2024-05-25T06:30:06.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.728+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173410e340 con 0x7f17340fe7d0 2024-05-25T06:30:06.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.728+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1734066a70 con 0x7f17340fe7d0 2024-05-25T06:30:06.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.729+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1724013590 con 0x7f17340fe7d0 2024-05-25T06:30:06.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.729+0000 7f172a7fc700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f172006e140 0x7f17200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:06.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.730+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f172408eda0 con 0x7f17340fe7d0 2024-05-25T06:30:06.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.730+0000 7f1739dee700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f172006e140 0x7f17200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:06.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.730+0000 7f1739dee700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f172006e140 0x7f17200705f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1730004f60 tx=0x7f173000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:06.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:06.735+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f172405bc00 con 0x7f17340fe7d0 2024-05-25T06:30:07.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.012+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1734109ed0 con 0x7f17340fe7d0 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.013+0000 7f172a7fc700 1 -- 172.21.15.160:0/3243402546 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f172405b790 con 0x7f17340fe7d0 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:07.014 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:30:07.015 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f172006e140 msgr2=0x7f17200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f172006e140 0x7f17200705f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1730004f60 tx=0x7f173000b040 comp rx=0 tx=0).stop 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 msgr2=0x7f1734113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734113f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f172400b3b0 tx=0x7f1724009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 shutdown_connections 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f172006e140 0x7f17200705f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17340fde30 0x7f17341139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 --2- 172.21.15.160:0/3243402546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17340fe7d0 0x7f1734113f30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 >> 172.21.15.160:0/3243402546 conn(0x7f17340f7870 msgr2=0x7f17340f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:07.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 shutdown_connections 2024-05-25T06:30:07.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.015+0000 7f173c052700 1 -- 172.21.15.160:0/3243402546 wait complete. 2024-05-25T06:30:07.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 -- 172.21.15.160:0/1213622346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 msgr2=0x7f5160102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 --2- 172.21.15.160:0/1213622346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f5160102160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f515c004940 tx=0x7f515c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:07.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 -- 172.21.15.160:0/1213622346 shutdown_connections 2024-05-25T06:30:07.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 --2- 172.21.15.160:0/1213622346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51600fbd40 0x7f51600fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 --2- 172.21.15.160:0/1213622346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f5160102160 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 -- 172.21.15.160:0/1213622346 >> 172.21.15.160:0/1213622346 conn(0x7f5160071d00 msgr2=0x7f5160072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:07.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 -- 172.21.15.160:0/1213622346 shutdown_connections 2024-05-25T06:30:07.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.117+0000 7f5167b4d700 1 -- 172.21.15.160:0/1213622346 wait complete. 2024-05-25T06:30:07.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f5167b4d700 1 Processor -- start 2024-05-25T06:30:07.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f5167b4d700 1 -- start start 2024-05-25T06:30:07.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f5167b4d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51600fbd40 0x7f516010eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f5167b4d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f516010f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f5167b4d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516010f740 con 0x7f5160101d90 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f5167b4d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516010f880 con 0x7f51600fbd40 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f516010f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f516010f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54454/0 (socket says 172.21.15.160:54454) 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 -- 172.21.15.160:0/3088816501 learned_addr learned my addr 172.21.15.160:0/3088816501 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51658e9700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51600fbd40 0x7f516010eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 -- 172.21.15.160:0/3088816501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51600fbd40 msgr2=0x7f516010eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51600fbd40 0x7f516010eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 -- 172.21.15.160:0/3088816501 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c005040 con 0x7f5160101d90 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.119+0000 7f51650e8700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f516010f060 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f515400b790 tx=0x7f5154009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:07.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.120+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515400f770 con 0x7f5160101d90 2024-05-25T06:30:07.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.120+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f515400fda0 con 0x7f5160101d90 2024-05-25T06:30:07.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.120+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5160117bf0 con 0x7f5160101d90 2024-05-25T06:30:07.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.120+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515400d310 con 0x7f5160101d90 2024-05-25T06:30:07.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.120+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5160118180 con 0x7f5160101d90 2024-05-25T06:30:07.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.121+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51600fd430 con 0x7f5160101d90 2024-05-25T06:30:07.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.121+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5154010420 con 0x7f5160101d90 2024-05-25T06:30:07.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.122+0000 7f51527fc700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f514c06e140 0x7f514c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.122+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f515408da60 con 0x7f5160101d90 2024-05-25T06:30:07.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.122+0000 7f51658e9700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f514c06e140 0x7f514c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.122+0000 7f51658e9700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f514c06e140 0x7f514c0705f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f515c000c00 tx=0x7f515c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:07.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.126+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5154056ed0 con 0x7f5160101d90 2024-05-25T06:30:07.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.368+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5160108ed0 con 0x7f5160101d90 2024-05-25T06:30:07.369 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:07 smithi160 ceph-mon[112094]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:07.369 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:07 smithi160 ceph-mon[112094]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:07.369 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:07 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3243402546' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:30:07.369 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:07 smithi160 ceph-mon[112094]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:07.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.369+0000 7f51527fc700 1 -- 172.21.15.160:0/3088816501 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f5154061020 con 0x7f5160101d90 2024-05-25T06:30:07.369 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:30:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:30:07.371 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:07.372 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:07.373 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:07.373 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f514c06e140 msgr2=0x7f514c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f514c06e140 0x7f514c0705f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f515c000c00 tx=0x7f515c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 msgr2=0x7f516010f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f516010f060 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f515400b790 tx=0x7f5154009250 comp rx=0 tx=0).stop 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 shutdown_connections 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f514c06e140 0x7f514c0705f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51600fbd40 0x7f516010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 --2- 172.21.15.160:0/3088816501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5160101d90 0x7f516010f060 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 >> 172.21.15.160:0/3088816501 conn(0x7f5160071d00 msgr2=0x7f51600fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 shutdown_connections 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.371+0000 7f5167b4d700 1 -- 172.21.15.160:0/3088816501 wait complete. 2024-05-25T06:30:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:30:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:07 smithi203 ceph-mon[83507]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:07 smithi203 ceph-mon[83507]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:07 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3243402546' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:30:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:07 smithi203 ceph-mon[83507]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:07.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 -- 172.21.15.160:0/529309671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 msgr2=0x7f54400fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 --2- 172.21.15.160:0/529309671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f54400fbfa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f543c004940 tx=0x7f543c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:07.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 -- 172.21.15.160:0/529309671 shutdown_connections 2024-05-25T06:30:07.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 --2- 172.21.15.160:0/529309671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 0x7f5440109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 --2- 172.21.15.160:0/529309671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f54400fbfa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 -- 172.21.15.160:0/529309671 >> 172.21.15.160:0/529309671 conn(0x7f54400f7860 msgr2=0x7f54400f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 -- 172.21.15.160:0/529309671 shutdown_connections 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 -- 172.21.15.160:0/529309671 wait complete. 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 Processor -- start 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.474+0000 7f5447d03700 1 -- start start 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5447d03700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f5440115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5447d03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 0x7f5440115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5447d03700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5440116660 con 0x7f54400fbbd0 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5447d03700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544010fbf0 con 0x7f54400fc4e0 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f5440115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f5440115a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54476/0 (socket says 172.21.15.160:54476) 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 -- 172.21.15.160:0/2116756145 learned_addr learned my addr 172.21.15.160:0/2116756145 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:07.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f544529e700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 0x7f5440115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 -- 172.21.15.160:0/2116756145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 msgr2=0x7f5440115f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 0x7f5440115f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 -- 172.21.15.160:0/2116756145 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c005040 con 0x7f54400fbbd0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f544529e700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 0x7f5440115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5445a9f700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f5440115a40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f543c009380 tx=0x7f543c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f543c00e040 con 0x7f54400fbbd0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f543c009af0 con 0x7f54400fbbd0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f543c01c630 con 0x7f54400fbbd0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544010fed0 con 0x7f54400fbbd0 2024-05-25T06:30:07.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.475+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5440110490 con 0x7f54400fbbd0 2024-05-25T06:30:07.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.476+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5440066af0 con 0x7f54400fbbd0 2024-05-25T06:30:07.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.477+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f543c00bbc0 con 0x7f54400fbbd0 2024-05-25T06:30:07.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.477+0000 7f54367fc700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542c06e140 0x7f542c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.477+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f543c08eda0 con 0x7f54400fbbd0 2024-05-25T06:30:07.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.478+0000 7f544529e700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542c06e140 0x7f542c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.478+0000 7f544529e700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542c06e140 0x7f542c0705f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5430006fd0 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:07.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.482+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f543c058210 con 0x7f54400fbbd0 2024-05-25T06:30:07.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.694+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54401110d0 con 0x7f542c06e140 2024-05-25T06:30:07.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.696+0000 7f54367fc700 1 -- 172.21.15.160:0/2116756145 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f54401110d0 con 0x7f542c06e140 2024-05-25T06:30:07.695 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:30:07.696 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542c06e140 msgr2=0x7f542c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542c06e140 0x7f542c0705f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5430006fd0 tx=0x7f5430008040 comp rx=0 tx=0).stop 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 msgr2=0x7f5440115a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f5440115a40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f543c009380 tx=0x7f543c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 shutdown_connections 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400fbbd0 0x7f5440115a40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f542c06e140 0x7f542c0705f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 --2- 172.21.15.160:0/2116756145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54400fc4e0 0x7f5440115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 >> 172.21.15.160:0/2116756145 conn(0x7f54400f7860 msgr2=0x7f54400f7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:07.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 shutdown_connections 2024-05-25T06:30:07.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.699+0000 7f5447d03700 1 -- 172.21.15.160:0/2116756145 wait complete. 2024-05-25T06:30:07.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 -- 172.21.15.160:0/2840058954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 msgr2=0x7f34201031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 --2- 172.21.15.160:0/2840058954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f34201031b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3410004940 tx=0x7f3410009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:07.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 -- 172.21.15.160:0/2840058954 shutdown_connections 2024-05-25T06:30:07.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 --2- 172.21.15.160:0/2840058954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3420103780 0x7f34200fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 --2- 172.21.15.160:0/2840058954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f34201031b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 -- 172.21.15.160:0/2840058954 >> 172.21.15.160:0/2840058954 conn(0x7f34200f89b0 msgr2=0x7f34200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:07.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 -- 172.21.15.160:0/2840058954 shutdown_connections 2024-05-25T06:30:07.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.801+0000 7f3425ca6700 1 -- 172.21.15.160:0/2840058954 wait complete. 2024-05-25T06:30:07.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3425ca6700 1 Processor -- start 2024-05-25T06:30:07.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3425ca6700 1 -- start start 2024-05-25T06:30:07.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3425ca6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f3420113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3425ca6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3420103780 0x7f34201140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3425ca6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420114710 con 0x7f3420102de0 2024-05-25T06:30:07.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3425ca6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342010db50 con 0x7f3420103780 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f341f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f3420113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f341f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f3420113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54490/0 (socket says 172.21.15.160:54490) 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f341f7fe700 1 -- 172.21.15.160:0/932020427 learned_addr learned my addr 172.21.15.160:0/932020427 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f3416ffd700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3420103780 0x7f34201140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f341f7fe700 1 -- 172.21.15.160:0/932020427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3420103780 msgr2=0x7f34201140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f341f7fe700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3420103780 0x7f34201140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.802+0000 7f341f7fe700 1 -- 172.21.15.160:0/932020427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410005040 con 0x7f3420102de0 2024-05-25T06:30:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.803+0000 7f341f7fe700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f3420113b80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f341000b8b0 tx=0x7f3410009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.803+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f341000e040 con 0x7f3420102de0 2024-05-25T06:30:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.803+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3410004290 con 0x7f3420102de0 2024-05-25T06:30:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.803+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342010ddd0 con 0x7f3420102de0 2024-05-25T06:30:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.803+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f341001c3f0 con 0x7f3420102de0 2024-05-25T06:30:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.803+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342010e390 con 0x7f3420102de0 2024-05-25T06:30:07.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.804+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34201000f0 con 0x7f3420102de0 2024-05-25T06:30:07.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.804+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3410013420 con 0x7f3420102de0 2024-05-25T06:30:07.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.805+0000 7f341cff9700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340c06e140 0x7f340c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:07.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.805+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f341008ed20 con 0x7f3420102de0 2024-05-25T06:30:07.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.805+0000 7f3416ffd700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340c06e140 0x7f340c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:07.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.805+0000 7f3416ffd700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340c06e140 0x7f340c0705f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f342010ecc0 tx=0x7f3408009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:07.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:07.809+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f341005bc00 con 0x7f3420102de0 2024-05-25T06:30:08.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.090+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f342010f1d0 con 0x7f3420102de0 2024-05-25T06:30:08.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.091+0000 7f341cff9700 1 -- 172.21.15.160:0/932020427 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f341005b790 con 0x7f3420102de0 2024-05-25T06:30:08.092 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:30:08.092 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:30:08.092 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:30:08.092 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:30:08.092 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:30:08.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340c06e140 msgr2=0x7f340c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:08.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340c06e140 0x7f340c0705f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f342010ecc0 tx=0x7f3408009040 comp rx=0 tx=0).stop 2024-05-25T06:30:08.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 msgr2=0x7f3420113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:08.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f3420113b80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f341000b8b0 tx=0x7f3410009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:08.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 shutdown_connections 2024-05-25T06:30:08.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3420102de0 0x7f3420113b80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:08.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f340c06e140 0x7f340c0705f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:08.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 --2- 172.21.15.160:0/932020427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3420103780 0x7f34201140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:08.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 >> 172.21.15.160:0/932020427 conn(0x7f34200f89b0 msgr2=0x7f3420107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:08.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 shutdown_connections 2024-05-25T06:30:08.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:08.093+0000 7f3425ca6700 1 -- 172.21.15.160:0/932020427 wait complete. 2024-05-25T06:30:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3088816501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:30:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:08 smithi203 ceph-mon[83507]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:08.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/932020427' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:30:08.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3088816501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:30:08.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:08 smithi160 ceph-mon[112094]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:08.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/932020427' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:30:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:09 smithi203 ceph-mon[83507]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:09 smithi160 ceph-mon[112094]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:11 smithi203 ceph-mon[83507]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:11 smithi160 ceph-mon[112094]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:13 smithi203 ceph-mon[83507]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:13 smithi160 ceph-mon[112094]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:15.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:15 smithi203 ceph-mon[83507]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:15 smithi160 ceph-mon[112094]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:17.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:17 smithi203 ceph-mon[83507]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:17 smithi160 ceph-mon[112094]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:19.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:19 smithi203 ceph-mon[83507]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:19 smithi160 ceph-mon[112094]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:21 smithi203 ceph-mon[83507]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:21 smithi160 ceph-mon[112094]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:23 smithi203 ceph-mon[83507]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:23 smithi160 ceph-mon[112094]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:25.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:25 smithi203 ceph-mon[83507]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:25 smithi160 ceph-mon[112094]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:27.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:27 smithi203 ceph-mon[83507]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:27 smithi160 ceph-mon[112094]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:29.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:29 smithi203 ceph-mon[83507]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:29 smithi160 ceph-mon[112094]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:31.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:31 smithi203 ceph-mon[83507]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:31 smithi160 ceph-mon[112094]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:33.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:33 smithi203 ceph-mon[83507]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:33 smithi160 ceph-mon[112094]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:35 smithi203 ceph-mon[83507]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:35 smithi160 ceph-mon[112094]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:37 smithi203 ceph-mon[83507]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:37 smithi160 ceph-mon[112094]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:38.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 -- 172.21.15.160:0/3692460727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3390101d90 msgr2=0x7f3390102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 --2- 172.21.15.160:0/3692460727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3390101d90 0x7f3390102160 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3384004940 tx=0x7f3384009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:38.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 -- 172.21.15.160:0/3692460727 shutdown_connections 2024-05-25T06:30:38.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 --2- 172.21.15.160:0/3692460727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33900fbd40 0x7f33900fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 --2- 172.21.15.160:0/3692460727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3390101d90 0x7f3390102160 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 -- 172.21.15.160:0/3692460727 >> 172.21.15.160:0/3692460727 conn(0x7f3390071d00 msgr2=0x7f3390072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 -- 172.21.15.160:0/3692460727 shutdown_connections 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.208+0000 7f3396c00700 1 -- 172.21.15.160:0/3692460727 wait complete. 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.209+0000 7f3396c00700 1 Processor -- start 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.209+0000 7f3396c00700 1 -- start start 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.209+0000 7f3396c00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 0x7f3390113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.209+0000 7f3396c00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3390101d90 0x7f33901140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.209+0000 7f3396c00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901147b0 con 0x7f33900fbd40 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.209+0000 7f3396c00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339010db10 con 0x7f3390101d90 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 0x7f3390113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 0x7f3390113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59216/0 (socket says 172.21.15.160:59216) 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 -- 172.21.15.160:0/2255986689 learned_addr learned my addr 172.21.15.160:0/2255986689 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f338ffff700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3390101d90 0x7f33901140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 -- 172.21.15.160:0/2255986689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3390101d90 msgr2=0x7f33901140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3390101d90 0x7f33901140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 -- 172.21.15.160:0/2255986689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384005040 con 0x7f33900fbd40 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f339499c700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 0x7f3390113b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f338400b3b0 tx=0x7f3384009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338400e040 con 0x7f33900fbd40 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f338400bbc0 con 0x7f33900fbd40 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339010dd90 con 0x7f33900fbd40 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338401c3f0 con 0x7f33900fbd40 2024-05-25T06:30:38.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.210+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339010e350 con 0x7f33900fbd40 2024-05-25T06:30:38.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.211+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33900fd430 con 0x7f33900fbd40 2024-05-25T06:30:38.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.212+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f338400bd30 con 0x7f33900fbd40 2024-05-25T06:30:38.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.212+0000 7f338d7fa700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f338006e140 0x7f33800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.213+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f338408eea0 con 0x7f33900fbd40 2024-05-25T06:30:38.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.213+0000 7f338ffff700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f338006e140 0x7f33800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.213+0000 7f338ffff700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f338006e140 0x7f33800705f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f337c006fd0 tx=0x7f337c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:38.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.217+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f338405bd00 con 0x7f33900fbd40 2024-05-25T06:30:38.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.432+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3390108ee0 con 0x7f338006e140 2024-05-25T06:30:38.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.433+0000 7f338d7fa700 1 -- 172.21.15.160:0/2255986689 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3390108ee0 con 0x7f338006e140 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.435+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f338006e140 msgr2=0x7f33800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.435+0000 7f3396c00700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f338006e140 0x7f33800705f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f337c006fd0 tx=0x7f337c008040 comp rx=0 tx=0).stop 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 msgr2=0x7f3390113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 0x7f3390113b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f338400b3b0 tx=0x7f3384009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 shutdown_connections 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33900fbd40 0x7f3390113b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f338006e140 0x7f33800705f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 --2- 172.21.15.160:0/2255986689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3390101d90 0x7f33901140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 >> 172.21.15.160:0/2255986689 conn(0x7f3390071d00 msgr2=0x7f33900fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 shutdown_connections 2024-05-25T06:30:38.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.436+0000 7f3396c00700 1 -- 172.21.15.160:0/2255986689 wait complete. 2024-05-25T06:30:38.446 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/4115743049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84101cf0 msgr2=0x7fcf841020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/4115743049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84101cf0 0x7fcf841020c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcf74004940 tx=0x7fcf74009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/4115743049 shutdown_connections 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/4115743049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84102690 0x7fcf840fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/4115743049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84101cf0 0x7fcf841020c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/4115743049 >> 172.21.15.160:0/4115743049 conn(0x7fcf840f7860 msgr2=0x7fcf840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:38.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/4115743049 shutdown_connections 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.540+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/4115743049 wait complete. 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8c7a9700 1 Processor -- start 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8c7a9700 1 -- start start 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8c7a9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84101cf0 0x7fcf8410eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8c7a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 0x7fcf8410f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8c7a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8410f5a0 con 0x7fcf84102690 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8c7a9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8410f6e0 con 0x7fcf84101cf0 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf89d44700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 0x7fcf8410f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf89d44700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 0x7fcf8410f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59244/0 (socket says 172.21.15.160:59244) 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf89d44700 1 -- 172.21.15.160:0/740446922 learned_addr learned my addr 172.21.15.160:0/740446922 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.541+0000 7fcf8a545700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84101cf0 0x7fcf8410eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf89d44700 1 -- 172.21.15.160:0/740446922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84101cf0 msgr2=0x7fcf8410eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf89d44700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84101cf0 0x7fcf8410eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf89d44700 1 -- 172.21.15.160:0/740446922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74005040 con 0x7fcf84102690 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf89d44700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 0x7fcf8410f060 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800bae0 tx=0x7fcf78009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf7800c8c0 con 0x7fcf84102690 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf78014410 con 0x7fcf84102690 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8410f970 con 0x7fcf84102690 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf78007610 con 0x7fcf84102690 2024-05-25T06:30:38.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.542+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf84117f90 con 0x7fcf84102690 2024-05-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.543+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf840fd440 con 0x7fcf84102690 2024-05-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.544+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcf78014580 con 0x7fcf84102690 2024-05-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.544+0000 7fcf82ffd700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf6c06e140 0x7fcf6c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.544+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcf7808dc60 con 0x7fcf84102690 2024-05-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.544+0000 7fcf8a545700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf6c06e140 0x7fcf6c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.544+0000 7fcf8a545700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf6c06e140 0x7fcf6c0705f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcf74004f90 tx=0x7fcf74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:38.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.549+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcf78056f90 con 0x7fcf84102690 2024-05-25T06:30:38.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.763+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf84108e60 con 0x7fcf6c06e140 2024-05-25T06:30:38.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.764+0000 7fcf82ffd700 1 -- 172.21.15.160:0/740446922 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcf84108e60 con 0x7fcf6c06e140 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf6c06e140 msgr2=0x7fcf6c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf6c06e140 0x7fcf6c0705f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcf74004f90 tx=0x7fcf74009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 msgr2=0x7fcf8410f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 0x7fcf8410f060 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800bae0 tx=0x7fcf78009810 comp rx=0 tx=0).stop 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 shutdown_connections 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fcf6c06e140 0x7fcf6c0705f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84101cf0 0x7fcf8410eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 --2- 172.21.15.160:0/740446922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf84102690 0x7fcf8410f060 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.766+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 >> 172.21.15.160:0/740446922 conn(0x7fcf840f7860 msgr2=0x7fcf840f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.767+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 shutdown_connections 2024-05-25T06:30:38.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.767+0000 7fcf8c7a9700 1 -- 172.21.15.160:0/740446922 wait complete. 2024-05-25T06:30:38.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 -- 172.21.15.160:0/4250354604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c102f60 msgr2=0x7f6f8c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 --2- 172.21.15.160:0/4250354604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c102f60 0x7f6f8c103330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009ce0 tx=0x7f6f88004d10 comp rx=0 tx=0).stop 2024-05-25T06:30:38.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 -- 172.21.15.160:0/4250354604 shutdown_connections 2024-05-25T06:30:38.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 --2- 172.21.15.160:0/4250354604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c0fb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 --2- 172.21.15.160:0/4250354604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c102f60 0x7f6f8c103330 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 -- 172.21.15.160:0/4250354604 >> 172.21.15.160:0/4250354604 conn(0x7f6f8c073350 msgr2=0x7f6f8c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:38.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 -- 172.21.15.160:0/4250354604 shutdown_connections 2024-05-25T06:30:38.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.870+0000 7f6f945a6700 1 -- 172.21.15.160:0/4250354604 wait complete. 2024-05-25T06:30:38.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f945a6700 1 Processor -- start 2024-05-25T06:30:38.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f945a6700 1 -- start start 2024-05-25T06:30:38.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f945a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f945a6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f8c102f60 0x7f6f8c114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f945a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c114680 con 0x7f6f8c0fb390 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f945a6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c10dc60 con 0x7f6f8c102f60 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f92342700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f92342700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59268/0 (socket says 172.21.15.160:59268) 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f92342700 1 -- 172.21.15.160:0/567162344 learned_addr learned my addr 172.21.15.160:0/567162344 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.871+0000 7f6f91b41700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f8c102f60 0x7f6f8c114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f92342700 1 -- 172.21.15.160:0/567162344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f8c102f60 msgr2=0x7f6f8c114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f92342700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f8c102f60 0x7f6f8c114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f92342700 1 -- 172.21.15.160:0/567162344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009940 con 0x7f6f8c0fb390 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f92342700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c113af0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6f88003400 tx=0x7f6f88004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8800e040 con 0x7f6f8c0fb390 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c10dee0 con 0x7f6f8c0fb390 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f88004610 con 0x7f6f8c0fb390 2024-05-25T06:30:38.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c10e420 con 0x7f6f8c0fb390 2024-05-25T06:30:38.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.872+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8801c640 con 0x7f6f8c0fb390 2024-05-25T06:30:38.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.873+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c100130 con 0x7f6f8c0fb390 2024-05-25T06:30:38.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.874+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6f88012a70 con 0x7f6f8c0fb390 2024-05-25T06:30:38.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.874+0000 7f6f82ffd700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6f7806e140 0x7f6f780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:38.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.874+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f8808f220 con 0x7f6f8c0fb390 2024-05-25T06:30:38.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.874+0000 7f6f91b41700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6f7806e140 0x7f6f780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:38.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.875+0000 7f6f91b41700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6f7806e140 0x7f6f780705f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c10f390 tx=0x7f6f7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:38.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:38.879+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6f8805c080 con 0x7f6f8c0fb390 2024-05-25T06:30:39.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.088+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f8c04c780 con 0x7f6f7806e140 2024-05-25T06:30:39.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.098+0000 7f6f82ffd700 1 -- 172.21.15.160:0/567162344 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6f8c04c780 con 0x7f6f7806e140 2024-05-25T06:30:39.098 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:30:39.098 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (16m) 9m ago 27m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:30:39.098 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (13m) 9m ago 27m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:30:39.098 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (13m) 9m ago 26m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:30:39.099 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (16m) 9m ago 27m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:30:39.099 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (10m) 9m ago 22m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:30:39.099 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (9m) 9m ago 22m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:30:39.099 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (9m) 9m ago 22m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:30:39.099 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (9m) 9m ago 22m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:30:39.099 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (18m) 9m ago 29m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (17m) 9m ago 26m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (15m) 9m ago 29m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (14m) 9m ago 26m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (17m) 9m ago 27m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (17m) 9m ago 26m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (13m) 9m ago 25m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (12m) 9m ago 25m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (12m) 9m ago 24m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (11m) 9m ago 24m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:30:39.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (11m) 9m ago 24m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:30:39.102 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (10m) 9m ago 23m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:30:39.102 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (16m) 9m ago 27m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:30:39.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.100+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6f7806e140 msgr2=0x7f6f780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.100+0000 7f6f945a6700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6f7806e140 0x7f6f780705f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c10f390 tx=0x7f6f7c00b040 comp rx=0 tx=0).stop 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.100+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 msgr2=0x7f6f8c113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.100+0000 7f6f945a6700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c113af0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6f88003400 tx=0x7f6f88004d10 comp rx=0 tx=0).stop 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.100+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 shutdown_connections 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.100+0000 7f6f945a6700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8c0fb390 0x7f6f8c113af0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.101+0000 7f6f945a6700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6f7806e140 0x7f6f780705f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.101+0000 7f6f945a6700 1 --2- 172.21.15.160:0/567162344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f8c102f60 0x7f6f8c114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.101+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 >> 172.21.15.160:0/567162344 conn(0x7f6f8c073350 msgr2=0x7f6f8c107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.101+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 shutdown_connections 2024-05-25T06:30:39.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.101+0000 7f6f945a6700 1 -- 172.21.15.160:0/567162344 wait complete. 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 -- 172.21.15.160:0/2671461229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 msgr2=0x7ff0ac1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 --2- 172.21.15.160:0/2671461229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1031b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff09c004940 tx=0x7ff09c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 -- 172.21.15.160:0/2671461229 shutdown_connections 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 --2- 172.21.15.160:0/2671461229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0ac103780 0x7ff0ac0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 --2- 172.21.15.160:0/2671461229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1031b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 -- 172.21.15.160:0/2671461229 >> 172.21.15.160:0/2671461229 conn(0x7ff0ac0f8990 msgr2=0x7ff0ac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 -- 172.21.15.160:0/2671461229 shutdown_connections 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.204+0000 7ff0b42de700 1 -- 172.21.15.160:0/2671461229 wait complete. 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b42de700 1 Processor -- start 2024-05-25T06:30:39.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b42de700 1 -- start start 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b42de700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b42de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0ac103780 0x7ff0ac113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b42de700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac114470 con 0x7ff0ac102de0 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b42de700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac10da70 con 0x7ff0ac103780 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b207a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b207a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59280/0 (socket says 172.21.15.160:59280) 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b207a700 1 -- 172.21.15.160:0/293029547 learned_addr learned my addr 172.21.15.160:0/293029547 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b1879700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0ac103780 0x7ff0ac113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b207a700 1 -- 172.21.15.160:0/293029547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0ac103780 msgr2=0x7ff0ac113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b207a700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0ac103780 0x7ff0ac113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.205+0000 7ff0b207a700 1 -- 172.21.15.160:0/293029547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c005040 con 0x7ff0ac102de0 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.206+0000 7ff0b207a700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1138e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009150 tx=0x7ff09c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.206+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff09c021070 con 0x7ff0ac102de0 2024-05-25T06:30:39.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.206+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac10dcf0 con 0x7ff0ac102de0 2024-05-25T06:30:39.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.206+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff09c013c00 con 0x7ff0ac102de0 2024-05-25T06:30:39.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.206+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff09c00e070 con 0x7ff0ac102de0 2024-05-25T06:30:39.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.206+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac10e1d0 con 0x7ff0ac102de0 2024-05-25T06:30:39.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.207+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac1000f0 con 0x7ff0ac102de0 2024-05-25T06:30:39.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.207+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff09c01c400 con 0x7ff0ac102de0 2024-05-25T06:30:39.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.208+0000 7ff0a2ffd700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff09806e140 0x7ff0980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.208+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff09c08f000 con 0x7ff0ac102de0 2024-05-25T06:30:39.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.208+0000 7ff0b1879700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff09806e140 0x7ff0980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.208+0000 7ff0b1879700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff09806e140 0x7ff0980705f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac10ec30 tx=0x7ff0a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:39.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.213+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff09c05bee0 con 0x7ff0ac102de0 2024-05-25T06:30:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:39 smithi203 ceph-mon[83507]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:39 smithi203 ceph-mon[83507]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:39 smithi203 ceph-mon[83507]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:39 smithi160 ceph-mon[112094]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:39 smithi160 ceph-mon[112094]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:39 smithi160 ceph-mon[112094]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:39.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.495+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0ac10ef30 con 0x7ff0ac102de0 2024-05-25T06:30:39.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.496+0000 7ff0a2ffd700 1 -- 172.21.15.160:0/293029547 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7ff09c004490 con 0x7ff0ac102de0 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:30:39.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:30:39.497 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:30:39.497 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:30:39.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff09806e140 msgr2=0x7ff0980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff09806e140 0x7ff0980705f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac10ec30 tx=0x7ff0a800b040 comp rx=0 tx=0).stop 2024-05-25T06:30:39.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 msgr2=0x7ff0ac1138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1138e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009150 tx=0x7ff09c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 shutdown_connections 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0ac102de0 0x7ff0ac1138e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff09806e140 0x7ff0980705f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 --2- 172.21.15.160:0/293029547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0ac103780 0x7ff0ac113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 >> 172.21.15.160:0/293029547 conn(0x7ff0ac0f8990 msgr2=0x7ff0ac0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 shutdown_connections 2024-05-25T06:30:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.498+0000 7ff0b42de700 1 -- 172.21.15.160:0/293029547 wait complete. 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 -- 172.21.15.160:0/671843524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd54101d90 msgr2=0x7fbd54102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/671843524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd54101d90 0x7fbd54102160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd44004940 tx=0x7fbd44009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 -- 172.21.15.160:0/671843524 shutdown_connections 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/671843524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd540fbd40 0x7fbd540fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/671843524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd54101d90 0x7fbd54102160 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 -- 172.21.15.160:0/671843524 >> 172.21.15.160:0/671843524 conn(0x7fbd54071d00 msgr2=0x7fbd54072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:39.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 -- 172.21.15.160:0/671843524 shutdown_connections 2024-05-25T06:30:39.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.601+0000 7fbd59b0a700 1 -- 172.21.15.160:0/671843524 wait complete. 2024-05-25T06:30:39.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd59b0a700 1 Processor -- start 2024-05-25T06:30:39.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd59b0a700 1 -- start start 2024-05-25T06:30:39.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd59b0a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 0x7fbd5410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd59b0a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd54101d90 0x7fbd5410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd59b0a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5410f6d0 con 0x7fbd540fbd40 2024-05-25T06:30:39.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd59b0a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5410f810 con 0x7fbd54101d90 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd537fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 0x7fbd5410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd537fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 0x7fbd5410eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59290/0 (socket says 172.21.15.160:59290) 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.602+0000 7fbd537fe700 1 -- 172.21.15.160:0/1944451737 learned_addr learned my addr 172.21.15.160:0/1944451737 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd52ffd700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd54101d90 0x7fbd5410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd537fe700 1 -- 172.21.15.160:0/1944451737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd54101d90 msgr2=0x7fbd5410f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd537fe700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd54101d90 0x7fbd5410f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd537fe700 1 -- 172.21.15.160:0/1944451737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44005040 con 0x7fbd540fbd40 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd52ffd700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd54101d90 0x7fbd5410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd537fe700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 0x7fbd5410eb40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400b3b0 tx=0x7fbd44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4400e040 con 0x7fbd540fbd40 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54117bb0 con 0x7fbd540fbd40 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd4400bbc0 con 0x7fbd540fbd40 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4401c3f0 con 0x7fbd540fbd40 2024-05-25T06:30:39.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.603+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd541180c0 con 0x7fbd540fbd40 2024-05-25T06:30:39.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.604+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd540fd430 con 0x7fbd540fbd40 2024-05-25T06:30:39.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.605+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbd4400bd30 con 0x7fbd540fbd40 2024-05-25T06:30:39.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.605+0000 7fbd3bfff700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbd3c06e140 0x7fbd3c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.605+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbd4408ee60 con 0x7fbd540fbd40 2024-05-25T06:30:39.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.605+0000 7fbd52ffd700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbd3c06e140 0x7fbd3c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.606+0000 7fbd52ffd700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbd3c06e140 0x7fbd3c0705f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009fd0 tx=0x7fbd4800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:39.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.610+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbd44058190 con 0x7fbd540fbd40 2024-05-25T06:30:39.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.853+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd54065d90 con 0x7fbd540fbd40 2024-05-25T06:30:39.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.854+0000 7fbd3bfff700 1 -- 172.21.15.160:0/1944451737 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fbd4405b850 con 0x7fbd540fbd40 2024-05-25T06:30:39.854 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:30:39.854 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:30:39.854 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:30:39.854 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:30:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:30:39.856 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:39.857 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:30:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbd3c06e140 msgr2=0x7fbd3c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbd3c06e140 0x7fbd3c0705f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009fd0 tx=0x7fbd4800b040 comp rx=0 tx=0).stop 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 msgr2=0x7fbd5410eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 0x7fbd5410eb40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400b3b0 tx=0x7fbd44009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 shutdown_connections 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd540fbd40 0x7fbd5410eb40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbd3c06e140 0x7fbd3c0705f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 --2- 172.21.15.160:0/1944451737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd54101d90 0x7fbd5410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 >> 172.21.15.160:0/1944451737 conn(0x7fbd54071d00 msgr2=0x7fbd540fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 shutdown_connections 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.857+0000 7fbd59b0a700 1 -- 172.21.15.160:0/1944451737 wait complete. 2024-05-25T06:30:39.859 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:30:39.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 -- 172.21.15.160:0/4185816227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 msgr2=0x7f1b280fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/4185816227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b280fb600 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1b24004940 tx=0x7f1b24009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 -- 172.21.15.160:0/4185816227 shutdown_connections 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/4185816227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b280fb600 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/4185816227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b28101cd0 0x7f1b281020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 -- 172.21.15.160:0/4185816227 >> 172.21.15.160:0/4185816227 conn(0x7f1b28071460 msgr2=0x7f1b28071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 -- 172.21.15.160:0/4185816227 shutdown_connections 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.961+0000 7f1b2fb77700 1 -- 172.21.15.160:0/4185816227 wait complete. 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2fb77700 1 Processor -- start 2024-05-25T06:30:39.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2fb77700 1 -- start start 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2fb77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b28101cd0 0x7f1b28113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2fb77700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b281140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2fb77700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28114740 con 0x7f1b28102670 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2fb77700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2810db30 con 0x7f1b28101cd0 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2d112700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b281140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2d112700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b281140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59322/0 (socket says 172.21.15.160:59322) 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.962+0000 7f1b2d112700 1 -- 172.21.15.160:0/2094908312 learned_addr learned my addr 172.21.15.160:0/2094908312 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b2d112700 1 -- 172.21.15.160:0/2094908312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b28101cd0 msgr2=0x7f1b28113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:39.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b2d112700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b28101cd0 0x7f1b28113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b2d112700 1 -- 172.21.15.160:0/2094908312 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24005040 con 0x7f1b28102670 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b2d112700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b281140f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400b3b0 tx=0x7f1b24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b2400e040 con 0x7f1b28102670 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2810ddb0 con 0x7f1b28102670 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b24013420 con 0x7f1b28102670 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b2401c630 con 0x7f1b28102670 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.963+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2810e370 con 0x7f1b28102670 2024-05-25T06:30:39.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.964+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b28065200 con 0x7f1b28102670 2024-05-25T06:30:39.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.964+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1b24013590 con 0x7f1b28102670 2024-05-25T06:30:39.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.965+0000 7f1b1e7fc700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1b1406e140 0x7f1b140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:39.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.965+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1b2408ed70 con 0x7f1b28102670 2024-05-25T06:30:39.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.965+0000 7f1b2d913700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1b1406e140 0x7f1b140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:39.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.965+0000 7f1b2d913700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1b1406e140 0x7f1b140705f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b18004d00 tx=0x7f1b18009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:39.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:39.970+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1b2405bd00 con 0x7f1b28102670 2024-05-25T06:30:40.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.187+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b28109ed0 con 0x7f1b1406e140 2024-05-25T06:30:40.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.189+0000 7f1b1e7fc700 1 -- 172.21.15.160:0/2094908312 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1b28109ed0 con 0x7f1b1406e140 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:30:40.189 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:30:40.190 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:30:40.190 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:30:40.190 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:30:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:30:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1b1406e140 msgr2=0x7f1b140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1b1406e140 0x7f1b140705f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b18004d00 tx=0x7f1b18009ce0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 msgr2=0x7f1b281140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b281140f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400b3b0 tx=0x7f1b24009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 shutdown_connections 2024-05-25T06:30:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1b1406e140 0x7f1b140705f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b28101cd0 0x7f1b28113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 --2- 172.21.15.160:0/2094908312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b28102670 0x7f1b281140f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 >> 172.21.15.160:0/2094908312 conn(0x7f1b28071460 msgr2=0x7f1b281009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 shutdown_connections 2024-05-25T06:30:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.191+0000 7f1b2fb77700 1 -- 172.21.15.160:0/2094908312 wait complete. 2024-05-25T06:30:40.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 -- 172.21.15.160:0/1123110052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 msgr2=0x7f78901020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 --2- 172.21.15.160:0/1123110052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f78901020d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f788c004940 tx=0x7f788c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 -- 172.21.15.160:0/1123110052 shutdown_connections 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 --2- 172.21.15.160:0/1123110052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 0x7f78900fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 --2- 172.21.15.160:0/1123110052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f78901020d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 -- 172.21.15.160:0/1123110052 >> 172.21.15.160:0/1123110052 conn(0x7f78900f7870 msgr2=0x7f78900f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 -- 172.21.15.160:0/1123110052 shutdown_connections 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.293+0000 7f7898756700 1 -- 172.21.15.160:0/1123110052 wait complete. 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7898756700 1 Processor -- start 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7898756700 1 -- start start 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7898756700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f7890112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:40.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7898756700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 0x7f7890112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7898756700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890113410 con 0x7f7890101d00 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7898756700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789010ca10 con 0x7f78901026a0 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f7890112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f7890112880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59340/0 (socket says 172.21.15.160:59340) 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 -- 172.21.15.160:0/4131573666 learned_addr learned my addr 172.21.15.160:0/4131573666 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7895cf1700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 0x7f7890112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 -- 172.21.15.160:0/4131573666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 msgr2=0x7f7890112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 0x7f7890112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 -- 172.21.15.160:0/4131573666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c005040 con 0x7f7890101d00 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f7895cf1700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 0x7f7890112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.294+0000 7f78964f2700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f7890112880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f788c00b3b0 tx=0x7f788c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:40.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.295+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f788c00e040 con 0x7f7890101d00 2024-05-25T06:30:40.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.295+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f788c00bbc0 con 0x7f7890101d00 2024-05-25T06:30:40.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.295+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f788c01c3f0 con 0x7f7890101d00 2024-05-25T06:30:40.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.295+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789010cc90 con 0x7f7890101d00 2024-05-25T06:30:40.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.295+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789010d250 con 0x7f7890101d00 2024-05-25T06:30:40.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.296+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78900fd450 con 0x7f7890101d00 2024-05-25T06:30:40.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.297+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f788c00bd30 con 0x7f7890101d00 2024-05-25T06:30:40.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.297+0000 7f7886ffd700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f787c06e140 0x7f787c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:30:40.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.297+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f788c08ed30 con 0x7f7890101d00 2024-05-25T06:30:40.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.297+0000 7f7895cf1700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f787c06e140 0x7f787c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:30:40.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.297+0000 7f7895cf1700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f787c06e140 0x7f787c0705f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7880009fd0 tx=0x7f7880009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:30:40.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.302+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f788c05bc10 con 0x7f7890101d00 2024-05-25T06:30:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:40 smithi203 ceph-mon[83507]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:40 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/293029547' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:30:40.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:40 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1944451737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:30:40.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:40 smithi160 ceph-mon[112094]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:40.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:40 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/293029547' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:30:40.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:40 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1944451737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:30:40.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.579+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f78900fd450 con 0x7f7890101d00 2024-05-25T06:30:40.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.579+0000 7f7886ffd700 1 -- 172.21.15.160:0/4131573666 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f788c05b7a0 con 0x7f7890101d00 2024-05-25T06:30:40.579 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:30:40.580 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:30:40.580 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:30:40.580 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:30:40.580 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:30:40.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.581+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f787c06e140 msgr2=0x7f787c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.581+0000 7f7898756700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f787c06e140 0x7f787c0705f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7880009fd0 tx=0x7f7880009ee0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.581+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 msgr2=0x7f7890112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.581+0000 7f7898756700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f7890112880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f788c00b3b0 tx=0x7f788c009e30 comp rx=0 tx=0).stop 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 shutdown_connections 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7890101d00 0x7f7890112880 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f787c06e140 0x7f787c0705f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 --2- 172.21.15.160:0/4131573666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901026a0 0x7f7890112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 >> 172.21.15.160:0/4131573666 conn(0x7f78900f7870 msgr2=0x7f78900f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 shutdown_connections 2024-05-25T06:30:40.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:30:40.582+0000 7f7898756700 1 -- 172.21.15.160:0/4131573666 wait complete. 2024-05-25T06:30:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:41 smithi203 ceph-mon[83507]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:41 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4131573666' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:30:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:41 smithi203 ceph-mon[83507]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:41 smithi160 ceph-mon[112094]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:30:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:41 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4131573666' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:30:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:41 smithi160 ceph-mon[112094]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:43 smithi203 ceph-mon[83507]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:43 smithi160 ceph-mon[112094]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:45 smithi203 ceph-mon[83507]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:45 smithi160 ceph-mon[112094]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:47 smithi203 ceph-mon[83507]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:47 smithi160 ceph-mon[112094]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:49 smithi203 ceph-mon[83507]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:49 smithi160 ceph-mon[112094]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:51 smithi203 ceph-mon[83507]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:51 smithi160 ceph-mon[112094]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:53 smithi203 ceph-mon[83507]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:53 smithi160 ceph-mon[112094]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:55 smithi203 ceph-mon[83507]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:55 smithi160 ceph-mon[112094]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:57 smithi203 ceph-mon[83507]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:57 smithi160 ceph-mon[112094]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:30:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:30:59 smithi203 ceph-mon[83507]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:30:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:30:59 smithi160 ceph-mon[112094]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:01 smithi203 ceph-mon[83507]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:01 smithi160 ceph-mon[112094]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:03 smithi203 ceph-mon[83507]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:03 smithi160 ceph-mon[112094]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:05 smithi203 ceph-mon[83507]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:05 smithi160 ceph-mon[112094]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:07 smithi203 ceph-mon[83507]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:07 smithi160 ceph-mon[112094]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:09 smithi203 ceph-mon[83507]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:09 smithi160 ceph-mon[112094]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 -- 172.21.15.160:0/1405711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fde60 msgr2=0x7fe4000fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 --2- 172.21.15.160:0/1405711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fde60 0x7fe4000fe230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0004940 tx=0x7fe3f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 -- 172.21.15.160:0/1405711524 shutdown_connections 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 --2- 172.21.15.160:0/1405711524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fe800 0x7fe400106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 --2- 172.21.15.160:0/1405711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fde60 0x7fe4000fe230 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 -- 172.21.15.160:0/1405711524 >> 172.21.15.160:0/1405711524 conn(0x7fe4000f78e0 msgr2=0x7fe4000f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 -- 172.21.15.160:0/1405711524 shutdown_connections 2024-05-25T06:31:10.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 -- 172.21.15.160:0/1405711524 wait complete. 2024-05-25T06:31:10.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.697+0000 7fe405488700 1 Processor -- start 2024-05-25T06:31:10.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe405488700 1 -- start start 2024-05-25T06:31:10.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe405488700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fde60 0x7fe400115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:10.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe405488700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 0x7fe4001160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:10.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe405488700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001166f0 con 0x7fe4000fe800 2024-05-25T06:31:10.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe405488700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40010fcd0 con 0x7fe4000fde60 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3f67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 0x7fe4001160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3f67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 0x7fe4001160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35338/0 (socket says 172.21.15.160:35338) 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3f67fc700 1 -- 172.21.15.160:0/29477553 learned_addr learned my addr 172.21.15.160:0/29477553 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3feffd700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fde60 0x7fe400115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3f67fc700 1 -- 172.21.15.160:0/29477553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fde60 msgr2=0x7fe400115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3f67fc700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fde60 0x7fe400115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3f67fc700 1 -- 172.21.15.160:0/29477553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0005040 con 0x7fe4000fe800 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.698+0000 7fe3feffd700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fde60 0x7fe400115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.699+0000 7fe3f67fc700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 0x7fe4001160a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800c830 tx=0x7fe3e8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.699+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3e800a760 con 0x7fe4000fe800 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.699+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3e800ad90 con 0x7fe4000fe800 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.699+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40010ffb0 con 0x7fe4000fe800 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.699+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3e80113a0 con 0x7fe4000fe800 2024-05-25T06:31:10.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.699+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4001104f0 con 0x7fe4000fe800 2024-05-25T06:31:10.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.700+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe400106990 con 0x7fe4000fe800 2024-05-25T06:31:10.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.700+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe3e8012400 con 0x7fe4000fe800 2024-05-25T06:31:10.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.701+0000 7fe3f7fff700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe3ec06e140 0x7fe3ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:10.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.701+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe3e808dad0 con 0x7fe4000fe800 2024-05-25T06:31:10.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.701+0000 7fe3feffd700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe3ec06e140 0x7fe3ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:10.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.701+0000 7fe3feffd700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe3ec06e140 0x7fe3ec0705f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b3b0 tx=0x7fe3f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:10.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.706+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe3e805aa70 con 0x7fe4000fe800 2024-05-25T06:31:10.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.922+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe400111220 con 0x7fe3ec06e140 2024-05-25T06:31:10.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.924+0000 7fe3f7fff700 1 -- 172.21.15.160:0/29477553 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe400111220 con 0x7fe3ec06e140 2024-05-25T06:31:10.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe3ec06e140 msgr2=0x7fe3ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:10.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe3ec06e140 0x7fe3ec0705f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b3b0 tx=0x7fe3f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:10.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 msgr2=0x7fe4001160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:10.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 0x7fe4001160a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800c830 tx=0x7fe3e8004470 comp rx=0 tx=0).stop 2024-05-25T06:31:10.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 shutdown_connections 2024-05-25T06:31:10.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe3ec06e140 0x7fe3ec0705f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:10.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4000fde60 0x7fe400115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:10.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 --2- 172.21.15.160:0/29477553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4000fe800 0x7fe4001160a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:10.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 >> 172.21.15.160:0/29477553 conn(0x7fe4000f78e0 msgr2=0x7fe4000fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:10.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 shutdown_connections 2024-05-25T06:31:10.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:10.926+0000 7fe405488700 1 -- 172.21.15.160:0/29477553 wait complete. 2024-05-25T06:31:10.937 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 -- 172.21.15.160:0/40111793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fb5a0 msgr2=0x7f7e5c0fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 --2- 172.21.15.160:0/40111793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c0fb970 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004940 tx=0x7f7e4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 -- 172.21.15.160:0/40111793 shutdown_connections 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 --2- 172.21.15.160:0/40111793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 --2- 172.21.15.160:0/40111793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c0fb970 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 -- 172.21.15.160:0/40111793 >> 172.21.15.160:0/40111793 conn(0x7f7e5c073350 msgr2=0x7f7e5c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 -- 172.21.15.160:0/40111793 shutdown_connections 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.031+0000 7f7e6168a700 1 -- 172.21.15.160:0/40111793 wait complete. 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e6168a700 1 Processor -- start 2024-05-25T06:31:11.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e6168a700 1 -- start start 2024-05-25T06:31:11.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e6168a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e6168a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e6168a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c1145c0 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e6168a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c10db50 con 0x7f7e5c0fb5a0 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e5a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e5a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35360/0 (socket says 172.21.15.160:35360) 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e5a7fc700 1 -- 172.21.15.160:0/3460651755 learned_addr learned my addr 172.21.15.160:0/3460651755 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.032+0000 7f7e5affd700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e5a7fc700 1 -- 172.21.15.160:0/3460651755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fb5a0 msgr2=0x7f7e5c1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e5a7fc700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e5a7fc700 1 -- 172.21.15.160:0/3460651755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c005040 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e5affd700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e5a7fc700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c113ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000b760 tx=0x7f7e50009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:11.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e5000f6f0 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e5000fd20 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c10de30 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e500103a0 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.033+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c10e370 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.034+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c0fe670 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.035+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7e50011400 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.035+0000 7f7e437fe700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7e4406e140 0x7f7e440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.035+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7e5008d9b0 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.035+0000 7f7e5affd700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7e4406e140 0x7f7e440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.035+0000 7f7e5affd700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7e4406e140 0x7f7e440705f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004f90 tx=0x7f7e4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:11.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.040+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7e50056d60 con 0x7f7e5c0fbf40 2024-05-25T06:31:11.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.258+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e5c109f90 con 0x7f7e4406e140 2024-05-25T06:31:11.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.260+0000 7f7e437fe700 1 -- 172.21.15.160:0/3460651755 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7e5c109f90 con 0x7f7e4406e140 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7e4406e140 msgr2=0x7f7e440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7e4406e140 0x7f7e440705f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004f90 tx=0x7f7e4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 msgr2=0x7f7e5c113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c113ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000b760 tx=0x7f7e50009220 comp rx=0 tx=0).stop 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 shutdown_connections 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7e4406e140 0x7f7e440705f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0fb5a0 0x7f7e5c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 --2- 172.21.15.160:0/3460651755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e5c0fbf40 0x7f7e5c113ee0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 >> 172.21.15.160:0/3460651755 conn(0x7f7e5c073350 msgr2=0x7f7e5c0f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 shutdown_connections 2024-05-25T06:31:11.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.262+0000 7f7e6168a700 1 -- 172.21.15.160:0/3460651755 wait complete. 2024-05-25T06:31:11.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 -- 172.21.15.160:0/2563621805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4102de0 msgr2=0x7ff4d41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/2563621805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4102de0 0x7ff4d41031b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004940 tx=0x7ff4c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 -- 172.21.15.160:0/2563621805 shutdown_connections 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/2563621805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4103780 0x7ff4d40fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/2563621805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4102de0 0x7ff4d41031b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 -- 172.21.15.160:0/2563621805 >> 172.21.15.160:0/2563621805 conn(0x7ff4d40f8990 msgr2=0x7ff4d40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 -- 172.21.15.160:0/2563621805 shutdown_connections 2024-05-25T06:31:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.364+0000 7ff4d9d73700 1 -- 172.21.15.160:0/2563621805 wait complete. 2024-05-25T06:31:11.366 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:11 smithi160 ceph-mon[112094]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:11.366 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:11 smithi160 ceph-mon[112094]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d9d73700 1 Processor -- start 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d9d73700 1 -- start start 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d9d73700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102de0 0x7ff4d410b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d9d73700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 0x7ff4d410be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d9d73700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410e840 con 0x7ff4d4103780 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d9d73700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410c350 con 0x7ff4d4102de0 2024-05-25T06:31:11.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 0x7ff4d410be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 0x7ff4d410be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35380/0 (socket says 172.21.15.160:35380) 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4cbfff700 1 -- 172.21.15.160:0/3096729411 learned_addr learned my addr 172.21.15.160:0/3096729411 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4d37fe700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102de0 0x7ff4d410b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4cbfff700 1 -- 172.21.15.160:0/3096729411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102de0 msgr2=0x7ff4d410b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.365+0000 7ff4cbfff700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102de0 0x7ff4d410b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4cbfff700 1 -- 172.21.15.160:0/3096729411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4005040 con 0x7ff4d4103780 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4cbfff700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 0x7ff4d410be10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c800 tx=0x7ff4bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc00a760 con 0x7ff4d4103780 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d410c630 con 0x7ff4d4103780 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4bc00ad90 con 0x7ff4d4103780 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc0113a0 con 0x7ff4d4103780 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.366+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d411e540 con 0x7ff4d4103780 2024-05-25T06:31:11.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.367+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d4106a30 con 0x7ff4d4103780 2024-05-25T06:31:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.368+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff4bc012400 con 0x7ff4d4103780 2024-05-25T06:31:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.368+0000 7ff4d0ff9700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4c006e140 0x7ff4c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.368+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff4bc08e630 con 0x7ff4d4103780 2024-05-25T06:31:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.368+0000 7ff4d37fe700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4c006e140 0x7ff4c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.369+0000 7ff4d37fe700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4c006e140 0x7ff4c00705f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004f90 tx=0x7ff4c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:11.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.374+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff4bc057a20 con 0x7ff4d4103780 2024-05-25T06:31:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:11 smithi203 ceph-mon[83507]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:11 smithi203 ceph-mon[83507]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:11.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.580+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4d4109ee0 con 0x7ff4c006e140 2024-05-25T06:31:11.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.590+0000 7ff4d0ff9700 1 -- 172.21.15.160:0/3096729411 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff4d4109ee0 con 0x7ff4c006e140 2024-05-25T06:31:11.591 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:31:11.591 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (17m) 10m ago 28m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:31:11.591 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (14m) 10m ago 28m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:31:11.591 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (14m) 9m ago 27m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (16m) 10m ago 27m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (10m) 10m ago 22m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (10m) 10m ago 22m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (9m) 9m ago 22m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (10m) 9m ago 22m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (18m) 10m ago 29m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (17m) 9m ago 27m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (15m) 10m ago 29m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (15m) 9m ago 27m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (17m) 10m ago 28m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (17m) 9m ago 27m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (14m) 10m ago 26m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (13m) 10m ago 25m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (12m) 10m ago 25m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:31:11.592 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (12m) 9m ago 25m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:31:11.593 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (11m) 9m ago 24m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:31:11.593 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (11m) 9m ago 24m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:31:11.593 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (17m) 10m ago 27m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4c006e140 msgr2=0x7ff4c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4c006e140 0x7ff4c00705f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004f90 tx=0x7ff4c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 msgr2=0x7ff4d410be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 0x7ff4d410be10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c800 tx=0x7ff4bc004470 comp rx=0 tx=0).stop 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 shutdown_connections 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff4c006e140 0x7ff4c00705f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102de0 0x7ff4d410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 --2- 172.21.15.160:0/3096729411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4d4103780 0x7ff4d410be10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 >> 172.21.15.160:0/3096729411 conn(0x7ff4d40f8990 msgr2=0x7ff4d4107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:11.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 shutdown_connections 2024-05-25T06:31:11.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.593+0000 7ff4d9d73700 1 -- 172.21.15.160:0/3096729411 wait complete. 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2098046987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec101d90 msgr2=0x7f5aec102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2098046987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec101d90 0x7f5aec102160 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4004940 tx=0x7f5ae4009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2098046987 shutdown_connections 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2098046987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec0fbd40 0x7f5aec0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2098046987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec101d90 0x7f5aec102160 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2098046987 >> 172.21.15.160:0/2098046987 conn(0x7f5aec071d00 msgr2=0x7f5aec072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:11.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2098046987 shutdown_connections 2024-05-25T06:31:11.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.694+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2098046987 wait complete. 2024-05-25T06:31:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af2f1d700 1 Processor -- start 2024-05-25T06:31:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af2f1d700 1 -- start start 2024-05-25T06:31:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af2f1d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 0x7f5aec113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af2f1d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec101d90 0x7f5aec1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af2f1d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec114740 con 0x7f5aec0fbd40 2024-05-25T06:31:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af2f1d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec10db30 con 0x7f5aec101d90 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af0cb9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 0x7f5aec113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af0cb9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 0x7f5aec113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35402/0 (socket says 172.21.15.160:35402) 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5af0cb9700 1 -- 172.21.15.160:0/2273802420 learned_addr learned my addr 172.21.15.160:0/2273802420 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.695+0000 7f5aebfff700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec101d90 0x7f5aec1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5af0cb9700 1 -- 172.21.15.160:0/2273802420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec101d90 msgr2=0x7f5aec1140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5af0cb9700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec101d90 0x7f5aec1140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5af0cb9700 1 -- 172.21.15.160:0/2273802420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4005040 con 0x7f5aec0fbd40 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5aebfff700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec101d90 0x7f5aec1140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5af0cb9700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 0x7f5aec113bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400b3b0 tx=0x7f5ae4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ae400e040 con 0x7f5aec0fbd40 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ae400bbc0 con 0x7f5aec0fbd40 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ae401c3f0 con 0x7f5aec0fbd40 2024-05-25T06:31:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec10ddb0 con 0x7f5aec0fbd40 2024-05-25T06:31:11.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.696+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec10e370 con 0x7f5aec0fbd40 2024-05-25T06:31:11.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.697+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec0fd430 con 0x7f5aec0fbd40 2024-05-25T06:31:11.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.698+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5ae400bd30 con 0x7f5aec0fbd40 2024-05-25T06:31:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.698+0000 7f5ae97fa700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5ad806e140 0x7f5ad80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.698+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5ae408edc0 con 0x7f5aec0fbd40 2024-05-25T06:31:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.698+0000 7f5aebfff700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5ad806e140 0x7f5ad80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.699+0000 7f5aebfff700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5ad806e140 0x7f5ad80705f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5aec10ecf0 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.703+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5ae405bca0 con 0x7f5aec0fbd40 2024-05-25T06:31:11.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.993+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5aec108df0 con 0x7f5aec0fbd40 2024-05-25T06:31:11.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.993+0000 7f5ae97fa700 1 -- 172.21.15.160:0/2273802420 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f5ae405b830 con 0x7f5aec0fbd40 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:11.995 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:31:11.996 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:31:11.996 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:31:11.996 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:31:11.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5ad806e140 msgr2=0x7f5ad80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5ad806e140 0x7f5ad80705f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5aec10ecf0 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 msgr2=0x7f5aec113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 0x7f5aec113bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400b3b0 tx=0x7f5ae4009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 shutdown_connections 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5aec0fbd40 0x7f5aec113bb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5ad806e140 0x7f5ad80705f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 --2- 172.21.15.160:0/2273802420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5aec101d90 0x7f5aec1140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 >> 172.21.15.160:0/2273802420 conn(0x7f5aec071d00 msgr2=0x7f5aec0655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 shutdown_connections 2024-05-25T06:31:11.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:11.996+0000 7f5af2f1d700 1 -- 172.21.15.160:0/2273802420 wait complete. 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 -- 172.21.15.160:0/3277118070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 msgr2=0x7f559c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 --2- 172.21.15.160:0/3277118070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c0fcdf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5584004940 tx=0x7f5584009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 -- 172.21.15.160:0/3277118070 shutdown_connections 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 --2- 172.21.15.160:0/3277118070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c0fcdf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 --2- 172.21.15.160:0/3277118070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f559c102de0 0x7f559c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 -- 172.21.15.160:0/3277118070 >> 172.21.15.160:0/3277118070 conn(0x7f559c0f8990 msgr2=0x7f559c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 -- 172.21.15.160:0/3277118070 shutdown_connections 2024-05-25T06:31:12.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.099+0000 7f55a304a700 1 -- 172.21.15.160:0/3277118070 wait complete. 2024-05-25T06:31:12.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a304a700 1 Processor -- start 2024-05-25T06:31:12.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a304a700 1 -- start start 2024-05-25T06:31:12.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a304a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f559c102de0 0x7f559c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a304a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a304a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c1145a0 con 0x7f559c103780 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a304a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c10db60 con 0x7f559c102de0 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35426/0 (socket says 172.21.15.160:35426) 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 -- 172.21.15.160:0/1904172661 learned_addr learned my addr 172.21.15.160:0/1904172661 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f55a0de6700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f559c102de0 0x7f559c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 -- 172.21.15.160:0/1904172661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f559c102de0 msgr2=0x7f559c113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f559c102de0 0x7f559c113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 -- 172.21.15.160:0/1904172661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5584005040 con 0x7f559c103780 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.100+0000 7f559bfff700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c113ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5584009100 tx=0x7f5584009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.101+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558400e040 con 0x7f559c103780 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.101+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c10dde0 con 0x7f559c103780 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.101+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5584013420 con 0x7f559c103780 2024-05-25T06:31:12.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.101+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558401c610 con 0x7f559c103780 2024-05-25T06:31:12.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.101+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c10e3a0 con 0x7f559c103780 2024-05-25T06:31:12.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.102+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c1000f0 con 0x7f559c103780 2024-05-25T06:31:12.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.102+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5584013590 con 0x7f559c103780 2024-05-25T06:31:12.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.103+0000 7f55997fa700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f558806e140 0x7f55880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.103+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f558408ee10 con 0x7f559c103780 2024-05-25T06:31:12.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.103+0000 7f55a0de6700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f558806e140 0x7f55880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.103+0000 7f55a0de6700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f558806e140 0x7f55880705f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f558c009f50 tx=0x7f558c009590 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:12.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.107+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f558405bcf0 con 0x7f559c103780 2024-05-25T06:31:12.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.350+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f559c0fe530 con 0x7f559c103780 2024-05-25T06:31:12.352 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:12 smithi160 ceph-mon[112094]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:12.352 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:12 smithi160 ceph-mon[112094]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:12.352 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:12 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2273802420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:31:12.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.351+0000 7f55997fa700 1 -- 172.21.15.160:0/1904172661 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f558405b880 con 0x7f559c103780 2024-05-25T06:31:12.352 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:31:12.353 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:31:12.354 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:12.355 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f558806e140 msgr2=0x7f55880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f558806e140 0x7f55880705f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f558c009f50 tx=0x7f558c009590 comp rx=0 tx=0).stop 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 msgr2=0x7f559c113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c113ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5584009100 tx=0x7f5584009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 shutdown_connections 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f558806e140 0x7f55880705f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f559c102de0 0x7f559c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 --2- 172.21.15.160:0/1904172661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f559c103780 0x7f559c113ec0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.353+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 >> 172.21.15.160:0/1904172661 conn(0x7f559c0f8990 msgr2=0x7f559c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:12.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.354+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 shutdown_connections 2024-05-25T06:31:12.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.354+0000 7f55a304a700 1 -- 172.21.15.160:0/1904172661 wait complete. 2024-05-25T06:31:12.357 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:31:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:12 smithi203 ceph-mon[83507]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:12 smithi203 ceph-mon[83507]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:12.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:12 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2273802420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 -- 172.21.15.160:0/910885121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c0fbd40 msgr2=0x7fa31c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 --2- 172.21.15.160:0/910885121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c0fbd40 0x7fa31c0fc1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa304004940 tx=0x7fa304009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 -- 172.21.15.160:0/910885121 shutdown_connections 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 --2- 172.21.15.160:0/910885121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c0fbd40 0x7fa31c0fc1b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 --2- 172.21.15.160:0/910885121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c101d90 0x7fa31c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 -- 172.21.15.160:0/910885121 >> 172.21.15.160:0/910885121 conn(0x7fa31c071d00 msgr2=0x7fa31c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 -- 172.21.15.160:0/910885121 shutdown_connections 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.457+0000 7fa3222c5700 1 -- 172.21.15.160:0/910885121 wait complete. 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa3222c5700 1 Processor -- start 2024-05-25T06:31:12.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa3222c5700 1 -- start start 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa3222c5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0fbd40 0x7fa31c112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa3222c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 0x7fa31c112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa3222c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c113640 con 0x7fa31c101d90 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa3222c5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c10ca30 con 0x7fa31c0fbd40 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa313fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 0x7fa31c112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa313fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 0x7fa31c112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35434/0 (socket says 172.21.15.160:35434) 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.458+0000 7fa313fff700 1 -- 172.21.15.160:0/1722786317 learned_addr learned my addr 172.21.15.160:0/1722786317 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa31bfff700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0fbd40 0x7fa31c112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa313fff700 1 -- 172.21.15.160:0/1722786317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0fbd40 msgr2=0x7fa31c112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa313fff700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0fbd40 0x7fa31c112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa313fff700 1 -- 172.21.15.160:0/1722786317 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304005040 con 0x7fa31c101d90 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa313fff700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 0x7fa31c112ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b3b0 tx=0x7fa304009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30400e040 con 0x7fa31c101d90 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa30400bbc0 con 0x7fa31c101d90 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30401c3f0 con 0x7fa31c101d90 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c10ccb0 con 0x7fa31c101d90 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.459+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c10d270 con 0x7fa31c101d90 2024-05-25T06:31:12.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.460+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c0fd430 con 0x7fa31c101d90 2024-05-25T06:31:12.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.461+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa30400bd30 con 0x7fa31c101d90 2024-05-25T06:31:12.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.461+0000 7fa3197fa700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa30806e140 0x7fa3080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.461+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa30408edb0 con 0x7fa31c101d90 2024-05-25T06:31:12.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.461+0000 7fa31bfff700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa30806e140 0x7fa3080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.462+0000 7fa31bfff700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa30806e140 0x7fa3080705f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa30c004d60 tx=0x7fa30c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:12.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.466+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa30405bc90 con 0x7fa31c101d90 2024-05-25T06:31:12.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.680+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa31c10e1a0 con 0x7fa30806e140 2024-05-25T06:31:12.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.682+0000 7fa3197fa700 1 -- 172.21.15.160:0/1722786317 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa31c10e1a0 con 0x7fa30806e140 2024-05-25T06:31:12.682 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:31:12.683 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:31:12.684 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa30806e140 msgr2=0x7fa3080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa30806e140 0x7fa3080705f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa30c004d60 tx=0x7fa30c00b040 comp rx=0 tx=0).stop 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 msgr2=0x7fa31c112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 0x7fa31c112ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b3b0 tx=0x7fa304009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 shutdown_connections 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa30806e140 0x7fa3080705f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0fbd40 0x7fa31c112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 --2- 172.21.15.160:0/1722786317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa31c101d90 0x7fa31c112ff0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 >> 172.21.15.160:0/1722786317 conn(0x7fa31c071d00 msgr2=0x7fa31c0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:12.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 shutdown_connections 2024-05-25T06:31:12.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.684+0000 7fa3222c5700 1 -- 172.21.15.160:0/1722786317 wait complete. 2024-05-25T06:31:12.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 -- 172.21.15.160:0/1828184913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fb5a0 msgr2=0x7fb6680fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/1828184913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fb5a0 0x7fb6680fb970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb65c004940 tx=0x7fb65c009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 -- 172.21.15.160:0/1828184913 shutdown_connections 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/1828184913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6680fbf40 0x7fb668101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/1828184913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fb5a0 0x7fb6680fb970 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 -- 172.21.15.160:0/1828184913 >> 172.21.15.160:0/1828184913 conn(0x7fb668073350 msgr2=0x7fb668073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 -- 172.21.15.160:0/1828184913 shutdown_connections 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.786+0000 7fb66ecd6700 1 -- 172.21.15.160:0/1828184913 wait complete. 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb66ecd6700 1 Processor -- start 2024-05-25T06:31:12.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb66ecd6700 1 -- start start 2024-05-25T06:31:12.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb66ecd6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6680fb5a0 0x7fb668113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb66ecd6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 0x7fb668113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb66ecd6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb668114510 con 0x7fb6680fbf40 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb66ecd6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66810db30 con 0x7fb6680fb5a0 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb667fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 0x7fb668113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb667fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 0x7fb668113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35458/0 (socket says 172.21.15.160:35458) 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb667fff700 1 -- 172.21.15.160:0/2045795631 learned_addr learned my addr 172.21.15.160:0/2045795631 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb667fff700 1 -- 172.21.15.160:0/2045795631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6680fb5a0 msgr2=0x7fb668113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb667fff700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6680fb5a0 0x7fb668113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.787+0000 7fb667fff700 1 -- 172.21.15.160:0/2045795631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c005040 con 0x7fb6680fbf40 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.788+0000 7fb667fff700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 0x7fb668113ec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb654009b80 tx=0x7fb654004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:12.790 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.788+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb654007cc0 con 0x7fb6680fbf40 2024-05-25T06:31:12.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.788+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb65400f040 con 0x7fb6680fbf40 2024-05-25T06:31:12.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.788+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66810de10 con 0x7fb6680fbf40 2024-05-25T06:31:12.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.788+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb65400dab0 con 0x7fb6680fbf40 2024-05-25T06:31:12.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.788+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66810e350 con 0x7fb6680fbf40 2024-05-25T06:31:12.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.789+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6680fe670 con 0x7fb6680fbf40 2024-05-25T06:31:12.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.789+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb65400dd30 con 0x7fb6680fbf40 2024-05-25T06:31:12.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.790+0000 7fb6657fa700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb65806e140 0x7fb6580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:12.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.790+0000 7fb66ca72700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb65806e140 0x7fb6580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:12.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.790+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb65408da10 con 0x7fb6680fbf40 2024-05-25T06:31:12.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.790+0000 7fb66ca72700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb65806e140 0x7fb6580705f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb65c004f90 tx=0x7fb65c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:12.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:12.795+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb654056cc0 con 0x7fb6680fbf40 2024-05-25T06:31:13.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.080+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb668109f40 con 0x7fb6680fbf40 2024-05-25T06:31:13.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.080+0000 7fb6657fa700 1 -- 172.21.15.160:0/2045795631 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb65405a380 con 0x7fb6680fbf40 2024-05-25T06:31:13.081 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:31:13.082 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:31:13.082 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:31:13.082 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:31:13.082 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb65806e140 msgr2=0x7fb6580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb65806e140 0x7fb6580705f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb65c004f90 tx=0x7fb65c009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 msgr2=0x7fb668113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 0x7fb668113ec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb654009b80 tx=0x7fb654004600 comp rx=0 tx=0).stop 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 shutdown_connections 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb65806e140 0x7fb6580705f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6680fb5a0 0x7fb668113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 --2- 172.21.15.160:0/2045795631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6680fbf40 0x7fb668113ec0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 >> 172.21.15.160:0/2045795631 conn(0x7fb668073350 msgr2=0x7fb668100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 shutdown_connections 2024-05-25T06:31:13.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:13.083+0000 7fb66ecd6700 1 -- 172.21.15.160:0/2045795631 wait complete. 2024-05-25T06:31:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1904172661' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:31:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:13 smithi203 ceph-mon[83507]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2045795631' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:31:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1904172661' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:31:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:13 smithi160 ceph-mon[112094]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2045795631' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:31:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:14 smithi203 ceph-mon[83507]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:14 smithi160 ceph-mon[112094]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:16.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:16 smithi203 ceph-mon[83507]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:16 smithi160 ceph-mon[112094]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:18 smithi160 ceph-mon[112094]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:18 smithi203 ceph-mon[83507]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:20 smithi160 ceph-mon[112094]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:20 smithi203 ceph-mon[83507]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:22.451 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:22 smithi203 ceph-mon[83507]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:22 smithi160 ceph-mon[112094]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:24 smithi160 ceph-mon[112094]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:24 smithi203 ceph-mon[83507]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:26 smithi160 ceph-mon[112094]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:26 smithi203 ceph-mon[83507]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:28 smithi160 ceph-mon[112094]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:28 smithi203 ceph-mon[83507]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:30 smithi160 ceph-mon[112094]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:30 smithi203 ceph-mon[83507]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:32 smithi160 ceph-mon[112094]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:32 smithi203 ceph-mon[83507]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:34 smithi160 ceph-mon[112094]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:34 smithi203 ceph-mon[83507]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:36 smithi160 ceph-mon[112094]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:36 smithi203 ceph-mon[83507]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:38.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:38 smithi160 ceph-mon[112094]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:38 smithi203 ceph-mon[83507]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:40.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:40 smithi160 ceph-mon[112094]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:40 smithi203 ceph-mon[83507]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:42 smithi160 ceph-mon[112094]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:42.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:42 smithi203 ceph-mon[83507]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:43.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 -- 172.21.15.160:0/3315139413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4101d70 msgr2=0x7f0ed4102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/3315139413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4101d70 0x7f0ed4102140 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4004940 tx=0x7f0ec4009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 -- 172.21.15.160:0/3315139413 shutdown_connections 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/3315139413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed4102710 0x7f0ed40fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/3315139413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4101d70 0x7f0ed4102140 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 -- 172.21.15.160:0/3315139413 >> 172.21.15.160:0/3315139413 conn(0x7f0ed40f78e0 msgr2=0x7f0ed40f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 -- 172.21.15.160:0/3315139413 shutdown_connections 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 -- 172.21.15.160:0/3315139413 wait complete. 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 Processor -- start 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.213+0000 7f0ed95f3700 1 -- start start 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ed95f3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed4101d70 0x7f0ed410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ed95f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 0x7f0ed410f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ed95f3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4067160 con 0x7f0ed4102710 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ed95f3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed410f5e0 con 0x7f0ed4101d70 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 0x7f0ed410f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 0x7f0ed410f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33116/0 (socket says 172.21.15.160:33116) 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 -- 172.21.15.160:0/579410648 learned_addr learned my addr 172.21.15.160:0/579410648 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ed2ffd700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed4101d70 0x7f0ed410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 -- 172.21.15.160:0/579410648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed4101d70 msgr2=0x7f0ed410eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed4101d70 0x7f0ed410eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 -- 172.21.15.160:0/579410648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4005040 con 0x7f0ed4102710 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.214+0000 7f0ecbfff700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 0x7f0ed410f0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc002a50 tx=0x7f0ebc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.215+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ebc007cc0 con 0x7f0ed4102710 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.215+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed410f860 con 0x7f0ed4102710 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.215+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ebc00f040 con 0x7f0ed4102710 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.215+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ebc00d9c0 con 0x7f0ed4102710 2024-05-25T06:31:43.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.215+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed4117ec0 con 0x7f0ed4102710 2024-05-25T06:31:43.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.216+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed40ff080 con 0x7f0ed4102710 2024-05-25T06:31:43.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.216+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0ebc009730 con 0x7f0ed4102710 2024-05-25T06:31:43.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.217+0000 7f0ecb7fe700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ec006e140 0x7f0ec00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.217+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0ebc08d8c0 con 0x7f0ed4102710 2024-05-25T06:31:43.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.217+0000 7f0ed2ffd700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ec006e140 0x7f0ec00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.217+0000 7f0ed2ffd700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ec006e140 0x7f0ec00705f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4004f90 tx=0x7f0ec4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:43.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.221+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0ebc05a720 con 0x7f0ed4102710 2024-05-25T06:31:43.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.438+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ed4108e70 con 0x7f0ec006e140 2024-05-25T06:31:43.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.439+0000 7f0ecb7fe700 1 -- 172.21.15.160:0/579410648 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0ed4108e70 con 0x7f0ec006e140 2024-05-25T06:31:43.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ec006e140 msgr2=0x7f0ec00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ec006e140 0x7f0ec00705f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4004f90 tx=0x7f0ec4009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 msgr2=0x7f0ed410f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 0x7f0ed410f0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc002a50 tx=0x7f0ebc004600 comp rx=0 tx=0).stop 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 shutdown_connections 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0ec006e140 0x7f0ec00705f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed4101d70 0x7f0ed410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 --2- 172.21.15.160:0/579410648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4102710 0x7f0ed410f0a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 >> 172.21.15.160:0/579410648 conn(0x7f0ed40f78e0 msgr2=0x7f0ed4106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 shutdown_connections 2024-05-25T06:31:43.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.441+0000 7f0ed95f3700 1 -- 172.21.15.160:0/579410648 wait complete. 2024-05-25T06:31:43.452 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:31:43.452 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:43.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 -- 172.21.15.160:0/81766224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 msgr2=0x7f925c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 --2- 172.21.15.160:0/81766224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c0fcdf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9244004940 tx=0x7f9244009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 -- 172.21.15.160:0/81766224 shutdown_connections 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 --2- 172.21.15.160:0/81766224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c0fcdf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 --2- 172.21.15.160:0/81766224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c102de0 0x7f925c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 -- 172.21.15.160:0/81766224 >> 172.21.15.160:0/81766224 conn(0x7f925c0f8950 msgr2=0x7f925c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 -- 172.21.15.160:0/81766224 shutdown_connections 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.546+0000 7f9262374700 1 -- 172.21.15.160:0/81766224 wait complete. 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9262374700 1 Processor -- start 2024-05-25T06:31:43.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9262374700 1 -- start start 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9262374700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c102de0 0x7f925c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9262374700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9262374700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c114730 con 0x7f925c103780 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9262374700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c10db20 con 0x7f925c102de0 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9253fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9253fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33134/0 (socket says 172.21.15.160:33134) 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9253fff700 1 -- 172.21.15.160:0/589668822 learned_addr learned my addr 172.21.15.160:0/589668822 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f925bfff700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c102de0 0x7f925c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9253fff700 1 -- 172.21.15.160:0/589668822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c102de0 msgr2=0x7f925c113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9253fff700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c102de0 0x7f925c113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.547+0000 7f9253fff700 1 -- 172.21.15.160:0/589668822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244005040 con 0x7f925c103780 2024-05-25T06:31:43.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.548+0000 7f9253fff700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c1140e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9244009100 tx=0x7f9244009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:43.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.548+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924400e040 con 0x7f925c103780 2024-05-25T06:31:43.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.548+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9244013420 con 0x7f925c103780 2024-05-25T06:31:43.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.548+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c10dda0 con 0x7f925c103780 2024-05-25T06:31:43.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.548+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924401c610 con 0x7f925c103780 2024-05-25T06:31:43.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.548+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c10e360 con 0x7f925c103780 2024-05-25T06:31:43.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.549+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c0fe530 con 0x7f925c103780 2024-05-25T06:31:43.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.549+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9244013590 con 0x7f925c103780 2024-05-25T06:31:43.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.550+0000 7f92597fa700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f924806e140 0x7f92480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.550+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f924408ee10 con 0x7f925c103780 2024-05-25T06:31:43.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.550+0000 7f925bfff700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f924806e140 0x7f92480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.550+0000 7f925bfff700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f924806e140 0x7f92480705f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f924c007fd0 tx=0x7f924c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:43.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.554+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f924405bcf0 con 0x7f925c103780 2024-05-25T06:31:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:43.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.768+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f925c109ec0 con 0x7f924806e140 2024-05-25T06:31:43.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.769+0000 7f92597fa700 1 -- 172.21.15.160:0/589668822 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f925c109ec0 con 0x7f924806e140 2024-05-25T06:31:43.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f924806e140 msgr2=0x7f92480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f924806e140 0x7f92480705f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f924c007fd0 tx=0x7f924c009040 comp rx=0 tx=0).stop 2024-05-25T06:31:43.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 msgr2=0x7f925c1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c1140e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9244009100 tx=0x7f9244009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 shutdown_connections 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f924806e140 0x7f92480705f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c102de0 0x7f925c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 --2- 172.21.15.160:0/589668822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f925c103780 0x7f925c1140e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 >> 172.21.15.160:0/589668822 conn(0x7f925c0f8950 msgr2=0x7f925c0f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.771+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 shutdown_connections 2024-05-25T06:31:43.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.772+0000 7f9262374700 1 -- 172.21.15.160:0/589668822 wait complete. 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 -- 172.21.15.160:0/19856714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 msgr2=0x7f1cd8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 --2- 172.21.15.160:0/19856714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8106e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8004940 tx=0x7f1cc8009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 -- 172.21.15.160:0/19856714 shutdown_connections 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 --2- 172.21.15.160:0/19856714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8106e40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 --2- 172.21.15.160:0/19856714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd80fde60 0x7f1cd80fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 -- 172.21.15.160:0/19856714 >> 172.21.15.160:0/19856714 conn(0x7f1cd80f78d0 msgr2=0x7f1cd80f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 -- 172.21.15.160:0/19856714 shutdown_connections 2024-05-25T06:31:43.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.875+0000 7f1ce0696700 1 -- 172.21.15.160:0/19856714 wait complete. 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1ce0696700 1 Processor -- start 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1ce0696700 1 -- start start 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1ce0696700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd80fde60 0x7f1cd8113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1ce0696700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1ce0696700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8114640 con 0x7f1cd80fe800 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1ce0696700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd810da30 con 0x7f1cd80fde60 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33150/0 (socket says 172.21.15.160:33150) 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 -- 172.21.15.160:0/576586450 learned_addr learned my addr 172.21.15.160:0/576586450 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:43.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cde432700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd80fde60 0x7f1cd8113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 -- 172.21.15.160:0/576586450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd80fde60 msgr2=0x7f1cd8113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd80fde60 0x7f1cd8113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 -- 172.21.15.160:0/576586450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8005040 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.876+0000 7f1cddc31700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8113ff0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.877+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc8021070 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.877+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cc8013c00 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.877+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd810dcb0 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.877+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc800e070 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.877+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd810e1f0 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.878+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd80666f0 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.878+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1cc801c400 con 0x7f1cd80fe800 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.879+0000 7f1cceffd700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1cc406e140 0x7f1cc40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:43.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.879+0000 7f1cde432700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1cc406e140 0x7f1cc40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:43.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.879+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1cc808f020 con 0x7f1cd80fe800 2024-05-25T06:31:43.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.879+0000 7f1cde432700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1cc406e140 0x7f1cc40705f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4001470 tx=0x7f1cd400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:43.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:43.884+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1cc805bf00 con 0x7f1cd80fe800 2024-05-25T06:31:44.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.089+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cd8109e90 con 0x7f1cc406e140 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.100+0000 7f1cceffd700 1 -- 172.21.15.160:0/576586450 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1cd8109e90 con 0x7f1cc406e140 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (17m) 10m ago 28m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (15m) 10m ago 28m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (14m) 10m ago 27m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (17m) 10m ago 28m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (11m) 10m ago 23m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (11m) 10m ago 23m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (10m) 10m ago 23m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:31:44.100 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (10m) 10m ago 23m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (19m) 10m ago 30m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (18m) 10m ago 27m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (16m) 10m ago 30m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (15m) 10m ago 27m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (18m) 10m ago 28m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (18m) 10m ago 27m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (14m) 10m ago 26m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (14m) 10m ago 26m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (13m) 10m ago 26m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (12m) 10m ago 25m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (12m) 10m ago 25m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:31:44.101 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (11m) 10m ago 24m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:31:44.102 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (17m) 10m ago 28m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:31:44.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1cc406e140 msgr2=0x7f1cc40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1cc406e140 0x7f1cc40705f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4001470 tx=0x7f1cd400b040 comp rx=0 tx=0).stop 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 msgr2=0x7f1cd8113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8113ff0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc8009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 shutdown_connections 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1cc406e140 0x7f1cc40705f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd80fde60 0x7f1cd8113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 --2- 172.21.15.160:0/576586450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cd80fe800 0x7f1cd8113ff0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 >> 172.21.15.160:0/576586450 conn(0x7f1cd80f78d0 msgr2=0x7f1cd80f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 shutdown_connections 2024-05-25T06:31:44.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.102+0000 7f1ce0696700 1 -- 172.21.15.160:0/576586450 wait complete. 2024-05-25T06:31:44.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 -- 172.21.15.160:0/2425866709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 msgr2=0x7fae3c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 --2- 172.21.15.160:0/2425866709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c101a30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fae30004940 tx=0x7fae30009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 -- 172.21.15.160:0/2425866709 shutdown_connections 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 --2- 172.21.15.160:0/2425866709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c101a30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 --2- 172.21.15.160:0/2425866709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c0fb5a0 0x7fae3c0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 -- 172.21.15.160:0/2425866709 >> 172.21.15.160:0/2425866709 conn(0x7fae3c073350 msgr2=0x7fae3c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 -- 172.21.15.160:0/2425866709 shutdown_connections 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.204+0000 7fae41c71700 1 -- 172.21.15.160:0/2425866709 wait complete. 2024-05-25T06:31:44.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae41c71700 1 Processor -- start 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae41c71700 1 -- start start 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae41c71700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c0fb5a0 0x7fae3c0fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae41c71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c0fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae41c71700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0fe480 con 0x7fae3c0fbf40 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae41c71700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0fcfb0 con 0x7fae3c0fb5a0 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae3affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c0fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae3affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c0fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33178/0 (socket says 172.21.15.160:33178) 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.205+0000 7fae3affd700 1 -- 172.21.15.160:0/1551595545 learned_addr learned my addr 172.21.15.160:0/1551595545 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae3b7fe700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c0fb5a0 0x7fae3c0fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae3affd700 1 -- 172.21.15.160:0/1551595545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c0fb5a0 msgr2=0x7fae3c0fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae3affd700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c0fb5a0 0x7fae3c0fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae3affd700 1 -- 172.21.15.160:0/1551595545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30005040 con 0x7fae3c0fbf40 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae3affd700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c0fca70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b3b0 tx=0x7fae30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3000e040 con 0x7fae3c0fbf40 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0fd230 con 0x7fae3c0fbf40 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae30013420 con 0x7fae3c0fbf40 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3001c630 con 0x7fae3c0fbf40 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.206+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c11a0b0 con 0x7fae3c0fbf40 2024-05-25T06:31:44.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.207+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c0fe670 con 0x7fae3c0fbf40 2024-05-25T06:31:44.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.208+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fae30013590 con 0x7fae3c0fbf40 2024-05-25T06:31:44.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.208+0000 7fae23fff700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fae2406e140 0x7fae240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.208+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fae3008ed70 con 0x7fae3c0fbf40 2024-05-25T06:31:44.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.208+0000 7fae3b7fe700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fae2406e140 0x7fae240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.209+0000 7fae3b7fe700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fae2406e140 0x7fae240705f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fae2c004990 tx=0x7fae2c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:44.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.213+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fae300581d0 con 0x7fae3c0fbf40 2024-05-25T06:31:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:44 smithi160 ceph-mon[112094]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:44 smithi160 ceph-mon[112094]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:44 smithi160 ceph-mon[112094]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.494+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae3c109f00 con 0x7fae3c0fbf40 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.495+0000 7fae23fff700 1 -- 172.21.15.160:0/1551595545 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fae30013840 con 0x7fae3c0fbf40 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:31:44.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:31:44.496 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fae2406e140 msgr2=0x7fae240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fae2406e140 0x7fae240705f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fae2c004990 tx=0x7fae2c009ee0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 msgr2=0x7fae3c0fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c0fca70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b3b0 tx=0x7fae30009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 shutdown_connections 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fae2406e140 0x7fae240705f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c0fb5a0 0x7fae3c0fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 --2- 172.21.15.160:0/1551595545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3c0fbf40 0x7fae3c0fca70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 >> 172.21.15.160:0/1551595545 conn(0x7fae3c073350 msgr2=0x7fae3c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 shutdown_connections 2024-05-25T06:31:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.497+0000 7fae41c71700 1 -- 172.21.15.160:0/1551595545 wait complete. 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.599+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/2841115454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 msgr2=0x7fa7cc1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/2841115454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc1020e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc004940 tx=0x7fa7bc009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/2841115454 shutdown_connections 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/2841115454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7cc1026b0 0x7fa7cc0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/2841115454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc1020e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/2841115454 >> 172.21.15.160:0/2841115454 conn(0x7fa7cc0f7860 msgr2=0x7fa7cc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/2841115454 shutdown_connections 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/2841115454 wait complete. 2024-05-25T06:31:44.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 Processor -- start 2024-05-25T06:31:44.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- start start 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7cc1026b0 0x7fa7cc1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc104b10 con 0x7fa7cc101d10 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.600+0000 7fa7d0e6f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1035f0 con 0x7fa7cc1026b0 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33196/0 (socket says 172.21.15.160:33196) 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 -- 172.21.15.160:0/933046422 learned_addr learned my addr 172.21.15.160:0/933046422 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 -- 172.21.15.160:0/933046422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7cc1026b0 msgr2=0x7fa7cc1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7cc1026b0 0x7fa7cc1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 -- 172.21.15.160:0/933046422 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc005040 con 0x7fa7cc101d10 2024-05-25T06:31:44.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7ca59c700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc102b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00b3b0 tx=0x7fa7bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7bc00e040 con 0x7fa7cc101d10 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7bc00bbc0 con 0x7fa7cc101d10 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc103840 con 0x7fa7cc101d10 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7bc01c3f0 con 0x7fa7cc101d10 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.601+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc11a100 con 0x7fa7cc101d10 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.602+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc0fd460 con 0x7fa7cc101d10 2024-05-25T06:31:44.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.603+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa7bc00bd30 con 0x7fa7cc101d10 2024-05-25T06:31:44.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.603+0000 7fa7baffd700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa7b406e140 0x7fa7b40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.603+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa7bc08edf0 con 0x7fa7cc101d10 2024-05-25T06:31:44.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.603+0000 7fa7c9d9b700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa7b406e140 0x7fa7b40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.604+0000 7fa7c9d9b700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa7b406e140 0x7fa7b40705f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc1042a0 tx=0x7fa7c0007cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:44.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.608+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa7bc058120 con 0x7fa7cc101d10 2024-05-25T06:31:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:44 smithi203 ceph-mon[83507]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:44 smithi203 ceph-mon[83507]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:44 smithi203 ceph-mon[83507]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.844+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa7cc11a350 con 0x7fa7cc101d10 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.844+0000 7fa7baffd700 1 -- 172.21.15.160:0/933046422 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fa7bc05b7e0 con 0x7fa7cc101d10 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:44.844 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:31:44.845 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:31:44.846 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:44.847 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa7b406e140 msgr2=0x7fa7b40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa7b406e140 0x7fa7b40705f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc1042a0 tx=0x7fa7c0007cc0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 msgr2=0x7fa7cc102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc102b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00b3b0 tx=0x7fa7bc009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 shutdown_connections 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7cc101d10 0x7fa7cc102b70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa7b406e140 0x7fa7b40705f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 --2- 172.21.15.160:0/933046422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7cc1026b0 0x7fa7cc1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 >> 172.21.15.160:0/933046422 conn(0x7fa7cc0f7860 msgr2=0x7fa7cc066270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 shutdown_connections 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.846+0000 7fa7d0e6f700 1 -- 172.21.15.160:0/933046422 wait complete. 2024-05-25T06:31:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:31:44.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/4129511527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58102670 msgr2=0x7f7f580fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/4129511527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58102670 0x7f7f580fbce0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c004940 tx=0x7f7f4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/4129511527 shutdown_connections 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/4129511527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58102670 0x7f7f580fbce0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/4129511527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f58101cd0 0x7f7f581020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/4129511527 >> 172.21.15.160:0/4129511527 conn(0x7f7f580f7860 msgr2=0x7f7f580f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.949+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/4129511527 shutdown_connections 2024-05-25T06:31:44.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.950+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/4129511527 wait complete. 2024-05-25T06:31:44.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.950+0000 7f7f5d3e9700 1 Processor -- start 2024-05-25T06:31:44.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.950+0000 7f7f5d3e9700 1 -- start start 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.950+0000 7f7f5d3e9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 0x7f7f5810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.950+0000 7f7f5d3e9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f58102670 0x7f7f5810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f5d3e9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5810f570 con 0x7f7f58101cd0 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f5d3e9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5810f6b0 con 0x7f7f58102670 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 0x7f7f5810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 0x7f7f5810eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33204/0 (socket says 172.21.15.160:33204) 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 -- 172.21.15.160:0/1240604316 learned_addr learned my addr 172.21.15.160:0/1240604316 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f567fc700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f58102670 0x7f7f5810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 -- 172.21.15.160:0/1240604316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f58102670 msgr2=0x7f7f5810f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f58102670 0x7f7f5810f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 -- 172.21.15.160:0/1240604316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c005040 con 0x7f7f58101cd0 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f56ffd700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 0x7f7f5810eaf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f48002a50 tx=0x7f7f480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f48007cc0 con 0x7f7f58101cd0 2024-05-25T06:31:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f58117b60 con 0x7f7f58101cd0 2024-05-25T06:31:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f4800f040 con 0x7f7f58101cd0 2024-05-25T06:31:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4800da30 con 0x7f7f58101cd0 2024-05-25T06:31:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.951+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f58117f30 con 0x7f7f58101cd0 2024-05-25T06:31:44.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.952+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f580fd420 con 0x7f7f58101cd0 2024-05-25T06:31:44.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.953+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7f4800db90 con 0x7f7f58101cd0 2024-05-25T06:31:44.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.953+0000 7f7f3f7fe700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f4006e140 0x7f7f400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:44.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.954+0000 7f7f567fc700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f4006e140 0x7f7f400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:44.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.954+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f4808d910 con 0x7f7f58101cd0 2024-05-25T06:31:44.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.954+0000 7f7f567fc700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f4006e140 0x7f7f400705f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f58066a80 tx=0x7f7f4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:44.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:44.958+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7f48056c40 con 0x7f7f58101cd0 2024-05-25T06:31:45.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.176+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f580668a0 con 0x7f7f4006e140 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.177+0000 7f7f3f7fe700 1 -- 172.21.15.160:0/1240604316 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f580668a0 con 0x7f7f4006e140 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:31:45.177 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:31:45.178 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:31:45.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f4006e140 msgr2=0x7f7f400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:45.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f4006e140 0x7f7f400705f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f58066a80 tx=0x7f7f4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:45.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 msgr2=0x7f7f5810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:45.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 0x7f7f5810eaf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f48002a50 tx=0x7f7f480046b0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 shutdown_connections 2024-05-25T06:31:45.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f58101cd0 0x7f7f5810eaf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7f4006e140 0x7f7f400705f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 --2- 172.21.15.160:0/1240604316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f58102670 0x7f7f5810f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.179+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 >> 172.21.15.160:0/1240604316 conn(0x7f7f580f7860 msgr2=0x7f7f580f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:45.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.180+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 shutdown_connections 2024-05-25T06:31:45.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.180+0000 7f7f5d3e9700 1 -- 172.21.15.160:0/1240604316 wait complete. 2024-05-25T06:31:45.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.281+0000 7fcd096c9700 1 -- 172.21.15.160:0/1484189544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 msgr2=0x7fcd040fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.281+0000 7fcd096c9700 1 --2- 172.21.15.160:0/1484189544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd040fc1b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fccf8004940 tx=0x7fccf8009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.281+0000 7fcd096c9700 1 -- 172.21.15.160:0/1484189544 shutdown_connections 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.281+0000 7fcd096c9700 1 --2- 172.21.15.160:0/1484189544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd040fc1b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.281+0000 7fcd096c9700 1 --2- 172.21.15.160:0/1484189544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101d90 0x7fcd04102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.281+0000 7fcd096c9700 1 -- 172.21.15.160:0/1484189544 >> 172.21.15.160:0/1484189544 conn(0x7fcd04071d00 msgr2=0x7fcd04072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 -- 172.21.15.160:0/1484189544 shutdown_connections 2024-05-25T06:31:45.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 -- 172.21.15.160:0/1484189544 wait complete. 2024-05-25T06:31:45.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 Processor -- start 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 -- start start 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd04106b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101d90 0x7fcd041070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041076c0 con 0x7fcd040fbd40 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.282+0000 7fcd096c9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04107800 con 0x7fcd04101d90 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd04106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd04106b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33230/0 (socket says 172.21.15.160:33230) 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 -- 172.21.15.160:0/3944957794 learned_addr learned my addr 172.21.15.160:0/3944957794 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd027fc700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101d90 0x7fcd041070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 -- 172.21.15.160:0/3944957794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101d90 msgr2=0x7fcd041070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101d90 0x7fcd041070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 -- 172.21.15.160:0/3944957794 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8005040 con 0x7fcd040fbd40 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd02ffd700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd04106b60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fccf400b790 tx=0x7fccf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf400f770 con 0x7fcd040fbd40 2024-05-25T06:31:45.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf400fda0 con 0x7fcd040fbd40 2024-05-25T06:31:45.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd04119ea0 con 0x7fcd040fbd40 2024-05-25T06:31:45.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf400d310 con 0x7fcd040fbd40 2024-05-25T06:31:45.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.283+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0411a380 con 0x7fcd040fbd40 2024-05-25T06:31:45.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.284+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd040fd430 con 0x7fcd040fbd40 2024-05-25T06:31:45.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.285+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fccf4010420 con 0x7fcd040fbd40 2024-05-25T06:31:45.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.285+0000 7fcceb7fe700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fccec06e150 0x7fccec070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:31:45.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.285+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fccf408db90 con 0x7fcd040fbd40 2024-05-25T06:31:45.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.285+0000 7fcd027fc700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fccec06e150 0x7fccec070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:31:45.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.286+0000 7fcd027fc700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fccec06e150 0x7fccec070600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd041084a0 tx=0x7fccf8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:31:45.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.290+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fccf4056f40 con 0x7fcd040fbd40 2024-05-25T06:31:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:45 smithi160 ceph-mon[112094]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:45 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1551595545' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:31:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:45 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/933046422' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.568+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcd0404bf60 con 0x7fcd040fbd40 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.568+0000 7fcceb7fe700 1 -- 172.21.15.160:0/3944957794 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fccf405a600 con 0x7fcd040fbd40 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:31:45.568 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:31:45.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fccec06e150 msgr2=0x7fccec070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:45.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fccec06e150 0x7fccec070600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd041084a0 tx=0x7fccf8009e30 comp rx=0 tx=0).stop 2024-05-25T06:31:45.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 msgr2=0x7fcd04106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd04106b60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fccf400b790 tx=0x7fccf4009250 comp rx=0 tx=0).stop 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 shutdown_connections 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd040fbd40 0x7fcd04106b60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fccec06e150 0x7fccec070600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 --2- 172.21.15.160:0/3944957794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101d90 0x7fcd041070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 >> 172.21.15.160:0/3944957794 conn(0x7fcd04071d00 msgr2=0x7fcd040fa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 shutdown_connections 2024-05-25T06:31:45.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:31:45.570+0000 7fcd096c9700 1 -- 172.21.15.160:0/3944957794 wait complete. 2024-05-25T06:31:45.571 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:45 smithi203 ceph-mon[83507]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:45.572 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:45 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1551595545' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:31:45.572 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:45 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/933046422' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:31:46.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:46 smithi160 ceph-mon[112094]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:46.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:46 smithi160 ceph-mon[112094]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:46.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:46 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3944957794' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:31:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:46 smithi203 ceph-mon[83507]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:46 smithi203 ceph-mon[83507]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:31:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:46 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3944957794' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:31:48.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:48 smithi160 ceph-mon[112094]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:48 smithi203 ceph-mon[83507]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:50.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:50 smithi160 ceph-mon[112094]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:50 smithi203 ceph-mon[83507]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:52.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:52 smithi160 ceph-mon[112094]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:52 smithi203 ceph-mon[83507]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:54.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:54 smithi160 ceph-mon[112094]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:54 smithi203 ceph-mon[83507]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:56 smithi160 ceph-mon[112094]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:56 smithi203 ceph-mon[83507]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:58 smithi160 ceph-mon[112094]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:31:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:31:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:58 smithi203 ceph-mon[83507]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:31:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:31:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:00 smithi203 ceph-mon[83507]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:00.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:00 smithi160 ceph-mon[112094]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:02 smithi203 ceph-mon[83507]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:02.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:02 smithi160 ceph-mon[112094]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:04 smithi203 ceph-mon[83507]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:04.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:04 smithi160 ceph-mon[112094]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:06.521 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:06 smithi160 ceph-mon[112094]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:06 smithi203 ceph-mon[83507]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:08 smithi203 ceph-mon[83507]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:08 smithi160 ceph-mon[112094]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:10 smithi203 ceph-mon[83507]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:10 smithi160 ceph-mon[112094]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:12 smithi203 ceph-mon[83507]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:12 smithi160 ceph-mon[112094]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:14 smithi203 ceph-mon[83507]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:14.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:14 smithi160 ceph-mon[112094]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:15.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.685+0000 7f08930d3700 1 -- 172.21.15.160:0/2146431251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c103760 msgr2=0x7f088c0fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:15.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.685+0000 7f08930d3700 1 --2- 172.21.15.160:0/2146431251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c103760 0x7f088c0fcdd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0878004940 tx=0x7f0878009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:15.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 -- 172.21.15.160:0/2146431251 shutdown_connections 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 --2- 172.21.15.160:0/2146431251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c103760 0x7f088c0fcdd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 --2- 172.21.15.160:0/2146431251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088c102dc0 0x7f088c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 -- 172.21.15.160:0/2146431251 >> 172.21.15.160:0/2146431251 conn(0x7f088c0f8930 msgr2=0x7f088c0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 -- 172.21.15.160:0/2146431251 shutdown_connections 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 -- 172.21.15.160:0/2146431251 wait complete. 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 Processor -- start 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.686+0000 7f08930d3700 1 -- start start 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08930d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 0x7f088c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08930d3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088c103760 0x7f088c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08930d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0721b0 con 0x7f088c102dc0 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08930d3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c070ce0 con 0x7f088c103760 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 0x7f088c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:15.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 0x7f088c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60048/0 (socket says 172.21.15.160:60048) 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 -- 172.21.15.160:0/345843053 learned_addr learned my addr 172.21.15.160:0/345843053 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f088bfff700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088c103760 0x7f088c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 -- 172.21.15.160:0/345843053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088c103760 msgr2=0x7f088c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088c103760 0x7f088c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 -- 172.21.15.160:0/345843053 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878005040 con 0x7f088c102dc0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f0890e6f700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 0x7f088c070260 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0880002a50 tx=0x7f08800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0880007cc0 con 0x7f088c102dc0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c070fc0 con 0x7f088c102dc0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f088000f040 con 0x7f088c102dc0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.687+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088000da30 con 0x7f088c102dc0 2024-05-25T06:32:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.688+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c11a0a0 con 0x7f088c102dc0 2024-05-25T06:32:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.688+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c0fe510 con 0x7f088c102dc0 2024-05-25T06:32:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.689+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f088000db90 con 0x7f088c102dc0 2024-05-25T06:32:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.690+0000 7f08897fa700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f087c06e140 0x7f087c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.690+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f088008d920 con 0x7f088c102dc0 2024-05-25T06:32:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.690+0000 7f088bfff700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f087c06e140 0x7f087c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.690+0000 7f088bfff700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f087c06e140 0x7f087c0705f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0878004890 tx=0x7f0878009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:15.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.694+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0880056c50 con 0x7f088c102dc0 2024-05-25T06:32:15.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.911+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f088c0fa000 con 0x7f087c06e140 2024-05-25T06:32:15.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.913+0000 7f08897fa700 1 -- 172.21.15.160:0/345843053 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f088c0fa000 con 0x7f087c06e140 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f087c06e140 msgr2=0x7f087c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f087c06e140 0x7f087c0705f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0878004890 tx=0x7f0878009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 msgr2=0x7f088c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 0x7f088c070260 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0880002a50 tx=0x7f08800046b0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 shutdown_connections 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f088c102dc0 0x7f088c070260 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f087c06e140 0x7f087c0705f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 --2- 172.21.15.160:0/345843053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088c103760 0x7f088c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 >> 172.21.15.160:0/345843053 conn(0x7f088c0f8930 msgr2=0x7f088c0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 shutdown_connections 2024-05-25T06:32:15.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:15.915+0000 7f08930d3700 1 -- 172.21.15.160:0/345843053 wait complete. 2024-05-25T06:32:15.925 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:32:16.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 -- 172.21.15.160:0/2866410969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c103780 msgr2=0x7f571c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 --2- 172.21.15.160:0/2866410969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c103780 0x7f571c0fcdf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5710004940 tx=0x7f5710009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:16.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 -- 172.21.15.160:0/2866410969 shutdown_connections 2024-05-25T06:32:16.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 --2- 172.21.15.160:0/2866410969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c103780 0x7f571c0fcdf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 --2- 172.21.15.160:0/2866410969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c102de0 0x7f571c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 -- 172.21.15.160:0/2866410969 >> 172.21.15.160:0/2866410969 conn(0x7f571c0f8950 msgr2=0x7f571c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 -- 172.21.15.160:0/2866410969 shutdown_connections 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 -- 172.21.15.160:0/2866410969 wait complete. 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.020+0000 7f57209b2700 1 Processor -- start 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f57209b2700 1 -- start start 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f57209b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 0x7f571c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f57209b2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c103780 0x7f571c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f57209b2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c114730 con 0x7f571c102de0 2024-05-25T06:32:16.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f57209b2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10db20 con 0x7f571c103780 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 0x7f571c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 0x7f571c113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60052/0 (socket says 172.21.15.160:60052) 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 -- 172.21.15.160:0/2665006744 learned_addr learned my addr 172.21.15.160:0/2665006744 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f5719d9b700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c103780 0x7f571c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 -- 172.21.15.160:0/2665006744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c103780 msgr2=0x7f571c1140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c103780 0x7f571c1140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 -- 172.21.15.160:0/2665006744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5710005040 con 0x7f571c102de0 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.021+0000 7f571a59c700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 0x7f571c113ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c8c0 tx=0x7f570c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:16.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.022+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f570c00a760 con 0x7f571c102de0 2024-05-25T06:32:16.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.022+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f570c00ad90 con 0x7f571c102de0 2024-05-25T06:32:16.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.022+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c10de00 con 0x7f571c102de0 2024-05-25T06:32:16.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.022+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f570c0113a0 con 0x7f571c102de0 2024-05-25T06:32:16.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.022+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c10e390 con 0x7f571c102de0 2024-05-25T06:32:16.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.022+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c0fe530 con 0x7f571c102de0 2024-05-25T06:32:16.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.023+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f570c012400 con 0x7f571c102de0 2024-05-25T06:32:16.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.023+0000 7f570affd700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570406e140 0x7f57040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.024+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f570c08db60 con 0x7f571c102de0 2024-05-25T06:32:16.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.024+0000 7f5719d9b700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570406e140 0x7f57040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.024+0000 7f5719d9b700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570406e140 0x7f57040705f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f571c10ee00 tx=0x7f5710009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:16.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.028+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f570c05aa40 con 0x7f571c102de0 2024-05-25T06:32:16.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.242+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f571c10df90 con 0x7f570406e140 2024-05-25T06:32:16.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.243+0000 7f570affd700 1 -- 172.21.15.160:0/2665006744 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f571c10df90 con 0x7f570406e140 2024-05-25T06:32:16.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570406e140 msgr2=0x7f57040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570406e140 0x7f57040705f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f571c10ee00 tx=0x7f5710009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:16.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 msgr2=0x7f571c113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 0x7f571c113ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c8c0 tx=0x7f570c0044c0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 shutdown_connections 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f571c102de0 0x7f571c113ba0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f570406e140 0x7f57040705f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 --2- 172.21.15.160:0/2665006744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c103780 0x7f571c1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.245+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 >> 172.21.15.160:0/2665006744 conn(0x7f571c0f8950 msgr2=0x7f571c0f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.246+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 shutdown_connections 2024-05-25T06:32:16.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.246+0000 7f57209b2700 1 -- 172.21.15.160:0/2665006744 wait complete. 2024-05-25T06:32:16.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 -- 172.21.15.160:0/2881395203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 msgr2=0x7f6ec8109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.349 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 --2- 172.21.15.160:0/2881395203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8109150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc004970 tx=0x7f6ebc009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:16.349 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:16 smithi160 ceph-mon[112094]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:16.349 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:16 smithi160 ceph-mon[112094]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 -- 172.21.15.160:0/2881395203 shutdown_connections 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 --2- 172.21.15.160:0/2881395203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8109150 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 --2- 172.21.15.160:0/2881395203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec80702a0 0x7f6ec8070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 -- 172.21.15.160:0/2881395203 >> 172.21.15.160:0/2881395203 conn(0x7f6ec80f8990 msgr2=0x7f6ec80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 -- 172.21.15.160:0/2881395203 shutdown_connections 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.349+0000 7f6eccd80700 1 -- 172.21.15.160:0/2881395203 wait complete. 2024-05-25T06:32:16.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6eccd80700 1 Processor -- start 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6eccd80700 1 -- start start 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6eccd80700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec80702a0 0x7f6ec8115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6eccd80700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6eccd80700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec81166e0 con 0x7f6ec8070bb0 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6eccd80700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec810fcc0 con 0x7f6ec80702a0 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60070/0 (socket says 172.21.15.160:60070) 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec5d9b700 1 -- 172.21.15.160:0/277996164 learned_addr learned my addr 172.21.15.160:0/277996164 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec659c700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec80702a0 0x7f6ec8115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec5d9b700 1 -- 172.21.15.160:0/277996164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec80702a0 msgr2=0x7f6ec8115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec5d9b700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec80702a0 0x7f6ec8115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.350+0000 7f6ec5d9b700 1 -- 172.21.15.160:0/277996164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc005040 con 0x7f6ec8070bb0 2024-05-25T06:32:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.351+0000 7f6ec5d9b700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8116090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc000c00 tx=0x7f6ebc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.351+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ebc00e040 con 0x7f6ec8070bb0 2024-05-25T06:32:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.351+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ebc00bc10 con 0x7f6ec8070bb0 2024-05-25T06:32:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.351+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec810ff40 con 0x7f6ec8070bb0 2024-05-25T06:32:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.351+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ebc01c3f0 con 0x7f6ec8070bb0 2024-05-25T06:32:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.351+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec8110480 con 0x7f6ec8070bb0 2024-05-25T06:32:16.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.352+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec80719e0 con 0x7f6ec8070bb0 2024-05-25T06:32:16.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.353+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6ebc00bd80 con 0x7f6ec8070bb0 2024-05-25T06:32:16.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.353+0000 7f6eb6ffd700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6eb006e140 0x7f6eb00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.353+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6ebc08ee90 con 0x7f6ec8070bb0 2024-05-25T06:32:16.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.353+0000 7f6ec659c700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6eb006e140 0x7f6eb00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.353+0000 7f6ec659c700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6eb006e140 0x7f6eb00705f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8007fd0 tx=0x7f6eb8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:16.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.357+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6ebc05bd70 con 0x7f6ec8070bb0 2024-05-25T06:32:16.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.562+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ec8111210 con 0x7f6eb006e140 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.572+0000 7f6eb6ffd700 1 -- 172.21.15.160:0/277996164 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ec8111210 con 0x7f6eb006e140 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (18m) 11m ago 29m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (15m) 11m ago 29m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (15m) 11m ago 28m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (17m) 11m ago 28m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:32:16.573 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (11m) 11m ago 23m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (11m) 11m ago 23m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (11m) 11m ago 23m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (11m) 11m ago 23m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (19m) 11m ago 30m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (19m) 11m ago 28m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (16m) 11m ago 30m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (16m) 11m ago 28m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (18m) 11m ago 29m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (18m) 11m ago 28m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (15m) 11m ago 27m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (14m) 11m ago 27m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:32:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (13m) 11m ago 26m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:32:16.575 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (13m) 11m ago 26m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:32:16.575 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (12m) 11m ago 25m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:32:16.575 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (12m) 11m ago 25m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:32:16.575 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (18m) 11m ago 28m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:32:16.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6eb006e140 msgr2=0x7f6eb00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6eb006e140 0x7f6eb00705f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8007fd0 tx=0x7f6eb8009040 comp rx=0 tx=0).stop 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 msgr2=0x7f6ec8116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8116090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc000c00 tx=0x7f6ebc009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 shutdown_connections 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6eb006e140 0x7f6eb00705f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec80702a0 0x7f6ec8115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 --2- 172.21.15.160:0/277996164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec8070bb0 0x7f6ec8116090 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 >> 172.21.15.160:0/277996164 conn(0x7f6ec80f8990 msgr2=0x7f6ec80f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 shutdown_connections 2024-05-25T06:32:16.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.574+0000 7f6eccd80700 1 -- 172.21.15.160:0/277996164 wait complete. 2024-05-25T06:32:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:16 smithi203 ceph-mon[83507]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:16 smithi203 ceph-mon[83507]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.675+0000 7fb2e9674700 1 -- 172.21.15.160:0/1154265092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fe800 msgr2=0x7fb2e4106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.675+0000 7fb2e9674700 1 --2- 172.21.15.160:0/1154265092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fe800 0x7fb2e4106e40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8004940 tx=0x7fb2d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.675+0000 7fb2e9674700 1 -- 172.21.15.160:0/1154265092 shutdown_connections 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.675+0000 7fb2e9674700 1 --2- 172.21.15.160:0/1154265092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fe800 0x7fb2e4106e40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.675+0000 7fb2e9674700 1 --2- 172.21.15.160:0/1154265092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40fde60 0x7fb2e40fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.675+0000 7fb2e9674700 1 -- 172.21.15.160:0/1154265092 >> 172.21.15.160:0/1154265092 conn(0x7fb2e40f7810 msgr2=0x7fb2e40f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 -- 172.21.15.160:0/1154265092 shutdown_connections 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 -- 172.21.15.160:0/1154265092 wait complete. 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 Processor -- start 2024-05-25T06:32:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 -- start start 2024-05-25T06:32:16.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 0x7fb2e4113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40fe800 0x7fb2e4113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4114490 con 0x7fb2e40fde60 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.676+0000 7fb2e9674700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e410da90 con 0x7fb2e40fe800 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 0x7fb2e4113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 0x7fb2e4113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60082/0 (socket says 172.21.15.160:60082) 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 -- 172.21.15.160:0/4259607583 learned_addr learned my addr 172.21.15.160:0/4259607583 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e27fc700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40fe800 0x7fb2e4113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 -- 172.21.15.160:0/4259607583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40fe800 msgr2=0x7fb2e4113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40fe800 0x7fb2e4113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 -- 172.21.15.160:0/4259607583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8005040 con 0x7fb2e40fde60 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e2ffd700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 0x7fb2e4113900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c830 tx=0x7fb2d4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d400a760 con 0x7fb2e40fde60 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2d400ad90 con 0x7fb2e40fde60 2024-05-25T06:32:16.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e410dd70 con 0x7fb2e40fde60 2024-05-25T06:32:16.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d40113a0 con 0x7fb2e40fde60 2024-05-25T06:32:16.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.677+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e410e2b0 con 0x7fb2e40fde60 2024-05-25T06:32:16.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.678+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e4066a70 con 0x7fb2e40fde60 2024-05-25T06:32:16.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.679+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb2d4011610 con 0x7fb2e40fde60 2024-05-25T06:32:16.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.679+0000 7fb2cb7fe700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb2cc06e140 0x7fb2cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:16.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.679+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb2d408dbb0 con 0x7fb2e40fde60 2024-05-25T06:32:16.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.679+0000 7fb2e27fc700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb2cc06e140 0x7fb2cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:16.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.680+0000 7fb2e27fc700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb2cc06e140 0x7fb2cc0705f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2e410ec50 tx=0x7fb2d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:16.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.684+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb2d405abe0 con 0x7fb2e40fde60 2024-05-25T06:32:16.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.964+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2e4109e20 con 0x7fb2e40fde60 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.965+0000 7fb2cb7fe700 1 -- 172.21.15.160:0/4259607583 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fb2d405a770 con 0x7fb2e40fde60 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:32:16.966 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:32:16.967 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:32:16.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb2cc06e140 msgr2=0x7fb2cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb2cc06e140 0x7fb2cc0705f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2e410ec50 tx=0x7fb2d8009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:16.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 msgr2=0x7fb2e4113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 0x7fb2e4113900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c830 tx=0x7fb2d4004470 comp rx=0 tx=0).stop 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 shutdown_connections 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2e40fde60 0x7fb2e4113900 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb2cc06e140 0x7fb2cc0705f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 --2- 172.21.15.160:0/4259607583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40fe800 0x7fb2e4113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.967+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 >> 172.21.15.160:0/4259607583 conn(0x7fb2e40f7810 msgr2=0x7fb2e40f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.968+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 shutdown_connections 2024-05-25T06:32:16.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:16.968+0000 7fb2e9674700 1 -- 172.21.15.160:0/4259607583 wait complete. 2024-05-25T06:32:17.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 -- 172.21.15.160:0/1110549111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670102610 msgr2=0x7fe6700fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 --2- 172.21.15.160:0/1110549111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670102610 0x7fe6700fbc80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe658004940 tx=0x7fe658009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 -- 172.21.15.160:0/1110549111 shutdown_connections 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 --2- 172.21.15.160:0/1110549111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670102610 0x7fe6700fbc80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 --2- 172.21.15.160:0/1110549111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe670101c70 0x7fe670102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 -- 172.21.15.160:0/1110549111 >> 172.21.15.160:0/1110549111 conn(0x7fe670072b60 msgr2=0x7fe670072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 -- 172.21.15.160:0/1110549111 shutdown_connections 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.070+0000 7fe675ec2700 1 -- 172.21.15.160:0/1110549111 wait complete. 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe675ec2700 1 Processor -- start 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe675ec2700 1 -- start start 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe675ec2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 0x7fe67010ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe675ec2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe670102610 0x7fe67010efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe675ec2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67010f5f0 con 0x7fe670101c70 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe675ec2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67010f730 con 0x7fe670102610 2024-05-25T06:32:17.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 0x7fe67010ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 0x7fe67010ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60100/0 (socket says 172.21.15.160:60100) 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 -- 172.21.15.160:0/3329325717 learned_addr learned my addr 172.21.15.160:0/3329325717 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe667fff700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe670102610 0x7fe67010efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 -- 172.21.15.160:0/3329325717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe670102610 msgr2=0x7fe67010efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe670102610 0x7fe67010efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 -- 172.21.15.160:0/3329325717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe658005040 con 0x7fe670101c70 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.071+0000 7fe66f7fe700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 0x7fe67010ea60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe660002a50 tx=0x7fe6600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.072+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe660007cc0 con 0x7fe670101c70 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.072+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe66000f040 con 0x7fe670101c70 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.072+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe670117ad0 con 0x7fe670101c70 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.072+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66000da30 con 0x7fe670101c70 2024-05-25T06:32:17.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.072+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe670118000 con 0x7fe670101c70 2024-05-25T06:32:17.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.073+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6700fd3c0 con 0x7fe670101c70 2024-05-25T06:32:17.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.073+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe66000dbb0 con 0x7fe670101c70 2024-05-25T06:32:17.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.074+0000 7fe66cff9700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe65c06e140 0x7fe65c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.074+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe66008d830 con 0x7fe670101c70 2024-05-25T06:32:17.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.074+0000 7fe667fff700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe65c06e140 0x7fe65c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.074+0000 7fe667fff700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe65c06e140 0x7fe65c0705f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe658004890 tx=0x7fe658009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:17.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.078+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe660056be0 con 0x7fe670101c70 2024-05-25T06:32:17.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.323+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe67004bf00 con 0x7fe670101c70 2024-05-25T06:32:17.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.323+0000 7fe66cff9700 1 -- 172.21.15.160:0/3329325717 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fe66008be60 con 0x7fe670101c70 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:32:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:32:17.326 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:32:17.328 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:17.328 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:17.328 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:17.328 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:17.328 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe65c06e140 msgr2=0x7fe65c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe65c06e140 0x7fe65c0705f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe658004890 tx=0x7fe658009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 msgr2=0x7fe67010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 0x7fe67010ea60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe660002a50 tx=0x7fe6600046b0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 shutdown_connections 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe670101c70 0x7fe67010ea60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe65c06e140 0x7fe65c0705f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 --2- 172.21.15.160:0/3329325717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe670102610 0x7fe67010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 >> 172.21.15.160:0/3329325717 conn(0x7fe670072b60 msgr2=0x7fe6700ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:17.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 shutdown_connections 2024-05-25T06:32:17.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.326+0000 7fe675ec2700 1 -- 172.21.15.160:0/3329325717 wait complete. 2024-05-25T06:32:17.330 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:32:17.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.429+0000 7fe85f65d700 1 -- 172.21.15.160:0/2085496981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 msgr2=0x7fe8580fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.429+0000 7fe85f65d700 1 --2- 172.21.15.160:0/2085496981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe8580fcdf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe848004940 tx=0x7fe848009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- 172.21.15.160:0/2085496981 shutdown_connections 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 --2- 172.21.15.160:0/2085496981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe8580fcdf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 --2- 172.21.15.160:0/2085496981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe858102de0 0x7fe8581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- 172.21.15.160:0/2085496981 >> 172.21.15.160:0/2085496981 conn(0x7fe8580f8990 msgr2=0x7fe8580fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- 172.21.15.160:0/2085496981 shutdown_connections 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- 172.21.15.160:0/2085496981 wait complete. 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 Processor -- start 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- start start 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe858102de0 0x7fe858113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe858113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581144e0 con 0x7fe858103780 2024-05-25T06:32:17.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.430+0000 7fe85f65d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85810dae0 con 0x7fe858102de0 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe858113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe858113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60120/0 (socket says 172.21.15.160:60120) 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 -- 172.21.15.160:0/1999724438 learned_addr learned my addr 172.21.15.160:0/1999724438 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85d3f9700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe858102de0 0x7fe858113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 -- 172.21.15.160:0/1999724438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe858102de0 msgr2=0x7fe858113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe858102de0 0x7fe858113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 -- 172.21.15.160:0/1999724438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848005040 con 0x7fe858103780 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85cbf8700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe858113e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe848009100 tx=0x7fe848009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe84800e040 con 0x7fe858103780 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe848013420 con 0x7fe858103780 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85810dd60 con 0x7fe858103780 2024-05-25T06:32:17.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe84801c610 con 0x7fe858103780 2024-05-25T06:32:17.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.431+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85810e320 con 0x7fe858103780 2024-05-25T06:32:17.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.432+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580fe530 con 0x7fe858103780 2024-05-25T06:32:17.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.433+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe848013590 con 0x7fe858103780 2024-05-25T06:32:17.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.433+0000 7fe84dffb700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe84406e140 0x7fe8440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.433+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe84808ee00 con 0x7fe858103780 2024-05-25T06:32:17.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.433+0000 7fe85d3f9700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe84406e140 0x7fe8440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.434+0000 7fe85d3f9700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe84406e140 0x7fe8440705f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe854009fd0 tx=0x7fe854009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:17.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.438+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe84805bce0 con 0x7fe858103780 2024-05-25T06:32:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:17 smithi203 ceph-mon[83507]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:17 smithi203 ceph-mon[83507]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:17 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4259607583' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:32:17.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.652+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe85810f010 con 0x7fe84406e140 2024-05-25T06:32:17.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:17 smithi160 ceph-mon[112094]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:17.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:17 smithi160 ceph-mon[112094]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:17.653 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:17 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4259607583' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.653+0000 7fe84dffb700 1 -- 172.21.15.160:0/1999724438 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe85810f010 con 0x7fe84406e140 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:32:17.654 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:32:17.655 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe84406e140 msgr2=0x7fe8440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe84406e140 0x7fe8440705f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe854009fd0 tx=0x7fe854009e50 comp rx=0 tx=0).stop 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 msgr2=0x7fe858113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe858113e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe848009100 tx=0x7fe848009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 shutdown_connections 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe84406e140 0x7fe8440705f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe858102de0 0x7fe858113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 --2- 172.21.15.160:0/1999724438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe858103780 0x7fe858113e90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 >> 172.21.15.160:0/1999724438 conn(0x7fe8580f8990 msgr2=0x7fe8580f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:17.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 shutdown_connections 2024-05-25T06:32:17.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.656+0000 7fe85f65d700 1 -- 172.21.15.160:0/1999724438 wait complete. 2024-05-25T06:32:17.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.758+0000 7fa24b127700 1 -- 172.21.15.160:0/1768245830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244102de0 msgr2=0x7fa2441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.758+0000 7fa24b127700 1 --2- 172.21.15.160:0/1768245830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244102de0 0x7fa2441031b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa238004940 tx=0x7fa238009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:17.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 -- 172.21.15.160:0/1768245830 shutdown_connections 2024-05-25T06:32:17.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 --2- 172.21.15.160:0/1768245830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa244103780 0x7fa2440fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 --2- 172.21.15.160:0/1768245830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244102de0 0x7fa2441031b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 -- 172.21.15.160:0/1768245830 >> 172.21.15.160:0/1768245830 conn(0x7fa2440f8990 msgr2=0x7fa2440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:17.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 -- 172.21.15.160:0/1768245830 shutdown_connections 2024-05-25T06:32:17.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 -- 172.21.15.160:0/1768245830 wait complete. 2024-05-25T06:32:17.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 Processor -- start 2024-05-25T06:32:17.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.759+0000 7fa24b127700 1 -- start start 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa24b127700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa244102de0 0x7fa244070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa24b127700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 0x7fa2440707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa24b127700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440721b0 con 0x7fa244103780 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa24b127700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa244070ce0 con 0x7fa244102de0 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 0x7fa2440707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 0x7fa2440707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60138/0 (socket says 172.21.15.160:60138) 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 -- 172.21.15.160:0/1542075348 learned_addr learned my addr 172.21.15.160:0/1542075348 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa248ec3700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa244102de0 0x7fa244070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 -- 172.21.15.160:0/1542075348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa244102de0 msgr2=0x7fa244070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa244102de0 0x7fa244070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 -- 172.21.15.160:0/1542075348 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238005040 con 0x7fa244103780 2024-05-25T06:32:17.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa243fff700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 0x7fa2440707a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa23000eb40 tx=0x7fa23000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:17.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa230007cc0 con 0x7fa244103780 2024-05-25T06:32:17.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa23000f040 con 0x7fa244103780 2024-05-25T06:32:17.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa244070f60 con 0x7fa244103780 2024-05-25T06:32:17.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa23000c9d0 con 0x7fa244103780 2024-05-25T06:32:17.763 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.760+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa244119fb0 con 0x7fa244103780 2024-05-25T06:32:17.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.761+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2441000f0 con 0x7fa244103780 2024-05-25T06:32:17.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.762+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa23000cc40 con 0x7fa244103780 2024-05-25T06:32:17.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.762+0000 7fa2417fa700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa23406e140 0x7fa2340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:17.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.763+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa23008d5c0 con 0x7fa244103780 2024-05-25T06:32:17.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.763+0000 7fa248ec3700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa23406e140 0x7fa2340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:17.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.763+0000 7fa248ec3700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa23406e140 0x7fa2340705f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa238009b30 tx=0x7fa238009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:17.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:17.767+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa2300569a0 con 0x7fa244103780 2024-05-25T06:32:18.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.052+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa244109ee0 con 0x7fa244103780 2024-05-25T06:32:18.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.052+0000 7fa2417fa700 1 -- 172.21.15.160:0/1542075348 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa23005a060 con 0x7fa244103780 2024-05-25T06:32:18.053 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:32:18.054 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:32:18.054 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:32:18.054 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:32:18.054 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa23406e140 msgr2=0x7fa2340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa23406e140 0x7fa2340705f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa238009b30 tx=0x7fa238009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 msgr2=0x7fa2440707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 0x7fa2440707a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa23000eb40 tx=0x7fa23000b040 comp rx=0 tx=0).stop 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 shutdown_connections 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa23406e140 0x7fa2340705f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa244102de0 0x7fa244070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 --2- 172.21.15.160:0/1542075348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa244103780 0x7fa2440707a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 >> 172.21.15.160:0/1542075348 conn(0x7fa2440f8990 msgr2=0x7fa2440fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 shutdown_connections 2024-05-25T06:32:18.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:18.055+0000 7fa24b127700 1 -- 172.21.15.160:0/1542075348 wait complete. 2024-05-25T06:32:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:18 smithi203 ceph-mon[83507]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:18 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3329325717' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:32:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:18 smithi203 ceph-mon[83507]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:18 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1542075348' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:32:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:18 smithi160 ceph-mon[112094]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:18 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3329325717' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:32:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:18 smithi160 ceph-mon[112094]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:18 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1542075348' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:32:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:20 smithi203 ceph-mon[83507]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:20.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:20 smithi160 ceph-mon[112094]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:22 smithi203 ceph-mon[83507]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:22.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:22 smithi160 ceph-mon[112094]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:24 smithi203 ceph-mon[83507]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:24 smithi160 ceph-mon[112094]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:26 smithi203 ceph-mon[83507]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:26 smithi160 ceph-mon[112094]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:28 smithi203 ceph-mon[83507]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:28 smithi160 ceph-mon[112094]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:30 smithi203 ceph-mon[83507]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:30 smithi160 ceph-mon[112094]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:32 smithi203 ceph-mon[83507]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:32.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:32 smithi160 ceph-mon[112094]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:34 smithi203 ceph-mon[83507]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:34 smithi160 ceph-mon[112094]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:36 smithi203 ceph-mon[83507]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:36 smithi160 ceph-mon[112094]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:38 smithi203 ceph-mon[83507]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:38 smithi160 ceph-mon[112094]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:40 smithi203 ceph-mon[83507]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:40 smithi160 ceph-mon[112094]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:42.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:42 smithi203 ceph-mon[83507]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:42.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:42 smithi160 ceph-mon[112094]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:43.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:44 smithi203 ceph-mon[83507]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:44.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:44 smithi160 ceph-mon[112094]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:46 smithi203 ceph-mon[83507]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:46.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:46 smithi160 ceph-mon[112094]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:48.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 -- 172.21.15.160:0/2978905457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854102de0 msgr2=0x7f08541031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 --2- 172.21.15.160:0/2978905457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854102de0 0x7f08541031b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0844004940 tx=0x7f0844009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:48.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 -- 172.21.15.160:0/2978905457 shutdown_connections 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 --2- 172.21.15.160:0/2978905457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854103780 0x7f08540fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 --2- 172.21.15.160:0/2978905457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854102de0 0x7f08541031b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 -- 172.21.15.160:0/2978905457 >> 172.21.15.160:0/2978905457 conn(0x7f08540f8990 msgr2=0x7f08540fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 -- 172.21.15.160:0/2978905457 shutdown_connections 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 -- 172.21.15.160:0/2978905457 wait complete. 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.172+0000 7f085941b700 1 Processor -- start 2024-05-25T06:32:48.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f085941b700 1 -- start start 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f085941b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854102de0 0x7f0854070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f085941b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 0x7f08540707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f085941b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854072160 con 0x7f0854103780 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f085941b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854070ce0 con 0x7f0854102de0 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f084a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 0x7f08540707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f084a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 0x7f08540707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48648/0 (socket says 172.21.15.160:48648) 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f084a7fc700 1 -- 172.21.15.160:0/2868588135 learned_addr learned my addr 172.21.15.160:0/2868588135 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f0852ffd700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854102de0 0x7f0854070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f084a7fc700 1 -- 172.21.15.160:0/2868588135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854102de0 msgr2=0x7f0854070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f084a7fc700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854102de0 0x7f0854070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.173+0000 7f084a7fc700 1 -- 172.21.15.160:0/2868588135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844005040 con 0x7f0854103780 2024-05-25T06:32:48.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.174+0000 7f084a7fc700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 0x7f08540707a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f083c002a50 tx=0x7f083c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:48.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.174+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083c007cc0 con 0x7f0854103780 2024-05-25T06:32:48.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.174+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f083c00f040 con 0x7f0854103780 2024-05-25T06:32:48.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.174+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0854070fc0 con 0x7f0854103780 2024-05-25T06:32:48.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.174+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083c00da40 con 0x7f0854103780 2024-05-25T06:32:48.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.174+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085411a160 con 0x7f0854103780 2024-05-25T06:32:48.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.175+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08541000f0 con 0x7f0854103780 2024-05-25T06:32:48.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.175+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f083c0097c0 con 0x7f0854103780 2024-05-25T06:32:48.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.176+0000 7f084bfff700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f084006e140 0x7f08400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.176+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f083c08d980 con 0x7f0854103780 2024-05-25T06:32:48.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.176+0000 7f0852ffd700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f084006e140 0x7f08400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.176+0000 7f0852ffd700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f084006e140 0x7f08400705f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0844004f90 tx=0x7f0844009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:48.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.181+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f083c056cb0 con 0x7f0854103780 2024-05-25T06:32:48.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.397+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0854109ee0 con 0x7f084006e140 2024-05-25T06:32:48.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.398+0000 7f084bfff700 1 -- 172.21.15.160:0/2868588135 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0854109ee0 con 0x7f084006e140 2024-05-25T06:32:48.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f084006e140 msgr2=0x7f08400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f084006e140 0x7f08400705f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0844004f90 tx=0x7f0844009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:48.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 msgr2=0x7f08540707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 0x7f08540707a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f083c002a50 tx=0x7f083c004600 comp rx=0 tx=0).stop 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 shutdown_connections 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f084006e140 0x7f08400705f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854102de0 0x7f0854070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 --2- 172.21.15.160:0/2868588135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0854103780 0x7f08540707a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.400+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 >> 172.21.15.160:0/2868588135 conn(0x7f08540f8990 msgr2=0x7f0854107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.401+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 shutdown_connections 2024-05-25T06:32:48.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.401+0000 7f085941b700 1 -- 172.21.15.160:0/2868588135 wait complete. 2024-05-25T06:32:48.411 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:32:48.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 -- 172.21.15.160:0/3819021709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 msgr2=0x7febb4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3819021709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4109350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feba8004940 tx=0x7feba8009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:48.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 -- 172.21.15.160:0/3819021709 shutdown_connections 2024-05-25T06:32:48.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3819021709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4109350 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3819021709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb40fc790 0x7febb40fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 -- 172.21.15.160:0/3819021709 >> 172.21.15.160:0/3819021709 conn(0x7febb4071d00 msgr2=0x7febb4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:48.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 -- 172.21.15.160:0/3819021709 shutdown_connections 2024-05-25T06:32:48.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.503+0000 7febb9dfa700 1 -- 172.21.15.160:0/3819021709 wait complete. 2024-05-25T06:32:48.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb9dfa700 1 Processor -- start 2024-05-25T06:32:48.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb9dfa700 1 -- start start 2024-05-25T06:32:48.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb9dfa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb40fc790 0x7febb4115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb9dfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb9dfa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4116950 con 0x7febb40fd0a0 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb9dfa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb410fd20 con 0x7febb40fc790 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48654/0 (socket says 172.21.15.160:48654) 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb2ffd700 1 -- 172.21.15.160:0/3933132543 learned_addr learned my addr 172.21.15.160:0/3933132543 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.504+0000 7febb37fe700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb40fc790 0x7febb4115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7febb2ffd700 1 -- 172.21.15.160:0/3933132543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb40fc790 msgr2=0x7febb4115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7febb2ffd700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb40fc790 0x7febb4115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7febb2ffd700 1 -- 172.21.15.160:0/3933132543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8005040 con 0x7febb40fd0a0 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7febb2ffd700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4116300 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feba800b8b0 tx=0x7feba8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feba800e040 con 0x7febb40fd0a0 2024-05-25T06:32:48.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feba8004290 con 0x7febb40fd0a0 2024-05-25T06:32:48.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb410ffa0 con 0x7febb40fd0a0 2024-05-25T06:32:48.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feba801c3f0 con 0x7febb40fd0a0 2024-05-25T06:32:48.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.505+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb41104e0 con 0x7febb40fd0a0 2024-05-25T06:32:48.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.506+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb40fc330 con 0x7febb40fd0a0 2024-05-25T06:32:48.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.506+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7feba8013420 con 0x7febb40fd0a0 2024-05-25T06:32:48.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.507+0000 7feb9bfff700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feb9c06e140 0x7feb9c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.507+0000 7febb37fe700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feb9c06e140 0x7feb9c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.507+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7feba808ed90 con 0x7febb40fd0a0 2024-05-25T06:32:48.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.507+0000 7febb37fe700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feb9c06e140 0x7feb9c0705f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feba4009fd0 tx=0x7feba400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:48.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.512+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feba805bc70 con 0x7febb40fd0a0 2024-05-25T06:32:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:48 smithi203 ceph-mon[83507]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:48.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:48 smithi160 ceph-mon[112094]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:48.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.728+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febb41114b0 con 0x7feb9c06e140 2024-05-25T06:32:48.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.729+0000 7feb9bfff700 1 -- 172.21.15.160:0/3933132543 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7febb41114b0 con 0x7feb9c06e140 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feb9c06e140 msgr2=0x7feb9c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feb9c06e140 0x7feb9c0705f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feba4009fd0 tx=0x7feba400b040 comp rx=0 tx=0).stop 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 msgr2=0x7febb4116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4116300 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feba800b8b0 tx=0x7feba8009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 shutdown_connections 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feb9c06e140 0x7feb9c0705f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb40fc790 0x7febb4115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 --2- 172.21.15.160:0/3933132543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febb40fd0a0 0x7febb4116300 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 >> 172.21.15.160:0/3933132543 conn(0x7febb4071d00 msgr2=0x7febb40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:48.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 shutdown_connections 2024-05-25T06:32:48.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.732+0000 7febb9dfa700 1 -- 172.21.15.160:0/3933132543 wait complete. 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 -- 172.21.15.160:0/1143515722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 msgr2=0x7fa4880fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/1143515722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa4880fbd10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa470004940 tx=0x7fa470009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 -- 172.21.15.160:0/1143515722 shutdown_connections 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/1143515722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa4880fbd10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/1143515722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa488101d00 0x7fa4881020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 -- 172.21.15.160:0/1143515722 >> 172.21.15.160:0/1143515722 conn(0x7fa4880f7870 msgr2=0x7fa4880f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 -- 172.21.15.160:0/1143515722 shutdown_connections 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.835+0000 7fa48ccf8700 1 -- 172.21.15.160:0/1143515722 wait complete. 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48ccf8700 1 Processor -- start 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48ccf8700 1 -- start start 2024-05-25T06:32:48.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48ccf8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa488101d00 0x7fa4881128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48ccf8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa488112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48ccf8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4881134f0 con 0x7fa4881026a0 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48ccf8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48810ca60 con 0x7fa488101d00 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa485d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa488112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa485d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa488112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48670/0 (socket says 172.21.15.160:48670) 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa485d9b700 1 -- 172.21.15.160:0/526643533 learned_addr learned my addr 172.21.15.160:0/526643533 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa48659c700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa488101d00 0x7fa4881128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa485d9b700 1 -- 172.21.15.160:0/526643533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa488101d00 msgr2=0x7fa4881128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa485d9b700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa488101d00 0x7fa4881128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.836+0000 7fa485d9b700 1 -- 172.21.15.160:0/526643533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470005040 con 0x7fa4881026a0 2024-05-25T06:32:48.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.837+0000 7fa485d9b700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa488112e10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa470009100 tx=0x7fa470009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:48.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.837+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47000e040 con 0x7fa4881026a0 2024-05-25T06:32:48.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.837+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa470009af0 con 0x7fa4881026a0 2024-05-25T06:32:48.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.837+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48810cce0 con 0x7fa4881026a0 2024-05-25T06:32:48.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.837+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47001c3f0 con 0x7fa4881026a0 2024-05-25T06:32:48.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.837+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48810d2a0 con 0x7fa4881026a0 2024-05-25T06:32:48.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.838+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4880fd450 con 0x7fa4881026a0 2024-05-25T06:32:48.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.838+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa47000baf0 con 0x7fa4881026a0 2024-05-25T06:32:48.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.839+0000 7fa47effd700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa47406e140 0x7fa4740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:48.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.839+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa47008edb0 con 0x7fa4881026a0 2024-05-25T06:32:48.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.839+0000 7fa48659c700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa47406e140 0x7fa4740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:48.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.839+0000 7fa48659c700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa47406e140 0x7fa4740705f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa478009b60 tx=0x7fa4780091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:48.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:48.843+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa47005bc90 con 0x7fa4881026a0 2024-05-25T06:32:49.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.051+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa488108e50 con 0x7fa47406e140 2024-05-25T06:32:49.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.061+0000 7fa47effd700 1 -- 172.21.15.160:0/526643533 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa488108e50 con 0x7fa47406e140 2024-05-25T06:32:49.061 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:32:49.061 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (18m) 12m ago 29m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:32:49.061 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (16m) 12m ago 29m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (16m) 11m ago 29m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (18m) 12m ago 29m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (12m) 12m ago 24m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (12m) 12m ago 24m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (11m) 11m ago 24m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (11m) 11m ago 24m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (20m) 12m ago 31m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (19m) 11m ago 28m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (17m) 12m ago 31m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (16m) 11m ago 28m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:32:49.062 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (19m) 12m ago 29m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (19m) 11m ago 28m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (15m) 12m ago 28m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (15m) 12m ago 27m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (14m) 12m ago 27m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (13m) 11m ago 26m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (13m) 11m ago 26m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (12m) 11m ago 25m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:32:49.063 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (18m) 12m ago 29m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:32:49.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa47406e140 msgr2=0x7fa4740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa47406e140 0x7fa4740705f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa478009b60 tx=0x7fa4780091a0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 msgr2=0x7fa488112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa488112e10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa470009100 tx=0x7fa470009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:49.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 shutdown_connections 2024-05-25T06:32:49.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fa47406e140 0x7fa4740705f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa488101d00 0x7fa4881128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 --2- 172.21.15.160:0/526643533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4881026a0 0x7fa488112e10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 >> 172.21.15.160:0/526643533 conn(0x7fa4880f7870 msgr2=0x7fa4880f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:49.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 shutdown_connections 2024-05-25T06:32:49.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.064+0000 7fa48ccf8700 1 -- 172.21.15.160:0/526643533 wait complete. 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2393474038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0102690 msgr2=0x7fc1f00fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2393474038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0102690 0x7fc1f00fbd00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0004940 tx=0x7fc1e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2393474038 shutdown_connections 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2393474038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0102690 0x7fc1f00fbd00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2393474038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f0101cf0 0x7fc1f01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2393474038 >> 172.21.15.160:0/2393474038 conn(0x7fc1f00f7860 msgr2=0x7fc1f00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2393474038 shutdown_connections 2024-05-25T06:32:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.165+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2393474038 wait complete. 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f7b3a700 1 Processor -- start 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f7b3a700 1 -- start start 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f7b3a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 0x7fc1f010c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f7b3a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f0102690 0x7fc1f010ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f7b3a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0067100 con 0x7fc1f0101cf0 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f7b3a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f010d380 con 0x7fc1f0102690 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f58d6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 0x7fc1f010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f58d6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 0x7fc1f010c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48696/0 (socket says 172.21.15.160:48696) 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f58d6700 1 -- 172.21.15.160:0/2967009015 learned_addr learned my addr 172.21.15.160:0/2967009015 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f50d5700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f0102690 0x7fc1f010ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f58d6700 1 -- 172.21.15.160:0/2967009015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f0102690 msgr2=0x7fc1f010ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f58d6700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f0102690 0x7fc1f010ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.166+0000 7fc1f58d6700 1 -- 172.21.15.160:0/2967009015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0005040 con 0x7fc1f0101cf0 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.167+0000 7fc1f58d6700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 0x7fc1f010c900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec002a50 tx=0x7fc1ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.167+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ec007cc0 con 0x7fc1f0101cf0 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.167+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1ec00f040 con 0x7fc1f0101cf0 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.167+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f010d660 con 0x7fc1f0101cf0 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.167+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ec009640 con 0x7fc1f0101cf0 2024-05-25T06:32:49.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.167+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f011a1b0 con 0x7fc1f0101cf0 2024-05-25T06:32:49.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.168+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f00fd440 con 0x7fc1f0101cf0 2024-05-25T06:32:49.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.169+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc1ec0097a0 con 0x7fc1f0101cf0 2024-05-25T06:32:49.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.169+0000 7fc1e67fc700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc1dc06e140 0x7fc1dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.169+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc1ec08d970 con 0x7fc1f0101cf0 2024-05-25T06:32:49.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.169+0000 7fc1f50d5700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc1dc06e140 0x7fc1dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.170+0000 7fc1f50d5700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc1dc06e140 0x7fc1dc0705f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009fd0 tx=0x7fc1e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:49.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.174+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc1ec05a7d0 con 0x7fc1f0101cf0 2024-05-25T06:32:49.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.452+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1f011a340 con 0x7fc1f0101cf0 2024-05-25T06:32:49.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.452+0000 7fc1e67fc700 1 -- 172.21.15.160:0/2967009015 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fc1ec05a360 con 0x7fc1f0101cf0 2024-05-25T06:32:49.452 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:32:49.452 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:32:49.453 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:32:49.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc1dc06e140 msgr2=0x7fc1dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc1dc06e140 0x7fc1dc0705f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009fd0 tx=0x7fc1e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:49.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 msgr2=0x7fc1f010c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 0x7fc1f010c900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec002a50 tx=0x7fc1ec0046b0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 shutdown_connections 2024-05-25T06:32:49.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1f0101cf0 0x7fc1f010c900 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc1dc06e140 0x7fc1dc0705f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 --2- 172.21.15.160:0/2967009015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f0102690 0x7fc1f010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.454+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 >> 172.21.15.160:0/2967009015 conn(0x7fc1f00f7860 msgr2=0x7fc1f00f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:49.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.455+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 shutdown_connections 2024-05-25T06:32:49.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.455+0000 7fc1f7b3a700 1 -- 172.21.15.160:0/2967009015 wait complete. 2024-05-25T06:32:49.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.556+0000 7f7f10e80700 1 -- 172.21.15.160:0/1601057002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c102f60 msgr2=0x7f7f0c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.556+0000 7f7f10e80700 1 --2- 172.21.15.160:0/1601057002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c102f60 0x7f7f0c103330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7f00004940 tx=0x7f7f00009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:49.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.556+0000 7f7f10e80700 1 -- 172.21.15.160:0/1601057002 shutdown_connections 2024-05-25T06:32:49.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.556+0000 7f7f10e80700 1 --2- 172.21.15.160:0/1601057002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.556+0000 7f7f10e80700 1 --2- 172.21.15.160:0/1601057002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c102f60 0x7f7f0c103330 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.556+0000 7f7f10e80700 1 -- 172.21.15.160:0/1601057002 >> 172.21.15.160:0/1601057002 conn(0x7f7f0c0714d0 msgr2=0x7f7f0c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 -- 172.21.15.160:0/1601057002 shutdown_connections 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 -- 172.21.15.160:0/1601057002 wait complete. 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 Processor -- start 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 -- start start 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c102f60 0x7f7f0c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c114530 con 0x7f7f0c0fb520 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f10e80700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c10db30 con 0x7f7f0c102f60 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f0a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f0a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48722/0 (socket says 172.21.15.160:48722) 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.557+0000 7f7f0a59c700 1 -- 172.21.15.160:0/3884144027 learned_addr learned my addr 172.21.15.160:0/3884144027 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f09d9b700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c102f60 0x7f7f0c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f0a59c700 1 -- 172.21.15.160:0/3884144027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c102f60 msgr2=0x7f7f0c113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f0a59c700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c102f60 0x7f7f0c113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f0a59c700 1 -- 172.21.15.160:0/3884144027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00005040 con 0x7f7f0c0fb520 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f0a59c700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c1139a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b3b0 tx=0x7f7f00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0000e040 con 0x7f7f0c0fb520 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c10ddb0 con 0x7f7f0c0fb520 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f0000bbc0 con 0x7f7f0c0fb520 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0001c3f0 con 0x7f7f0c0fb520 2024-05-25T06:32:49.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.558+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c10e2f0 con 0x7f7f0c0fb520 2024-05-25T06:32:49.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.559+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c100250 con 0x7f7f0c0fb520 2024-05-25T06:32:49.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.560+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7f0000bd30 con 0x7f7f0c0fb520 2024-05-25T06:32:49.561 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:49 smithi160 ceph-mon[112094]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:49.561 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:49 smithi160 ceph-mon[112094]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:49.561 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:49 smithi160 ceph-mon[112094]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:49.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.560+0000 7f7efaffd700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef406e140 0x7f7ef40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.560+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f0008ee10 con 0x7f7f0c0fb520 2024-05-25T06:32:49.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.560+0000 7f7f09d9b700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef406e140 0x7f7ef40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.561+0000 7f7f09d9b700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef406e140 0x7f7ef40705f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c10ecf0 tx=0x7f7efc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:49.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.565+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7f0005bc70 con 0x7f7f0c0fb520 2024-05-25T06:32:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:49 smithi203 ceph-mon[83507]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:49 smithi203 ceph-mon[83507]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:49 smithi203 ceph-mon[83507]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:49.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.802+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f0c10eff0 con 0x7f7f0c0fb520 2024-05-25T06:32:49.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.803+0000 7f7efaffd700 1 -- 172.21.15.160:0/3884144027 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f7f0005b800 con 0x7f7f0c0fb520 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:32:49.803 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:32:49.804 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:49.805 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:49.806 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:32:49.806 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:32:49.806 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:49.806 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:49.806 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:49.806 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef406e140 msgr2=0x7f7ef40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef406e140 0x7f7ef40705f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c10ecf0 tx=0x7f7efc009ee0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 msgr2=0x7f7f0c1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c1139a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b3b0 tx=0x7f7f00009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 shutdown_connections 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f0c0fb520 0x7f7f0c1139a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7ef406e140 0x7f7ef40705f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 --2- 172.21.15.160:0/3884144027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c102f60 0x7f7f0c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 >> 172.21.15.160:0/3884144027 conn(0x7f7f0c0714d0 msgr2=0x7f7f0c0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:49.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 shutdown_connections 2024-05-25T06:32:49.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.805+0000 7f7f10e80700 1 -- 172.21.15.160:0/3884144027 wait complete. 2024-05-25T06:32:49.808 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 -- 172.21.15.160:0/593095579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8103780 msgr2=0x7f65a80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 --2- 172.21.15.160:0/593095579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8103780 0x7f65a80fcdf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6594004940 tx=0x7f6594009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 -- 172.21.15.160:0/593095579 shutdown_connections 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 --2- 172.21.15.160:0/593095579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8103780 0x7f65a80fcdf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 --2- 172.21.15.160:0/593095579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65a8102de0 0x7f65a81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 -- 172.21.15.160:0/593095579 >> 172.21.15.160:0/593095579 conn(0x7f65a80f8990 msgr2=0x7f65a80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 -- 172.21.15.160:0/593095579 shutdown_connections 2024-05-25T06:32:49.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 -- 172.21.15.160:0/593095579 wait complete. 2024-05-25T06:32:49.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 Processor -- start 2024-05-25T06:32:49.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.910+0000 7f65af17a700 1 -- start start 2024-05-25T06:32:49.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65af17a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 0x7f65a8113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65af17a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65a8103780 0x7f65a8113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65af17a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81144e0 con 0x7f65a8102de0 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65af17a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a810dae0 con 0x7f65a8103780 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 0x7f65a8113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 0x7f65a8113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48750/0 (socket says 172.21.15.160:48750) 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 -- 172.21.15.160:0/1184410553 learned_addr learned my addr 172.21.15.160:0/1184410553 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65a7fff700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65a8103780 0x7f65a8113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 -- 172.21.15.160:0/1184410553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65a8103780 msgr2=0x7f65a8113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65a8103780 0x7f65a8113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 -- 172.21.15.160:0/1184410553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6594005040 con 0x7f65a8102de0 2024-05-25T06:32:49.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65acf16700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 0x7f65a8113950 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f659c002a50 tx=0x7f659c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:49.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659c007cc0 con 0x7f65a8102de0 2024-05-25T06:32:49.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f659c00f040 con 0x7f65a8102de0 2024-05-25T06:32:49.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.911+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659c00da30 con 0x7f65a8102de0 2024-05-25T06:32:49.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.912+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a810ddc0 con 0x7f65a8102de0 2024-05-25T06:32:49.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.912+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a810e300 con 0x7f65a8102de0 2024-05-25T06:32:49.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.912+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a80fe530 con 0x7f65a8102de0 2024-05-25T06:32:49.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.913+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f659c00db90 con 0x7f65a8102de0 2024-05-25T06:32:49.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.914+0000 7f65a57fa700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f659806e140 0x7f65980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:49.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.914+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f659c08d910 con 0x7f65a8102de0 2024-05-25T06:32:49.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.914+0000 7f65a7fff700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f659806e140 0x7f65980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:49.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.914+0000 7f65a7fff700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f659806e140 0x7f65980705f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6594004890 tx=0x7f6594009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:49.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:49.918+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f659c05a770 con 0x7f65a8102de0 2024-05-25T06:32:50.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.135+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65a8109f60 con 0x7f659806e140 2024-05-25T06:32:50.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.136+0000 7f65a57fa700 1 -- 172.21.15.160:0/1184410553 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f659400b5c0 con 0x7f659806e140 2024-05-25T06:32:50.136 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:32:50.136 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:32:50.136 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:32:50.136 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:32:50.137 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f659806e140 msgr2=0x7f65980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f659806e140 0x7f65980705f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6594004890 tx=0x7f6594009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 msgr2=0x7f65a8113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 0x7f65a8113950 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f659c002a50 tx=0x7f659c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 shutdown_connections 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a8102de0 0x7f65a8113950 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f659806e140 0x7f65980705f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 --2- 172.21.15.160:0/1184410553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65a8103780 0x7f65a8113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 >> 172.21.15.160:0/1184410553 conn(0x7f65a80f8990 msgr2=0x7f65a80f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:50.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 shutdown_connections 2024-05-25T06:32:50.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.139+0000 7f65af17a700 1 -- 172.21.15.160:0/1184410553 wait complete. 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 -- 172.21.15.160:0/678424271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 msgr2=0x7f9570106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 --2- 172.21.15.160:0/678424271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570106e20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f955c004940 tx=0x7f955c009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 -- 172.21.15.160:0/678424271 shutdown_connections 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 --2- 172.21.15.160:0/678424271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570106e20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 --2- 172.21.15.160:0/678424271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 0x7f95700fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 -- 172.21.15.160:0/678424271 >> 172.21.15.160:0/678424271 conn(0x7f95700f7870 msgr2=0x7f95700f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 -- 172.21.15.160:0/678424271 shutdown_connections 2024-05-25T06:32:50.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.241+0000 7f9576ba5700 1 -- 172.21.15.160:0/678424271 wait complete. 2024-05-25T06:32:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f9576ba5700 1 Processor -- start 2024-05-25T06:32:50.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f9576ba5700 1 -- start start 2024-05-25T06:32:50.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f9576ba5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 0x7f95701138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f9576ba5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f9576ba5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570114480 con 0x7f95700fe7e0 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f9576ba5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957010da80 con 0x7f95700fde40 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.242+0000 7f956ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48776/0 (socket says 172.21.15.160:48776) 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956ffff700 1 -- 172.21.15.160:0/4273919043 learned_addr learned my addr 172.21.15.160:0/4273919043 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f9574941700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 0x7f95701138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956ffff700 1 -- 172.21.15.160:0/4273919043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 msgr2=0x7f95701138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956ffff700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 0x7f95701138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956ffff700 1 -- 172.21.15.160:0/4273919043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c005040 con 0x7f95700fe7e0 2024-05-25T06:32:50.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f9574941700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 0x7f95701138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956ffff700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570113e30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f955c009380 tx=0x7f955c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955c00e040 con 0x7f95700fe7e0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f955c009af0 con 0x7f95700fe7e0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957010dd60 con 0x7f95700fe7e0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955c01c630 con 0x7f95700fe7e0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.243+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957010e2a0 con 0x7f95700fe7e0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.244+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570066b30 con 0x7f95700fe7e0 2024-05-25T06:32:50.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.245+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f955c00bbc0 con 0x7f95700fe7e0 2024-05-25T06:32:50.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.245+0000 7f956d7fa700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f956006e140 0x7f95600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:32:50.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.245+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f955c08ee70 con 0x7f95700fe7e0 2024-05-25T06:32:50.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.245+0000 7f9574941700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f956006e140 0x7f95600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:32:50.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.246+0000 7f9574941700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f956006e140 0x7f95600705f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9564007fd0 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:32:50.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.250+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f955c05bd50 con 0x7f95700fe7e0 2024-05-25T06:32:50.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.529+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9570109e00 con 0x7f95700fe7e0 2024-05-25T06:32:50.529 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:50 smithi160 ceph-mon[112094]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:50.529 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:50 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2967009015' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:32:50.530 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:50 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3884144027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:32:50.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.529+0000 7f956d7fa700 1 -- 172.21.15.160:0/4273919043 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f955c05b8e0 con 0x7f95700fe7e0 2024-05-25T06:32:50.530 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:32:50.530 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:32:50.530 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:32:50.530 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:32:50.530 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f956006e140 msgr2=0x7f95600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f956006e140 0x7f95600705f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9564007fd0 tx=0x7f9564009040 comp rx=0 tx=0).stop 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 msgr2=0x7f9570113e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570113e30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f955c009380 tx=0x7f955c009e30 comp rx=0 tx=0).stop 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 shutdown_connections 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f956006e140 0x7f95600705f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95700fde40 0x7f95701138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 --2- 172.21.15.160:0/4273919043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95700fe7e0 0x7f9570113e30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:32:50.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 >> 172.21.15.160:0/4273919043 conn(0x7f95700f7870 msgr2=0x7f95700f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:32:50.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 shutdown_connections 2024-05-25T06:32:50.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:32:50.532+0000 7f9576ba5700 1 -- 172.21.15.160:0/4273919043 wait complete. 2024-05-25T06:32:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:50 smithi203 ceph-mon[83507]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:50 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2967009015' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:32:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:50 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3884144027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:32:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:51 smithi203 ceph-mon[83507]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:51 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4273919043' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:32:51.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:51 smithi160 ceph-mon[112094]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:32:51.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:51 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4273919043' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:32:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:52 smithi203 ceph-mon[83507]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:52.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:52 smithi160 ceph-mon[112094]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:54 smithi203 ceph-mon[83507]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:54 smithi160 ceph-mon[112094]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:56 smithi203 ceph-mon[83507]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:56.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:56 smithi160 ceph-mon[112094]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:58 smithi203 ceph-mon[83507]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:32:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:32:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:58 smithi160 ceph-mon[112094]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:32:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:32:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:00 smithi203 ceph-mon[83507]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:00.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:00 smithi160 ceph-mon[112094]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:02 smithi203 ceph-mon[83507]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:02.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:02 smithi160 ceph-mon[112094]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:04 smithi203 ceph-mon[83507]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:04.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:04 smithi160 ceph-mon[112094]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:06 smithi203 ceph-mon[83507]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:06 smithi160 ceph-mon[112094]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:08 smithi203 ceph-mon[83507]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:08 smithi160 ceph-mon[112094]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:10 smithi203 ceph-mon[83507]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:10 smithi160 ceph-mon[112094]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:12 smithi203 ceph-mon[83507]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:12 smithi160 ceph-mon[112094]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:14 smithi203 ceph-mon[83507]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:14.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:14 smithi160 ceph-mon[112094]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:16 smithi203 ceph-mon[83507]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:16 smithi160 ceph-mon[112094]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:18 smithi203 ceph-mon[83507]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:18 smithi160 ceph-mon[112094]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:20.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 -- 172.21.15.160:0/847122569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8101ce0 msgr2=0x7fe1c81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:20.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/847122569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8101ce0 0x7fe1c81020b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009ce0 tx=0x7fe1c4004d10 comp rx=0 tx=0).stop 2024-05-25T06:33:20.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 -- 172.21.15.160:0/847122569 shutdown_connections 2024-05-25T06:33:20.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/847122569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c8102680 0x7fe1c80fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/847122569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8101ce0 0x7fe1c81020b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 -- 172.21.15.160:0/847122569 >> 172.21.15.160:0/847122569 conn(0x7fe1c80f7850 msgr2=0x7fe1c80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:20.649 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:20 smithi160 ceph-mon[112094]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:20.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 -- 172.21.15.160:0/847122569 shutdown_connections 2024-05-25T06:33:20.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.648+0000 7fe1ceadc700 1 -- 172.21.15.160:0/847122569 wait complete. 2024-05-25T06:33:20.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.649+0000 7fe1ceadc700 1 Processor -- start 2024-05-25T06:33:20.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.649+0000 7fe1ceadc700 1 -- start start 2024-05-25T06:33:20.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.649+0000 7fe1ceadc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c8101ce0 0x7fe1c8102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:20.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.649+0000 7fe1ceadc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 0x7fe1c8103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.649+0000 7fe1ceadc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8103610 con 0x7fe1c8102680 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.649+0000 7fe1ceadc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8103750 con 0x7fe1c8101ce0 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 0x7fe1c8103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 0x7fe1c8103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45110/0 (socket says 172.21.15.160:45110) 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 -- 172.21.15.160:0/331351007 learned_addr learned my addr 172.21.15.160:0/331351007 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1cc878700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c8101ce0 0x7fe1c8102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 -- 172.21.15.160:0/331351007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c8101ce0 msgr2=0x7fe1c8102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c8101ce0 0x7fe1c8102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 -- 172.21.15.160:0/331351007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4009940 con 0x7fe1c8102680 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bffff700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 0x7fe1c8103080 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8002a50 tx=0x7fe1b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b8007cc0 con 0x7fe1c8102680 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1b800f040 con 0x7fe1c8102680 2024-05-25T06:33:20.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c8119fc0 con 0x7fe1c8102680 2024-05-25T06:33:20.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b800d9c0 con 0x7fe1c8102680 2024-05-25T06:33:20.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.650+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c811a440 con 0x7fe1c8102680 2024-05-25T06:33:20.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.651+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c80fd430 con 0x7fe1c8102680 2024-05-25T06:33:20.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:20 smithi203 ceph-mon[83507]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:20.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.652+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe1b8009730 con 0x7fe1c8102680 2024-05-25T06:33:20.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.652+0000 7fe1bd7fa700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe1b006e140 0x7fe1b00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:20.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.652+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe1b808d8c0 con 0x7fe1c8102680 2024-05-25T06:33:20.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.652+0000 7fe1cc878700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe1b006e140 0x7fe1b00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:20.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.653+0000 7fe1cc878700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe1b006e140 0x7fe1b00705f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400b740 tx=0x7fe1c4004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:20.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.657+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe1b8056bf0 con 0x7fe1c8102680 2024-05-25T06:33:20.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.873+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1c804bed0 con 0x7fe1b006e140 2024-05-25T06:33:20.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.874+0000 7fe1bd7fa700 1 -- 172.21.15.160:0/331351007 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe1c804bed0 con 0x7fe1b006e140 2024-05-25T06:33:20.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe1b006e140 msgr2=0x7fe1b00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe1b006e140 0x7fe1b00705f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400b740 tx=0x7fe1c4004d10 comp rx=0 tx=0).stop 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 msgr2=0x7fe1c8103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 0x7fe1c8103080 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8002a50 tx=0x7fe1b8004600 comp rx=0 tx=0).stop 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 shutdown_connections 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe1b006e140 0x7fe1b00705f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c8101ce0 0x7fe1c8102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 --2- 172.21.15.160:0/331351007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c8102680 0x7fe1c8103080 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 >> 172.21.15.160:0/331351007 conn(0x7fe1c80f7850 msgr2=0x7fe1c80f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 shutdown_connections 2024-05-25T06:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.877+0000 7fe1ceadc700 1 -- 172.21.15.160:0/331351007 wait complete. 2024-05-25T06:33:20.887 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:33:20.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 -- 172.21.15.160:0/2039039843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 msgr2=0x7f24981031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 --2- 172.21.15.160:0/2039039843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f24981031b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2488004940 tx=0x7f2488009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 -- 172.21.15.160:0/2039039843 shutdown_connections 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 --2- 172.21.15.160:0/2039039843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2498103780 0x7f24980fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 --2- 172.21.15.160:0/2039039843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f24981031b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 -- 172.21.15.160:0/2039039843 >> 172.21.15.160:0/2039039843 conn(0x7f24980f8990 msgr2=0x7f24980fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 -- 172.21.15.160:0/2039039843 shutdown_connections 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 -- 172.21.15.160:0/2039039843 wait complete. 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.984+0000 7f249d174700 1 Processor -- start 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f249d174700 1 -- start start 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f249d174700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f2498070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f249d174700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2498103780 0x7f24980707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:20.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f249d174700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498072160 con 0x7f2498102de0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f249d174700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498070ce0 con 0x7f2498103780 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f2498070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f2498070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45126/0 (socket says 172.21.15.160:45126) 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 -- 172.21.15.160:0/2634675596 learned_addr learned my addr 172.21.15.160:0/2634675596 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f248e59c700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2498103780 0x7f24980707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 -- 172.21.15.160:0/2634675596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2498103780 msgr2=0x7f24980707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2498103780 0x7f24980707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 -- 172.21.15.160:0/2634675596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2488005040 con 0x7f2498102de0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.985+0000 7f2496d9d700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f2498070260 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f248800b3b0 tx=0x7f2488009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.986+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248800e040 con 0x7f2498102de0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.986+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f248800bbc0 con 0x7f2498102de0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.986+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248801c3f0 con 0x7f2498102de0 2024-05-25T06:33:20.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.986+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498070f60 con 0x7f2498102de0 2024-05-25T06:33:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.986+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249811a110 con 0x7f2498102de0 2024-05-25T06:33:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.987+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24981000f0 con 0x7f2498102de0 2024-05-25T06:33:20.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.987+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f248800bd30 con 0x7f2498102de0 2024-05-25T06:33:20.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.988+0000 7f248ffff700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f248406e140 0x7f24840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:20.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.988+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f248808edc0 con 0x7f2498102de0 2024-05-25T06:33:20.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.988+0000 7f248e59c700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f248406e140 0x7f24840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:20.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.988+0000 7f248e59c700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f248406e140 0x7f24840705f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2480007fd0 tx=0x7f2480009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:20.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:20.992+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f24880580f0 con 0x7f2498102de0 2024-05-25T06:33:21.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.208+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24980716f0 con 0x7f248406e140 2024-05-25T06:33:21.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.210+0000 7f248ffff700 1 -- 172.21.15.160:0/2634675596 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f24980716f0 con 0x7f248406e140 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f248406e140 msgr2=0x7f24840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f248406e140 0x7f24840705f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2480007fd0 tx=0x7f2480009040 comp rx=0 tx=0).stop 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 msgr2=0x7f2498070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f2498070260 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f248800b3b0 tx=0x7f2488009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 shutdown_connections 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2498102de0 0x7f2498070260 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f248406e140 0x7f24840705f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 --2- 172.21.15.160:0/2634675596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2498103780 0x7f24980707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 >> 172.21.15.160:0/2634675596 conn(0x7f24980f8990 msgr2=0x7f2498107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:21.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 shutdown_connections 2024-05-25T06:33:21.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.212+0000 7f249d174700 1 -- 172.21.15.160:0/2634675596 wait complete. 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 -- 172.21.15.160:0/1161410807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fb3c0 msgr2=0x7f333c0fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 --2- 172.21.15.160:0/1161410807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fb3c0 0x7f333c0fb830 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3330004940 tx=0x7f3330009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 -- 172.21.15.160:0/1161410807 shutdown_connections 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 --2- 172.21.15.160:0/1161410807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fb3c0 0x7f333c0fb830 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 --2- 172.21.15.160:0/1161410807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f333c0fdbf0 0x7f333c0fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 -- 172.21.15.160:0/1161410807 >> 172.21.15.160:0/1161410807 conn(0x7f333c072380 msgr2=0x7f333c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 -- 172.21.15.160:0/1161410807 shutdown_connections 2024-05-25T06:33:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.315+0000 7f3342a42700 1 -- 172.21.15.160:0/1161410807 wait complete. 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f3342a42700 1 Processor -- start 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f3342a42700 1 -- start start 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f3342a42700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f333c0fb3c0 0x7f333c115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f3342a42700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 0x7f333c116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f3342a42700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c1166c0 con 0x7f333c0fdbf0 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f3342a42700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c10fcf0 con 0x7f333c0fb3c0 2024-05-25T06:33:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f333b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 0x7f333c116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f333b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 0x7f333c116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45154/0 (socket says 172.21.15.160:45154) 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f333b7fe700 1 -- 172.21.15.160:0/3104303676 learned_addr learned my addr 172.21.15.160:0/3104303676 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.316+0000 7f333bfff700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f333c0fb3c0 0x7f333c115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f333b7fe700 1 -- 172.21.15.160:0/3104303676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f333c0fb3c0 msgr2=0x7f333c115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f333b7fe700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f333c0fb3c0 0x7f333c115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f333b7fe700 1 -- 172.21.15.160:0/3104303676 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330005040 con 0x7f333c0fdbf0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f333b7fe700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 0x7f333c116070 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f333000b8b0 tx=0x7f3330009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333000e040 con 0x7f333c0fdbf0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3330004290 con 0x7f333c0fdbf0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c10ff70 con 0x7f333c0fdbf0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333001c3f0 con 0x7f333c0fdbf0 2024-05-25T06:33:21.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.317+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c110530 con 0x7f333c0fdbf0 2024-05-25T06:33:21.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.318+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c0fc3d0 con 0x7f333c0fdbf0 2024-05-25T06:33:21.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.319+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3330013420 con 0x7f333c0fdbf0 2024-05-25T06:33:21.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.319+0000 7f3338ff9700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f332406e140 0x7f33240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:21.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.319+0000 7f333bfff700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f332406e140 0x7f33240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:21.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.319+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f333008ed90 con 0x7f333c0fdbf0 2024-05-25T06:33:21.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.320+0000 7f333bfff700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f332406e140 0x7f33240705f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f332c004f60 tx=0x7f332c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:21.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.324+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f333005bc70 con 0x7f333c0fdbf0 2024-05-25T06:33:21.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.529+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f333c111220 con 0x7f332406e140 2024-05-25T06:33:21.539 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.540+0000 7f3338ff9700 1 -- 172.21.15.160:0/3104303676 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f333c111220 con 0x7f332406e140 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (19m) 12m ago 30m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (16m) 12m ago 30m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (16m) 12m ago 29m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (18m) 12m ago 30m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (13m) 12m ago 24m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (12m) 12m ago 24m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (12m) 12m ago 24m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (12m) 12m ago 24m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (20m) 12m ago 31m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (20m) 12m ago 29m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:33:21.540 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (17m) 12m ago 31m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (17m) 12m ago 29m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (19m) 12m ago 30m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (19m) 12m ago 29m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (16m) 12m ago 28m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (15m) 12m ago 28m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (15m) 12m ago 27m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (14m) 12m ago 27m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (13m) 12m ago 26m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (13m) 12m ago 26m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:33:21.541 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (19m) 12m ago 29m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.542+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f332406e140 msgr2=0x7f33240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.542+0000 7f3342a42700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f332406e140 0x7f33240705f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f332c004f60 tx=0x7f332c00b040 comp rx=0 tx=0).stop 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.542+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 msgr2=0x7f333c116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.542+0000 7f3342a42700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 0x7f333c116070 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f333000b8b0 tx=0x7f3330009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 shutdown_connections 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f332406e140 0x7f33240705f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f333c0fb3c0 0x7f333c115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 --2- 172.21.15.160:0/3104303676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f333c0fdbf0 0x7f333c116070 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 >> 172.21.15.160:0/3104303676 conn(0x7f333c072380 msgr2=0x7f333c0f8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 shutdown_connections 2024-05-25T06:33:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.543+0000 7f3342a42700 1 -- 172.21.15.160:0/3104303676 wait complete. 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 -- 172.21.15.160:0/309739970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 msgr2=0x7f13a40fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/309739970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a40fbfc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f13a0004940 tx=0x7f13a0009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 -- 172.21.15.160:0/309739970 shutdown_connections 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/309739970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a40fc590 0x7f13a4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/309739970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a40fbfc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 -- 172.21.15.160:0/309739970 >> 172.21.15.160:0/309739970 conn(0x7f13a40f7870 msgr2=0x7f13a40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 -- 172.21.15.160:0/309739970 shutdown_connections 2024-05-25T06:33:21.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 -- 172.21.15.160:0/309739970 wait complete. 2024-05-25T06:33:21.645 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:21 smithi160 ceph-mon[112094]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:21.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.644+0000 7f13ab1a5700 1 Processor -- start 2024-05-25T06:33:21.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13ab1a5700 1 -- start start 2024-05-25T06:33:21.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13ab1a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a4119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:21.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13ab1a5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a40fc590 0x7f13a411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13ab1a5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a411ab10 con 0x7f13a40fbbf0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13ab1a5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a41140f0 con 0x7f13a40fc590 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13a8f41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a4119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13a8f41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a4119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45164/0 (socket says 172.21.15.160:45164) 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13a8f41700 1 -- 172.21.15.160:0/585139280 learned_addr learned my addr 172.21.15.160:0/585139280 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f139bfff700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a40fc590 0x7f13a411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13a8f41700 1 -- 172.21.15.160:0/585139280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a40fc590 msgr2=0x7f13a411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13a8f41700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a40fc590 0x7f13a411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.645+0000 7f13a8f41700 1 -- 172.21.15.160:0/585139280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0005040 con 0x7f13a40fbbf0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.646+0000 7f13a8f41700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a4119f80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b3b0 tx=0x7f13a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.646+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a000e040 con 0x7f13a40fbbf0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.646+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13a000bbc0 con 0x7f13a40fbbf0 2024-05-25T06:33:21.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.646+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a4114370 con 0x7f13a40fbbf0 2024-05-25T06:33:21.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.646+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a001c3f0 con 0x7f13a40fbbf0 2024-05-25T06:33:21.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.646+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a41148b0 con 0x7f13a40fbbf0 2024-05-25T06:33:21.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.647+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a4066b30 con 0x7f13a40fbbf0 2024-05-25T06:33:21.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.648+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f13a000bd30 con 0x7f13a40fbbf0 2024-05-25T06:33:21.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.648+0000 7f13997fa700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f138c06e140 0x7f138c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:21.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.648+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f13a008ee60 con 0x7f13a40fbbf0 2024-05-25T06:33:21.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.648+0000 7f139bfff700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f138c06e140 0x7f138c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:21.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.649+0000 7f139bfff700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f138c06e140 0x7f138c0705f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1394009fd0 tx=0x7f1394009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:21 smithi203 ceph-mon[83507]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:21.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.652+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f13a00bd850 con 0x7f13a40fbbf0 2024-05-25T06:33:21.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.928+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f13a404c710 con 0x7f13a40fbbf0 2024-05-25T06:33:21.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.928+0000 7f13997fa700 1 -- 172.21.15.160:0/585139280 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f13a005be00 con 0x7f13a40fbbf0 2024-05-25T06:33:21.928 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:33:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:33:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:33:21.930 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:33:21.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.930+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f138c06e140 msgr2=0x7f138c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.930+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f138c06e140 0x7f138c0705f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1394009fd0 tx=0x7f1394009f60 comp rx=0 tx=0).stop 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.930+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 msgr2=0x7f13a4119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.930+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a4119f80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b3b0 tx=0x7f13a0009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 shutdown_connections 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13a40fbbf0 0x7f13a4119f80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f138c06e140 0x7f138c0705f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 --2- 172.21.15.160:0/585139280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a40fc590 0x7f13a411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 >> 172.21.15.160:0/585139280 conn(0x7f13a40f7870 msgr2=0x7f13a410cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 shutdown_connections 2024-05-25T06:33:21.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:21.931+0000 7f13ab1a5700 1 -- 172.21.15.160:0/585139280 wait complete. 2024-05-25T06:33:22.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 -- 172.21.15.160:0/1901881191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43501026a0 msgr2=0x7f43500fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 --2- 172.21.15.160:0/1901881191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43501026a0 0x7f43500fbd10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4340004940 tx=0x7f4340009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:22.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 -- 172.21.15.160:0/1901881191 shutdown_connections 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 --2- 172.21.15.160:0/1901881191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43501026a0 0x7f43500fbd10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 --2- 172.21.15.160:0/1901881191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4350101d00 0x7f43501020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 -- 172.21.15.160:0/1901881191 >> 172.21.15.160:0/1901881191 conn(0x7f43500f7850 msgr2=0x7f43500f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 -- 172.21.15.160:0/1901881191 shutdown_connections 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.033+0000 7f4357da2700 1 -- 172.21.15.160:0/1901881191 wait complete. 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4357da2700 1 Processor -- start 2024-05-25T06:33:22.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4357da2700 1 -- start start 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4357da2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 0x7f4350112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4357da2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43501026a0 0x7f4350112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4357da2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43501133f0 con 0x7f4350101d00 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4357da2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435010c9f0 con 0x7f43501026a0 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4355b3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 0x7f4350112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4355b3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 0x7f4350112860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45184/0 (socket says 172.21.15.160:45184) 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4355b3e700 1 -- 172.21.15.160:0/3624015634 learned_addr learned my addr 172.21.15.160:0/3624015634 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f435533d700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43501026a0 0x7f4350112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4355b3e700 1 -- 172.21.15.160:0/3624015634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43501026a0 msgr2=0x7f4350112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4355b3e700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43501026a0 0x7f4350112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.034+0000 7f4355b3e700 1 -- 172.21.15.160:0/3624015634 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340005040 con 0x7f4350101d00 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.035+0000 7f4355b3e700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 0x7f4350112860 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f434c002a80 tx=0x7f434c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:22.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.035+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f434c007cc0 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.035+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f434c00f040 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.035+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435010ccd0 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.035+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435010d210 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.035+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f434c0096f0 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.036+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43500fd450 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.036+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f434c00d690 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.037+0000 7f43467fc700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f433c070350 0x7f433c072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.037+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f434c08d320 con 0x7f4350101d00 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.037+0000 7f435533d700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f433c070350 0x7f433c072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.037+0000 7f435533d700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f433c070350 0x7f433c072800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f434000b3b0 tx=0x7f4340009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:22.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.042+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f434c05a180 con 0x7f4350101d00 2024-05-25T06:33:22.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.283+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4350108de0 con 0x7f4350101d00 2024-05-25T06:33:22.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.283+0000 7f43467fc700 1 -- 172.21.15.160:0/3624015634 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f434c059d10 con 0x7f4350101d00 2024-05-25T06:33:22.283 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:33:22.283 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:33:22.284 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:33:22.285 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:22.286 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:22.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f433c070350 msgr2=0x7f433c072800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f433c070350 0x7f433c072800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f434000b3b0 tx=0x7f4340009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:22.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 msgr2=0x7f4350112860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 0x7f4350112860 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f434c002a80 tx=0x7f434c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 shutdown_connections 2024-05-25T06:33:22.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4350101d00 0x7f4350112860 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f433c070350 0x7f433c072800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 --2- 172.21.15.160:0/3624015634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43501026a0 0x7f4350112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 >> 172.21.15.160:0/3624015634 conn(0x7f43500f7850 msgr2=0x7f43500ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:22.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 shutdown_connections 2024-05-25T06:33:22.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.286+0000 7f4357da2700 1 -- 172.21.15.160:0/3624015634 wait complete. 2024-05-25T06:33:22.288 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 -- 172.21.15.160:0/1020453312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 msgr2=0x7f57d8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 --2- 172.21.15.160:0/1020453312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8101a30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f57d4004940 tx=0x7f57d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 -- 172.21.15.160:0/1020453312 shutdown_connections 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 --2- 172.21.15.160:0/1020453312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8101a30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 --2- 172.21.15.160:0/1020453312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d80fb5a0 0x7f57d80fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 -- 172.21.15.160:0/1020453312 >> 172.21.15.160:0/1020453312 conn(0x7f57d8073350 msgr2=0x7f57d8073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 -- 172.21.15.160:0/1020453312 shutdown_connections 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.391+0000 7f57dfe72700 1 -- 172.21.15.160:0/1020453312 wait complete. 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dfe72700 1 Processor -- start 2024-05-25T06:33:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dfe72700 1 -- start start 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dfe72700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d80fb5a0 0x7f57d8113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dfe72700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dfe72700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81145c0 con 0x7f57d80fbf40 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dfe72700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d810dbf0 con 0x7f57d80fb5a0 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dd40d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dd40d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45204/0 (socket says 172.21.15.160:45204) 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57dd40d700 1 -- 172.21.15.160:0/3813672360 learned_addr learned my addr 172.21.15.160:0/3813672360 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.392+0000 7f57ddc0e700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d80fb5a0 0x7f57d8113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57dd40d700 1 -- 172.21.15.160:0/3813672360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d80fb5a0 msgr2=0x7f57d8113a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57dd40d700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d80fb5a0 0x7f57d8113a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57dd40d700 1 -- 172.21.15.160:0/3813672360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4005040 con 0x7f57d80fbf40 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57dd40d700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8113f70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f57d400b3b0 tx=0x7f57d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d400e040 con 0x7f57d80fbf40 2024-05-25T06:33:22.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d810de70 con 0x7f57d80fbf40 2024-05-25T06:33:22.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57d4013420 con 0x7f57d80fbf40 2024-05-25T06:33:22.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d401c630 con 0x7f57d80fbf40 2024-05-25T06:33:22.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.393+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d810e3b0 con 0x7f57d80fbf40 2024-05-25T06:33:22.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.394+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d80fe670 con 0x7f57d80fbf40 2024-05-25T06:33:22.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.394+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f57d4013590 con 0x7f57d80fbf40 2024-05-25T06:33:22.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.395+0000 7f57ca7fc700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f57c406e140 0x7f57c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.395+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f57d408ec90 con 0x7f57d80fbf40 2024-05-25T06:33:22.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.395+0000 7f57ddc0e700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f57c406e140 0x7f57c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.395+0000 7f57ddc0e700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f57c406e140 0x7f57c40705f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57cc004f60 tx=0x7f57cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:22.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.399+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f57d405bc20 con 0x7f57d80fbf40 2024-05-25T06:33:22.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.615+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57d8109f90 con 0x7f57c406e140 2024-05-25T06:33:22.615 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:22 smithi160 ceph-mon[112094]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:22.615 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:22 smithi160 ceph-mon[112094]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:22.615 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:22 smithi160 ceph-mon[112094]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:22.615 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:22 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/585139280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:33:22.615 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:22 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3624015634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.616+0000 7f57ca7fc700 1 -- 172.21.15.160:0/3813672360 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f57d8109f90 con 0x7f57c406e140 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:33:22.616 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:33:22.617 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:33:22.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f57c406e140 msgr2=0x7f57c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f57c406e140 0x7f57c40705f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57cc004f60 tx=0x7f57cc00b040 comp rx=0 tx=0).stop 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 msgr2=0x7f57d8113f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8113f70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f57d400b3b0 tx=0x7f57d4009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 shutdown_connections 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f57c406e140 0x7f57c40705f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d80fb5a0 0x7f57d8113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 --2- 172.21.15.160:0/3813672360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57d80fbf40 0x7f57d8113f70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 >> 172.21.15.160:0/3813672360 conn(0x7f57d8073350 msgr2=0x7f57d8100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:22.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 shutdown_connections 2024-05-25T06:33:22.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.618+0000 7f57dfe72700 1 -- 172.21.15.160:0/3813672360 wait complete. 2024-05-25T06:33:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:22 smithi203 ceph-mon[83507]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:22 smithi203 ceph-mon[83507]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:22 smithi203 ceph-mon[83507]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:22 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/585139280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:33:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:22 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3624015634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:33:22.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 -- 172.21.15.160:0/426804216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fd0a0 msgr2=0x7f8bd4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/426804216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4109350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc004940 tx=0x7f8bbc009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 -- 172.21.15.160:0/426804216 shutdown_connections 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/426804216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4109350 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/426804216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fc790 0x7f8bd40fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 -- 172.21.15.160:0/426804216 >> 172.21.15.160:0/426804216 conn(0x7f8bd4071d00 msgr2=0x7f8bd4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 -- 172.21.15.160:0/426804216 shutdown_connections 2024-05-25T06:33:22.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.721+0000 7f8bd8a98700 1 -- 172.21.15.160:0/426804216 wait complete. 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd8a98700 1 Processor -- start 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd8a98700 1 -- start start 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd8a98700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 0x7f8bd4115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd8a98700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd8a98700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd41169c0 con 0x7f8bd40fc790 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd8a98700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd410fd90 con 0x7f8bd40fd0a0 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 0x7f8bd4115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 0x7f8bd4115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:45212/0 (socket says 172.21.15.160:45212) 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 -- 172.21.15.160:0/2983441556 learned_addr learned my addr 172.21.15.160:0/2983441556 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd1d9b700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 -- 172.21.15.160:0/2983441556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fd0a0 msgr2=0x7f8bd4116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4116370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 -- 172.21.15.160:0/2983441556 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc005040 con 0x7f8bd40fc790 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd1d9b700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.722+0000 7f8bd259c700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 0x7f8bd4115e30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c8c0 tx=0x7f8bcc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.723+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bcc00a980 con 0x7f8bd40fc790 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.723+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bcc00ce60 con 0x7f8bd40fc790 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.723+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd4110070 con 0x7f8bd40fc790 2024-05-25T06:33:22.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.723+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd41105b0 con 0x7f8bd40fc790 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.723+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bcc0115e0 con 0x7f8bd40fc790 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.724+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb40052b0 con 0x7f8bd40fc790 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.724+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8bcc012870 con 0x7f8bd40fc790 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.725+0000 7f8bcaffd700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc006e140 0x7f8bc00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.725+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8bcc08db10 con 0x7f8bd40fc790 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.725+0000 7f8bd1d9b700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc006e140 0x7f8bc00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:22.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.725+0000 7f8bd1d9b700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc006e140 0x7f8bc00705f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc004f90 tx=0x7f8bbc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:22.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:22.729+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8bcc05aab0 con 0x7f8bd40fc790 2024-05-25T06:33:23.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.006+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8bb4005560 con 0x7f8bd40fc790 2024-05-25T06:33:23.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.006+0000 7f8bcaffd700 1 -- 172.21.15.160:0/2983441556 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8bcc05a640 con 0x7f8bd40fc790 2024-05-25T06:33:23.006 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:33:23.006 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:33:23.006 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:33:23.007 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:33:23.007 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:33:23.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.008+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc006e140 msgr2=0x7f8bc00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:23.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.008+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc006e140 0x7f8bc00705f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc004f90 tx=0x7f8bbc009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:23.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.008+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 msgr2=0x7f8bd4115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:23.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.008+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 0x7f8bd4115e30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c8c0 tx=0x7f8bcc0044c0 comp rx=0 tx=0).stop 2024-05-25T06:33:23.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 shutdown_connections 2024-05-25T06:33:23.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd40fc790 0x7f8bd4115e30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:23.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc006e140 0x7f8bc00705f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:23.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 --2- 172.21.15.160:0/2983441556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd40fd0a0 0x7f8bd4116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:23.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 >> 172.21.15.160:0/2983441556 conn(0x7f8bd4071d00 msgr2=0x7f8bd40f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:23.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 shutdown_connections 2024-05-25T06:33:23.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:23.009+0000 7f8bd8a98700 1 -- 172.21.15.160:0/2983441556 wait complete. 2024-05-25T06:33:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:23 smithi203 ceph-mon[83507]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:23 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2983441556' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:33:23.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:23 smithi160 ceph-mon[112094]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:23.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:23 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2983441556' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:33:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:24 smithi203 ceph-mon[83507]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:24 smithi160 ceph-mon[112094]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:26 smithi203 ceph-mon[83507]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:26.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:26 smithi160 ceph-mon[112094]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:28 smithi203 ceph-mon[83507]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:28 smithi160 ceph-mon[112094]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:30 smithi203 ceph-mon[83507]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:30 smithi160 ceph-mon[112094]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:32 smithi203 ceph-mon[83507]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:32.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:32 smithi160 ceph-mon[112094]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:34 smithi203 ceph-mon[83507]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:34 smithi160 ceph-mon[112094]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:36 smithi203 ceph-mon[83507]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:36 smithi160 ceph-mon[112094]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:39 smithi203 ceph-mon[83507]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:39 smithi160 ceph-mon[112094]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:41 smithi203 ceph-mon[83507]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:41 smithi160 ceph-mon[112094]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:43 smithi203 ceph-mon[83507]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:43 smithi160 ceph-mon[112094]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:45 smithi203 ceph-mon[83507]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:45 smithi160 ceph-mon[112094]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:47 smithi203 ceph-mon[83507]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:47 smithi160 ceph-mon[112094]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:49 smithi203 ceph-mon[83507]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:49 smithi160 ceph-mon[112094]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:51 smithi203 ceph-mon[83507]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:51 smithi160 ceph-mon[112094]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:53.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 -- 172.21.15.160:0/1353114100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 msgr2=0x7f3dc81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 --2- 172.21.15.160:0/1353114100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc81031b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3db8004940 tx=0x7f3db8009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:53.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 -- 172.21.15.160:0/1353114100 shutdown_connections 2024-05-25T06:33:53.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 --2- 172.21.15.160:0/1353114100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 0x7f3dc80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 --2- 172.21.15.160:0/1353114100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc81031b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 -- 172.21.15.160:0/1353114100 >> 172.21.15.160:0/1353114100 conn(0x7f3dc80f8990 msgr2=0x7f3dc80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:53.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 -- 172.21.15.160:0/1353114100 shutdown_connections 2024-05-25T06:33:53.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 -- 172.21.15.160:0/1353114100 wait complete. 2024-05-25T06:33:53.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.127+0000 7f3dce304700 1 Processor -- start 2024-05-25T06:33:53.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dce304700 1 -- start start 2024-05-25T06:33:53.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dce304700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dce304700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 0x7f3dc8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dce304700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8114500 con 0x7f3dc8102de0 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dce304700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810db50 con 0x7f3dc8103780 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc8113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60316/0 (socket says 172.21.15.160:60316) 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc7fff700 1 -- 172.21.15.160:0/4120307945 learned_addr learned my addr 172.21.15.160:0/4120307945 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc77fe700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 0x7f3dc8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc7fff700 1 -- 172.21.15.160:0/4120307945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 msgr2=0x7f3dc8113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc7fff700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 0x7f3dc8113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc7fff700 1 -- 172.21.15.160:0/4120307945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8005040 con 0x7f3dc8102de0 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.128+0000 7f3dc77fe700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 0x7f3dc8113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.129+0000 7f3dc7fff700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc8113970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b3b0 tx=0x7f3db8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.129+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db800e040 con 0x7f3dc8102de0 2024-05-25T06:33:53.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.129+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3db800bbc0 con 0x7f3dc8102de0 2024-05-25T06:33:53.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.129+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc810ddd0 con 0x7f3dc8102de0 2024-05-25T06:33:53.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.129+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db801c3f0 con 0x7f3dc8102de0 2024-05-25T06:33:53.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.129+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc810e390 con 0x7f3dc8102de0 2024-05-25T06:33:53.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.130+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc81000f0 con 0x7f3dc8102de0 2024-05-25T06:33:53.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.130+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3db801c660 con 0x7f3dc8102de0 2024-05-25T06:33:53.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.131+0000 7f3dc4ff9700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3db006e140 0x7f3db00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.131+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3db808ee70 con 0x7f3dc8102de0 2024-05-25T06:33:53.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.131+0000 7f3dc77fe700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3db006e140 0x7f3db00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.131+0000 7f3dc77fe700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3db006e140 0x7f3db00705f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007fd0 tx=0x7f3dbc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:53.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.136+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3db805bd40 con 0x7f3dc8102de0 2024-05-25T06:33:53.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.353+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dc810f030 con 0x7f3db006e140 2024-05-25T06:33:53.353 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:53 smithi160 ceph-mon[112094]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:53.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.354+0000 7f3dc4ff9700 1 -- 172.21.15.160:0/4120307945 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3dc810f030 con 0x7f3db006e140 2024-05-25T06:33:53.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.356+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3db006e140 msgr2=0x7f3db00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.356+0000 7f3dce304700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3db006e140 0x7f3db00705f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007fd0 tx=0x7f3dbc009040 comp rx=0 tx=0).stop 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.356+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 msgr2=0x7f3dc8113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.356+0000 7f3dce304700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc8113970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b3b0 tx=0x7f3db8009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 shutdown_connections 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc8102de0 0x7f3dc8113970 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3db006e140 0x7f3db00705f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 --2- 172.21.15.160:0/4120307945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc8103780 0x7f3dc8113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 >> 172.21.15.160:0/4120307945 conn(0x7f3dc80f8990 msgr2=0x7f3dc8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 shutdown_connections 2024-05-25T06:33:53.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.357+0000 7f3dce304700 1 -- 172.21.15.160:0/4120307945 wait complete. 2024-05-25T06:33:53.367 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:33:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:53 smithi203 ceph-mon[83507]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:53.460 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.459+0000 7ff554f9e700 1 -- 172.21.15.160:0/1869098872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 msgr2=0x7ff5501031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.459+0000 7ff554f9e700 1 --2- 172.21.15.160:0/1869098872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff5501031b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff540004940 tx=0x7ff540009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- 172.21.15.160:0/1869098872 shutdown_connections 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 --2- 172.21.15.160:0/1869098872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff550103780 0x7ff5500fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 --2- 172.21.15.160:0/1869098872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff5501031b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- 172.21.15.160:0/1869098872 >> 172.21.15.160:0/1869098872 conn(0x7ff5500f8990 msgr2=0x7ff5500fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- 172.21.15.160:0/1869098872 shutdown_connections 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- 172.21.15.160:0/1869098872 wait complete. 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 Processor -- start 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- start start 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff550113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff550103780 0x7ff550113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff550114500 con 0x7ff550102de0 2024-05-25T06:33:53.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff554f9e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55010db50 con 0x7ff550103780 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff54e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff550113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.460+0000 7ff54e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff550113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60330/0 (socket says 172.21.15.160:60330) 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff54e59c700 1 -- 172.21.15.160:0/4005007224 learned_addr learned my addr 172.21.15.160:0/4005007224 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff54dd9b700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff550103780 0x7ff550113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff54e59c700 1 -- 172.21.15.160:0/4005007224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff550103780 msgr2=0x7ff550113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff54e59c700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff550103780 0x7ff550113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff54e59c700 1 -- 172.21.15.160:0/4005007224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540005040 con 0x7ff550102de0 2024-05-25T06:33:53.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff54e59c700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff550113970 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff54000b8b0 tx=0x7ff540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:53.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54000e040 con 0x7ff550102de0 2024-05-25T06:33:53.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55010ddd0 con 0x7ff550102de0 2024-05-25T06:33:53.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff540004290 con 0x7ff550102de0 2024-05-25T06:33:53.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54001c3f0 con 0x7ff550102de0 2024-05-25T06:33:53.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.461+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55010e390 con 0x7ff550102de0 2024-05-25T06:33:53.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.462+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5501000f0 con 0x7ff550102de0 2024-05-25T06:33:53.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.463+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff5400137b0 con 0x7ff550102de0 2024-05-25T06:33:53.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.463+0000 7ff53effd700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff53806e140 0x7ff5380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.463+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff54008eea0 con 0x7ff550102de0 2024-05-25T06:33:53.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.463+0000 7ff54dd9b700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff53806e140 0x7ff5380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.464+0000 7ff54dd9b700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff53806e140 0x7ff5380705f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff55010ecc0 tx=0x7ff544009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:53.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.468+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff54005bd80 con 0x7ff550102de0 2024-05-25T06:33:53.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.685+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff55010f030 con 0x7ff53806e140 2024-05-25T06:33:53.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.686+0000 7ff53effd700 1 -- 172.21.15.160:0/4005007224 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff55010f030 con 0x7ff53806e140 2024-05-25T06:33:53.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.689+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff53806e140 msgr2=0x7ff5380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.689+0000 7ff554f9e700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff53806e140 0x7ff5380705f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff55010ecc0 tx=0x7ff544009ee0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.689+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 msgr2=0x7ff550113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.689+0000 7ff554f9e700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff550113970 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff54000b8b0 tx=0x7ff540009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 shutdown_connections 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff550102de0 0x7ff550113970 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff53806e140 0x7ff5380705f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 --2- 172.21.15.160:0/4005007224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff550103780 0x7ff550113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 >> 172.21.15.160:0/4005007224 conn(0x7ff5500f8990 msgr2=0x7ff550107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 shutdown_connections 2024-05-25T06:33:53.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.690+0000 7ff554f9e700 1 -- 172.21.15.160:0/4005007224 wait complete. 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 -- 172.21.15.160:0/2682712461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714102de0 msgr2=0x7f77141031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 --2- 172.21.15.160:0/2682712461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714102de0 0x7f77141031b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7704004940 tx=0x7f7704009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 -- 172.21.15.160:0/2682712461 shutdown_connections 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 --2- 172.21.15.160:0/2682712461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714103780 0x7f77140fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 --2- 172.21.15.160:0/2682712461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714102de0 0x7f77141031b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 -- 172.21.15.160:0/2682712461 >> 172.21.15.160:0/2682712461 conn(0x7f77140f8990 msgr2=0x7f77140fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:53.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 -- 172.21.15.160:0/2682712461 shutdown_connections 2024-05-25T06:33:53.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 -- 172.21.15.160:0/2682712461 wait complete. 2024-05-25T06:33:53.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.792+0000 7f7719bf2700 1 Processor -- start 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f7719bf2700 1 -- start start 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f7719bf2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714102de0 0x7f7714070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f7719bf2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 0x7f77140707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f7719bf2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7714072160 con 0x7f7714103780 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f7719bf2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7714070ce0 con 0x7f7714102de0 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 0x7f77140707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 0x7f77140707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60342/0 (socket says 172.21.15.160:60342) 2024-05-25T06:33:53.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 -- 172.21.15.160:0/1627217407 learned_addr learned my addr 172.21.15.160:0/1627217407 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f77137fe700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714102de0 0x7f7714070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 -- 172.21.15.160:0/1627217407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714102de0 msgr2=0x7f7714070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714102de0 0x7f7714070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 -- 172.21.15.160:0/1627217407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704005040 con 0x7f7714103780 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.793+0000 7f770affd700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 0x7f77140707a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c800 tx=0x7f76fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.794+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76fc00a760 con 0x7f7714103780 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.794+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7714070fc0 con 0x7f7714103780 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.794+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76fc00ad90 con 0x7f7714103780 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.794+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76fc0113a0 con 0x7f7714103780 2024-05-25T06:33:53.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.794+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771411a160 con 0x7f7714103780 2024-05-25T06:33:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.795+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77141000f0 con 0x7f7714103780 2024-05-25T06:33:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.795+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f76fc012400 con 0x7f7714103780 2024-05-25T06:33:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.796+0000 7f7710ff9700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f770006e140 0x7f77000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.796+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f76fc08db90 con 0x7f7714103780 2024-05-25T06:33:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.796+0000 7f77137fe700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f770006e140 0x7f77000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.796+0000 7f77137fe700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f770006e140 0x7f77000705f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7704004f90 tx=0x7f7704009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:53.800+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f76fc056f40 con 0x7f7714103780 2024-05-25T06:33:54.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.006+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7714109ee0 con 0x7f770006e140 2024-05-25T06:33:54.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.016+0000 7f7710ff9700 1 -- 172.21.15.160:0/1627217407 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7714109ee0 con 0x7f770006e140 2024-05-25T06:33:54.016 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:33:54.016 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (19m) 13m ago 30m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (17m) 13m ago 31m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (17m) 12m ago 30m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (19m) 13m ago 30m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (13m) 13m ago 25m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (13m) 13m ago 25m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (12m) 12m ago 25m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (12m) 12m ago 25m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (21m) 13m ago 32m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (20m) 12m ago 29m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (18m) 13m ago 32m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (17m) 12m ago 29m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (20m) 13m ago 30m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (20m) 12m ago 30m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:33:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (16m) 13m ago 29m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:33:54.018 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (16m) 13m ago 28m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:33:54.018 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (15m) 13m ago 28m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:33:54.018 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (14m) 12m ago 27m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:33:54.018 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (14m) 12m ago 27m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:33:54.018 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (13m) 12m ago 26m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:33:54.018 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (20m) 13m ago 30m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f770006e140 msgr2=0x7f77000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f770006e140 0x7f77000705f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7704004f90 tx=0x7f7704009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 msgr2=0x7f77140707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 0x7f77140707a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c800 tx=0x7f76fc004470 comp rx=0 tx=0).stop 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 shutdown_connections 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f770006e140 0x7f77000705f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714102de0 0x7f7714070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 --2- 172.21.15.160:0/1627217407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7714103780 0x7f77140707a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 >> 172.21.15.160:0/1627217407 conn(0x7f77140f8990 msgr2=0x7f7714107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:54.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 shutdown_connections 2024-05-25T06:33:54.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.019+0000 7f7719bf2700 1 -- 172.21.15.160:0/1627217407 wait complete. 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/1662362362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 msgr2=0x7ff6ac0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/1662362362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac0fb990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff698004940 tx=0x7ff698009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/1662362362 shutdown_connections 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/1662362362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac0fb990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/1662362362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 0x7ff6ac103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/1662362362 >> 172.21.15.160:0/1662362362 conn(0x7ff6ac0714d0 msgr2=0x7ff6ac0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:54.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/1662362362 shutdown_connections 2024-05-25T06:33:54.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/1662362362 wait complete. 2024-05-25T06:33:54.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 Processor -- start 2024-05-25T06:33:54.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.122+0000 7ff6b2d8c700 1 -- start start 2024-05-25T06:33:54.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b2d8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b2d8c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 0x7ff6ac113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b2d8c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac114510 con 0x7ff6ac0fb520 2024-05-25T06:33:54.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b2d8c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac10db10 con 0x7ff6ac102f60 2024-05-25T06:33:54.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60350/0 (socket says 172.21.15.160:60350) 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 -- 172.21.15.160:0/2975989845 learned_addr learned my addr 172.21.15.160:0/2975989845 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6abfff700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 0x7ff6ac113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 -- 172.21.15.160:0/2975989845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 msgr2=0x7ff6ac113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 0x7ff6ac113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 -- 172.21.15.160:0/2975989845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698005040 con 0x7ff6ac0fb520 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6abfff700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 0x7ff6ac113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b0b28700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac113980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c8c0 tx=0x7ff6a00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a000a980 con 0x7ff6ac0fb520 2024-05-25T06:33:54.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6a000ce60 con 0x7ff6ac0fb520 2024-05-25T06:33:54.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac10ddf0 con 0x7ff6ac0fb520 2024-05-25T06:33:54.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a00115e0 con 0x7ff6ac0fb520 2024-05-25T06:33:54.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.123+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac10e330 con 0x7ff6ac0fb520 2024-05-25T06:33:54.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.124+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac100250 con 0x7ff6ac0fb520 2024-05-25T06:33:54.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.125+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff6a0012400 con 0x7ff6ac0fb520 2024-05-25T06:33:54.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.125+0000 7ff6a97fa700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff69c06e140 0x7ff69c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.125+0000 7ff6abfff700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff69c06e140 0x7ff69c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.125+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff6a008db50 con 0x7ff6ac0fb520 2024-05-25T06:33:54.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.126+0000 7ff6abfff700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff69c06e140 0x7ff69c0705f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff69800b3b0 tx=0x7ff698009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:54.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.130+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff6a005aaf0 con 0x7ff6ac0fb520 2024-05-25T06:33:54.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.410+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff6ac10efd0 con 0x7ff6ac0fb520 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.411+0000 7ff6a97fa700 1 -- 172.21.15.160:0/2975989845 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7ff6a00126b0 con 0x7ff6ac0fb520 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:33:54.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:33:54.412 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:33:54.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff69c06e140 msgr2=0x7ff69c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff69c06e140 0x7ff69c0705f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff69800b3b0 tx=0x7ff698009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:54.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 msgr2=0x7ff6ac113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac113980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c8c0 tx=0x7ff6a00044c0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 shutdown_connections 2024-05-25T06:33:54.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6ac0fb520 0x7ff6ac113980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff69c06e140 0x7ff69c0705f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 --2- 172.21.15.160:0/2975989845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ac102f60 0x7ff6ac113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 >> 172.21.15.160:0/2975989845 conn(0x7ff6ac0714d0 msgr2=0x7ff6ac0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:54.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 shutdown_connections 2024-05-25T06:33:54.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.413+0000 7ff6b2d8c700 1 -- 172.21.15.160:0/2975989845 wait complete. 2024-05-25T06:33:54.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 -- 172.21.15.160:0/3818777847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8102de0 msgr2=0x7f4fc81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 --2- 172.21.15.160:0/3818777847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8102de0 0x7f4fc81031b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004940 tx=0x7f4fb8009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:54.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 -- 172.21.15.160:0/3818777847 shutdown_connections 2024-05-25T06:33:54.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 --2- 172.21.15.160:0/3818777847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8103780 0x7f4fc80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 --2- 172.21.15.160:0/3818777847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8102de0 0x7f4fc81031b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 -- 172.21.15.160:0/3818777847 >> 172.21.15.160:0/3818777847 conn(0x7f4fc80f8990 msgr2=0x7f4fc80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:54.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 -- 172.21.15.160:0/3818777847 shutdown_connections 2024-05-25T06:33:54.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.515+0000 7f4fcd286700 1 -- 172.21.15.160:0/3818777847 wait complete. 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fcd286700 1 Processor -- start 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fcd286700 1 -- start start 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fcd286700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8102de0 0x7f4fc8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fcd286700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 0x7f4fc8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fcd286700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8114500 con 0x7f4fc8103780 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fcd286700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc810db50 con 0x7f4fc8102de0 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fc67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 0x7f4fc8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fc67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 0x7f4fc8113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60370/0 (socket says 172.21.15.160:60370) 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.516+0000 7f4fc67fc700 1 -- 172.21.15.160:0/2495444208 learned_addr learned my addr 172.21.15.160:0/2495444208 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fc6ffd700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8102de0 0x7f4fc8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fc67fc700 1 -- 172.21.15.160:0/2495444208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8102de0 msgr2=0x7f4fc8113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fc67fc700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8102de0 0x7f4fc8113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fc67fc700 1 -- 172.21.15.160:0/2495444208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8005040 con 0x7f4fc8103780 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fc6ffd700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8102de0 0x7f4fc8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fc67fc700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 0x7f4fc8113eb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c800 tx=0x7f4fbc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fbc00a760 con 0x7f4fc8103780 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc810de30 con 0x7f4fc8103780 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fbc00ad90 con 0x7f4fc8103780 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fbc0113a0 con 0x7f4fc8103780 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.517+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc810e3f0 con 0x7f4fc8103780 2024-05-25T06:33:54.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.518+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc81000f0 con 0x7f4fc8103780 2024-05-25T06:33:54.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.519+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4fbc012400 con 0x7f4fc8103780 2024-05-25T06:33:54.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.519+0000 7f4faf7fe700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4fb006e140 0x7f4fb00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.519+0000 7f4fc6ffd700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4fb006e140 0x7f4fb00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.519+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4fbc08db90 con 0x7f4fc8103780 2024-05-25T06:33:54.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.520+0000 7f4fc6ffd700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4fb006e140 0x7f4fb00705f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004560 tx=0x7f4fb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:54.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.524+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4fbc05aa70 con 0x7f4fc8103780 2024-05-25T06:33:54.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.764+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4fc810efc0 con 0x7f4fc8103780 2024-05-25T06:33:54.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.765+0000 7f4faf7fe700 1 -- 172.21.15.160:0/2495444208 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f4fbc05a600 con 0x7f4fc8103780 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:33:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:33:54.766 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:33:54.767 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:33:54.768 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:54.768 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:54.768 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:54.768 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:33:54.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4fb006e140 msgr2=0x7f4fb00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4fb006e140 0x7f4fb00705f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004560 tx=0x7f4fb8009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 msgr2=0x7f4fc8113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 0x7f4fc8113eb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c800 tx=0x7f4fbc004470 comp rx=0 tx=0).stop 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 shutdown_connections 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4fb006e140 0x7f4fb00705f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc8102de0 0x7f4fc8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 --2- 172.21.15.160:0/2495444208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fc8103780 0x7f4fc8113eb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 >> 172.21.15.160:0/2495444208 conn(0x7f4fc80f8990 msgr2=0x7f4fc8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 shutdown_connections 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.767+0000 7f4fcd286700 1 -- 172.21.15.160:0/2495444208 wait complete. 2024-05-25T06:33:54.769 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 -- 172.21.15.160:0/2209839758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 msgr2=0x7f2388103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 --2- 172.21.15.160:0/2209839758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388103220 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2384009ce0 tx=0x7f2384004d10 comp rx=0 tx=0).stop 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 -- 172.21.15.160:0/2209839758 shutdown_connections 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 --2- 172.21.15.160:0/2209839758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2388065640 0x7f2388065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 --2- 172.21.15.160:0/2209839758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388103220 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 -- 172.21.15.160:0/2209839758 >> 172.21.15.160:0/2209839758 conn(0x7f2388071d00 msgr2=0x7f2388072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 -- 172.21.15.160:0/2209839758 shutdown_connections 2024-05-25T06:33:54.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.870+0000 7f238f41b700 1 -- 172.21.15.160:0/2209839758 wait complete. 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238f41b700 1 Processor -- start 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238f41b700 1 -- start start 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238f41b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2388065640 0x7f2388113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238f41b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238f41b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388114880 con 0x7f2388102e50 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238f41b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238810dc50 con 0x7f2388065640 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388114230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60382/0 (socket says 172.21.15.160:60382) 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 -- 172.21.15.160:0/3885554527 learned_addr learned my addr 172.21.15.160:0/3885554527 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238d1b7700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2388065640 0x7f2388113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 -- 172.21.15.160:0/3885554527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2388065640 msgr2=0x7f2388113cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2388065640 0x7f2388113cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 -- 172.21.15.160:0/3885554527 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384009940 con 0x7f2388102e50 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.871+0000 7f238c9b6700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388114230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f237800c830 tx=0x7f2378004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.872+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23780106f0 con 0x7f2388102e50 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.872+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238810df30 con 0x7f2388102e50 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.872+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2378010d20 con 0x7f2388102e50 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.872+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23780113a0 con 0x7f2388102e50 2024-05-25T06:33:54.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.872+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238810e4f0 con 0x7f2388102e50 2024-05-25T06:33:54.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.873+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2388100100 con 0x7f2388102e50 2024-05-25T06:33:54.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.873+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2378012400 con 0x7f2388102e50 2024-05-25T06:33:54.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.874+0000 7f237dffb700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f237406e140 0x7f23740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:54.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.874+0000 7f238d1b7700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f237406e140 0x7f23740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:54.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.874+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f237808dac0 con 0x7f2388102e50 2024-05-25T06:33:54.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.875+0000 7f238d1b7700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f237406e140 0x7f23740705f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f238400b740 tx=0x7f2384004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:54.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:54.879+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f237805a9a0 con 0x7f2388102e50 2024-05-25T06:33:55.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.096+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f238810f340 con 0x7f237406e140 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.097+0000 7f237dffb700 1 -- 172.21.15.160:0/3885554527 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f238810f340 con 0x7f237406e140 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:33:55.097 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:33:55.098 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:33:55.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.099+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f237406e140 msgr2=0x7f23740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:55.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.099+0000 7f238f41b700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f237406e140 0x7f23740705f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f238400b740 tx=0x7f2384004d10 comp rx=0 tx=0).stop 2024-05-25T06:33:55.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.099+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 msgr2=0x7f2388114230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.099+0000 7f238f41b700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388114230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f237800c830 tx=0x7f2378004470 comp rx=0 tx=0).stop 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 shutdown_connections 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f237406e140 0x7f23740705f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2388065640 0x7f2388113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 --2- 172.21.15.160:0/3885554527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2388102e50 0x7f2388114230 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 >> 172.21.15.160:0/3885554527 conn(0x7f2388071d00 msgr2=0x7f23880fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 shutdown_connections 2024-05-25T06:33:55.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.100+0000 7f238f41b700 1 -- 172.21.15.160:0/3885554527 wait complete. 2024-05-25T06:33:55.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 -- 172.21.15.160:0/1410580383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 msgr2=0x7efd3c070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:55.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 --2- 172.21.15.160:0/1410580383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c070cb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efd30004970 tx=0x7efd30009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:55.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 -- 172.21.15.160:0/1410580383 shutdown_connections 2024-05-25T06:33:55.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 --2- 172.21.15.160:0/1410580383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c070cb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 --2- 172.21.15.160:0/1410580383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c071f30 0x7efd3c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 -- 172.21.15.160:0/1410580383 >> 172.21.15.160:0/1410580383 conn(0x7efd3c0f8990 msgr2=0x7efd3c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:55.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 -- 172.21.15.160:0/1410580383 shutdown_connections 2024-05-25T06:33:55.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.203+0000 7efd4185f700 1 -- 172.21.15.160:0/1410580383 wait complete. 2024-05-25T06:33:55.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd4185f700 1 Processor -- start 2024-05-25T06:33:55.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd4185f700 1 -- start start 2024-05-25T06:33:55.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd4185f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:55.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd4185f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c071f30 0x7efd3c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:55.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd4185f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c11aad0 con 0x7efd3c070840 2024-05-25T06:33:55.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd4185f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c1140b0 con 0x7efd3c071f30 2024-05-25T06:33:55.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd3affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:55.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd3affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60402/0 (socket says 172.21.15.160:60402) 2024-05-25T06:33:55.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.204+0000 7efd3affd700 1 -- 172.21.15.160:0/2577482600 learned_addr learned my addr 172.21.15.160:0/2577482600 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:33:55.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd3a7fc700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c071f30 0x7efd3c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:55.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd3affd700 1 -- 172.21.15.160:0/2577482600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c071f30 msgr2=0x7efd3c11a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd3affd700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c071f30 0x7efd3c11a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd3affd700 1 -- 172.21.15.160:0/2577482600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd30005040 con 0x7efd3c070840 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd3affd700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c119f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efd2c002a50 tx=0x7efd2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd2c007cc0 con 0x7efd3c070840 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd2c00f040 con 0x7efd3c070840 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c114390 con 0x7efd3c070840 2024-05-25T06:33:55.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd2c00da30 con 0x7efd3c070840 2024-05-25T06:33:55.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.205+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c1148a0 con 0x7efd3c070840 2024-05-25T06:33:55.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.206+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c071a40 con 0x7efd3c070840 2024-05-25T06:33:55.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.207+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7efd2c00db90 con 0x7efd3c070840 2024-05-25T06:33:55.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.207+0000 7efd237fe700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7efd2406e140 0x7efd240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:33:55.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.207+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efd2c08d910 con 0x7efd3c070840 2024-05-25T06:33:55.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.207+0000 7efd3a7fc700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7efd2406e140 0x7efd240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:33:55.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.208+0000 7efd3a7fc700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7efd2406e140 0x7efd240705f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efd30004f50 tx=0x7efd30009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:33:55.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.212+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd2c05a770 con 0x7efd3c070840 2024-05-25T06:33:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:55 smithi203 ceph-mon[83507]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:55 smithi203 ceph-mon[83507]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:55 smithi203 ceph-mon[83507]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:55 smithi203 ceph-mon[83507]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:55 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2975989845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:33:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:55 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2495444208' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:33:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:55 smithi160 ceph-mon[112094]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:55 smithi160 ceph-mon[112094]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:55 smithi160 ceph-mon[112094]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:55 smithi160 ceph-mon[112094]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:55.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:55 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2975989845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:33:55.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:55 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2495444208' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:33:55.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.489+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd3c04c710 con 0x7efd3c070840 2024-05-25T06:33:55.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.490+0000 7efd237fe700 1 -- 172.21.15.160:0/2577482600 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7efd2c05a300 con 0x7efd3c070840 2024-05-25T06:33:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:33:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:33:55.490 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:33:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:33:55.490 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7efd2406e140 msgr2=0x7efd240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7efd2406e140 0x7efd240705f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efd30004f50 tx=0x7efd30009e30 comp rx=0 tx=0).stop 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 msgr2=0x7efd3c119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c119f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efd2c002a50 tx=0x7efd2c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 shutdown_connections 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd3c070840 0x7efd3c119f40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7efd2406e140 0x7efd240705f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 --2- 172.21.15.160:0/2577482600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c071f30 0x7efd3c11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:33:55.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 >> 172.21.15.160:0/2577482600 conn(0x7efd3c0f8990 msgr2=0x7efd3c0f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:33:55.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 shutdown_connections 2024-05-25T06:33:55.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:33:55.492+0000 7efd4185f700 1 -- 172.21.15.160:0/2577482600 wait complete. 2024-05-25T06:33:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:56 smithi203 ceph-mon[83507]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:56.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:56 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2577482600' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:33:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:56 smithi160 ceph-mon[112094]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:33:56.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:56 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2577482600' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:33:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:57 smithi203 ceph-mon[83507]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:57 smithi160 ceph-mon[112094]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:58.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:33:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:33:59 smithi203 ceph-mon[83507]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:33:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:33:59 smithi160 ceph-mon[112094]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:01 smithi203 ceph-mon[83507]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:01 smithi160 ceph-mon[112094]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:03.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:03 smithi203 ceph-mon[83507]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:03 smithi160 ceph-mon[112094]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:05 smithi203 ceph-mon[83507]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:05 smithi160 ceph-mon[112094]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:07 smithi203 ceph-mon[83507]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:07 smithi160 ceph-mon[112094]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:09 smithi203 ceph-mon[83507]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:09 smithi160 ceph-mon[112094]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:10 smithi160 ceph-mon[112094]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:10 smithi203 ceph-mon[83507]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:12 smithi160 ceph-mon[112094]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:12 smithi203 ceph-mon[83507]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:14 smithi160 ceph-mon[112094]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:14 smithi203 ceph-mon[83507]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:16 smithi160 ceph-mon[112094]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:16 smithi203 ceph-mon[83507]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:18 smithi160 ceph-mon[112094]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:18 smithi203 ceph-mon[83507]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:20 smithi160 ceph-mon[112094]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:20 smithi203 ceph-mon[83507]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:22 smithi160 ceph-mon[112094]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:22 smithi203 ceph-mon[83507]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:24.451 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:24 smithi203 ceph-mon[83507]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:24 smithi160 ceph-mon[112094]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 -- 172.21.15.160:0/2857273096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 msgr2=0x7f69ec0fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2857273096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0fcdd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f69dc004940 tx=0x7f69dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 -- 172.21.15.160:0/2857273096 shutdown_connections 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2857273096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0fcdd0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2857273096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ec102dc0 0x7f69ec103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 -- 172.21.15.160:0/2857273096 >> 172.21.15.160:0/2857273096 conn(0x7f69ec0f8930 msgr2=0x7f69ec0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 -- 172.21.15.160:0/2857273096 shutdown_connections 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.609+0000 7f69f34ac700 1 -- 172.21.15.160:0/2857273096 wait complete. 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f34ac700 1 Processor -- start 2024-05-25T06:34:25.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f34ac700 1 -- start start 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f34ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ec102dc0 0x7f69ec070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f34ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f34ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0721b0 con 0x7f69ec103760 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f34ac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec070ce0 con 0x7f69ec102dc0 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f0a47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f0a47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47422/0 (socket says 172.21.15.160:47422) 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f0a47700 1 -- 172.21.15.160:0/2127343613 learned_addr learned my addr 172.21.15.160:0/2127343613 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f1248700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ec102dc0 0x7f69ec070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f0a47700 1 -- 172.21.15.160:0/2127343613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ec102dc0 msgr2=0x7f69ec070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f0a47700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ec102dc0 0x7f69ec070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.610+0000 7f69f0a47700 1 -- 172.21.15.160:0/2127343613 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc005040 con 0x7f69ec103760 2024-05-25T06:34:25.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.611+0000 7f69f0a47700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0707a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009100 tx=0x7f69dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:25.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.611+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69dc00e040 con 0x7f69ec103760 2024-05-25T06:34:25.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.611+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69dc009af0 con 0x7f69ec103760 2024-05-25T06:34:25.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.611+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec070f60 con 0x7f69ec103760 2024-05-25T06:34:25.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.611+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69dc01c3f0 con 0x7f69ec103760 2024-05-25T06:34:25.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.611+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec11a0a0 con 0x7f69ec103760 2024-05-25T06:34:25.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.612+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec0fe510 con 0x7f69ec103760 2024-05-25T06:34:25.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.613+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f69dc00baf0 con 0x7f69ec103760 2024-05-25T06:34:25.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.613+0000 7f69e1ffb700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f69d806e140 0x7f69d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:25.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.613+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f69dc08edb0 con 0x7f69ec103760 2024-05-25T06:34:25.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.613+0000 7f69f1248700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f69d806e140 0x7f69d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:25.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.614+0000 7f69f1248700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f69d806e140 0x7f69d80705f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69e8004eb0 tx=0x7f69e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:25.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.618+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f69dc058160 con 0x7f69ec103760 2024-05-25T06:34:25.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.836+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69ec109f10 con 0x7f69d806e140 2024-05-25T06:34:25.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.838+0000 7f69e1ffb700 1 -- 172.21.15.160:0/2127343613 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f69ec109f10 con 0x7f69d806e140 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f69d806e140 msgr2=0x7f69d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f69d806e140 0x7f69d80705f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69e8004eb0 tx=0x7f69e800b040 comp rx=0 tx=0).stop 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 msgr2=0x7f69ec0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0707a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009100 tx=0x7f69dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 shutdown_connections 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f69d806e140 0x7f69d80705f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ec102dc0 0x7f69ec070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 --2- 172.21.15.160:0/2127343613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69ec103760 0x7f69ec0707a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 >> 172.21.15.160:0/2127343613 conn(0x7f69ec0f8930 msgr2=0x7f69ec0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:25.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 shutdown_connections 2024-05-25T06:34:25.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.840+0000 7f69f34ac700 1 -- 172.21.15.160:0/2127343613 wait complete. 2024-05-25T06:34:25.851 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:34:25.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 -- 172.21.15.160:0/1339879364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4102680 msgr2=0x7fb3f40fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:25.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/1339879364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4102680 0x7fb3f40fbcf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc004940 tx=0x7fb3dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:25.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 -- 172.21.15.160:0/1339879364 shutdown_connections 2024-05-25T06:34:25.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/1339879364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4102680 0x7fb3f40fbcf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/1339879364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4101ce0 0x7fb3f41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 -- 172.21.15.160:0/1339879364 >> 172.21.15.160:0/1339879364 conn(0x7fb3f40f7850 msgr2=0x7fb3f40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:25.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 -- 172.21.15.160:0/1339879364 shutdown_connections 2024-05-25T06:34:25.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.943+0000 7fb3f8d37700 1 -- 172.21.15.160:0/1339879364 wait complete. 2024-05-25T06:34:25.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f8d37700 1 Processor -- start 2024-05-25T06:34:25.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f8d37700 1 -- start start 2024-05-25T06:34:25.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f8d37700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 0x7fb3f4113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f8d37700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4102680 0x7fb3f41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f8d37700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41147a0 con 0x7fb3f4101ce0 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f8d37700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f410db00 con 0x7fb3f4102680 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 0x7fb3f4113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 0x7fb3f4113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47430/0 (socket says 172.21.15.160:47430) 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.944+0000 7fb3f259c700 1 -- 172.21.15.160:0/58520818 learned_addr learned my addr 172.21.15.160:0/58520818 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3e9d9b700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4102680 0x7fb3f41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3f259c700 1 -- 172.21.15.160:0/58520818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4102680 msgr2=0x7fb3f41140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3f259c700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4102680 0x7fb3f41140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3f259c700 1 -- 172.21.15.160:0/58520818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc005040 con 0x7fb3f4101ce0 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3e9d9b700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4102680 0x7fb3f41140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3f259c700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 0x7fb3f4113b80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b790 tx=0x7fb3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:25.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e400f770 con 0x7fb3f4101ce0 2024-05-25T06:34:25.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3e400fda0 con 0x7fb3f4101ce0 2024-05-25T06:34:25.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f410dde0 con 0x7fb3f4101ce0 2024-05-25T06:34:25.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e400d310 con 0x7fb3f4101ce0 2024-05-25T06:34:25.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.945+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f410e3a0 con 0x7fb3f4101ce0 2024-05-25T06:34:25.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.946+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f40fd430 con 0x7fb3f4101ce0 2024-05-25T06:34:25.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.947+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb3e4010420 con 0x7fb3f4101ce0 2024-05-25T06:34:25.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.947+0000 7fb3eb7fe700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb3e006e140 0x7fb3e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:25.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.947+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb3e408e1d0 con 0x7fb3f4101ce0 2024-05-25T06:34:25.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.947+0000 7fb3e9d9b700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb3e006e140 0x7fb3e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:25.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.948+0000 7fb3e9d9b700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb3e006e140 0x7fb3e00705f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b3b0 tx=0x7fb3dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:25.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:25.951+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb3e405b170 con 0x7fb3f4101ce0 2024-05-25T06:34:26.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.167+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3f404bed0 con 0x7fb3e006e140 2024-05-25T06:34:26.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.168+0000 7fb3eb7fe700 1 -- 172.21.15.160:0/58520818 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb3f404bed0 con 0x7fb3e006e140 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb3e006e140 msgr2=0x7fb3e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb3e006e140 0x7fb3e00705f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b3b0 tx=0x7fb3dc009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 msgr2=0x7fb3f4113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 0x7fb3f4113b80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b790 tx=0x7fb3e4009250 comp rx=0 tx=0).stop 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 shutdown_connections 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f4101ce0 0x7fb3f4113b80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb3e006e140 0x7fb3e00705f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 --2- 172.21.15.160:0/58520818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3f4102680 0x7fb3f41140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 >> 172.21.15.160:0/58520818 conn(0x7fb3f40f7850 msgr2=0x7fb3f40f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:26.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 shutdown_connections 2024-05-25T06:34:26.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.170+0000 7fb3f8d37700 1 -- 172.21.15.160:0/58520818 wait complete. 2024-05-25T06:34:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:26 smithi160 ceph-mon[112094]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:26.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:26 smithi160 ceph-mon[112094]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.273+0000 7f7c57d61700 1 -- 172.21.15.160:0/3845090805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 msgr2=0x7f7c501020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.273+0000 7f7c57d61700 1 --2- 172.21.15.160:0/3845090805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501020d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c004940 tx=0x7f7c4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.273+0000 7f7c57d61700 1 -- 172.21.15.160:0/3845090805 shutdown_connections 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.273+0000 7f7c57d61700 1 --2- 172.21.15.160:0/3845090805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c501026a0 0x7f7c500fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.274+0000 7f7c57d61700 1 --2- 172.21.15.160:0/3845090805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501020d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.274+0000 7f7c57d61700 1 -- 172.21.15.160:0/3845090805 >> 172.21.15.160:0/3845090805 conn(0x7f7c500f7850 msgr2=0x7f7c500f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.274+0000 7f7c57d61700 1 -- 172.21.15.160:0/3845090805 shutdown_connections 2024-05-25T06:34:26.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.274+0000 7f7c57d61700 1 -- 172.21.15.160:0/3845090805 wait complete. 2024-05-25T06:34:26.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.274+0000 7f7c57d61700 1 Processor -- start 2024-05-25T06:34:26.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.274+0000 7f7c57d61700 1 -- start start 2024-05-25T06:34:26.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c57d61700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501128f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:26.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c57d61700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c501026a0 0x7f7c50112e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:26.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c57d61700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50113480 con 0x7f7c50101d00 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c57d61700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5010ca80 con 0x7f7c501026a0 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47446/0 (socket says 172.21.15.160:47446) 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 -- 172.21.15.160:0/2678137265 learned_addr learned my addr 172.21.15.160:0/2678137265 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c552fc700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c501026a0 0x7f7c50112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 -- 172.21.15.160:0/2678137265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c501026a0 msgr2=0x7f7c50112e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c501026a0 0x7f7c50112e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 -- 172.21.15.160:0/2678137265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c005040 con 0x7f7c50101d00 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c55afd700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501128f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b3b0 tx=0x7f7c4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4c00e040 con 0x7f7c50101d00 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c4c00bbc0 con 0x7f7c50101d00 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4c01c3f0 con 0x7f7c50101d00 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5010cd00 con 0x7f7c50101d00 2024-05-25T06:34:26.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.275+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5010d2c0 con 0x7f7c50101d00 2024-05-25T06:34:26.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.276+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c500fd450 con 0x7f7c50101d00 2024-05-25T06:34:26.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.277+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7c4c00bd30 con 0x7f7c50101d00 2024-05-25T06:34:26.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.277+0000 7f7c467fc700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7c3c06e140 0x7f7c3c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:26.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.277+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7c4c08ed30 con 0x7f7c50101d00 2024-05-25T06:34:26.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.277+0000 7f7c552fc700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7c3c06e140 0x7f7c3c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:26.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.278+0000 7f7c552fc700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7c3c06e140 0x7f7c3c0705f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009fd0 tx=0x7f7c40009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:26.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.282+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7c4c05bc10 con 0x7f7c50101d00 2024-05-25T06:34:26.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.488+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c500fd450 con 0x7f7c3c06e140 2024-05-25T06:34:26.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.499+0000 7f7c467fc700 1 -- 172.21.15.160:0/2678137265 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7c500fd450 con 0x7f7c3c06e140 2024-05-25T06:34:26.499 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:34:26.499 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (20m) 13m ago 31m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:34:26.499 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (17m) 13m ago 31m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:34:26.500 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (17m) 13m ago 30m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:34:26.500 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (19m) 13m ago 31m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:34:26.500 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (14m) 13m ago 26m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:34:26.500 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (13m) 13m ago 25m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:34:26.500 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (13m) 13m ago 25m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:34:26.500 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (13m) 13m ago 25m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (21m) 13m ago 32m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (21m) 13m ago 30m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (18m) 13m ago 32m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (18m) 13m ago 30m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (20m) 13m ago 31m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (20m) 13m ago 30m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (17m) 13m ago 29m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:34:26.501 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (16m) 13m ago 29m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:34:26.502 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (16m) 13m ago 28m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:34:26.502 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (15m) 13m ago 28m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:34:26.502 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (14m) 13m ago 27m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:34:26.502 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (14m) 13m ago 27m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:34:26.502 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (20m) 13m ago 30m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:34:26.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7c3c06e140 msgr2=0x7f7c3c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7c3c06e140 0x7f7c3c0705f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009fd0 tx=0x7f7c40009f20 comp rx=0 tx=0).stop 2024-05-25T06:34:26.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 msgr2=0x7f7c501128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501128f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b3b0 tx=0x7f7c4c009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:26.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 shutdown_connections 2024-05-25T06:34:26.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c50101d00 0x7f7c501128f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7c3c06e140 0x7f7c3c0705f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 --2- 172.21.15.160:0/2678137265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c501026a0 0x7f7c50112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 >> 172.21.15.160:0/2678137265 conn(0x7f7c500f7850 msgr2=0x7f7c500f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:26.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 shutdown_connections 2024-05-25T06:34:26.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.501+0000 7f7c57d61700 1 -- 172.21.15.160:0/2678137265 wait complete. 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 -- 172.21.15.160:0/573166169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94103780 msgr2=0x7f5c940fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 --2- 172.21.15.160:0/573166169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94103780 0x7f5c940fcdf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c88004940 tx=0x7f5c88009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 -- 172.21.15.160:0/573166169 shutdown_connections 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 --2- 172.21.15.160:0/573166169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94103780 0x7f5c940fcdf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 --2- 172.21.15.160:0/573166169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c94102de0 0x7f5c941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 -- 172.21.15.160:0/573166169 >> 172.21.15.160:0/573166169 conn(0x7f5c940f8950 msgr2=0x7f5c940fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 -- 172.21.15.160:0/573166169 shutdown_connections 2024-05-25T06:34:26.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 -- 172.21.15.160:0/573166169 wait complete. 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.602+0000 7f5c9a577700 1 Processor -- start 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c9a577700 1 -- start start 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c9a577700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 0x7f5c94115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c9a577700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c94103780 0x7f5c94116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c9a577700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94116990 con 0x7f5c94102de0 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c9a577700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9410fd60 con 0x7f5c94103780 2024-05-25T06:34:26.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c93fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 0x7f5c94115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c93fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 0x7f5c94115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47466/0 (socket says 172.21.15.160:47466) 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c93fff700 1 -- 172.21.15.160:0/44047598 learned_addr learned my addr 172.21.15.160:0/44047598 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c937fe700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c94103780 0x7f5c94116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c93fff700 1 -- 172.21.15.160:0/44047598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c94103780 msgr2=0x7f5c94116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c93fff700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c94103780 0x7f5c94116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.603+0000 7f5c93fff700 1 -- 172.21.15.160:0/44047598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c88005040 con 0x7f5c94102de0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.604+0000 7f5c93fff700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 0x7f5c94115e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400c8c0 tx=0x7f5c840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.604+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8400a980 con 0x7f5c94102de0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.604+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c94110040 con 0x7f5c94102de0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.604+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c8400ce60 con 0x7f5c94102de0 2024-05-25T06:34:26.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.604+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c840115e0 con 0x7f5c94102de0 2024-05-25T06:34:26.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.604+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c94110580 con 0x7f5c94102de0 2024-05-25T06:34:26.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.605+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c940fe530 con 0x7f5c94102de0 2024-05-25T06:34:26.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.605+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5c84012400 con 0x7f5c94102de0 2024-05-25T06:34:26.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.606+0000 7f5c90ff9700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:26.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.606+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5c8408dc40 con 0x7f5c94102de0 2024-05-25T06:34:26.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.606+0000 7f5c937fe700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:26.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.606+0000 7f5c937fe700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800b3b0 tx=0x7f5c88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:26.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.613+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c84093020 con 0x7f5c94102de0 2024-05-25T06:34:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:26 smithi203 ceph-mon[83507]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:26 smithi203 ceph-mon[83507]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:26.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.900+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5c9404bed0 con 0x7f5c94102de0 2024-05-25T06:34:26.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.901+0000 7f5c90ff9700 1 -- 172.21.15.160:0/44047598 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f5c8405ab20 con 0x7f5c94102de0 2024-05-25T06:34:26.901 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:34:26.901 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:34:26.901 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:34:26.901 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:26.901 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:34:26.902 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:34:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 msgr2=0x7f5c7c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800b3b0 tx=0x7f5c88009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 msgr2=0x7f5c94115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 0x7f5c94115e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400c8c0 tx=0x7f5c840044c0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 shutdown_connections 2024-05-25T06:34:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c94102de0 0x7f5c94115e00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f5c7c06e140 0x7f5c7c0705f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 --2- 172.21.15.160:0/44047598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c94103780 0x7f5c94116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:26.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.903+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 >> 172.21.15.160:0/44047598 conn(0x7f5c940f8950 msgr2=0x7f5c940f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:26.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.904+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 shutdown_connections 2024-05-25T06:34:26.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:26.904+0000 7f5c9a577700 1 -- 172.21.15.160:0/44047598 wait complete. 2024-05-25T06:34:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 -- 172.21.15.160:0/2887027894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 msgr2=0x7f7dac1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 --2- 172.21.15.160:0/2887027894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac1020d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c004940 tx=0x7f7d9c009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 -- 172.21.15.160:0/2887027894 shutdown_connections 2024-05-25T06:34:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 --2- 172.21.15.160:0/2887027894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 0x7f7dac0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 --2- 172.21.15.160:0/2887027894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac1020d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 -- 172.21.15.160:0/2887027894 >> 172.21.15.160:0/2887027894 conn(0x7f7dac0f7870 msgr2=0x7f7dac0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 -- 172.21.15.160:0/2887027894 shutdown_connections 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 -- 172.21.15.160:0/2887027894 wait complete. 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 Processor -- start 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.006+0000 7f7db084f700 1 -- start start 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7db084f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7db084f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 0x7f7dac1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7db084f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac116710 con 0x7f7dac101d00 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7db084f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac10fd30 con 0x7f7dac1026a0 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47484/0 (socket says 172.21.15.160:47484) 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 -- 172.21.15.160:0/3646080211 learned_addr learned my addr 172.21.15.160:0/3646080211 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:27.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7da9d9b700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 0x7f7dac1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 -- 172.21.15.160:0/3646080211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 msgr2=0x7f7dac1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 0x7f7dac1160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 -- 172.21.15.160:0/3646080211 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c005040 con 0x7f7dac101d00 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7da9d9b700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 0x7f7dac1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7daa59c700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac115b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b3b0 tx=0x7f7d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c00e040 con 0x7f7dac101d00 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac10ffb0 con 0x7f7dac101d00 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d9c00bbc0 con 0x7f7dac101d00 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c01c3f0 con 0x7f7dac101d00 2024-05-25T06:34:27.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.007+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac110570 con 0x7f7dac101d00 2024-05-25T06:34:27.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.008+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac0fd450 con 0x7f7dac101d00 2024-05-25T06:34:27.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.009+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7d9c00bd30 con 0x7f7dac101d00 2024-05-25T06:34:27.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.009+0000 7f7d9affd700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d9406e140 0x7f7d940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.009+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7d9c08ee10 con 0x7f7dac101d00 2024-05-25T06:34:27.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.009+0000 7f7da9d9b700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d9406e140 0x7f7d940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.010+0000 7f7da9d9b700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d9406e140 0x7f7d940705f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009fd0 tx=0x7f7da0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:27.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.014+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7d9c058140 con 0x7f7dac101d00 2024-05-25T06:34:27.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.259+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7dac108e90 con 0x7f7dac101d00 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.260+0000 7f7d9affd700 1 -- 172.21.15.160:0/3646080211 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f7d9c05b800 con 0x7f7dac101d00 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:34:27.260 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:34:27.261 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:34:27.262 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:27.263 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d9406e140 msgr2=0x7f7d940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d9406e140 0x7f7d940705f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009fd0 tx=0x7f7da0009ee0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 msgr2=0x7f7dac115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac115b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b3b0 tx=0x7f7d9c009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 shutdown_connections 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dac101d00 0x7f7dac115b80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f7d9406e140 0x7f7d940705f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 --2- 172.21.15.160:0/3646080211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac1026a0 0x7f7dac1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 >> 172.21.15.160:0/3646080211 conn(0x7f7dac0f7870 msgr2=0x7f7dac066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:27.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 shutdown_connections 2024-05-25T06:34:27.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.262+0000 7f7db084f700 1 -- 172.21.15.160:0/3646080211 wait complete. 2024-05-25T06:34:27.265 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 -- 172.21.15.160:0/484374375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 msgr2=0x7fb7900fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 --2- 172.21.15.160:0/484374375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb7900fcdf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb780004940 tx=0x7fb780009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 -- 172.21.15.160:0/484374375 shutdown_connections 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 --2- 172.21.15.160:0/484374375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb7900fcdf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 --2- 172.21.15.160:0/484374375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb790102de0 0x7fb7901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 -- 172.21.15.160:0/484374375 >> 172.21.15.160:0/484374375 conn(0x7fb7900f8990 msgr2=0x7fb7900fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 -- 172.21.15.160:0/484374375 shutdown_connections 2024-05-25T06:34:27.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 -- 172.21.15.160:0/484374375 wait complete. 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 Processor -- start 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.366+0000 7fb797596700 1 -- start start 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb797596700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb790102de0 0x7fb790113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb797596700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb790113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb797596700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7901144e0 con 0x7fb790103780 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb797596700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79010dae0 con 0x7fb790102de0 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb790113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb790113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47514/0 (socket says 172.21.15.160:47514) 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 -- 172.21.15.160:0/2376015292 learned_addr learned my addr 172.21.15.160:0/2376015292 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb795332700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb790102de0 0x7fb790113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 -- 172.21.15.160:0/2376015292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb790102de0 msgr2=0x7fb790113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb790102de0 0x7fb790113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 -- 172.21.15.160:0/2376015292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb780005040 con 0x7fb790103780 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb794b31700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb790113e90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb780009100 tx=0x7fb780009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb78000e040 con 0x7fb790103780 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb780013420 con 0x7fb790103780 2024-05-25T06:34:27.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.367+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79010dd60 con 0x7fb790103780 2024-05-25T06:34:27.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.368+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79010e320 con 0x7fb790103780 2024-05-25T06:34:27.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.368+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb78001c610 con 0x7fb790103780 2024-05-25T06:34:27.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.369+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7900fe530 con 0x7fb790103780 2024-05-25T06:34:27.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.369+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb780013590 con 0x7fb790103780 2024-05-25T06:34:27.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.370+0000 7fb785ffb700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb77c06e140 0x7fb77c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.370+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb78008eca0 con 0x7fb790103780 2024-05-25T06:34:27.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.370+0000 7fb795332700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb77c06e140 0x7fb77c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.370+0000 7fb795332700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb77c06e140 0x7fb77c0705f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009fd0 tx=0x7fb78c009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:27.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.374+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb780093070 con 0x7fb790103780 2024-05-25T06:34:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:27 smithi160 ceph-mon[112094]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:27 smithi160 ceph-mon[112094]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:27 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/44047598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:34:27.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.591+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb790109ec0 con 0x7fb77c06e140 2024-05-25T06:34:27.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.592+0000 7fb785ffb700 1 -- 172.21.15.160:0/2376015292 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb790109ec0 con 0x7fb77c06e140 2024-05-25T06:34:27.592 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:34:27.592 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:34:27.593 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb77c06e140 msgr2=0x7fb77c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb77c06e140 0x7fb77c0705f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009fd0 tx=0x7fb78c009e50 comp rx=0 tx=0).stop 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 msgr2=0x7fb790113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb790113e90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb780009100 tx=0x7fb780009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 shutdown_connections 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb77c06e140 0x7fb77c0705f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb790102de0 0x7fb790113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 --2- 172.21.15.160:0/2376015292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb790103780 0x7fb790113e90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 >> 172.21.15.160:0/2376015292 conn(0x7fb7900f8990 msgr2=0x7fb7900f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 shutdown_connections 2024-05-25T06:34:27.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.595+0000 7fb797596700 1 -- 172.21.15.160:0/2376015292 wait complete. 2024-05-25T06:34:27.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:27 smithi203 ceph-mon[83507]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:27.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:27 smithi203 ceph-mon[83507]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:27.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:27 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/44047598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:34:27.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.696+0000 7fc34caae700 1 -- 172.21.15.160:0/2283575999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 msgr2=0x7fc3480fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.696+0000 7fc34caae700 1 --2- 172.21.15.160:0/2283575999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fb990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc33c004940 tx=0x7fc33c009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:27.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 -- 172.21.15.160:0/2283575999 shutdown_connections 2024-05-25T06:34:27.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 --2- 172.21.15.160:0/2283575999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fb990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 --2- 172.21.15.160:0/2283575999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 0x7fc348103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 -- 172.21.15.160:0/2283575999 >> 172.21.15.160:0/2283575999 conn(0x7fc3480714d0 msgr2=0x7fc3480718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 -- 172.21.15.160:0/2283575999 shutdown_connections 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 -- 172.21.15.160:0/2283575999 wait complete. 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 Processor -- start 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 -- start start 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 0x7fc3480fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.697+0000 7fc34caae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480fe650 con 0x7fc3480fb520 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34caae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480fd180 con 0x7fc348102f60 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:47532/0 (socket says 172.21.15.160:47532) 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 -- 172.21.15.160:0/4134306869 learned_addr learned my addr 172.21.15.160:0/4134306869 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc345d9b700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 0x7fc3480fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 -- 172.21.15.160:0/4134306869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 msgr2=0x7fc3480fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 0x7fc3480fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 -- 172.21.15.160:0/4134306869 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c005040 con 0x7fc3480fb520 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc345d9b700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 0x7fc3480fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34659c700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fc700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c8c0 tx=0x7fc3380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33800a980 con 0x7fc3480fb520 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc33800ce60 con 0x7fc3480fb520 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3480fd460 con 0x7fc3480fb520 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3380115e0 con 0x7fc3480fb520 2024-05-25T06:34:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.698+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34811a0b0 con 0x7fc3480fb520 2024-05-25T06:34:27.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.699+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc348100250 con 0x7fc3480fb520 2024-05-25T06:34:27.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.700+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc338012400 con 0x7fc3480fb520 2024-05-25T06:34:27.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.700+0000 7fc336ffd700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc33006e140 0x7fc3300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:27.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.700+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc33808db60 con 0x7fc3480fb520 2024-05-25T06:34:27.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.700+0000 7fc345d9b700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc33006e140 0x7fc3300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:27.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.701+0000 7fc345d9b700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc33006e140 0x7fc3300705f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc33c004f90 tx=0x7fc33c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:27.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.705+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc338056fd0 con 0x7fc3480fb520 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.985+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc3480fdbe0 con 0x7fc3480fb520 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.985+0000 7fc336ffd700 1 -- 172.21.15.160:0/4134306869 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc3380126b0 con 0x7fc3480fb520 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:34:27.985 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc33006e140 msgr2=0x7fc3300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc33006e140 0x7fc3300705f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc33c004f90 tx=0x7fc33c009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 msgr2=0x7fc3480fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fc700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c8c0 tx=0x7fc3380044c0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 shutdown_connections 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3480fb520 0x7fc3480fc700 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc33006e140 0x7fc3300705f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 --2- 172.21.15.160:0/4134306869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc348102f60 0x7fc3480fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:27.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 >> 172.21.15.160:0/4134306869 conn(0x7fc3480714d0 msgr2=0x7fc3480fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:27.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 shutdown_connections 2024-05-25T06:34:27.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:27.987+0000 7fc34caae700 1 -- 172.21.15.160:0/4134306869 wait complete. 2024-05-25T06:34:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:28 smithi160 ceph-mon[112094]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:28 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3646080211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:34:28.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:28 smithi160 ceph-mon[112094]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:28.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:28.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:28 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4134306869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:34:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:28 smithi203 ceph-mon[83507]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:28 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3646080211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:34:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:28 smithi203 ceph-mon[83507]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:34:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:28 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4134306869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:34:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:30 smithi160 ceph-mon[112094]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:30 smithi203 ceph-mon[83507]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:32 smithi160 ceph-mon[112094]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:32 smithi203 ceph-mon[83507]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:35 smithi203 ceph-mon[83507]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:35 smithi160 ceph-mon[112094]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:37 smithi203 ceph-mon[83507]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:37 smithi160 ceph-mon[112094]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:39 smithi203 ceph-mon[83507]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:39 smithi160 ceph-mon[112094]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:41 smithi203 ceph-mon[83507]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:41 smithi160 ceph-mon[112094]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:43 smithi203 ceph-mon[83507]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:43 smithi160 ceph-mon[112094]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:45.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:45 smithi203 ceph-mon[83507]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:45 smithi160 ceph-mon[112094]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:47 smithi203 ceph-mon[83507]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:47 smithi160 ceph-mon[112094]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:49.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:49 smithi203 ceph-mon[83507]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:49 smithi160 ceph-mon[112094]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:51.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:51 smithi203 ceph-mon[83507]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:51 smithi160 ceph-mon[112094]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:53.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:53 smithi203 ceph-mon[83507]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:53 smithi160 ceph-mon[112094]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:55.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:55 smithi203 ceph-mon[83507]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:55 smithi160 ceph-mon[112094]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:57.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:57 smithi203 ceph-mon[83507]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:57 smithi160 ceph-mon[112094]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 -- 172.21.15.160:0/881918118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500102df0 msgr2=0x7f35001031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 --2- 172.21.15.160:0/881918118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500102df0 0x7f35001031c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f34f4004940 tx=0x7f34f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 -- 172.21.15.160:0/881918118 shutdown_connections 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 --2- 172.21.15.160:0/881918118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3500103790 0x7f35000fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 --2- 172.21.15.160:0/881918118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500102df0 0x7f35001031c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 -- 172.21.15.160:0/881918118 >> 172.21.15.160:0/881918118 conn(0x7f35000f8980 msgr2=0x7f35000fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 -- 172.21.15.160:0/881918118 shutdown_connections 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.105+0000 7f35085a6700 1 -- 172.21.15.160:0/881918118 wait complete. 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.106+0000 7f35085a6700 1 Processor -- start 2024-05-25T06:34:58.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.106+0000 7f35085a6700 1 -- start start 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.106+0000 7f35085a6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3500102df0 0x7f3500113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.106+0000 7f35085a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 0x7f3500114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.106+0000 7f35085a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3500114750 con 0x7f3500103790 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.106+0000 7f35085a6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350010daf0 con 0x7f3500102df0 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 0x7f3500114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 0x7f3500114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43700/0 (socket says 172.21.15.160:43700) 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3506342700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3500102df0 0x7f3500113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 -- 172.21.15.160:0/4129612610 learned_addr learned my addr 172.21.15.160:0/4129612610 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 -- 172.21.15.160:0/4129612610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3500102df0 msgr2=0x7f3500113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3500102df0 0x7f3500113b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 -- 172.21.15.160:0/4129612610 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4005040 con 0x7f3500103790 2024-05-25T06:34:58.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.107+0000 7f3505b41700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 0x7f3500114070 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00b790 tx=0x7f34fc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:58.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.108+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34fc00f6f0 con 0x7f3500103790 2024-05-25T06:34:58.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.108+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34fc00fd20 con 0x7f3500103790 2024-05-25T06:34:58.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.108+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350010ddd0 con 0x7f3500103790 2024-05-25T06:34:58.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.109+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34fc0103a0 con 0x7f3500103790 2024-05-25T06:34:58.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.109+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350010e390 con 0x7f3500103790 2024-05-25T06:34:58.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.110+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35000fe540 con 0x7f3500103790 2024-05-25T06:34:58.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.111+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f34fc011400 con 0x7f3500103790 2024-05-25T06:34:58.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.111+0000 7f34f2ffd700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f34ec06e140 0x7f34ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.111+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f34fc08d810 con 0x7f3500103790 2024-05-25T06:34:58.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.111+0000 7f3506342700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f34ec06e140 0x7f34ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.112+0000 7f3506342700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f34ec06e140 0x7f34ec0705f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34f4004f90 tx=0x7f34f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:58.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.117+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f34fc05a7b0 con 0x7f3500103790 2024-05-25T06:34:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:58.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.334+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3500109f60 con 0x7f34ec06e140 2024-05-25T06:34:58.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.335+0000 7f34f2ffd700 1 -- 172.21.15.160:0/4129612610 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3500109f60 con 0x7f34ec06e140 2024-05-25T06:34:58.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f34ec06e140 msgr2=0x7f34ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f34ec06e140 0x7f34ec0705f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34f4004f90 tx=0x7f34f4009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:58.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 msgr2=0x7f3500114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 0x7f3500114070 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00b790 tx=0x7f34fc009220 comp rx=0 tx=0).stop 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 shutdown_connections 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f34ec06e140 0x7f34ec0705f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3500102df0 0x7f3500113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 --2- 172.21.15.160:0/4129612610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3500103790 0x7f3500114070 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 >> 172.21.15.160:0/4129612610 conn(0x7f35000f8980 msgr2=0x7f35000f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 shutdown_connections 2024-05-25T06:34:58.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.337+0000 7f35085a6700 1 -- 172.21.15.160:0/4129612610 wait complete. 2024-05-25T06:34:58.347 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:34:58.402 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:34:58.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.440+0000 7f84ace47700 1 -- 172.21.15.160:0/3494174802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8103780 msgr2=0x7f84a80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.440+0000 7f84ace47700 1 --2- 172.21.15.160:0/3494174802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8103780 0x7f84a80fcdf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8490009ce0 tx=0x7f8490004d10 comp rx=0 tx=0).stop 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.440+0000 7f84ace47700 1 -- 172.21.15.160:0/3494174802 shutdown_connections 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.440+0000 7f84ace47700 1 --2- 172.21.15.160:0/3494174802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8103780 0x7f84a80fcdf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.440+0000 7f84ace47700 1 --2- 172.21.15.160:0/3494174802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a8102de0 0x7f84a81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.440+0000 7f84ace47700 1 -- 172.21.15.160:0/3494174802 >> 172.21.15.160:0/3494174802 conn(0x7f84a80f8990 msgr2=0x7f84a80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 -- 172.21.15.160:0/3494174802 shutdown_connections 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 -- 172.21.15.160:0/3494174802 wait complete. 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 Processor -- start 2024-05-25T06:34:58.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 -- start start 2024-05-25T06:34:58.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 0x7f84a8113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a8103780 0x7f84a8114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a8114670 con 0x7f84a8102de0 2024-05-25T06:34:58.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84ace47700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a810dc50 con 0x7f84a8103780 2024-05-25T06:34:58.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84a659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 0x7f84a8113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84a659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 0x7f84a8113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43710/0 (socket says 172.21.15.160:43710) 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84a659c700 1 -- 172.21.15.160:0/4087835048 learned_addr learned my addr 172.21.15.160:0/4087835048 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.441+0000 7f84a5d9b700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a8103780 0x7f84a8114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f84a659c700 1 -- 172.21.15.160:0/4087835048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a8103780 msgr2=0x7f84a8114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f84a659c700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a8103780 0x7f84a8114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f84a659c700 1 -- 172.21.15.160:0/4087835048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490009940 con 0x7f84a8102de0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f84a659c700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 0x7f84a8113ae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84a0002a50 tx=0x7f84a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a0007cc0 con 0x7f84a8102de0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a810df30 con 0x7f84a8102de0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a000f040 con 0x7f84a8102de0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a0009640 con 0x7f84a8102de0 2024-05-25T06:34:58.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.442+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a810e4f0 con 0x7f84a8102de0 2024-05-25T06:34:58.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.443+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a80fe530 con 0x7f84a8102de0 2024-05-25T06:34:58.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.444+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f84a00097a0 con 0x7f84a8102de0 2024-05-25T06:34:58.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.444+0000 7f849effd700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f849406e140 0x7f84940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.444+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f84a008d990 con 0x7f84a8102de0 2024-05-25T06:34:58.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.444+0000 7f84a5d9b700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f849406e140 0x7f84940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.445+0000 7f84a5d9b700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f849406e140 0x7f84940705f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84a810f380 tx=0x7f8490004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:58.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.449+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f84a005a7f0 con 0x7f84a8102de0 2024-05-25T06:34:58.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.664+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84a810f1a0 con 0x7f849406e140 2024-05-25T06:34:58.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.665+0000 7f849effd700 1 -- 172.21.15.160:0/4087835048 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f84a810f1a0 con 0x7f849406e140 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.667+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f849406e140 msgr2=0x7f84940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.667+0000 7f84ace47700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f849406e140 0x7f84940705f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84a810f380 tx=0x7f8490004d10 comp rx=0 tx=0).stop 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.667+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 msgr2=0x7f84a8113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.667+0000 7f84ace47700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 0x7f84a8113ae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84a0002a50 tx=0x7f84a00046b0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 shutdown_connections 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84a8102de0 0x7f84a8113ae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f849406e140 0x7f84940705f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 --2- 172.21.15.160:0/4087835048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a8103780 0x7f84a8114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 >> 172.21.15.160:0/4087835048 conn(0x7f84a80f8990 msgr2=0x7f84a80f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:58.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 shutdown_connections 2024-05-25T06:34:58.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.668+0000 7f84ace47700 1 -- 172.21.15.160:0/4087835048 wait complete. 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 -- 172.21.15.160:0/3509401886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d01026b0 msgr2=0x7f09d00fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/3509401886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d01026b0 0x7f09d00fbd20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004940 tx=0x7f09cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 -- 172.21.15.160:0/3509401886 shutdown_connections 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/3509401886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d01026b0 0x7f09d00fbd20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/3509401886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d0101d10 0x7f09d01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 -- 172.21.15.160:0/3509401886 >> 172.21.15.160:0/3509401886 conn(0x7f09d00f78e0 msgr2=0x7f09d00f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 -- 172.21.15.160:0/3509401886 shutdown_connections 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.771+0000 7f09d7b6f700 1 -- 172.21.15.160:0/3509401886 wait complete. 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d7b6f700 1 Processor -- start 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d7b6f700 1 -- start start 2024-05-25T06:34:58.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d7b6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 0x7f09d0113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d7b6f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d01026b0 0x7f09d0113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d7b6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0114580 con 0x7f09d0101d10 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d7b6f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d010daf0 con 0x7f09d01026b0 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d590b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 0x7f09d0113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d590b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 0x7f09d0113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43728/0 (socket says 172.21.15.160:43728) 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.772+0000 7f09d590b700 1 -- 172.21.15.160:0/1060105331 learned_addr learned my addr 172.21.15.160:0/1060105331 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d510a700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d01026b0 0x7f09d0113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d590b700 1 -- 172.21.15.160:0/1060105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d01026b0 msgr2=0x7f09d0113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d590b700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d01026b0 0x7f09d0113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d590b700 1 -- 172.21.15.160:0/1060105331 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc005040 con 0x7f09d0101d10 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d590b700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 0x7f09d0113960 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b790 tx=0x7f09c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c000f770 con 0x7f09d0101d10 2024-05-25T06:34:58.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09c000fda0 con 0x7f09d0101d10 2024-05-25T06:34:58.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d010ddd0 con 0x7f09d0101d10 2024-05-25T06:34:58.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c000d310 con 0x7f09d0101d10 2024-05-25T06:34:58.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.773+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d010e390 con 0x7f09d0101d10 2024-05-25T06:34:58.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.774+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d00fd460 con 0x7f09d0101d10 2024-05-25T06:34:58.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.774+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f09c0010420 con 0x7f09d0101d10 2024-05-25T06:34:58.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.775+0000 7f09c67fc700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09bc06e140 0x7f09bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:58.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.775+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f09c008da80 con 0x7f09d0101d10 2024-05-25T06:34:58.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.775+0000 7f09d510a700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09bc06e140 0x7f09bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:58.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.775+0000 7f09d510a700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09bc06e140 0x7f09bc0705f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b3b0 tx=0x7f09cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:58.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.780+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f09c005aa20 con 0x7f09d0101d10 2024-05-25T06:34:58.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.986+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f09d0108ed0 con 0x7f09bc06e140 2024-05-25T06:34:58.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.996+0000 7f09c67fc700 1 -- 172.21.15.160:0/1060105331 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f09d0108ed0 con 0x7f09bc06e140 2024-05-25T06:34:58.996 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:34:58.996 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (20m) 14m ago 31m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:34:58.996 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (18m) 14m ago 32m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (18m) 13m ago 31m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (20m) 14m ago 31m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (14m) 14m ago 26m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (14m) 14m ago 26m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (13m) 13m ago 26m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (14m) 13m ago 26m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (22m) 14m ago 33m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (21m) 13m ago 31m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (19m) 14m ago 33m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (18m) 13m ago 30m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (21m) 14m ago 32m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:34:58.997 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (21m) 13m ago 31m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (17m) 14m ago 30m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (17m) 14m ago 29m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (16m) 14m ago 29m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (16m) 13m ago 28m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (15m) 13m ago 28m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (14m) 13m ago 28m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:34:58.998 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (21m) 14m ago 31m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09bc06e140 msgr2=0x7f09bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09bc06e140 0x7f09bc0705f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b3b0 tx=0x7f09cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 msgr2=0x7f09d0113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 0x7f09d0113960 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b790 tx=0x7f09c0009250 comp rx=0 tx=0).stop 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 shutdown_connections 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d0101d10 0x7f09d0113960 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f09bc06e140 0x7f09bc0705f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 --2- 172.21.15.160:0/1060105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d01026b0 0x7f09d0113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 >> 172.21.15.160:0/1060105331 conn(0x7f09d00f78e0 msgr2=0x7f09d00f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 shutdown_connections 2024-05-25T06:34:58.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:58.998+0000 7f09d7b6f700 1 -- 172.21.15.160:0/1060105331 wait complete. 2024-05-25T06:34:59.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 -- 172.21.15.160:0/1864096071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664101d90 msgr2=0x7f2664102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 --2- 172.21.15.160:0/1864096071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664101d90 0x7f2664102160 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2654004940 tx=0x7f2654009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:59.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 -- 172.21.15.160:0/1864096071 shutdown_connections 2024-05-25T06:34:59.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 --2- 172.21.15.160:0/1864096071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2664102730 0x7f26640fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 --2- 172.21.15.160:0/1864096071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664101d90 0x7f2664102160 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 -- 172.21.15.160:0/1864096071 >> 172.21.15.160:0/1864096071 conn(0x7f26640f7940 msgr2=0x7f26640f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 -- 172.21.15.160:0/1864096071 shutdown_connections 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 -- 172.21.15.160:0/1864096071 wait complete. 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 Processor -- start 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.100+0000 7f266a9b8700 1 -- start start 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f266a9b8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2664101d90 0x7f2664113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f266a9b8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 0x7f2664113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f266a9b8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2664114520 con 0x7f2664102730 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f266a9b8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266410db20 con 0x7f2664101d90 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 0x7f2664113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 0x7f2664113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43752/0 (socket says 172.21.15.160:43752) 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 -- 172.21.15.160:0/3762339864 learned_addr learned my addr 172.21.15.160:0/3762339864 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f2663fff700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2664101d90 0x7f2664113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 -- 172.21.15.160:0/3762339864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2664101d90 msgr2=0x7f2664113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2664101d90 0x7f2664113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 -- 172.21.15.160:0/3762339864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2654005040 con 0x7f2664102730 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f26637fe700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 0x7f2664113ed0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f264c00c890 tx=0x7f264c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264c00a700 con 0x7f2664102730 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f264c00ad30 con 0x7f2664102730 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266410de00 con 0x7f2664102730 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264c0113a0 con 0x7f2664102730 2024-05-25T06:34:59.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.101+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266410e3c0 con 0x7f2664102730 2024-05-25T06:34:59.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.102+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26640fd4e0 con 0x7f2664102730 2024-05-25T06:34:59.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.103+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f264c012400 con 0x7f2664102730 2024-05-25T06:34:59.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.103+0000 7f2660ff9700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f265006e140 0x7f26500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.104+0000 7f2663fff700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f265006e140 0x7f26500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.104+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f264c08dbf0 con 0x7f2664102730 2024-05-25T06:34:59.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.104+0000 7f2663fff700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f265006e140 0x7f26500705f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2654004f90 tx=0x7f2654009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:59.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.108+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f264c094080 con 0x7f2664102730 2024-05-25T06:34:59.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.392+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2664108e90 con 0x7f2664102730 2024-05-25T06:34:59.392 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:34:59 smithi160 ceph-mon[112094]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.393+0000 7f2660ff9700 1 -- 172.21.15.160:0/3762339864 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f264c05aad0 con 0x7f2664102730 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:34:59.393 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:34:59.394 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.395+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f265006e140 msgr2=0x7f26500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.395+0000 7f266a9b8700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f265006e140 0x7f26500705f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2654004f90 tx=0x7f2654009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.395+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 msgr2=0x7f2664113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.395+0000 7f266a9b8700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 0x7f2664113ed0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f264c00c890 tx=0x7f264c0044c0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.395+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 shutdown_connections 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.396+0000 7f266a9b8700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f265006e140 0x7f26500705f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.396+0000 7f266a9b8700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2664101d90 0x7f2664113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.396+0000 7f266a9b8700 1 --2- 172.21.15.160:0/3762339864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664102730 0x7f2664113ed0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.396+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 >> 172.21.15.160:0/3762339864 conn(0x7f26640f7940 msgr2=0x7f2664066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.396+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 shutdown_connections 2024-05-25T06:34:59.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.396+0000 7f266a9b8700 1 -- 172.21.15.160:0/3762339864 wait complete. 2024-05-25T06:34:59.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:34:59 smithi203 ceph-mon[83507]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 -- 172.21.15.160:0/3471137184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4101d10 msgr2=0x7fd0c41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 --2- 172.21.15.160:0/3471137184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4101d10 0x7fd0c41020e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4004940 tx=0x7fd0b4009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 -- 172.21.15.160:0/3471137184 shutdown_connections 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 --2- 172.21.15.160:0/3471137184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0c41026b0 0x7fd0c40fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 --2- 172.21.15.160:0/3471137184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4101d10 0x7fd0c41020e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 -- 172.21.15.160:0/3471137184 >> 172.21.15.160:0/3471137184 conn(0x7fd0c40f7860 msgr2=0x7fd0c40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 -- 172.21.15.160:0/3471137184 shutdown_connections 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.497+0000 7fd0cc111700 1 -- 172.21.15.160:0/3471137184 wait complete. 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0cc111700 1 Processor -- start 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0cc111700 1 -- start start 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0cc111700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0c4101d10 0x7fd0c410eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0cc111700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 0x7fd0c410f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0cc111700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c410f610 con 0x7fd0c41026b0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0cc111700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c410f750 con 0x7fd0c4101d10 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c96ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 0x7fd0c410f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c96ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 0x7fd0c410f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43774/0 (socket says 172.21.15.160:43774) 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c96ac700 1 -- 172.21.15.160:0/21996856 learned_addr learned my addr 172.21.15.160:0/21996856 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c9ead700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0c4101d10 0x7fd0c410eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c96ac700 1 -- 172.21.15.160:0/21996856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0c4101d10 msgr2=0x7fd0c410eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c96ac700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0c4101d10 0x7fd0c410eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.498+0000 7fd0c96ac700 1 -- 172.21.15.160:0/21996856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4005040 con 0x7fd0c41026b0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.499+0000 7fd0c96ac700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 0x7fd0c410f0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0002a80 tx=0x7fd0c0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.499+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c0007cc0 con 0x7fd0c41026b0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.499+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0c000f040 con 0x7fd0c41026b0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.499+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c4117c00 con 0x7fd0c41026b0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.499+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c000daf0 con 0x7fd0c41026b0 2024-05-25T06:34:59.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.499+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c4117fd0 con 0x7fd0c41026b0 2024-05-25T06:34:59.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.500+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c40fd460 con 0x7fd0c41026b0 2024-05-25T06:34:59.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.500+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd0c000dc90 con 0x7fd0c41026b0 2024-05-25T06:34:59.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.501+0000 7fd0ba7fc700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd0b006e140 0x7fd0b00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.501 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.501+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd0c008d220 con 0x7fd0c41026b0 2024-05-25T06:34:59.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.501+0000 7fd0c9ead700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd0b006e140 0x7fd0b00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.501+0000 7fd0c9ead700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd0b006e140 0x7fd0b00705f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4004f90 tx=0x7fd0b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:59.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.505+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd0c0056550 con 0x7fd0c41026b0 2024-05-25T06:34:59.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.744+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd0c410f910 con 0x7fd0c41026b0 2024-05-25T06:34:59.744 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.745+0000 7fd0ba7fc700 1 -- 172.21.15.160:0/21996856 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fd0c410f910 con 0x7fd0c41026b0 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:34:59.745 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:34:59.746 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:59.747 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:59.748 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:34:59.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd0b006e140 msgr2=0x7fd0b00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd0b006e140 0x7fd0b00705f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4004f90 tx=0x7fd0b4009e30 comp rx=0 tx=0).stop 2024-05-25T06:34:59.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 msgr2=0x7fd0c410f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 0x7fd0c410f0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0002a80 tx=0x7fd0c0004630 comp rx=0 tx=0).stop 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 shutdown_connections 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd0b006e140 0x7fd0b00705f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0c4101d10 0x7fd0c410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 --2- 172.21.15.160:0/21996856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41026b0 0x7fd0c410f0d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 >> 172.21.15.160:0/21996856 conn(0x7fd0c40f7860 msgr2=0x7fd0c40f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 shutdown_connections 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.747+0000 7fd0cc111700 1 -- 172.21.15.160:0/21996856 wait complete. 2024-05-25T06:34:59.749 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:34:59.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2764731902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4102df0 msgr2=0x7f09a41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2764731902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4102df0 0x7f09a41031c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ce0 tx=0x7f09a0004d10 comp rx=0 tx=0).stop 2024-05-25T06:34:59.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2764731902 shutdown_connections 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2764731902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4103790 0x7f09a40fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2764731902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4102df0 0x7f09a41031c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2764731902 >> 172.21.15.160:0/2764731902 conn(0x7f09a40f8980 msgr2=0x7f09a40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2764731902 shutdown_connections 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.851+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2764731902 wait complete. 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09ab6f1700 1 Processor -- start 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09ab6f1700 1 -- start start 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09ab6f1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4102df0 0x7f09a4113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09ab6f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 0x7f09a4114260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09ab6f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41148b0 con 0x7f09a4103790 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09ab6f1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a410dc80 con 0x7f09a4102df0 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a8c8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 0x7f09a4114260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a8c8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 0x7f09a4114260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43796/0 (socket says 172.21.15.160:43796) 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a8c8c700 1 -- 172.21.15.160:0/2801091291 learned_addr learned my addr 172.21.15.160:0/2801091291 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a948d700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4102df0 0x7f09a4113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a8c8c700 1 -- 172.21.15.160:0/2801091291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4102df0 msgr2=0x7f09a4113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a8c8c700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4102df0 0x7f09a4113d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.852+0000 7f09a8c8c700 1 -- 172.21.15.160:0/2801091291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0009940 con 0x7f09a4103790 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.853+0000 7f09a8c8c700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 0x7f09a4114260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0994002a80 tx=0x7f0994004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.853+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0994007cc0 con 0x7f09a4103790 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.853+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a410df60 con 0x7f09a4103790 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.853+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f099400f040 con 0x7f09a4103790 2024-05-25T06:34:59.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.853+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099400daf0 con 0x7f09a4103790 2024-05-25T06:34:59.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.853+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a410e520 con 0x7f09a4103790 2024-05-25T06:34:59.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.854+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a4100100 con 0x7f09a4103790 2024-05-25T06:34:59.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.854+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f099400dc90 con 0x7f09a4103790 2024-05-25T06:34:59.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.855+0000 7f0999ffb700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f099006e140 0x7f09900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:34:59.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.855+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f099408d270 con 0x7f09a4103790 2024-05-25T06:34:59.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.855+0000 7f09a948d700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f099006e140 0x7f09900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:34:59.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.855+0000 7f09a948d700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f099006e140 0x7f09900705f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f09a0015040 tx=0x7f09a0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:34:59.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:34:59.860+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f099405a0d0 con 0x7f09a4103790 2024-05-25T06:35:00.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.078+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09a410f370 con 0x7f099006e140 2024-05-25T06:35:00.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.079+0000 7f0999ffb700 1 -- 172.21.15.160:0/2801091291 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f09a410f370 con 0x7f099006e140 2024-05-25T06:35:00.079 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:35:00.079 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:35:00.080 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f099006e140 msgr2=0x7f09900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f099006e140 0x7f09900705f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f09a0015040 tx=0x7f09a0004d10 comp rx=0 tx=0).stop 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 msgr2=0x7f09a4114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 0x7f09a4114260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0994002a80 tx=0x7f0994004630 comp rx=0 tx=0).stop 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 shutdown_connections 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f099006e140 0x7f09900705f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4102df0 0x7f09a4113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 --2- 172.21.15.160:0/2801091291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a4103790 0x7f09a4114260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 >> 172.21.15.160:0/2801091291 conn(0x7f09a40f8980 msgr2=0x7f09a4107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:00.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 shutdown_connections 2024-05-25T06:35:00.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.082+0000 7f09ab6f1700 1 -- 172.21.15.160:0/2801091291 wait complete. 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 -- 172.21.15.160:0/85253211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 msgr2=0x7f58dc103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 --2- 172.21.15.160:0/85253211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc103220 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004940 tx=0x7f58cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 -- 172.21.15.160:0/85253211 shutdown_connections 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 --2- 172.21.15.160:0/85253211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 0x7f58dc065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 --2- 172.21.15.160:0/85253211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc103220 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 -- 172.21.15.160:0/85253211 >> 172.21.15.160:0/85253211 conn(0x7f58dc071d00 msgr2=0x7f58dc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 -- 172.21.15.160:0/85253211 shutdown_connections 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.186+0000 7f58e2540700 1 -- 172.21.15.160:0/85253211 wait complete. 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58e2540700 1 Processor -- start 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58e2540700 1 -- start start 2024-05-25T06:35:00.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58e2540700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 0x7f58dc113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58e2540700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58e2540700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc114710 con 0x7f58dc102e50 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58e2540700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10db00 con 0x7f58dc065640 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc1140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43814/0 (socket says 172.21.15.160:43814) 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58db7fe700 1 -- 172.21.15.160:0/1404792770 learned_addr learned my addr 172.21.15.160:0/1404792770 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.187+0000 7f58dbfff700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 0x7f58dc113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58db7fe700 1 -- 172.21.15.160:0/1404792770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 msgr2=0x7f58dc113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58db7fe700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 0x7f58dc113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58db7fe700 1 -- 172.21.15.160:0/1404792770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc005040 con 0x7f58dc102e50 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58dbfff700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 0x7f58dc113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58db7fe700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc1140c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f58d0002a50 tx=0x7f58d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d0007cc0 con 0x7f58dc102e50 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc10dde0 con 0x7f58dc102e50 2024-05-25T06:35:00.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58d000f040 con 0x7f58dc102e50 2024-05-25T06:35:00.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d0009640 con 0x7f58dc102e50 2024-05-25T06:35:00.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.188+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc10e3a0 con 0x7f58dc102e50 2024-05-25T06:35:00.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.189+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc100100 con 0x7f58dc102e50 2024-05-25T06:35:00.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.189+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f58d00097a0 con 0x7f58dc102e50 2024-05-25T06:35:00.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.190+0000 7f58d8ff9700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58c406e140 0x7f58c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:00.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.190+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f58d008d990 con 0x7f58dc102e50 2024-05-25T06:35:00.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.190+0000 7f58dbfff700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58c406e140 0x7f58c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:00.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.190+0000 7f58dbfff700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58c406e140 0x7f58c40705f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004560 tx=0x7f58cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:00.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.195+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f58d005a7f0 con 0x7f58dc102e50 2024-05-25T06:35:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:00 smithi203 ceph-mon[83507]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:00 smithi203 ceph-mon[83507]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:00 smithi203 ceph-mon[83507]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:00 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3762339864' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:35:00.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:00 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/21996856' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:35:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:00 smithi160 ceph-mon[112094]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:00 smithi160 ceph-mon[112094]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:00 smithi160 ceph-mon[112094]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:00 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3762339864' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:35:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:00 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/21996856' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.474+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f58dc0fe540 con 0x7f58dc102e50 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.474+0000 7f58d8ff9700 1 -- 172.21.15.160:0/1404792770 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f58d005a380 con 0x7f58dc102e50 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:35:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:35:00.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58c406e140 msgr2=0x7f58c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:00.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58c406e140 0x7f58c40705f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004560 tx=0x7f58cc009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 msgr2=0x7f58dc1140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc1140c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f58d0002a50 tx=0x7f58d00046b0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 shutdown_connections 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f58c406e140 0x7f58c40705f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58dc065640 0x7f58dc113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 --2- 172.21.15.160:0/1404792770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102e50 0x7f58dc1140c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.476+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 >> 172.21.15.160:0/1404792770 conn(0x7f58dc071d00 msgr2=0x7f58dc0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.477+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 shutdown_connections 2024-05-25T06:35:00.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:00.477+0000 7f58e2540700 1 -- 172.21.15.160:0/1404792770 wait complete. 2024-05-25T06:35:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:01 smithi203 ceph-mon[83507]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:01 smithi203 ceph-mon[83507]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:01.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:01 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1404792770' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:35:01.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:01 smithi160 ceph-mon[112094]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:01.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:01 smithi160 ceph-mon[112094]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:01.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:01 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1404792770' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:35:02.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:02 smithi203 ceph-mon[83507]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:02.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:02 smithi160 ceph-mon[112094]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:04 smithi160 ceph-mon[112094]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:04 smithi203 ceph-mon[83507]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:06.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:06 smithi160 ceph-mon[112094]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:06 smithi203 ceph-mon[83507]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:08.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:08 smithi160 ceph-mon[112094]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:08 smithi203 ceph-mon[83507]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:10 smithi160 ceph-mon[112094]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:10 smithi203 ceph-mon[83507]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:12.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:12 smithi160 ceph-mon[112094]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:12 smithi203 ceph-mon[83507]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:14 smithi160 ceph-mon[112094]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:14 smithi203 ceph-mon[83507]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:16 smithi160 ceph-mon[112094]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:16 smithi203 ceph-mon[83507]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:18 smithi160 ceph-mon[112094]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:18 smithi203 ceph-mon[83507]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:20 smithi203 ceph-mon[83507]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:20.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:20 smithi160 ceph-mon[112094]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:22 smithi203 ceph-mon[83507]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:22.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:22 smithi160 ceph-mon[112094]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:24 smithi203 ceph-mon[83507]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:24 smithi160 ceph-mon[112094]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:26 smithi203 ceph-mon[83507]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:26 smithi160 ceph-mon[112094]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:28 smithi203 ceph-mon[83507]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:28 smithi160 ceph-mon[112094]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:30.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 -- 172.21.15.160:0/1290861655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc103780 msgr2=0x7f71bc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:30.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 --2- 172.21.15.160:0/1290861655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc103780 0x7f71bc0fcdf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f71b0004940 tx=0x7f71b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:30.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 -- 172.21.15.160:0/1290861655 shutdown_connections 2024-05-25T06:35:30.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 --2- 172.21.15.160:0/1290861655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc103780 0x7f71bc0fcdf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 --2- 172.21.15.160:0/1290861655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc102de0 0x7f71bc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 -- 172.21.15.160:0/1290861655 >> 172.21.15.160:0/1290861655 conn(0x7f71bc0f89b0 msgr2=0x7f71bc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:30.594 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:30 smithi160 ceph-mon[112094]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:30.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 -- 172.21.15.160:0/1290861655 shutdown_connections 2024-05-25T06:35:30.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.593+0000 7f71c2060700 1 -- 172.21.15.160:0/1290861655 wait complete. 2024-05-25T06:35:30.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71c2060700 1 Processor -- start 2024-05-25T06:35:30.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71c2060700 1 -- start start 2024-05-25T06:35:30.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71c2060700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 0x7f71bc113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:30.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71c2060700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc103780 0x7f71bc1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:30.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71c2060700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc114710 con 0x7f71bc102de0 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71c2060700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc10db00 con 0x7f71bc103780 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 0x7f71bc113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 0x7f71bc113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34102/0 (socket says 172.21.15.160:34102) 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71bb7fe700 1 -- 172.21.15.160:0/432804702 learned_addr learned my addr 172.21.15.160:0/432804702 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71baffd700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc103780 0x7f71bc1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71bb7fe700 1 -- 172.21.15.160:0/432804702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc103780 msgr2=0x7f71bc1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:30.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71bb7fe700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc103780 0x7f71bc1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.594+0000 7f71bb7fe700 1 -- 172.21.15.160:0/432804702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0005040 con 0x7f71bc102de0 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.595+0000 7f71bb7fe700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 0x7f71bc113b80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f71ac001d80 tx=0x7f71ac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.595+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ac007d10 con 0x7f71bc102de0 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.595+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71ac001e80 con 0x7f71bc102de0 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.595+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc10dde0 con 0x7f71bc102de0 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.595+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ac014640 con 0x7f71bc102de0 2024-05-25T06:35:30.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.595+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc10e370 con 0x7f71bc102de0 2024-05-25T06:35:30.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.596+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc0fe530 con 0x7f71bc102de0 2024-05-25T06:35:30.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.596+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f71ac00e960 con 0x7f71bc102de0 2024-05-25T06:35:30.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.597+0000 7f71a3fff700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f71a406e140 0x7f71a40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:30.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.597+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f71ac08d820 con 0x7f71bc102de0 2024-05-25T06:35:30.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.597+0000 7f71baffd700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f71a406e140 0x7f71a40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:30.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.597+0000 7f71baffd700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f71a406e140 0x7f71a40705f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f71b0004890 tx=0x7f71b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:30.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.602+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f71ac05a7b0 con 0x7f71bc102de0 2024-05-25T06:35:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:30 smithi203 ceph-mon[83507]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:30.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.816+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71bc04bed0 con 0x7f71a406e140 2024-05-25T06:35:30.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.817+0000 7f71a3fff700 1 -- 172.21.15.160:0/432804702 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f71bc04bed0 con 0x7f71a406e140 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f71a406e140 msgr2=0x7f71a40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f71a406e140 0x7f71a40705f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f71b0004890 tx=0x7f71b0009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 msgr2=0x7f71bc113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 0x7f71bc113b80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f71ac001d80 tx=0x7f71ac009ec0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 shutdown_connections 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71bc102de0 0x7f71bc113b80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f71a406e140 0x7f71a40705f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 --2- 172.21.15.160:0/432804702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc103780 0x7f71bc1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 >> 172.21.15.160:0/432804702 conn(0x7f71bc0f89b0 msgr2=0x7f71bc0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:30.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 shutdown_connections 2024-05-25T06:35:30.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.820+0000 7f71c2060700 1 -- 172.21.15.160:0/432804702 wait complete. 2024-05-25T06:35:30.830 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:35:30.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 -- 172.21.15.160:0/1753392550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 msgr2=0x7ff5300fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:30.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 --2- 172.21.15.160:0/1753392550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff5300fcdf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff518004940 tx=0x7ff518009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 -- 172.21.15.160:0/1753392550 shutdown_connections 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 --2- 172.21.15.160:0/1753392550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff5300fcdf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 --2- 172.21.15.160:0/1753392550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff530102de0 0x7ff5301031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 -- 172.21.15.160:0/1753392550 >> 172.21.15.160:0/1753392550 conn(0x7ff5300f8990 msgr2=0x7ff5300fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 -- 172.21.15.160:0/1753392550 shutdown_connections 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.923+0000 7ff535c59700 1 -- 172.21.15.160:0/1753392550 wait complete. 2024-05-25T06:35:30.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff535c59700 1 Processor -- start 2024-05-25T06:35:30.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff535c59700 1 -- start start 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff535c59700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff530102de0 0x7ff530113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff535c59700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff530113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff535c59700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5301144e0 con 0x7ff530103780 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff535c59700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53010dae0 con 0x7ff530102de0 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff52effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff530113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff52effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff530113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34120/0 (socket says 172.21.15.160:34120) 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.924+0000 7ff52effd700 1 -- 172.21.15.160:0/1894982388 learned_addr learned my addr 172.21.15.160:0/1894982388 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff52f7fe700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff530102de0 0x7ff530113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff52effd700 1 -- 172.21.15.160:0/1894982388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff530102de0 msgr2=0x7ff530113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff52effd700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff530102de0 0x7ff530113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff52effd700 1 -- 172.21.15.160:0/1894982388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518005040 con 0x7ff530103780 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff52effd700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff530113e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff518009100 tx=0x7ff518009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:30.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51800e040 con 0x7ff530103780 2024-05-25T06:35:30.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53010dd60 con 0x7ff530103780 2024-05-25T06:35:30.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff518004290 con 0x7ff530103780 2024-05-25T06:35:30.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51801c3f0 con 0x7ff530103780 2024-05-25T06:35:30.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.925+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53010e320 con 0x7ff530103780 2024-05-25T06:35:30.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.926+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5300fe530 con 0x7ff530103780 2024-05-25T06:35:30.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.927+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff518013790 con 0x7ff530103780 2024-05-25T06:35:30.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.927+0000 7ff527fff700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff51c06e140 0x7ff51c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:30.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.928+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff51808ed00 con 0x7ff530103780 2024-05-25T06:35:30.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.928+0000 7ff52f7fe700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff51c06e140 0x7ff51c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:30.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.928+0000 7ff52f7fe700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff51c06e140 0x7ff51c0705f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff520004fa0 tx=0x7ff52000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:30.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:30.932+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff518092030 con 0x7ff530103780 2024-05-25T06:35:31.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.150+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff53010f010 con 0x7ff51c06e140 2024-05-25T06:35:31.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.151+0000 7ff527fff700 1 -- 172.21.15.160:0/1894982388 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff53010f010 con 0x7ff51c06e140 2024-05-25T06:35:31.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff51c06e140 msgr2=0x7ff51c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff51c06e140 0x7ff51c0705f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff520004fa0 tx=0x7ff52000b040 comp rx=0 tx=0).stop 2024-05-25T06:35:31.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 msgr2=0x7ff530113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff530113e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff518009100 tx=0x7ff518009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 shutdown_connections 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff51c06e140 0x7ff51c0705f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff530102de0 0x7ff530113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 --2- 172.21.15.160:0/1894982388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff530103780 0x7ff530113e90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.153+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 >> 172.21.15.160:0/1894982388 conn(0x7ff5300f8990 msgr2=0x7ff5300f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.154+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 shutdown_connections 2024-05-25T06:35:31.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.154+0000 7ff535c59700 1 -- 172.21.15.160:0/1894982388 wait complete. 2024-05-25T06:35:31.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 -- 172.21.15.160:0/3702985275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 msgr2=0x7fde701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3702985275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde701031b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009ce0 tx=0x7fde6c004d10 comp rx=0 tx=0).stop 2024-05-25T06:35:31.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 -- 172.21.15.160:0/3702985275 shutdown_connections 2024-05-25T06:35:31.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3702985275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70103780 0x7fde700fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3702985275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde701031b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 -- 172.21.15.160:0/3702985275 >> 172.21.15.160:0/3702985275 conn(0x7fde700f8990 msgr2=0x7fde700fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:31.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 -- 172.21.15.160:0/3702985275 shutdown_connections 2024-05-25T06:35:31.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 -- 172.21.15.160:0/3702985275 wait complete. 2024-05-25T06:35:31.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.259+0000 7fde77aa3700 1 Processor -- start 2024-05-25T06:35:31.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde77aa3700 1 -- start start 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde77aa3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde70113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde77aa3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70103780 0x7fde70113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde77aa3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701146d0 con 0x7fde70102de0 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde77aa3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7010dc20 con 0x7fde70103780 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde70113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde70113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34140/0 (socket says 172.21.15.160:34140) 2024-05-25T06:35:31.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 -- 172.21.15.160:0/3720676164 learned_addr learned my addr 172.21.15.160:0/3720676164 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7503e700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70103780 0x7fde70113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 -- 172.21.15.160:0/3720676164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70103780 msgr2=0x7fde70113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70103780 0x7fde70113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 -- 172.21.15.160:0/3720676164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c009940 con 0x7fde70102de0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde7583f700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde70113ab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00be90 tx=0x7fde6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.260+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde6c00e040 con 0x7fde70102de0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.261+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde6c008ea0 con 0x7fde70102de0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.261+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7010de40 con 0x7fde70102de0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.261+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde6c01c810 con 0x7fde70102de0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.261+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7010e400 con 0x7fde70102de0 2024-05-25T06:35:31.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.262+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde701000f0 con 0x7fde70102de0 2024-05-25T06:35:31.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.262+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fde6c012c00 con 0x7fde70102de0 2024-05-25T06:35:31.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.262+0000 7fde667fc700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fde5c06e140 0x7fde5c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.263+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fde6c08f790 con 0x7fde70102de0 2024-05-25T06:35:31.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.263+0000 7fde7503e700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fde5c06e140 0x7fde5c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.263+0000 7fde7503e700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fde5c06e140 0x7fde5c0705f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fde60006fd0 tx=0x7fde60008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:31.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.267+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fde6c05c5f0 con 0x7fde70102de0 2024-05-25T06:35:31.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.478+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde70109f50 con 0x7fde5c06e140 2024-05-25T06:35:31.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.489+0000 7fde667fc700 1 -- 172.21.15.160:0/3720676164 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fde70109f50 con 0x7fde5c06e140 2024-05-25T06:35:31.488 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:35:31.488 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (21m) 14m ago 32m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (18m) 14m ago 32m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (18m) 14m ago 31m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (21m) 14m ago 32m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (15m) 14m ago 27m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (14m) 14m ago 26m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (14m) 14m ago 26m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (14m) 14m ago 27m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (23m) 14m ago 33m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (22m) 14m ago 31m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (19m) 14m ago 34m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (19m) 14m ago 31m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (22m) 14m ago 32m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (21m) 14m ago 31m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (18m) 14m ago 30m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:35:31.489 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (17m) 14m ago 30m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:35:31.490 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (17m) 14m ago 29m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:35:31.490 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (16m) 14m ago 29m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:35:31.490 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (16m) 14m ago 29m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:35:31.490 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (15m) 14m ago 28m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:35:31.490 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (21m) 14m ago 32m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fde5c06e140 msgr2=0x7fde5c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fde5c06e140 0x7fde5c0705f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fde60006fd0 tx=0x7fde60008040 comp rx=0 tx=0).stop 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 msgr2=0x7fde70113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde70113ab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00be90 tx=0x7fde6c004d10 comp rx=0 tx=0).stop 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 shutdown_connections 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde70102de0 0x7fde70113ab0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fde5c06e140 0x7fde5c0705f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 --2- 172.21.15.160:0/3720676164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70103780 0x7fde70113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 >> 172.21.15.160:0/3720676164 conn(0x7fde700f8990 msgr2=0x7fde700fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 shutdown_connections 2024-05-25T06:35:31.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.491+0000 7fde77aa3700 1 -- 172.21.15.160:0/3720676164 wait complete. 2024-05-25T06:35:31.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 -- 172.21.15.160:0/2568775357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420065640 msgr2=0x7f4420065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 --2- 172.21.15.160:0/2568775357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420065640 0x7f4420065ab0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4414004940 tx=0x7f4414009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:31.592 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:31 smithi160 ceph-mon[112094]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:31.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 -- 172.21.15.160:0/2568775357 shutdown_connections 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 --2- 172.21.15.160:0/2568775357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420065640 0x7f4420065ab0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 --2- 172.21.15.160:0/2568775357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4420102e50 0x7f4420103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 -- 172.21.15.160:0/2568775357 >> 172.21.15.160:0/2568775357 conn(0x7f4420071d00 msgr2=0x7f4420072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 -- 172.21.15.160:0/2568775357 shutdown_connections 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.592+0000 7f442869a700 1 -- 172.21.15.160:0/2568775357 wait complete. 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f442869a700 1 Processor -- start 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f442869a700 1 -- start start 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f442869a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4420065640 0x7f44200fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f442869a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 0x7f44200fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f442869a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200fc880 con 0x7f4420102e50 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f442869a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200fc9c0 con 0x7f4420065640 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 0x7f44200fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 0x7f44200fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34164/0 (socket says 172.21.15.160:34164) 2024-05-25T06:35:31.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 -- 172.21.15.160:0/2015556574 learned_addr learned my addr 172.21.15.160:0/2015556574 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4426436700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4420065640 0x7f44200fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 -- 172.21.15.160:0/2015556574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4420065640 msgr2=0x7f44200fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4420065640 0x7f44200fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 -- 172.21.15.160:0/2015556574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414005040 con 0x7f4420102e50 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.593+0000 7f4425c35700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 0x7f44200fc2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f441400b3b0 tx=0x7f4414009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.594+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441400e040 con 0x7f4420102e50 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.594+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44200fcc10 con 0x7f4420102e50 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.594+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4414013420 con 0x7f4420102e50 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.594+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441401c630 con 0x7f4420102e50 2024-05-25T06:35:31.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.594+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442011a2a0 con 0x7f4420102e50 2024-05-25T06:35:31.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.595+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420100100 con 0x7f4420102e50 2024-05-25T06:35:31.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.595+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4414013590 con 0x7f4420102e50 2024-05-25T06:35:31.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.596+0000 7f441effd700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f440806e140 0x7f44080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.596+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f441408ed10 con 0x7f4420102e50 2024-05-25T06:35:31.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.596+0000 7f4426436700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f440806e140 0x7f44080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.597+0000 7f4426436700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f440806e140 0x7f44080705f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4418004f40 tx=0x7f441800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:31.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.601+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f44140580c0 con 0x7f4420102e50 2024-05-25T06:35:31.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:31 smithi203 ceph-mon[83507]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:31.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.876+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f44200fe540 con 0x7f4420102e50 2024-05-25T06:35:31.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.877+0000 7f441effd700 1 -- 172.21.15.160:0/2015556574 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f441405b780 con 0x7f4420102e50 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:35:31.877 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:35:31.878 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:35:31.878 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:35:31.878 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:35:31.878 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:35:31.878 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:35:31.878 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:35:31.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.879+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f440806e140 msgr2=0x7f44080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.879+0000 7f442869a700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f440806e140 0x7f44080705f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4418004f40 tx=0x7f441800b040 comp rx=0 tx=0).stop 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.879+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 msgr2=0x7f44200fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.879+0000 7f442869a700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 0x7f44200fc2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f441400b3b0 tx=0x7f4414009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 shutdown_connections 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f440806e140 0x7f44080705f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4420065640 0x7f44200fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 --2- 172.21.15.160:0/2015556574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4420102e50 0x7f44200fc2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 >> 172.21.15.160:0/2015556574 conn(0x7f4420071d00 msgr2=0x7f44200fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 shutdown_connections 2024-05-25T06:35:31.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.880+0000 7f442869a700 1 -- 172.21.15.160:0/2015556574 wait complete. 2024-05-25T06:35:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 -- 172.21.15.160:0/4294026010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 msgr2=0x7f4a78106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/4294026010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78106e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c004940 tx=0x7f4a6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 -- 172.21.15.160:0/4294026010 shutdown_connections 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/4294026010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78106e20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/4294026010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a780fde40 0x7f4a780fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 -- 172.21.15.160:0/4294026010 >> 172.21.15.160:0/4294026010 conn(0x7f4a780f7850 msgr2=0x7f4a780f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 -- 172.21.15.160:0/4294026010 shutdown_connections 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.982+0000 7f4a7faa8700 1 -- 172.21.15.160:0/4294026010 wait complete. 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7faa8700 1 Processor -- start 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7faa8700 1 -- start start 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7faa8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a780fde40 0x7f4a78113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7faa8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7faa8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781145a0 con 0x7f4a780fe7e0 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7faa8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7810d990 con 0x7f4a780fde40 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34186/0 (socket says 172.21.15.160:34186) 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 -- 172.21.15.160:0/3195980575 learned_addr learned my addr 172.21.15.160:0/3195980575 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d844700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a780fde40 0x7f4a78113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 -- 172.21.15.160:0/3195980575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a780fde40 msgr2=0x7f4a78113a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a780fde40 0x7f4a78113a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 -- 172.21.15.160:0/3195980575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c005040 con 0x7f4a780fe7e0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.983+0000 7f4a7d043700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78113f50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c004890 tx=0x7f4a6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.984+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a6c00e040 con 0x7f4a780fe7e0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.984+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a6c009af0 con 0x7f4a780fe7e0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.984+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a6c01c3f0 con 0x7f4a780fe7e0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.984+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7810dc70 con 0x7f4a780fe7e0 2024-05-25T06:35:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.984+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7810e1b0 con 0x7f4a780fe7e0 2024-05-25T06:35:31.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.985+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a780666f0 con 0x7f4a780fe7e0 2024-05-25T06:35:31.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.985+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4a6c00baf0 con 0x7f4a780fe7e0 2024-05-25T06:35:31.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.986+0000 7f4a767fc700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4a6006e140 0x7f4a600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:31.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.986+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4a6c08ed40 con 0x7f4a780fe7e0 2024-05-25T06:35:31.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.986+0000 7f4a7d844700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4a6006e140 0x7f4a600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:31.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.986+0000 7f4a7d844700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4a6006e140 0x7f4a600705f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007b20 tx=0x7f4a70009e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:31.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:31.990+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4a6c092030 con 0x7f4a780fe7e0 2024-05-25T06:35:32.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.236+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4a78109e00 con 0x7f4a780fe7e0 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.236+0000 7f4a767fc700 1 -- 172.21.15.160:0/3195980575 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f4a6c05bc20 con 0x7f4a780fe7e0 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:35:32.236 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:35:32.237 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:35:32.238 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:35:32.239 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4a6006e140 msgr2=0x7f4a600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4a6006e140 0x7f4a600705f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007b20 tx=0x7f4a70009e20 comp rx=0 tx=0).stop 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 msgr2=0x7f4a78113f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78113f50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c004890 tx=0x7f4a6c009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 shutdown_connections 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4a6006e140 0x7f4a600705f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a780fde40 0x7f4a78113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 --2- 172.21.15.160:0/3195980575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a780fe7e0 0x7f4a78113f50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 >> 172.21.15.160:0/3195980575 conn(0x7f4a780f7850 msgr2=0x7f4a780f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:32.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 shutdown_connections 2024-05-25T06:35:32.241 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.238+0000 7f4a7faa8700 1 -- 172.21.15.160:0/3195980575 wait complete. 2024-05-25T06:35:32.241 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:35:32.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 -- 172.21.15.160:0/4043739043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fde10 msgr2=0x7f05380fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 --2- 172.21.15.160:0/4043739043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fde10 0x7f05380fe1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0528004940 tx=0x7f0528009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:32.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 -- 172.21.15.160:0/4043739043 shutdown_connections 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 --2- 172.21.15.160:0/4043739043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05380fe7b0 0x7f0538106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 --2- 172.21.15.160:0/4043739043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fde10 0x7f05380fe1e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 -- 172.21.15.160:0/4043739043 >> 172.21.15.160:0/4043739043 conn(0x7f05380f7860 msgr2=0x7f05380f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 -- 172.21.15.160:0/4043739043 shutdown_connections 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 -- 172.21.15.160:0/4043739043 wait complete. 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 Processor -- start 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.342+0000 7f053e969700 1 -- start start 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f053e969700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05380fde10 0x7f05381139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f053e969700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 0x7f0538113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f053e969700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538114530 con 0x7f05380fe7b0 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f053e969700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053810db30 con 0x7f05380fde10 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 0x7f0538113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:32.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 0x7f0538113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34200/0 (socket says 172.21.15.160:34200) 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 -- 172.21.15.160:0/267513100 learned_addr learned my addr 172.21.15.160:0/267513100 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f0537fff700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05380fde10 0x7f05381139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 -- 172.21.15.160:0/267513100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05380fde10 msgr2=0x7f05381139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05380fde10 0x7f05381139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 -- 172.21.15.160:0/267513100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528005040 con 0x7f05380fe7b0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f05377fe700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 0x7f0538113ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c830 tx=0x7f052c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052c00a760 con 0x7f05380fe7b0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f052c00ad90 con 0x7f05380fe7b0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.343+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053810de10 con 0x7f05380fe7b0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.344+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052c0113a0 con 0x7f05380fe7b0 2024-05-25T06:35:32.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.344+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053810e350 con 0x7f05380fe7b0 2024-05-25T06:35:32.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.344+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0538066b50 con 0x7f05380fe7b0 2024-05-25T06:35:32.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.345+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f052c012400 con 0x7f05380fe7b0 2024-05-25T06:35:32.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.346+0000 7f0534ff9700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f052006e140 0x7f05200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:32.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.346+0000 7f0537fff700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f052006e140 0x7f05200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:32.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.346+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f052c08dbb0 con 0x7f05380fe7b0 2024-05-25T06:35:32.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.346+0000 7f0537fff700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f052006e140 0x7f05200705f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0528004f90 tx=0x7f0528009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:32.350 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.350+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f052c05aa90 con 0x7f05380fe7b0 2024-05-25T06:35:32.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.566+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0538109ea0 con 0x7f052006e140 2024-05-25T06:35:32.566 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:32 smithi160 ceph-mon[112094]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:32.566 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:32 smithi160 ceph-mon[112094]: from='client.35182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:32.566 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:32 smithi160 ceph-mon[112094]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:32.566 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:32 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2015556574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:35:32.566 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:32 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3195980575' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.567+0000 7f0534ff9700 1 -- 172.21.15.160:0/267513100 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0538109ea0 con 0x7f052006e140 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:35:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:35:32.568 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:35:32.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f052006e140 msgr2=0x7f05200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f052006e140 0x7f05200705f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0528004f90 tx=0x7f0528009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 msgr2=0x7f0538113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 0x7f0538113ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c830 tx=0x7f052c004470 comp rx=0 tx=0).stop 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 shutdown_connections 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f052006e140 0x7f05200705f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05380fde10 0x7f05381139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 --2- 172.21.15.160:0/267513100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05380fe7b0 0x7f0538113ee0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.569+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 >> 172.21.15.160:0/267513100 conn(0x7f05380f7860 msgr2=0x7f05380f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.570+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 shutdown_connections 2024-05-25T06:35:32.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.570+0000 7f053e969700 1 -- 172.21.15.160:0/267513100 wait complete. 2024-05-25T06:35:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:32 smithi203 ceph-mon[83507]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:32 smithi203 ceph-mon[83507]: from='client.35182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:32 smithi203 ceph-mon[83507]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:32 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2015556574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:35:32.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:32 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3195980575' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.670+0000 7f36595fa700 1 -- 172.21.15.160:0/1981887127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3654101d00 msgr2=0x7f36541020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.670+0000 7f36595fa700 1 --2- 172.21.15.160:0/1981887127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3654101d00 0x7f36541020d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3644004940 tx=0x7f3644009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.670+0000 7f36595fa700 1 -- 172.21.15.160:0/1981887127 shutdown_connections 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.670+0000 7f36595fa700 1 --2- 172.21.15.160:0/1981887127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36541026a0 0x7f36540fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.670+0000 7f36595fa700 1 --2- 172.21.15.160:0/1981887127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3654101d00 0x7f36541020d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.670+0000 7f36595fa700 1 -- 172.21.15.160:0/1981887127 >> 172.21.15.160:0/1981887127 conn(0x7f36540f7870 msgr2=0x7f36540f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.671+0000 7f36595fa700 1 -- 172.21.15.160:0/1981887127 shutdown_connections 2024-05-25T06:35:32.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.671+0000 7f36595fa700 1 -- 172.21.15.160:0/1981887127 wait complete. 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.671+0000 7f36595fa700 1 Processor -- start 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.671+0000 7f36595fa700 1 -- start start 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36595fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3654101d00 0x7f3654112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36595fa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 0x7f3654112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36595fa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3654113410 con 0x7f36541026a0 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36595fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365410ca10 con 0x7f3654101d00 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 0x7f3654112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 0x7f3654112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34220/0 (socket says 172.21.15.160:34220) 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 -- 172.21.15.160:0/1229540105 learned_addr learned my addr 172.21.15.160:0/1229540105 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:35:32.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f3652ffd700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3654101d00 0x7f3654112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 -- 172.21.15.160:0/1229540105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3654101d00 msgr2=0x7f3654112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3654101d00 0x7f3654112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 -- 172.21.15.160:0/1229540105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3644005040 con 0x7f36541026a0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f3652ffd700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3654101d00 0x7f3654112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36527fc700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 0x7f3654112dc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3648002a50 tx=0x7f3648004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3648007cc0 con 0x7f36541026a0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f364800f040 con 0x7f36541026a0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365410ccf0 con 0x7f36541026a0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.672+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f364800daf0 con 0x7f36541026a0 2024-05-25T06:35:32.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.673+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365410d2b0 con 0x7f36541026a0 2024-05-25T06:35:32.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.673+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36540fd450 con 0x7f36541026a0 2024-05-25T06:35:32.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.674+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3648009810 con 0x7f36541026a0 2024-05-25T06:35:32.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.674+0000 7f363b7fe700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f363c06e140 0x7f363c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:35:32.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.675+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f364808d940 con 0x7f36541026a0 2024-05-25T06:35:32.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.675+0000 7f3652ffd700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f363c06e140 0x7f363c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:35:32.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.675+0000 7f3652ffd700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f363c06e140 0x7f363c0705f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3644004f90 tx=0x7f3644009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:35:32.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.679+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f364805a7a0 con 0x7f36541026a0 2024-05-25T06:35:32.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.960+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3654108e30 con 0x7f36541026a0 2024-05-25T06:35:32.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.961+0000 7f363b7fe700 1 -- 172.21.15.160:0/1229540105 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f364805a330 con 0x7f36541026a0 2024-05-25T06:35:32.961 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:35:32.961 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:35:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:35:32.961 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:35:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f363c06e140 msgr2=0x7f363c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f363c06e140 0x7f363c0705f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3644004f90 tx=0x7f3644009e30 comp rx=0 tx=0).stop 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 msgr2=0x7f3654112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 0x7f3654112dc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3648002a50 tx=0x7f3648004600 comp rx=0 tx=0).stop 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 shutdown_connections 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f363c06e140 0x7f363c0705f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3654101d00 0x7f3654112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 --2- 172.21.15.160:0/1229540105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36541026a0 0x7f3654112dc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 >> 172.21.15.160:0/1229540105 conn(0x7f36540f7870 msgr2=0x7f36540f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:35:32.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 shutdown_connections 2024-05-25T06:35:32.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:35:32.963+0000 7f36595fa700 1 -- 172.21.15.160:0/1229540105 wait complete. 2024-05-25T06:35:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:33 smithi203 ceph-mon[83507]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:33 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1229540105' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:35:33.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:33 smithi160 ceph-mon[112094]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:35:33.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:33 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1229540105' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:35:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:34 smithi203 ceph-mon[83507]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:34 smithi160 ceph-mon[112094]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:36 smithi203 ceph-mon[83507]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:36 smithi160 ceph-mon[112094]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:38 smithi203 ceph-mon[83507]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:38 smithi160 ceph-mon[112094]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:40 smithi203 ceph-mon[83507]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:40 smithi160 ceph-mon[112094]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:42.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:42 smithi203 ceph-mon[83507]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:42.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:42 smithi160 ceph-mon[112094]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:43.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:44 smithi203 ceph-mon[83507]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:44.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:44 smithi160 ceph-mon[112094]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:46 smithi203 ceph-mon[83507]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:46.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:46 smithi160 ceph-mon[112094]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:48 smithi203 ceph-mon[83507]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:48.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:48 smithi160 ceph-mon[112094]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:50 smithi203 ceph-mon[83507]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:50 smithi160 ceph-mon[112094]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:52 smithi203 ceph-mon[83507]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:52.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:52 smithi160 ceph-mon[112094]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:54 smithi203 ceph-mon[83507]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:54 smithi160 ceph-mon[112094]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:56 smithi203 ceph-mon[83507]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:56.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:56 smithi160 ceph-mon[112094]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:58 smithi203 ceph-mon[83507]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:35:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:35:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:58 smithi160 ceph-mon[112094]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:35:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:35:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:00 smithi203 ceph-mon[83507]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:00.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:00 smithi160 ceph-mon[112094]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:02 smithi203 ceph-mon[83507]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:02.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:02 smithi160 ceph-mon[112094]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.079+0000 7fac94c05700 1 -- 172.21.15.160:0/2480684562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fe7e0 msgr2=0x7fac90106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.079+0000 7fac94c05700 1 --2- 172.21.15.160:0/2480684562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fe7e0 0x7fac90106e20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fac84004940 tx=0x7fac84009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 -- 172.21.15.160:0/2480684562 shutdown_connections 2024-05-25T06:36:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 --2- 172.21.15.160:0/2480684562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fe7e0 0x7fac90106e20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 --2- 172.21.15.160:0/2480684562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fde40 0x7fac900fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 -- 172.21.15.160:0/2480684562 >> 172.21.15.160:0/2480684562 conn(0x7fac900f7890 msgr2=0x7fac900f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 -- 172.21.15.160:0/2480684562 shutdown_connections 2024-05-25T06:36:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 -- 172.21.15.160:0/2480684562 wait complete. 2024-05-25T06:36:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 Processor -- start 2024-05-25T06:36:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.080+0000 7fac94c05700 1 -- start start 2024-05-25T06:36:03.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac94c05700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 0x7fac90113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac94c05700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fe7e0 0x7fac90113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac94c05700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901145e0 con 0x7fac900fde40 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac94c05700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9010d9d0 con 0x7fac900fe7e0 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 0x7fac90113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 0x7fac90113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44104/0 (socket says 172.21.15.160:44104) 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 -- 172.21.15.160:0/2530168800 learned_addr learned my addr 172.21.15.160:0/2530168800 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8dd9b700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fe7e0 0x7fac90113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 -- 172.21.15.160:0/2530168800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fe7e0 msgr2=0x7fac90113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fe7e0 0x7fac90113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 -- 172.21.15.160:0/2530168800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84005040 con 0x7fac900fde40 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8dd9b700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fe7e0 0x7fac90113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac8e59c700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 0x7fac90113a50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fac80002a80 tx=0x7fac800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:03.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.081+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac80007cc0 con 0x7fac900fde40 2024-05-25T06:36:03.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.082+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac8000f040 con 0x7fac900fde40 2024-05-25T06:36:03.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.082+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9010dcb0 con 0x7fac900fde40 2024-05-25T06:36:03.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.082+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac800096f0 con 0x7fac900fde40 2024-05-25T06:36:03.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.082+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9010e1c0 con 0x7fac900fde40 2024-05-25T06:36:03.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.083+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac900666f0 con 0x7fac900fde40 2024-05-25T06:36:03.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.083+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fac8000d690 con 0x7fac900fde40 2024-05-25T06:36:03.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.084+0000 7fac7effd700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac7806e150 0x7fac78070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.084+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fac8008d320 con 0x7fac900fde40 2024-05-25T06:36:03.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.084+0000 7fac8dd9b700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac7806e150 0x7fac78070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.084+0000 7fac8dd9b700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac7806e150 0x7fac78070600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac9010ecb0 tx=0x7fac84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:03.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.089+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fac8005a180 con 0x7fac900fde40 2024-05-25T06:36:03.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.308+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac90109e00 con 0x7fac7806e150 2024-05-25T06:36:03.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.309+0000 7fac7effd700 1 -- 172.21.15.160:0/2530168800 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fac90109e00 con 0x7fac7806e150 2024-05-25T06:36:03.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac7806e150 msgr2=0x7fac78070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac7806e150 0x7fac78070600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac9010ecb0 tx=0x7fac84009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 msgr2=0x7fac90113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 0x7fac90113a50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fac80002a80 tx=0x7fac800046b0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 shutdown_connections 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac900fde40 0x7fac90113a50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fac7806e150 0x7fac78070600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 --2- 172.21.15.160:0/2530168800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac900fe7e0 0x7fac90113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.311+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 >> 172.21.15.160:0/2530168800 conn(0x7fac900f7890 msgr2=0x7fac900f8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.312+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 shutdown_connections 2024-05-25T06:36:03.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.312+0000 7fac94c05700 1 -- 172.21.15.160:0/2530168800 wait complete. 2024-05-25T06:36:03.322 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.414+0000 7f26d0249700 1 -- 172.21.15.160:0/690560597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 msgr2=0x7f26c80fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.414+0000 7f26d0249700 1 --2- 172.21.15.160:0/690560597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c80fce00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f26bc004940 tx=0x7f26bc009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 -- 172.21.15.160:0/690560597 shutdown_connections 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 --2- 172.21.15.160:0/690560597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c80fce00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 --2- 172.21.15.160:0/690560597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26c8102df0 0x7f26c81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 -- 172.21.15.160:0/690560597 >> 172.21.15.160:0/690560597 conn(0x7f26c80f8980 msgr2=0x7f26c80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:03.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 -- 172.21.15.160:0/690560597 shutdown_connections 2024-05-25T06:36:03.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 -- 172.21.15.160:0/690560597 wait complete. 2024-05-25T06:36:03.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 Processor -- start 2024-05-25T06:36:03.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 -- start start 2024-05-25T06:36:03.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26c8102df0 0x7f26c8111950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.415+0000 7f26d0249700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c8111e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26d0249700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c81124e0 con 0x7f26c8103790 2024-05-25T06:36:03.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26d0249700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c810b8d0 con 0x7f26c8102df0 2024-05-25T06:36:03.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c8111e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c8111e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44116/0 (socket says 172.21.15.160:44116) 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 -- 172.21.15.160:0/219348793 learned_addr learned my addr 172.21.15.160:0/219348793 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cdfe5700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26c8102df0 0x7f26c8111950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 -- 172.21.15.160:0/219348793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26c8102df0 msgr2=0x7f26c8111950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26c8102df0 0x7f26c8111950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 -- 172.21.15.160:0/219348793 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc005040 con 0x7f26c8103790 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26cd7e4700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c8111e90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009100 tx=0x7f26bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26bc00e040 con 0x7f26c8103790 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c810bb50 con 0x7f26c8103790 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26bc009af0 con 0x7f26c8103790 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26bc01c3f0 con 0x7f26c8103790 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.416+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c810c110 con 0x7f26c8103790 2024-05-25T06:36:03.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.417+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c80fe540 con 0x7f26c8103790 2024-05-25T06:36:03.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.418+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f26bc00baf0 con 0x7f26c8103790 2024-05-25T06:36:03.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.419+0000 7f26c67fc700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f26b0072540 0x7f26b00749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.419+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f26bc08edb0 con 0x7f26c8103790 2024-05-25T06:36:03.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.419+0000 7f26cdfe5700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f26b0072540 0x7f26b00749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.419+0000 7f26cdfe5700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f26b0072540 0x7f26b00749f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007970 tx=0x7f26c0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.423+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f26bc05bc90 con 0x7f26c8103790 2024-05-25T06:36:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.639+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26c8071f90 con 0x7f26b0072540 2024-05-25T06:36:03.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.640+0000 7f26c67fc700 1 -- 172.21.15.160:0/219348793 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f26c8071f90 con 0x7f26b0072540 2024-05-25T06:36:03.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f26b0072540 msgr2=0x7f26b00749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f26b0072540 0x7f26b00749f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007970 tx=0x7f26c0009da0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 msgr2=0x7f26c8111e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c8111e90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009100 tx=0x7f26bc009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:03.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 shutdown_connections 2024-05-25T06:36:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f26b0072540 0x7f26b00749f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26c8102df0 0x7f26c8111950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 --2- 172.21.15.160:0/219348793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26c8103790 0x7f26c8111e90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 >> 172.21.15.160:0/219348793 conn(0x7f26c80f8980 msgr2=0x7f26c80f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 shutdown_connections 2024-05-25T06:36:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.642+0000 7f26d0249700 1 -- 172.21.15.160:0/219348793 wait complete. 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 -- 172.21.15.160:0/1961674806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4101cf0 msgr2=0x7f51f41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1961674806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4101cf0 0x7f51f41020c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004940 tx=0x7f51f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 -- 172.21.15.160:0/1961674806 shutdown_connections 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1961674806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4102690 0x7f51f40fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1961674806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4101cf0 0x7f51f41020c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 -- 172.21.15.160:0/1961674806 >> 172.21.15.160:0/1961674806 conn(0x7f51f40f7860 msgr2=0x7f51f40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 -- 172.21.15.160:0/1961674806 shutdown_connections 2024-05-25T06:36:03.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 -- 172.21.15.160:0/1961674806 wait complete. 2024-05-25T06:36:03.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 Processor -- start 2024-05-25T06:36:03.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 -- start start 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4101cf0 0x7f51f410c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.746+0000 7f51fbf70700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 0x7f51f410ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51fbf70700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4067160 con 0x7f51f4102690 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51fbf70700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f410d370 con 0x7f51f4101cf0 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 0x7f51f410ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 0x7f51f410ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44138/0 (socket says 172.21.15.160:44138) 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 -- 172.21.15.160:0/1208374735 learned_addr learned my addr 172.21.15.160:0/1208374735 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f9d0c700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4101cf0 0x7f51f410c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 -- 172.21.15.160:0/1208374735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4101cf0 msgr2=0x7f51f410c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4101cf0 0x7f51f410c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 -- 172.21.15.160:0/1208374735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0005040 con 0x7f51f4102690 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f9d0c700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4101cf0 0x7f51f410c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51f950b700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 0x7f51f410ce30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c890 tx=0x7f51e40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e400a7d0 con 0x7f51f4102690 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f410d650 con 0x7f51f4102690 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51e400ae00 con 0x7f51f4102690 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e40113a0 con 0x7f51f4102690 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.747+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f411a1a0 con 0x7f51f4102690 2024-05-25T06:36:03.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.748+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f40fd440 con 0x7f51f4102690 2024-05-25T06:36:03.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.749+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f51e4012400 con 0x7f51f4102690 2024-05-25T06:36:03.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.749+0000 7f51ea7fc700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51e006e140 0x7f51e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:03.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.749+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f51e408dc20 con 0x7f51f4102690 2024-05-25T06:36:03.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.749+0000 7f51f9d0c700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51e006e140 0x7f51e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:03.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.750+0000 7f51f9d0c700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51e006e140 0x7f51e00705f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004560 tx=0x7f51f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:03.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.754+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f51e405ab00 con 0x7f51f4102690 2024-05-25T06:36:03.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.961+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51f40f8ee0 con 0x7f51e006e140 2024-05-25T06:36:03.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.972+0000 7f51ea7fc700 1 -- 172.21.15.160:0/1208374735 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f51f40f8ee0 con 0x7f51e006e140 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (21m) 15m ago 32m 25.8M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (19m) 15m ago 33m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (19m) 14m ago 32m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (21m) 15m ago 32m 80.4M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (15m) 15m ago 27m 14.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (15m) 15m ago 27m 11.0M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (14m) 14m ago 27m 15.1M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (15m) 14m ago 27m 14.8M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (23m) 15m ago 34m 511M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (22m) 14m ago 32m 454M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (20m) 15m ago 34m 54.9M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:36:03.972 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (20m) 14m ago 32m 45.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (22m) 15m ago 33m 15.4M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (22m) 14m ago 32m 16.6M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (18m) 15m ago 31m 101M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (18m) 15m ago 30m 103M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (17m) 15m ago 30m 83.0M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (17m) 14m ago 29m 114M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (16m) 14m ago 29m 94.2M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (16m) 14m ago 29m 87.5M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:36:03.973 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (22m) 15m ago 32m 67.7M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51e006e140 msgr2=0x7f51e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51e006e140 0x7f51e00705f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004560 tx=0x7f51f0009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 msgr2=0x7f51f410ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 0x7f51f410ce30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c890 tx=0x7f51e40044c0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 shutdown_connections 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f51e006e140 0x7f51e00705f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51f4101cf0 0x7f51f410c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 --2- 172.21.15.160:0/1208374735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51f4102690 0x7f51f410ce30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 >> 172.21.15.160:0/1208374735 conn(0x7f51f40f7860 msgr2=0x7f51f40f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 shutdown_connections 2024-05-25T06:36:03.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:03.974+0000 7f51fbf70700 1 -- 172.21.15.160:0/1208374735 wait complete. 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 -- 172.21.15.160:0/1937897455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 msgr2=0x7f10340fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1937897455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f10340fc1b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1028004940 tx=0x7f1028009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 -- 172.21.15.160:0/1937897455 shutdown_connections 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1937897455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f10340fc1b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1937897455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1034101d90 0x7f1034102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 -- 172.21.15.160:0/1937897455 >> 172.21.15.160:0/1937897455 conn(0x7f1034071d00 msgr2=0x7f1034072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 -- 172.21.15.160:0/1937897455 shutdown_connections 2024-05-25T06:36:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.076+0000 7f103b8c3700 1 -- 172.21.15.160:0/1937897455 wait complete. 2024-05-25T06:36:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103b8c3700 1 Processor -- start 2024-05-25T06:36:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103b8c3700 1 -- start start 2024-05-25T06:36:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103b8c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f1034115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103b8c3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1034101d90 0x7f10341162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103b8c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034116910 con 0x7f10340fbd40 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103b8c3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103410fce0 con 0x7f1034101d90 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f1034115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f1034115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44168/0 (socket says 172.21.15.160:44168) 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 -- 172.21.15.160:0/1239039335 learned_addr learned my addr 172.21.15.160:0/1239039335 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 -- 172.21.15.160:0/1239039335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1034101d90 msgr2=0x7f10341162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1034101d90 0x7f10341162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 -- 172.21.15.160:0/1239039335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028005040 con 0x7f10340fbd40 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.077+0000 7f103965f700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f1034115d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f102c002a50 tx=0x7f102c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.078+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102c007cc0 con 0x7f10340fbd40 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.078+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f102c00f040 con 0x7f10340fbd40 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.078+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103410ffc0 con 0x7f10340fbd40 2024-05-25T06:36:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.079+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102c009640 con 0x7f10340fbd40 2024-05-25T06:36:04.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.079+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1034110580 con 0x7f10340fbd40 2024-05-25T06:36:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.079+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f102c0097a0 con 0x7f10340fbd40 2024-05-25T06:36:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.080+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10340fd430 con 0x7f10340fbd40 2024-05-25T06:36:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.080+0000 7f1031ffb700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f101c06e140 0x7f101c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.080+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f102c08d980 con 0x7f10340fbd40 2024-05-25T06:36:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.080+0000 7f1038e5e700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f101c06e140 0x7f101c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.081+0000 7f1038e5e700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f101c06e140 0x7f101c0705f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1028004540 tx=0x7f1028009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:04.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.085+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f102c05a7e0 con 0x7f10340fbd40 2024-05-25T06:36:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.364+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f103404bed0 con 0x7f10340fbd40 2024-05-25T06:36:04.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:04 smithi160 ceph-mon[112094]: pgmap v649: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:04.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:04 smithi160 ceph-mon[112094]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:04.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:04 smithi160 ceph-mon[112094]: from='client.35210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:04.365 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:04 smithi160 ceph-mon[112094]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.365+0000 7f1031ffb700 1 -- 172.21.15.160:0/1239039335 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f102c05a370 con 0x7f10340fbd40 2024-05-25T06:36:04.365 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:04.366 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:36:04.367 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:36:04.367 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:36:04.367 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:36:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f101c06e140 msgr2=0x7f101c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f101c06e140 0x7f101c0705f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1028004540 tx=0x7f1028009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 msgr2=0x7f1034115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f1034115d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f102c002a50 tx=0x7f102c0046b0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 shutdown_connections 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10340fbd40 0x7f1034115d80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f101c06e140 0x7f101c0705f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.367+0000 7f103b8c3700 1 --2- 172.21.15.160:0/1239039335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1034101d90 0x7f10341162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.368+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 >> 172.21.15.160:0/1239039335 conn(0x7f1034071d00 msgr2=0x7f10340655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.368+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 shutdown_connections 2024-05-25T06:36:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.368+0000 7f103b8c3700 1 -- 172.21.15.160:0/1239039335 wait complete. 2024-05-25T06:36:04.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.470+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1356304710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0103780 msgr2=0x7f0ba00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.470+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1356304710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0103780 0x7f0ba00fcdf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0b90004940 tx=0x7f0b90009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.470+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1356304710 shutdown_connections 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.470+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1356304710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0103780 0x7f0ba00fcdf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.470+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1356304710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ba0102de0 0x7f0ba01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.470+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1356304710 >> 172.21.15.160:0/1356304710 conn(0x7f0ba00f8990 msgr2=0x7f0ba00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1356304710 shutdown_connections 2024-05-25T06:36:04.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1356304710 wait complete. 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 Processor -- start 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 -- start start 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 0x7f0ba0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ba0103780 0x7f0ba0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0114520 con 0x7f0ba0102de0 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.471+0000 7f0ba4db3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba010db20 con 0x7f0ba0103780 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 0x7f0ba0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 0x7f0ba0113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44182/0 (socket says 172.21.15.160:44182) 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 -- 172.21.15.160:0/1939276709 learned_addr learned my addr 172.21.15.160:0/1939276709 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9dd9b700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ba0103780 0x7f0ba0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 -- 172.21.15.160:0/1939276709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ba0103780 msgr2=0x7f0ba0113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ba0103780 0x7f0ba0113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 -- 172.21.15.160:0/1939276709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90005040 con 0x7f0ba0102de0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b9e59c700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 0x7f0ba0113990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0b94002a50 tx=0x7f0b940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b94007cc0 con 0x7f0ba0102de0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b9400f040 con 0x7f0ba0102de0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba010de00 con 0x7f0ba0102de0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b94009640 con 0x7f0ba0102de0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.472+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba010e3c0 con 0x7f0ba0102de0 2024-05-25T06:36:04.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.473+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba00fe530 con 0x7f0ba0102de0 2024-05-25T06:36:04.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.474+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0b940097a0 con 0x7f0ba0102de0 2024-05-25T06:36:04.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.474+0000 7f0b8effd700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b8806e140 0x7f0b880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.474+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0b9408d990 con 0x7f0ba0102de0 2024-05-25T06:36:04.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.474+0000 7f0b9dd9b700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b8806e140 0x7f0b880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.475+0000 7f0b9dd9b700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b8806e140 0x7f0b880705f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ba010ece0 tx=0x7f0b90009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:04.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.479+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0b9405a7f0 con 0x7f0ba0102de0 2024-05-25T06:36:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:04 smithi203 ceph-mon[83507]: pgmap v649: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:04 smithi203 ceph-mon[83507]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:04 smithi203 ceph-mon[83507]: from='client.35210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:04 smithi203 ceph-mon[83507]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:04.718 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.718+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ba0109ec0 con 0x7f0ba0102de0 2024-05-25T06:36:04.718 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.718+0000 7f0b8effd700 1 -- 172.21.15.160:0/1939276709 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f0b9405a380 con 0x7f0ba0102de0 2024-05-25T06:36:04.718 INFO:teuthology.orchestra.run.smithi160.stdout:e37 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 28 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:20:11.153647+0000 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:36:04.719 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:up {} 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:failed 0,1 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:36:04.720 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 0 members: 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{-1:34416} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:36:04.721 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:36:04.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b8806e140 msgr2=0x7f0b880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b8806e140 0x7f0b880705f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ba010ece0 tx=0x7f0b90009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 msgr2=0x7f0ba0113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 0x7f0ba0113990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0b94002a50 tx=0x7f0b940046b0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 shutdown_connections 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ba0102de0 0x7f0ba0113990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0b8806e140 0x7f0b880705f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 --2- 172.21.15.160:0/1939276709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ba0103780 0x7f0ba0113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 >> 172.21.15.160:0/1939276709 conn(0x7f0ba00f8990 msgr2=0x7f0ba00f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 shutdown_connections 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.722+0000 7f0ba4db3700 1 -- 172.21.15.160:0/1939276709 wait complete. 2024-05-25T06:36:04.723 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:36:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 -- 172.21.15.160:0/1107369309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8070840 msgr2=0x7fbeb8070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 --2- 172.21.15.160:0/1107369309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8070840 0x7fbeb8070cb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbea8004970 tx=0x7fbea8009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 -- 172.21.15.160:0/1107369309 shutdown_connections 2024-05-25T06:36:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 --2- 172.21.15.160:0/1107369309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8070840 0x7fbeb8070cb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 --2- 172.21.15.160:0/1107369309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb8071f30 0x7fbeb8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 -- 172.21.15.160:0/1107369309 >> 172.21.15.160:0/1107369309 conn(0x7fbeb80f8990 msgr2=0x7fbeb80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 -- 172.21.15.160:0/1107369309 shutdown_connections 2024-05-25T06:36:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 -- 172.21.15.160:0/1107369309 wait complete. 2024-05-25T06:36:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.827+0000 7fbec0645700 1 Processor -- start 2024-05-25T06:36:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbec0645700 1 -- start start 2024-05-25T06:36:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbec0645700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb8070840 0x7fbeb8119f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbec0645700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 0x7fbeb811a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbec0645700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb811ab00 con 0x7fbeb8071f30 2024-05-25T06:36:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbec0645700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81140e0 con 0x7fbeb8070840 2024-05-25T06:36:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 0x7fbeb811a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 0x7fbeb811a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44190/0 (socket says 172.21.15.160:44190) 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 -- 172.21.15.160:0/9933539 learned_addr learned my addr 172.21.15.160:0/9933539 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebe3e1700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb8070840 0x7fbeb8119f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 -- 172.21.15.160:0/9933539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb8070840 msgr2=0x7fbeb8119f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb8070840 0x7fbeb8119f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 -- 172.21.15.160:0/9933539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8005040 con 0x7fbeb8071f30 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbebdbe0700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 0x7fbeb811a4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000c00 tx=0x7fbea8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.828+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbea800e040 con 0x7fbeb8071f30 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.829+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbea800bc10 con 0x7fbeb8071f30 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.829+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8114360 con 0x7fbeb8071f30 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.829+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbea801c630 con 0x7fbeb8071f30 2024-05-25T06:36:04.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.829+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb81148a0 con 0x7fbeb8071f30 2024-05-25T06:36:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.829+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb8071a40 con 0x7fbeb8071f30 2024-05-25T06:36:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.830+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbea8013420 con 0x7fbeb8071f30 2024-05-25T06:36:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.830+0000 7fbeaeffd700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbea406e140 0x7fbea40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.831+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbea808ed60 con 0x7fbeb8071f30 2024-05-25T06:36:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.831+0000 7fbebe3e1700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbea406e140 0x7fbea40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.831+0000 7fbebe3e1700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbea406e140 0x7fbea40705f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4004f40 tx=0x7fbeb400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:04.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:04.835+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbea805bd00 con 0x7fbeb8071f30 2024-05-25T06:36:05.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.053+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbeb8110500 con 0x7fbea406e140 2024-05-25T06:36:05.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.055+0000 7fbeaeffd700 1 -- 172.21.15.160:0/9933539 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbeb8110500 con 0x7fbea406e140 2024-05-25T06:36:05.054 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": true, 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [ 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "mon", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "mds", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "osd" 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": "16/21 daemons upgraded", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "Currently upgrading mds daemons", 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:36:05.055 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbea406e140 msgr2=0x7fbea40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbea406e140 0x7fbea40705f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4004f40 tx=0x7fbeb400b040 comp rx=0 tx=0).stop 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 msgr2=0x7fbeb811a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 0x7fbeb811a4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000c00 tx=0x7fbea8009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 shutdown_connections 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fbea406e140 0x7fbea40705f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb8070840 0x7fbeb8119f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 --2- 172.21.15.160:0/9933539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb8071f30 0x7fbeb811a4b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 >> 172.21.15.160:0/9933539 conn(0x7fbeb80f8990 msgr2=0x7fbeb80f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:05.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 shutdown_connections 2024-05-25T06:36:05.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.057+0000 7fbec0645700 1 -- 172.21.15.160:0/9933539 wait complete. 2024-05-25T06:36:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 -- 172.21.15.160:0/3960325735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654103790 msgr2=0x7ff6540fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:05.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3960325735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654103790 0x7ff6540fce00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff648004940 tx=0x7ff648009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 -- 172.21.15.160:0/3960325735 shutdown_connections 2024-05-25T06:36:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3960325735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654103790 0x7ff6540fce00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3960325735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654102df0 0x7ff6541031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 -- 172.21.15.160:0/3960325735 >> 172.21.15.160:0/3960325735 conn(0x7ff6540f8980 msgr2=0x7ff6540fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 -- 172.21.15.160:0/3960325735 shutdown_connections 2024-05-25T06:36:05.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.160+0000 7ff6599d9700 1 -- 172.21.15.160:0/3960325735 wait complete. 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff6599d9700 1 Processor -- start 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff6599d9700 1 -- start start 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff6599d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 0x7ff654113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff6599d9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654103790 0x7ff6541140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff6599d9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541147b0 con 0x7ff654102df0 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff6599d9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410db30 con 0x7ff654103790 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 0x7ff654113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 0x7ff654113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44204/0 (socket says 172.21.15.160:44204) 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.161+0000 7ff652ffd700 1 -- 172.21.15.160:0/3704711693 learned_addr learned my addr 172.21.15.160:0/3704711693 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff6527fc700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654103790 0x7ff6541140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:05.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff652ffd700 1 -- 172.21.15.160:0/3704711693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654103790 msgr2=0x7ff6541140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff652ffd700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654103790 0x7ff6541140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff652ffd700 1 -- 172.21.15.160:0/3704711693 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648005040 con 0x7ff654102df0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff6527fc700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654103790 0x7ff6541140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff652ffd700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 0x7ff654113b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff64400b790 tx=0x7ff644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64400f770 con 0x7ff654102df0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff64400fda0 con 0x7ff654102df0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65410de10 con 0x7ff654102df0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64400d310 con 0x7ff654102df0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.162+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65410e350 con 0x7ff654102df0 2024-05-25T06:36:05.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.163+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654100100 con 0x7ff654102df0 2024-05-25T06:36:05.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.164+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff644010420 con 0x7ff654102df0 2024-05-25T06:36:05.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.164+0000 7ff63b7fe700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff63c06e140 0x7ff63c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:05.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.164+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff64408db90 con 0x7ff654102df0 2024-05-25T06:36:05.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.164+0000 7ff6527fc700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff63c06e140 0x7ff63c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:05.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.164+0000 7ff6527fc700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff63c06e140 0x7ff63c0705f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff648009fd0 tx=0x7ff648009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:05.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.169+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff644056f40 con 0x7ff654102df0 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.448+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6540fe540 con 0x7ff654102df0 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.448+0000 7ff63b7fe700 1 -- 172.21.15.160:0/3704711693 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff64405a600 con 0x7ff654102df0 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is degraded 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-25T06:36:05.448 INFO:teuthology.orchestra.run.smithi160.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-25T06:36:05.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff63c06e140 msgr2=0x7ff63c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:05.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff63c06e140 0x7ff63c0705f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff648009fd0 tx=0x7ff648009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:05.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 msgr2=0x7ff654113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:05.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 0x7ff654113b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff64400b790 tx=0x7ff644009250 comp rx=0 tx=0).stop 2024-05-25T06:36:05.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 shutdown_connections 2024-05-25T06:36:05.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff654102df0 0x7ff654113b90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff63c06e140 0x7ff63c0705f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 --2- 172.21.15.160:0/3704711693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff654103790 0x7ff6541140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:05.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 >> 172.21.15.160:0/3704711693 conn(0x7ff6540f8980 msgr2=0x7ff6541073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:05.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 shutdown_connections 2024-05-25T06:36:05.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:05.450+0000 7ff6599d9700 1 -- 172.21.15.160:0/3704711693 wait complete. 2024-05-25T06:36:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:05 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1239039335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:05 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1939276709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:36:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:05 smithi203 ceph-mon[83507]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:05 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1239039335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:05 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1939276709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:36:05.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:05 smithi160 ceph-mon[112094]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:06 smithi203 ceph-mon[83507]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:06 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3704711693' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:36:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:06 smithi160 ceph-mon[112094]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:06 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3704711693' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:36:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:08 smithi203 ceph-mon[83507]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:08 smithi160 ceph-mon[112094]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:10 smithi203 ceph-mon[83507]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:10 smithi160 ceph-mon[112094]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:12 smithi203 ceph-mon[83507]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:12 smithi160 ceph-mon[112094]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:36:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all mds 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:13.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.rrhzig"}]': finished 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.tlctof"}]': finished 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.tndnrb"}]': finished 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.yltymd"}]': finished 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: Upgrade: Setting filesystem cephfs Joinable 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: Standby daemon mds.cephfs.smithi160.rrhzig assigned to filesystem cephfs as rank 0 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: Standby daemon mds.cephfs.smithi160.tlctof assigned to filesystem cephfs as rank 1 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:replay,1=cephfs.smithi160.tlctof=up:replay} 2 up:standby 2024-05-25T06:36:13.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-25T06:36:13.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:13.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all mds 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.rrhzig"}]: dispatch 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.rrhzig"}]': finished 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.tlctof"}]: dispatch 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi160.tlctof"}]': finished 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.tndnrb"}]: dispatch 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.tndnrb"}]': finished 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.yltymd"}]: dispatch 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi203.yltymd"}]': finished 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: Upgrade: Setting filesystem cephfs Joinable 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: Standby daemon mds.cephfs.smithi160.rrhzig assigned to filesystem cephfs as rank 0 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: Standby daemon mds.cephfs.smithi160.tlctof assigned to filesystem cephfs as rank 1 2024-05-25T06:36:13.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-25T06:36:13.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-25T06:36:13.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:replay,1=cephfs.smithi160.tlctof=up:replay} 2 up:standby 2024-05-25T06:36:13.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-25T06:36:13.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-25T06:36:14.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-05-25T06:36:14.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-25T06:36:14.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:replay,1=cephfs.smithi160.tlctof=up:replay} 2 up:standby 2024-05-25T06:36:14.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all rgw 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all rbd-mirror 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all ceph-exporter 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all iscsi 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all nfs 2024-05-25T06:36:14.651 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: Upgrade: Setting container_image for all nvmeof 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.652 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-25T06:36:14.653 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-25T06:36:14.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-25T06:36:14.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-25T06:36:14.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-25T06:36:14.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-25T06:36:14.654 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-25T06:36:14.655 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-25T06:36:14.656 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:14 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:replay,1=cephfs.smithi160.tlctof=up:replay} 2 up:standby 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all rgw 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all rbd-mirror 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.720 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all ceph-exporter 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all iscsi 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all nfs 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: Upgrade: Setting container_image for all nvmeof 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.721 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-25T06:36:14.722 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-25T06:36:14.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-25T06:36:14.723 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-25T06:36:14.724 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.725 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-25T06:36:14.726 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:14 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:36:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: Upgrade: Finalizing container_image settings 2024-05-25T06:36:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: Upgrade: Complete! 2024-05-25T06:36:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 3.4 MiB/s rd, 1 op/s 2024-05-25T06:36:15.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:15.900 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:15 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: Upgrade: Finalizing container_image settings 2024-05-25T06:36:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: Upgrade: Complete! 2024-05-25T06:36:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:16.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 3.4 MiB/s rd, 1 op/s 2024-05-25T06:36:16.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:16.220 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:15 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:18 smithi160 ceph-mon[112094]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-05-25T06:36:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:18 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:18 smithi203 ceph-mon[83507]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-05-25T06:36:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:18.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:18.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:18 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:20.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:20 smithi203 ceph-mon[83507]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-05-25T06:36:20.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:20 smithi160 ceph-mon[112094]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-05-25T06:36:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:21 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:resolve 2024-05-25T06:36:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:21 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:resolve 2024-05-25T06:36:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:21 smithi203 ceph-mon[83507]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:resolve,1=cephfs.smithi160.tlctof=up:resolve} 2 up:standby 2024-05-25T06:36:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:21.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:21 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:21 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:resolve 2024-05-25T06:36:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:21 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:resolve 2024-05-25T06:36:21.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:21 smithi160 ceph-mon[112094]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:resolve,1=cephfs.smithi160.tlctof=up:resolve} 2 up:standby 2024-05-25T06:36:21.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:21.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:21 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:22.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: reconnect by client.14534 192.168.0.1:0/1579019524 after 0.000999985 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: reconnect by client.14534 192.168.0.1:0/1579019524 after 0.000999985 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: reconnect by client.24325 192.168.0.1:0/2204786649 after 0.000999985 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: reconnect by client.24325 192.168.0.1:0/2204786649 after 0.000999985 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:reconnect 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:reconnect 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:reconnect,1=cephfs.smithi160.tlctof=up:reconnect} 2 up:standby 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:36:22.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:36:22.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:36:22.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:22.401 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:22 smithi203 ceph-mon[83507]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-05-25T06:36:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-05-25T06:36:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: reconnect by client.14534 192.168.0.1:0/1579019524 after 0.000999985 2024-05-25T06:36:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: reconnect by client.14534 192.168.0.1:0/1579019524 after 0.000999985 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: reconnect by client.24325 192.168.0.1:0/2204786649 after 0.000999985 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: reconnect by client.24325 192.168.0.1:0/2204786649 after 0.000999985 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:reconnect 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:reconnect 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:reconnect,1=cephfs.smithi160.tlctof=up:reconnect} 2 up:standby 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:36:22.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:22.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:22 smithi160 ceph-mon[112094]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-05-25T06:36:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:rejoin 2024-05-25T06:36:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:rejoin 2024-05-25T06:36:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:rejoin,1=cephfs.smithi160.tlctof=up:rejoin} 2 up:standby 2024-05-25T06:36:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: daemon mds.cephfs.smithi160.rrhzig is now active in filesystem cephfs as rank 0 2024-05-25T06:36:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: daemon mds.cephfs.smithi160.tlctof is now active in filesystem cephfs as rank 1 2024-05-25T06:36:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-25T06:36:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: Cluster is now healthy 2024-05-25T06:36:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:active 2024-05-25T06:36:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:active 2024-05-25T06:36:23.400 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:23 smithi203 ceph-mon[83507]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi160.tlctof=up:active} 2 up:standby-replay 2024-05-25T06:36:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:rejoin 2024-05-25T06:36:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:rejoin 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: fsmap cephfs:2/2 {0=cephfs.smithi160.rrhzig=up:rejoin,1=cephfs.smithi160.tlctof=up:rejoin} 2 up:standby 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: daemon mds.cephfs.smithi160.rrhzig is now active in filesystem cephfs as rank 0 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: daemon mds.cephfs.smithi160.tlctof is now active in filesystem cephfs as rank 1 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: Cluster is now healthy 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] up:active 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: mds.? [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] up:active 2024-05-25T06:36:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:23 smithi160 ceph-mon[112094]: fsmap cephfs:2 {0=cephfs.smithi160.rrhzig=up:active,1=cephfs.smithi160.tlctof=up:active} 2 up:standby-replay 2024-05-25T06:36:24.149 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:24 smithi203 ceph-mon[83507]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 99 B/s wr, 8 op/s 2024-05-25T06:36:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:24 smithi160 ceph-mon[112094]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 99 B/s wr, 8 op/s 2024-05-25T06:36:26.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:26 smithi203 ceph-mon[83507]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 599 B/s wr, 11 op/s 2024-05-25T06:36:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:26 smithi160 ceph-mon[112094]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 599 B/s wr, 11 op/s 2024-05-25T06:36:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:28 smithi160 ceph-mon[112094]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 699 B/s wr, 14 op/s 2024-05-25T06:36:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:28 smithi203 ceph-mon[83507]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 699 B/s wr, 14 op/s 2024-05-25T06:36:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:36:28.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:30.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:30 smithi203 ceph-mon[83507]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 699 B/s wr, 15 op/s 2024-05-25T06:36:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:30 smithi160 ceph-mon[112094]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 699 B/s wr, 15 op/s 2024-05-25T06:36:32.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:32 smithi203 ceph-mon[83507]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 10 KiB/s wr, 20 op/s 2024-05-25T06:36:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:32 smithi160 ceph-mon[112094]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 10 KiB/s wr, 20 op/s 2024-05-25T06:36:34.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:34 smithi203 ceph-mon[83507]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.8 KiB/s wr, 17 op/s 2024-05-25T06:36:34.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:34 smithi160 ceph-mon[112094]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.8 KiB/s wr, 17 op/s 2024-05-25T06:36:35.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:35 smithi203 ceph-mon[83507]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-05-25T06:36:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:35 smithi160 ceph-mon[112094]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-05-25T06:36:35.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 -- 172.21.15.160:0/2711516516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 msgr2=0x7f67d80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:35.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 --2- 172.21.15.160:0/2711516516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80fcdf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67c0004940 tx=0x7f67c0009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 -- 172.21.15.160:0/2711516516 shutdown_connections 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 --2- 172.21.15.160:0/2711516516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80fcdf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 --2- 172.21.15.160:0/2711516516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67d8102de0 0x7f67d81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 -- 172.21.15.160:0/2711516516 >> 172.21.15.160:0/2711516516 conn(0x7f67d80f8950 msgr2=0x7f67d80fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 -- 172.21.15.160:0/2711516516 shutdown_connections 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.567+0000 7f67de0f8700 1 -- 172.21.15.160:0/2711516516 wait complete. 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67de0f8700 1 Processor -- start 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67de0f8700 1 -- start start 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67de0f8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67d8102de0 0x7f67d8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67de0f8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:35.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67de0f8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8070d30 con 0x7f67d8103780 2024-05-25T06:36:35.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67de0f8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8070e70 con 0x7f67d8102de0 2024-05-25T06:36:35.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:35.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54116/0 (socket says 172.21.15.160:54116) 2024-05-25T06:36:35.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.568+0000 7f67d6ffd700 1 -- 172.21.15.160:0/3001485132 learned_addr learned my addr 172.21.15.160:0/3001485132 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:35.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67d77fe700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67d8102de0 0x7f67d8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:35.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67d6ffd700 1 -- 172.21.15.160:0/3001485132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67d8102de0 msgr2=0x7f67d8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:35.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67d6ffd700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67d8102de0 0x7f67d8070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:35.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67d6ffd700 1 -- 172.21.15.160:0/3001485132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0005040 con 0x7f67d8103780 2024-05-25T06:36:35.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67d6ffd700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80707a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009100 tx=0x7f67c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c000e040 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d80710c0 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67c0004290 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c001c3f0 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.569+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d811a290 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.570+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d80fe530 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.571+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f67c0013690 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.571+0000 7f67cffff700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f67c406e150 0x7f67c4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.571+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f67c008ec90 con 0x7f67d8103780 2024-05-25T06:36:35.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.571+0000 7f67d77fe700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f67c406e150 0x7f67c4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:35.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.572+0000 7f67d77fe700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f67c406e150 0x7f67c4070600 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f67d0004f40 tx=0x7f67d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:35.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.576+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f67c0058040 con 0x7f67d8103780 2024-05-25T06:36:35.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.794+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67d8109f60 con 0x7f67c406e150 2024-05-25T06:36:35.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.795+0000 7f67cffff700 1 -- 172.21.15.160:0/3001485132 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f67d000f040 con 0x7f67c406e150 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f67c406e150 msgr2=0x7f67c4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f67c406e150 0x7f67c4070600 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f67d0004f40 tx=0x7f67d000b040 comp rx=0 tx=0).stop 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 msgr2=0x7f67d80707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80707a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009100 tx=0x7f67c0009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 shutdown_connections 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f67c406e150 0x7f67c4070600 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67d8102de0 0x7f67d8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 --2- 172.21.15.160:0/3001485132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d8103780 0x7f67d80707a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 >> 172.21.15.160:0/3001485132 conn(0x7f67d80f8950 msgr2=0x7f67d80f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 shutdown_connections 2024-05-25T06:36:35.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:35.798+0000 7f67de0f8700 1 -- 172.21.15.160:0/3001485132 wait complete. 2024-05-25T06:36:36.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:36 smithi203 ceph-mon[83507]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:36.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:36 smithi160 ceph-mon[112094]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:36.546 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch ps' 2024-05-25T06:36:36.803 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:36:37.215 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:37 smithi160 ceph-mon[112094]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 8.3 KiB/s wr, 14 op/s 2024-05-25T06:36:37.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:37 smithi203 ceph-mon[83507]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 8.3 KiB/s wr, 14 op/s 2024-05-25T06:36:38.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 -- 172.21.15.160:0/409755997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec102de0 msgr2=0x7f2aec1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:38.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 --2- 172.21.15.160:0/409755997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec102de0 0x7f2aec1031b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2adc004940 tx=0x7f2adc009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:38.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 -- 172.21.15.160:0/409755997 shutdown_connections 2024-05-25T06:36:38.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 --2- 172.21.15.160:0/409755997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aec103780 0x7f2aec0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:38.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 --2- 172.21.15.160:0/409755997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec102de0 0x7f2aec1031b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:38.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 -- 172.21.15.160:0/409755997 >> 172.21.15.160:0/409755997 conn(0x7f2aec0f89b0 msgr2=0x7f2aec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:38.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 -- 172.21.15.160:0/409755997 shutdown_connections 2024-05-25T06:36:38.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.401+0000 7f2af133f700 1 -- 172.21.15.160:0/409755997 wait complete. 2024-05-25T06:36:38.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2af133f700 1 Processor -- start 2024-05-25T06:36:38.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2af133f700 1 -- start start 2024-05-25T06:36:38.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2af133f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aec102de0 0x7f2aec115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:38.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2af133f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 0x7f2aec1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:38.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2af133f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec116930 con 0x7f2aec103780 2024-05-25T06:36:38.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2af133f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec10fd00 con 0x7f2aec102de0 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2aea7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 0x7f2aec1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2aea7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 0x7f2aec1162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54134/0 (socket says 172.21.15.160:54134) 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2aea7fc700 1 -- 172.21.15.160:0/2816774597 learned_addr learned my addr 172.21.15.160:0/2816774597 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2aea7fc700 1 -- 172.21.15.160:0/2816774597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aec102de0 msgr2=0x7f2aec115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.402+0000 7f2aea7fc700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aec102de0 0x7f2aec115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2aea7fc700 1 -- 172.21.15.160:0/2816774597 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc005040 con 0x7f2aec103780 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2aea7fc700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 0x7f2aec1162e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0002a50 tx=0x7f2ae0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ae0007cc0 con 0x7f2aec103780 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ae000f040 con 0x7f2aec103780 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ae000da40 con 0x7f2aec103780 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec10ffe0 con 0x7f2aec103780 2024-05-25T06:36:38.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.403+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec1105a0 con 0x7f2aec103780 2024-05-25T06:36:38.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.404+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec106a30 con 0x7f2aec103780 2024-05-25T06:36:38.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.404+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2ae00097c0 con 0x7f2aec103780 2024-05-25T06:36:38.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.405+0000 7f2ad37fe700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2ad406e140 0x7f2ad40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:38.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.405+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2ae008d9a0 con 0x7f2aec103780 2024-05-25T06:36:38.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.405+0000 7f2aeaffd700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2ad406e140 0x7f2ad40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:38.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.405+0000 7f2aeaffd700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2ad406e140 0x7f2ad40705f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2adc004f90 tx=0x7f2adc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:38.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.410+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2ae005a800 con 0x7f2aec103780 2024-05-25T06:36:38.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.616+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aec111460 con 0x7f2ad406e140 2024-05-25T06:36:38.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.627+0000 7f2ad37fe700 1 -- 172.21.15.160:0/2816774597 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2aec111460 con 0x7f2ad406e140 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (22m) 18s ago 33m 24.5M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (19m) 18s ago 33m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (19m) 21s ago 32m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (22m) 18s ago 33m 80.5M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (16m) 18s ago 28m 99.5M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (15m) 18s ago 28m 131M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (15m) 21s ago 28m 20.6M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (15m) 21s ago 28m 20.3M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (24m) 18s ago 35m 515M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:36:38.627 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (23m) 21s ago 32m 460M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (21m) 18s ago 35m 71.0M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (20m) 21s ago 32m 52.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (23m) 18s ago 33m 15.3M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (23m) 21s ago 32m 16.7M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (19m) 18s ago 31m 108M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (18m) 18s ago 31m 115M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (18m) 18s ago 30m 91.3M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (17m) 21s ago 30m 125M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (17m) 21s ago 30m 102M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (16m) 21s ago 29m 97.9M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:36:38.628 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (22m) 18s ago 33m 78.1M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.629+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2ad406e140 msgr2=0x7f2ad40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2ad406e140 0x7f2ad40705f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2adc004f90 tx=0x7f2adc009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 msgr2=0x7f2aec1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 0x7f2aec1162e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0002a50 tx=0x7f2ae0004600 comp rx=0 tx=0).stop 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 shutdown_connections 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2ad406e140 0x7f2ad40705f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aec102de0 0x7f2aec115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 --2- 172.21.15.160:0/2816774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aec103780 0x7f2aec1162e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:38.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 >> 172.21.15.160:0/2816774597 conn(0x7f2aec0f89b0 msgr2=0x7f2aec107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:38.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 shutdown_connections 2024-05-25T06:36:38.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:38.630+0000 7f2af133f700 1 -- 172.21.15.160:0/2816774597 wait complete. 2024-05-25T06:36:39.226 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch upgrade status' 2024-05-25T06:36:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:39 smithi203 ceph-mon[83507]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:39 smithi203 ceph-mon[83507]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 8.8 MiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-25T06:36:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:39 smithi160 ceph-mon[112094]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:39.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:39 smithi160 ceph-mon[112094]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 8.8 MiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-25T06:36:39.499 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:36:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 -- 172.21.15.160:0/2328545804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04102e50 msgr2=0x7f6c04103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/2328545804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04102e50 0x7f6c04103220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4004940 tx=0x7f6bf4009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 -- 172.21.15.160:0/2328545804 shutdown_connections 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/2328545804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c04065640 0x7f6c04065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/2328545804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04102e50 0x7f6c04103220 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 -- 172.21.15.160:0/2328545804 >> 172.21.15.160:0/2328545804 conn(0x7f6c04071d00 msgr2=0x7f6c04072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 -- 172.21.15.160:0/2328545804 shutdown_connections 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.184+0000 7f6c09a8b700 1 -- 172.21.15.160:0/2328545804 wait complete. 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c09a8b700 1 Processor -- start 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c09a8b700 1 -- start start 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c09a8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 0x7f6c040fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c09a8b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c04102e50 0x7f6c040fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:41.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c09a8b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040fc880 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c09a8b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040fc9c0 con 0x7f6c04102e50 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.185+0000 7f6c037fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 0x7f6c040fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c037fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 0x7f6c040fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54142/0 (socket says 172.21.15.160:54142) 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c037fe700 1 -- 172.21.15.160:0/403888924 learned_addr learned my addr 172.21.15.160:0/403888924 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c02ffd700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c04102e50 0x7f6c040fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c037fe700 1 -- 172.21.15.160:0/403888924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c04102e50 msgr2=0x7f6c040fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c037fe700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c04102e50 0x7f6c040fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c037fe700 1 -- 172.21.15.160:0/403888924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4005040 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c037fe700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 0x7f6c040fbdb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400b8b0 tx=0x7f6bf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf400e040 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c040fcc10 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bf4004290 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf401c3f0 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.186+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0411a210 con 0x7f6c04065640 2024-05-25T06:36:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.187+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c04100100 con 0x7f6c04065640 2024-05-25T06:36:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.188+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6bf40137b0 con 0x7f6c04065640 2024-05-25T06:36:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.188+0000 7f6bebfff700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6bec06e140 0x7f6bec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.188+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6bf408eea0 con 0x7f6c04065640 2024-05-25T06:36:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.188+0000 7f6c02ffd700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6bec06e140 0x7f6bec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.189+0000 7f6c02ffd700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6bec06e140 0x7f6bec0705f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009fd0 tx=0x7f6bf8009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.193+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6bf4058250 con 0x7f6c04065640 2024-05-25T06:36:41.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:41 smithi160 ceph-mon[112094]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-05-25T06:36:41.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.412+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c0404bed0 con 0x7f6bec06e140 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.413+0000 7f6bebfff700 1 -- 172.21.15.160:0/403888924 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f6c0404bed0 con 0x7f6bec06e140 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout: "target_image": null, 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout: "in_progress": false, 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout: "which": "", 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout: "services_complete": [], 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout: "progress": null, 2024-05-25T06:36:41.413 INFO:teuthology.orchestra.run.smithi160.stdout: "message": "", 2024-05-25T06:36:41.414 INFO:teuthology.orchestra.run.smithi160.stdout: "is_paused": false 2024-05-25T06:36:41.414 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:36:41.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6bec06e140 msgr2=0x7f6bec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:41.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6bec06e140 0x7f6bec0705f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009fd0 tx=0x7f6bf8009f60 comp rx=0 tx=0).stop 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 msgr2=0x7f6c040fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 0x7f6c040fbdb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400b8b0 tx=0x7f6bf4009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 shutdown_connections 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04065640 0x7f6c040fbdb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6bec06e140 0x7f6bec0705f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.415+0000 7f6c09a8b700 1 --2- 172.21.15.160:0/403888924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c04102e50 0x7f6c040fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.416+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 >> 172.21.15.160:0/403888924 conn(0x7f6c04071d00 msgr2=0x7f6c040faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.416+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 shutdown_connections 2024-05-25T06:36:41.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:41.416+0000 7f6c09a8b700 1 -- 172.21.15.160:0/403888924 wait complete. 2024-05-25T06:36:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:41 smithi203 ceph-mon[83507]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-05-25T06:36:42.194 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph health detail' 2024-05-25T06:36:42.453 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:36:42.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:42 smithi160 ceph-mon[112094]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:42.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:42 smithi203 ceph-mon[83507]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:36:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:43 smithi160 ceph-mon[112094]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 3 op/s 2024-05-25T06:36:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:43 smithi203 ceph-mon[83507]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 3 op/s 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 -- 172.21.15.160:0/984105560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 msgr2=0x7f56d01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 --2- 172.21.15.160:0/984105560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d01020b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ce0 tx=0x7f56c8004d10 comp rx=0 tx=0).stop 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 -- 172.21.15.160:0/984105560 shutdown_connections 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 --2- 172.21.15.160:0/984105560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 0x7f56d00fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 --2- 172.21.15.160:0/984105560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d01020b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 -- 172.21.15.160:0/984105560 >> 172.21.15.160:0/984105560 conn(0x7f56d00f7850 msgr2=0x7f56d00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 -- 172.21.15.160:0/984105560 shutdown_connections 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.985+0000 7f56d4b05700 1 -- 172.21.15.160:0/984105560 wait complete. 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56d4b05700 1 Processor -- start 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56d4b05700 1 -- start start 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56d4b05700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d0113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56d4b05700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 0x7f56d0114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56d4b05700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0114910 con 0x7f56d0101ce0 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56d4b05700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d010dc50 con 0x7f56d0102680 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d0113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d0113cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:54148/0 (socket says 172.21.15.160:54148) 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56ce59c700 1 -- 172.21.15.160:0/4160267904 learned_addr learned my addr 172.21.15.160:0/4160267904 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56cdd9b700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 0x7f56d0114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56ce59c700 1 -- 172.21.15.160:0/4160267904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 msgr2=0x7f56d0114230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56ce59c700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 0x7f56d0114230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56ce59c700 1 -- 172.21.15.160:0/4160267904 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8009940 con 0x7f56d0101ce0 2024-05-25T06:36:43.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.986+0000 7f56cdd9b700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 0x7f56d0114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.987+0000 7f56ce59c700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d0113cf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f56c800be90 tx=0x7f56c8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.987+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c800e040 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.987+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56c8008ea0 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.987+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d010de70 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.987+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c801c810 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.987+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d010e430 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.988+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d00fd430 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.988+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f56c8012c00 con 0x7f56d0101ce0 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.989+0000 7f56c6ffd700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f56bc06e140 0x7f56bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.989+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f56c808f790 con 0x7f56d0101ce0 2024-05-25T06:36:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.989+0000 7f56cdd9b700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f56bc06e140 0x7f56bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.989+0000 7f56cdd9b700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f56bc06e140 0x7f56bc0705f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006fd0 tx=0x7f56b8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:43.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:43.993+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f56c805c5f0 con 0x7f56d0101ce0 2024-05-25T06:36:44.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.273+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f56d005f5c0 con 0x7f56d0101ce0 2024-05-25T06:36:44.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.274+0000 7f56c6ffd700 1 -- 172.21.15.160:0/4160267904 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f56c8004e10 con 0x7f56d0101ce0 2024-05-25T06:36:44.274 INFO:teuthology.orchestra.run.smithi160.stdout:HEALTH_OK 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f56bc06e140 msgr2=0x7f56bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f56bc06e140 0x7f56bc0705f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006fd0 tx=0x7f56b8008040 comp rx=0 tx=0).stop 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 msgr2=0x7f56d0113cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d0113cf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f56c800be90 tx=0x7f56c8004d10 comp rx=0 tx=0).stop 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 shutdown_connections 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56d0101ce0 0x7f56d0113cf0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f56bc06e140 0x7f56bc0705f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 --2- 172.21.15.160:0/4160267904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56d0102680 0x7f56d0114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 >> 172.21.15.160:0/4160267904 conn(0x7f56d00f7850 msgr2=0x7f56d00f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:44.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 shutdown_connections 2024-05-25T06:36:44.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:44.276+0000 7f56d4b05700 1 -- 172.21.15.160:0/4160267904 wait complete. 2024-05-25T06:36:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:44 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4160267904' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:36:44.693 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:44 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4160267904' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-25T06:36:44.880 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions' 2024-05-25T06:36:45.140 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:36:45.584 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:45 smithi160 ceph-mon[112094]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 4 op/s 2024-05-25T06:36:45.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:45 smithi203 ceph-mon[83507]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 4 op/s 2024-05-25T06:36:46.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 -- 172.21.15.160:0/3536485363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 msgr2=0x7fb8240fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 --2- 172.21.15.160:0/3536485363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb8240fbfc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb814004940 tx=0x7fb814009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 -- 172.21.15.160:0/3536485363 shutdown_connections 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 --2- 172.21.15.160:0/3536485363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8240fc590 0x7fb824100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 --2- 172.21.15.160:0/3536485363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb8240fbfc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 -- 172.21.15.160:0/3536485363 >> 172.21.15.160:0/3536485363 conn(0x7fb8240f7870 msgr2=0x7fb8240f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 -- 172.21.15.160:0/3536485363 shutdown_connections 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.667+0000 7fb82a69f700 1 -- 172.21.15.160:0/3536485363 wait complete. 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb82a69f700 1 Processor -- start 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb82a69f700 1 -- start start 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb82a69f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb82411a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb82a69f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8240fc590 0x7fb82411a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb82a69f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82411ac50 con 0x7fb8240fbbf0 2024-05-25T06:36:46.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb82a69f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb824114270 con 0x7fb8240fc590 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb82411a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb82411a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51780/0 (socket says 172.21.15.160:51780) 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 -- 172.21.15.160:0/268892336 learned_addr learned my addr 172.21.15.160:0/268892336 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb81bfff700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8240fc590 0x7fb82411a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 -- 172.21.15.160:0/268892336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8240fc590 msgr2=0x7fb82411a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8240fc590 0x7fb82411a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 -- 172.21.15.160:0/268892336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814005040 con 0x7fb8240fbbf0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.668+0000 7fb823fff700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb82411a0c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b8b0 tx=0x7fb814009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.669+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81400e040 con 0x7fb8240fbbf0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.669+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb814004290 con 0x7fb8240fbbf0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.669+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8241144f0 con 0x7fb8240fbbf0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.669+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81401c3f0 con 0x7fb8240fbbf0 2024-05-25T06:36:46.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.669+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb824114a30 con 0x7fb8240fbbf0 2024-05-25T06:36:46.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.670+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb824066b30 con 0x7fb8240fbbf0 2024-05-25T06:36:46.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.670+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb8140137b0 con 0x7fb8240fbbf0 2024-05-25T06:36:46.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.671+0000 7fb8217fa700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb81006e140 0x7fb8100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:36:46.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.671+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb81408eda0 con 0x7fb8240fbbf0 2024-05-25T06:36:46.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.671+0000 7fb81bfff700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb81006e140 0x7fb8100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:36:46.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.671+0000 7fb81bfff700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb81006e140 0x7fb8100705f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb80c007fd0 tx=0x7fb80c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:36:46.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.675+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb814058210 con 0x7fb8240fbbf0 2024-05-25T06:36:46.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.958+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb824115750 con 0x7fb8240fbbf0 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.959+0000 7fb8217fa700 1 -- 172.21.15.160:0/268892336 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fb81405b8d0 con 0x7fb8240fbbf0 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:46.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:36:46.960 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb81006e140 msgr2=0x7fb8100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb81006e140 0x7fb8100705f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb80c007fd0 tx=0x7fb80c009040 comp rx=0 tx=0).stop 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 msgr2=0x7fb82411a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb82411a0c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b8b0 tx=0x7fb814009e30 comp rx=0 tx=0).stop 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 shutdown_connections 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8240fbbf0 0x7fb82411a0c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:46.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb81006e140 0x7fb8100705f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:46.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 --2- 172.21.15.160:0/268892336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8240fc590 0x7fb82411a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:36:46.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 >> 172.21.15.160:0/268892336 conn(0x7fb8240f7870 msgr2=0x7fb8240f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:36:46.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 shutdown_connections 2024-05-25T06:36:46.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:36:46.961+0000 7fb82a69f700 1 -- 172.21.15.160:0/268892336 wait complete. 2024-05-25T06:36:47.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:47 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/268892336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:47.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:47 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/268892336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:36:47.638 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-25T06:36:47.897 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:36:48.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:48 smithi160 ceph-mon[112094]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:48.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:48 smithi203 ceph-mon[83507]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:49.298 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:49 smithi160 ceph-mon[112094]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:49.557 INFO:teuthology.orchestra.run.smithi160.stdout:wait for servicemap items w/ changing names to refresh 2024-05-25T06:36:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:49 smithi203 ceph-mon[83507]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:50.197 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'sleep 60' 2024-05-25T06:36:50.459 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:36:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:51 smithi160 ceph-mon[112094]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:36:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:51 smithi203 ceph-mon[83507]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:36:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:53 smithi160 ceph-mon[112094]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:53 smithi203 ceph-mon[83507]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:55 smithi160 ceph-mon[112094]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:36:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:55 smithi203 ceph-mon[83507]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:36:57.452 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:57 smithi203 ceph-mon[83507]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:57 smithi160 ceph-mon[112094]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:36:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:36:59 smithi160 ceph-mon[112094]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:36:59.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:36:59 smithi203 ceph-mon[83507]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:01 smithi160 ceph-mon[112094]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:01 smithi203 ceph-mon[83507]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:04.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:04 smithi203 ceph-mon[83507]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:04.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:04 smithi160 ceph-mon[112094]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:05.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:05 smithi160 ceph-mon[112094]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:05 smithi203 ceph-mon[83507]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:07 smithi160 ceph-mon[112094]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:07.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:07 smithi203 ceph-mon[83507]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:09.439 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:09 smithi160 ceph-mon[112094]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:09 smithi203 ceph-mon[83507]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:11 smithi160 ceph-mon[112094]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:11 smithi203 ceph-mon[83507]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:13 smithi203 ceph-mon[83507]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:13 smithi160 ceph-mon[112094]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:15 smithi160 ceph-mon[112094]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:15 smithi203 ceph-mon[83507]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:17 smithi160 ceph-mon[112094]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:17 smithi203 ceph-mon[83507]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:19 smithi160 ceph-mon[112094]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:19.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:19 smithi203 ceph-mon[83507]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:21 smithi160 ceph-mon[112094]: pgmap v689: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:21 smithi203 ceph-mon[83507]: pgmap v689: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:37:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:37:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:37:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:37:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:37:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:37:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:37:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:37:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:23 smithi160 ceph-mon[112094]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:23 smithi203 ceph-mon[83507]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:25 smithi160 ceph-mon[112094]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:25 smithi203 ceph-mon[83507]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:27.451 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:27 smithi203 ceph-mon[83507]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:27 smithi160 ceph-mon[112094]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:28.638 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:29 smithi160 ceph-mon[112094]: pgmap v693: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:29.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:29 smithi203 ceph-mon[83507]: pgmap v693: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:31 smithi160 ceph-mon[112094]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:31.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:31 smithi203 ceph-mon[83507]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:33 smithi160 ceph-mon[112094]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:33 smithi203 ceph-mon[83507]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:35 smithi160 ceph-mon[112094]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:35 smithi203 ceph-mon[83507]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:37 smithi160 ceph-mon[112094]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:37.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:37 smithi203 ceph-mon[83507]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:39 smithi160 ceph-mon[112094]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:39 smithi203 ceph-mon[83507]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:41 smithi160 ceph-mon[112094]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:41 smithi203 ceph-mon[83507]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:43 smithi203 ceph-mon[83507]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:43 smithi160 ceph-mon[112094]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:45 smithi160 ceph-mon[112094]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:45.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:45 smithi203 ceph-mon[83507]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:47 smithi160 ceph-mon[112094]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:47.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:47 smithi203 ceph-mon[83507]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:49 smithi160 ceph-mon[112094]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:49 smithi203 ceph-mon[83507]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:51 smithi160 ceph-mon[112094]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:51 smithi203 ceph-mon[83507]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:37:52.569 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph orch ps' 2024-05-25T06:37:52.831 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:37:53.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:53 smithi160 ceph-mon[112094]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:53 smithi203 ceph-mon[83507]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:54.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 -- 172.21.15.160:0/673906702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954103780 msgr2=0x7f19540fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:54.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 --2- 172.21.15.160:0/673906702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954103780 0x7f19540fcdf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1944004940 tx=0x7f1944009e30 comp rx=0 tx=0).stop 2024-05-25T06:37:54.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 -- 172.21.15.160:0/673906702 shutdown_connections 2024-05-25T06:37:54.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 --2- 172.21.15.160:0/673906702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954103780 0x7f19540fcdf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:54.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 --2- 172.21.15.160:0/673906702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954102de0 0x7f19541031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 -- 172.21.15.160:0/673906702 >> 172.21.15.160:0/673906702 conn(0x7f19540f8990 msgr2=0x7f19540fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 -- 172.21.15.160:0/673906702 shutdown_connections 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.399+0000 7f195c3b9700 1 -- 172.21.15.160:0/673906702 wait complete. 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195c3b9700 1 Processor -- start 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195c3b9700 1 -- start start 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195c3b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 0x7f1954113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195c3b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954103780 0x7f1954113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195c3b9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954114490 con 0x7f1954102de0 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195c3b9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195410da90 con 0x7f1954103780 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195a155700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 0x7f1954113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:54.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195a155700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 0x7f1954113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38594/0 (socket says 172.21.15.160:38594) 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195a155700 1 -- 172.21.15.160:0/1224953885 learned_addr learned my addr 172.21.15.160:0/1224953885 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f1959954700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954103780 0x7f1954113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195a155700 1 -- 172.21.15.160:0/1224953885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954103780 msgr2=0x7f1954113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195a155700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954103780 0x7f1954113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f195a155700 1 -- 172.21.15.160:0/1224953885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944005040 con 0x7f1954102de0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.400+0000 7f1959954700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954103780 0x7f1954113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.401+0000 7f195a155700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 0x7f1954113900 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f195000eb40 tx=0x7f195000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.401+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1950007cc0 con 0x7f1954102de0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.401+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f195000f040 con 0x7f1954102de0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.401+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1950009620 con 0x7f1954102de0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.401+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195410dd10 con 0x7f1954102de0 2024-05-25T06:37:54.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.401+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195410e200 con 0x7f1954102de0 2024-05-25T06:37:54.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.402+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19540fe530 con 0x7f1954102de0 2024-05-25T06:37:54.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.402+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1950009780 con 0x7f1954102de0 2024-05-25T06:37:54.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.403+0000 7f194affd700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f194006e140 0x7f19400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:54.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.403+0000 7f1959954700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f194006e140 0x7f19400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:54.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.403+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f195008d830 con 0x7f1954102de0 2024-05-25T06:37:54.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.403+0000 7f1959954700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f194006e140 0x7f19400705f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f194400b0d0 tx=0x7f1944009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:37:54.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.408+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f195005a740 con 0x7f1954102de0 2024-05-25T06:37:54.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.614+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 --> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1954109ec0 con 0x7f194006e140 2024-05-25T06:37:54.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.624+0000 7f194affd700 1 -- 172.21.15.160:0/1224953885 <== mgr.34104 v2:172.21.15.160:6800/427616578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1954109ec0 con 0x7f194006e140 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:alertmanager.smithi160 smithi160 *:9093,9094 running (23m) 94s ago 34m 24.5M - 0.25.0 c8568f914cd2 8abecc0cf20a 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi160 smithi160 running (21m) 94s ago 35m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 31883df7da8d 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:crash.smithi203 smithi203 running (21m) 97s ago 34m 6660k - 19.0.0-2843-gea0684c0 f9a9809cafd6 0a3865db747c 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:grafana.smithi160 smithi160 *:3000 running (23m) 94s ago 34m 80.5M - 9.4.12 2bacad6d85d8 febb492f65cd 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.rrhzig smithi160 running (17m) 94s ago 29m 99.5M - 19.0.0-2843-gea0684c0 f9a9809cafd6 8c7c7a1e5a44 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi160.tlctof smithi160 running (17m) 94s ago 29m 131M - 19.0.0-2843-gea0684c0 f9a9809cafd6 3aadf17b4efa 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.tndnrb smithi203 running (16m) 97s ago 29m 20.6M - 19.0.0-2843-gea0684c0 f9a9809cafd6 7acdee1cc0eb 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mds.cephfs.smithi203.yltymd smithi203 running (16m) 97s ago 29m 20.3M - 19.0.0-2843-gea0684c0 f9a9809cafd6 eaff4ab3adf1 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi160.nalzlg smithi160 *:8443,9283,8765 running (25m) 94s ago 36m 515M - 19.0.0-2843-gea0684c0 f9a9809cafd6 1b408c5eb134 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.smithi203.rotqex smithi203 *:8443,9283,8765 running (24m) 97s ago 33m 460M - 19.0.0-2843-gea0684c0 f9a9809cafd6 c6bb44cbcb63 2024-05-25T06:37:54.624 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi160 smithi160 running (22m) 94s ago 36m 71.0M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 f6abc627de68 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:mon.smithi203 smithi203 running (21m) 97s ago 33m 52.3M 2048M 19.0.0-2843-gea0684c0 f9a9809cafd6 bd246c6536a8 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi160 smithi160 *:9100 running (24m) 94s ago 34m 15.3M - 1.5.0 0da6a335fe13 67da51ad2f56 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:node-exporter.smithi203 smithi203 *:9100 running (24m) 97s ago 34m 16.7M - 1.5.0 0da6a335fe13 aab77de6e083 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (20m) 94s ago 33m 108M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 80bfe7bea4a3 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi160 running (20m) 94s ago 32m 115M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 544e0dd60296 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi160 running (19m) 94s ago 32m 91.3M 1557M 19.0.0-2843-gea0684c0 f9a9809cafd6 26bb5422342f 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi203 running (19m) 97s ago 31m 125M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 8e79c63db459 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi203 running (18m) 97s ago 31m 102M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 5331e6b134b3 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi203 running (17m) 97s ago 30m 97.9M 2581M 19.0.0-2843-gea0684c0 f9a9809cafd6 7f2ea24e28ec 2024-05-25T06:37:54.625 INFO:teuthology.orchestra.run.smithi160.stdout:prometheus.smithi160 smithi160 *:9095 running (24m) 94s ago 34m 78.1M - 2.43.0 a07b618ecd1d 9cad694f06cd 2024-05-25T06:37:54.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f194006e140 msgr2=0x7f19400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:54.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f194006e140 0x7f19400705f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f194400b0d0 tx=0x7f1944009e30 comp rx=0 tx=0).stop 2024-05-25T06:37:54.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 msgr2=0x7f1954113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:54.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 0x7f1954113900 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f195000eb40 tx=0x7f195000b040 comp rx=0 tx=0).stop 2024-05-25T06:37:54.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 shutdown_connections 2024-05-25T06:37:54.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1954102de0 0x7f1954113900 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:54.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f194006e140 0x7f19400705f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:54.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 --2- 172.21.15.160:0/1224953885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954103780 0x7f1954113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:54.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 >> 172.21.15.160:0/1224953885 conn(0x7f19540f8990 msgr2=0x7f19540f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:37:54.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 shutdown_connections 2024-05-25T06:37:54.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:54.626+0000 7f195c3b9700 1 -- 172.21.15.160:0/1224953885 wait complete. 2024-05-25T06:37:55.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions' 2024-05-25T06:37:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:55 smithi160 ceph-mon[112094]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:37:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:55 smithi160 ceph-mon[112094]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:55.542 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:37:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:55 smithi203 ceph-mon[83507]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-25T06:37:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:55 smithi203 ceph-mon[83507]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:37:57.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 -- 172.21.15.160:0/3357478746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fe760 msgr2=0x7fed10106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:57.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 --2- 172.21.15.160:0/3357478746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fe760 0x7fed10106e30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fed04004940 tx=0x7fed04009e30 comp rx=0 tx=0).stop 2024-05-25T06:37:57.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 -- 172.21.15.160:0/3357478746 shutdown_connections 2024-05-25T06:37:57.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 --2- 172.21.15.160:0/3357478746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fe760 0x7fed10106e30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 --2- 172.21.15.160:0/3357478746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed100fddc0 0x7fed100fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 -- 172.21.15.160:0/3357478746 >> 172.21.15.160:0/3357478746 conn(0x7fed100f7860 msgr2=0x7fed100f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 -- 172.21.15.160:0/3357478746 shutdown_connections 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 -- 172.21.15.160:0/3357478746 wait complete. 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 Processor -- start 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.197+0000 7fed1485e700 1 -- start start 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed1485e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 0x7fed100fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed1485e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed100fe760 0x7fed100fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed1485e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100fdbd0 con 0x7fed100fddc0 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed1485e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100fc700 con 0x7fed100fe760 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 0x7fed100fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 0x7fed100fbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49302/0 (socket says 172.21.15.160:49302) 2024-05-25T06:37:57.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 -- 172.21.15.160:0/2888513414 learned_addr learned my addr 172.21.15.160:0/2888513414 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:37:57.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0dd9b700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed100fe760 0x7fed100fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:57.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 -- 172.21.15.160:0/2888513414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed100fe760 msgr2=0x7fed100fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:57.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed100fe760 0x7fed100fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 -- 172.21.15.160:0/2888513414 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04005040 con 0x7fed100fddc0 2024-05-25T06:37:57.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.198+0000 7fed0e59c700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 0x7fed100fbc80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fed00002a50 tx=0x7fed000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:37:57.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.201+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed00007cc0 con 0x7fed100fddc0 2024-05-25T06:37:57.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.201+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed0000f040 con 0x7fed100fddc0 2024-05-25T06:37:57.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.201+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0000da30 con 0x7fed100fddc0 2024-05-25T06:37:57.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.201+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed100fc9e0 con 0x7fed100fddc0 2024-05-25T06:37:57.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.201+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1011a140 con 0x7fed100fddc0 2024-05-25T06:37:57.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.202+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed10066af0 con 0x7fed100fddc0 2024-05-25T06:37:57.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.202+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fed0000db90 con 0x7fed100fddc0 2024-05-25T06:37:57.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.203+0000 7fecfeffd700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf806e140 0x7fecf80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:57.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.203+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed0008d840 con 0x7fed100fddc0 2024-05-25T06:37:57.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.203+0000 7fed0dd9b700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf806e140 0x7fecf80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:57.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.203+0000 7fed0dd9b700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf806e140 0x7fecf80705f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fed100fd3b0 tx=0x7fed04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:37:57.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.208+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fed00056bf0 con 0x7fed100fddc0 2024-05-25T06:37:57.451 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:57 smithi203 ceph-mon[83507]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:57 smithi160 ceph-mon[112094]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:57.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.493+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fed1011a380 con 0x7fed100fddc0 2024-05-25T06:37:57.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.494+0000 7fecfeffd700 1 -- 172.21.15.160:0/2888513414 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fed0005a2b0 con 0x7fed100fddc0 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "mon": { 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr": { 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 2 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "osd": { 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 6 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:37:57.495 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-25T06:37:57.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 4 2024-05-25T06:37:57.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-25T06:37:57.496 INFO:teuthology.orchestra.run.smithi160.stdout: "overall": { 2024-05-25T06:37:57.496 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)": 14 2024-05-25T06:37:57.496 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-25T06:37:57.496 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf806e140 msgr2=0x7fecf80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf806e140 0x7fecf80705f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fed100fd3b0 tx=0x7fed04009e30 comp rx=0 tx=0).stop 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 msgr2=0x7fed100fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 0x7fed100fbc80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fed00002a50 tx=0x7fed000046b0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 shutdown_connections 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed100fddc0 0x7fed100fbc80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fecf806e140 0x7fecf80705f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 --2- 172.21.15.160:0/2888513414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed100fe760 0x7fed100fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 >> 172.21.15.160:0/2888513414 conn(0x7fed100f7860 msgr2=0x7fed100f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 shutdown_connections 2024-05-25T06:37:57.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:57.496+0000 7fed1485e700 1 -- 172.21.15.160:0/2888513414 wait complete. 2024-05-25T06:37:58.170 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-25T06:37:58.433 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:37:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:58 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2888513414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:37:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:58 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2888513414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:37:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:37:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:37:59 smithi160 ceph-mon[112094]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:59.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:37:59 smithi203 ceph-mon[83507]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:37:59.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 -- 172.21.15.160:0/3867569023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4102de0 msgr2=0x7f73d41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:59.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 --2- 172.21.15.160:0/3867569023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4102de0 0x7f73d41031b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f73c4004940 tx=0x7f73c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 -- 172.21.15.160:0/3867569023 shutdown_connections 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 --2- 172.21.15.160:0/3867569023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4103780 0x7f73d40fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 --2- 172.21.15.160:0/3867569023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4102de0 0x7f73d41031b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 -- 172.21.15.160:0/3867569023 >> 172.21.15.160:0/3867569023 conn(0x7f73d40f89b0 msgr2=0x7f73d40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 -- 172.21.15.160:0/3867569023 shutdown_connections 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.943+0000 7f73da8e3700 1 -- 172.21.15.160:0/3867569023 wait complete. 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73da8e3700 1 Processor -- start 2024-05-25T06:37:59.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73da8e3700 1 -- start start 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73da8e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4102de0 0x7f73d4113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73da8e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 0x7f73d41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73da8e3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4114710 con 0x7f73d4103780 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73da8e3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d410db50 con 0x7f73d4102de0 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 0x7f73d41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73cbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 0x7f73d41140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49310/0 (socket says 172.21.15.160:49310) 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.944+0000 7f73cbfff700 1 -- 172.21.15.160:0/1006372917 learned_addr learned my addr 172.21.15.160:0/1006372917 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73d3fff700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4102de0 0x7f73d4113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73cbfff700 1 -- 172.21.15.160:0/1006372917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4102de0 msgr2=0x7f73d4113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73cbfff700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4102de0 0x7f73d4113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:37:59.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73cbfff700 1 -- 172.21.15.160:0/1006372917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4005040 con 0x7f73d4103780 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73d3fff700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4102de0 0x7f73d4113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73cbfff700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 0x7f73d41140c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c800 tx=0x7f73bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc00a760 con 0x7f73d4103780 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73bc00ad90 con 0x7f73d4103780 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d410de30 con 0x7f73d4103780 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc0113a0 con 0x7f73d4103780 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.945+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d410e3f0 con 0x7f73d4103780 2024-05-25T06:37:59.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.946+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d41000f0 con 0x7f73d4103780 2024-05-25T06:37:59.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.946+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f73bc00a8c0 con 0x7f73d4103780 2024-05-25T06:37:59.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.947+0000 7f73d17fa700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f73c006e140 0x7f73c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:37:59.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.947+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f73bc08dbd0 con 0x7f73d4103780 2024-05-25T06:37:59.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.947+0000 7f73d3fff700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f73c006e140 0x7f73c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:37:59.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.947+0000 7f73d3fff700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f73c006e140 0x7f73c00705f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f73c4004560 tx=0x7f73c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:37:59.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:37:59.952+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f73bc05aab0 con 0x7f73d4103780 2024-05-25T06:38:00.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.239+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f73d410f1d0 con 0x7f73d4103780 2024-05-25T06:38:00.240 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.240+0000 7f73d17fa700 1 -- 172.21.15.160:0/1006372917 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f73bc05a640 con 0x7f73d4103780 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f73c006e140 msgr2=0x7f73c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f73c006e140 0x7f73c00705f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f73c4004560 tx=0x7f73c4009e30 comp rx=0 tx=0).stop 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 msgr2=0x7f73d41140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 0x7f73d41140c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c800 tx=0x7f73bc004470 comp rx=0 tx=0).stop 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 shutdown_connections 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f73c006e140 0x7f73c00705f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73d4102de0 0x7f73d4113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 --2- 172.21.15.160:0/1006372917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d4103780 0x7f73d41140c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 >> 172.21.15.160:0/1006372917 conn(0x7f73d40f89b0 msgr2=0x7f73d4107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 shutdown_connections 2024-05-25T06:38:00.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:00.242+0000 7f73da8e3700 1 -- 172.21.15.160:0/1006372917 wait complete. 2024-05-25T06:38:00.253 INFO:teuthology.orchestra.run.smithi160.stdout:true 2024-05-25T06:38:00.605 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:00 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1006372917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:38:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:00 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1006372917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:38:00.935 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -e sha1=ea0684c08cbcf83f4af9930de2267b3563ac5f78 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-25T06:38:01.198 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:38:01.634 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:01 smithi160 ceph-mon[112094]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:01 smithi203 ceph-mon[83507]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:02.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 -- 172.21.15.160:0/3725820897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 msgr2=0x7f6ca40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:02.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 --2- 172.21.15.160:0/3725820897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40fce10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009ce0 tx=0x7f6c8c004d10 comp rx=0 tx=0).stop 2024-05-25T06:38:02.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 -- 172.21.15.160:0/3725820897 shutdown_connections 2024-05-25T06:38:02.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 --2- 172.21.15.160:0/3725820897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40fce10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:02.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 --2- 172.21.15.160:0/3725820897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca4102e00 0x7f6ca41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:02.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 -- 172.21.15.160:0/3725820897 >> 172.21.15.160:0/3725820897 conn(0x7f6ca40f89b0 msgr2=0x7f6ca40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:02.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 -- 172.21.15.160:0/3725820897 shutdown_connections 2024-05-25T06:38:02.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.841+0000 7f6ca9713700 1 -- 172.21.15.160:0/3725820897 wait complete. 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca9713700 1 Processor -- start 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca9713700 1 -- start start 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca9713700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca4102e00 0x7f6ca4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca9713700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca9713700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40721b0 con 0x7f6ca41037a0 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca9713700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4070ce0 con 0x7f6ca4102e00 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49322/0 (socket says 172.21.15.160:49322) 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca27fc700 1 -- 172.21.15.160:0/636195079 learned_addr learned my addr 172.21.15.160:0/636195079 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:02.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.842+0000 7f6ca2ffd700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca4102e00 0x7f6ca4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6ca27fc700 1 -- 172.21.15.160:0/636195079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca4102e00 msgr2=0x7f6ca4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6ca27fc700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca4102e00 0x7f6ca4070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6ca27fc700 1 -- 172.21.15.160:0/636195079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c009940 con 0x7f6ca41037a0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6ca27fc700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40707a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00be40 tx=0x7f6c8c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8c00e040 con 0x7f6ca41037a0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c8c004610 con 0x7f6ca41037a0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca4070f60 con 0x7f6ca41037a0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8c012650 con 0x7f6ca41037a0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.843+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca411a280 con 0x7f6ca41037a0 2024-05-25T06:38:02.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.844+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca41052d0 con 0x7f6ca41037a0 2024-05-25T06:38:02.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.845+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6c8c007d10 con 0x7f6ca41037a0 2024-05-25T06:38:02.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.845+0000 7f6c9b7fe700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6c9006e140 0x7f6c900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:02.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.845+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6c8c08f770 con 0x7f6ca41037a0 2024-05-25T06:38:02.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.845+0000 7f6ca2ffd700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6c9006e140 0x7f6c900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:02.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.845+0000 7f6ca2ffd700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6c9006e140 0x7f6c900705f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c94004f40 tx=0x7f6c9400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:02.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:02.850+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6c8c058aa0 con 0x7f6ca41037a0 2024-05-25T06:38:03.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.133+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ca4071740 con 0x7f6ca41037a0 2024-05-25T06:38:03.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.133+0000 7f6c9b7fe700 1 -- 172.21.15.160:0/636195079 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f6c8c004e10 con 0x7f6ca41037a0 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6c9006e140 msgr2=0x7f6c900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6c9006e140 0x7f6c900705f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c94004f40 tx=0x7f6c9400b040 comp rx=0 tx=0).stop 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 msgr2=0x7f6ca40707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40707a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00be40 tx=0x7f6c8c004d10 comp rx=0 tx=0).stop 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 shutdown_connections 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f6c9006e140 0x7f6c900705f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca4102e00 0x7f6ca4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 --2- 172.21.15.160:0/636195079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca41037a0 0x7f6ca40707a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 >> 172.21.15.160:0/636195079 conn(0x7f6ca40f89b0 msgr2=0x7f6ca4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 shutdown_connections 2024-05-25T06:38:03.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:03.136+0000 7f6ca9713700 1 -- 172.21.15.160:0/636195079 wait complete. 2024-05-25T06:38:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph version 19.0.0-2843-gea0684c0 (ea0684c08cbcf83f4af9930de2267b3563ac5f78) squid (rc)" 2024-05-25T06:38:03.462 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:03 smithi160 ceph-mon[112094]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:03.462 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:03 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/636195079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:38:03.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:03 smithi203 ceph-mon[83507]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:03.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:03 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/636195079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-25T06:38:03.797 DEBUG:teuthology.parallel:result is None 2024-05-25T06:38:03.797 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-25T06:38:03.808 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:38:03.809 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-25T06:38:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/mon.smithi160/config 2024-05-25T06:38:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:05 smithi160 ceph-mon[112094]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:05 smithi203 ceph-mon[83507]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:05.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 -- 172.21.15.160:0/3826629848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8102680 msgr2=0x7f61f80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:05.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 --2- 172.21.15.160:0/3826629848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8102680 0x7f61f80fbcf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f61e0004940 tx=0x7f61e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:38:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 -- 172.21.15.160:0/3826629848 shutdown_connections 2024-05-25T06:38:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 --2- 172.21.15.160:0/3826629848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8102680 0x7f61f80fbcf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 --2- 172.21.15.160:0/3826629848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8101ce0 0x7f61f81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 -- 172.21.15.160:0/3826629848 >> 172.21.15.160:0/3826629848 conn(0x7f61f80f7850 msgr2=0x7f61f80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 -- 172.21.15.160:0/3826629848 shutdown_connections 2024-05-25T06:38:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.707+0000 7f61fd695700 1 -- 172.21.15.160:0/3826629848 wait complete. 2024-05-25T06:38:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61fd695700 1 Processor -- start 2024-05-25T06:38:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61fd695700 1 -- start start 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61fd695700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 0x7f61f8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61fd695700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8102680 0x7f61f81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61fd695700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81147a0 con 0x7f61f8101ce0 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61fd695700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f810db00 con 0x7f61f8102680 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 0x7f61f8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 0x7f61f8113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40900/0 (socket says 172.21.15.160:40900) 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.708+0000 7f61f6ffd700 1 -- 172.21.15.160:0/326218143 learned_addr learned my addr 172.21.15.160:0/326218143 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61f6ffd700 1 -- 172.21.15.160:0/326218143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8102680 msgr2=0x7f61f81140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61f67fc700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8102680 0x7f61f81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61f6ffd700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8102680 0x7f61f81140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61f6ffd700 1 -- 172.21.15.160:0/326218143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0005040 con 0x7f61f8101ce0 2024-05-25T06:38:05.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61f67fc700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8102680 0x7f61f81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61f6ffd700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 0x7f61f8113b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f61e800b790 tx=0x7f61e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e800f770 con 0x7f61f8101ce0 2024-05-25T06:38:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61e800fda0 con 0x7f61f8101ce0 2024-05-25T06:38:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f810dde0 con 0x7f61f8101ce0 2024-05-25T06:38:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e800d310 con 0x7f61f8101ce0 2024-05-25T06:38:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.709+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f810e3a0 con 0x7f61f8101ce0 2024-05-25T06:38:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.710+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f80fd430 con 0x7f61f8101ce0 2024-05-25T06:38:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.711+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f61e800f8d0 con 0x7f61f8101ce0 2024-05-25T06:38:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.711+0000 7f61ef7fe700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61e406e140 0x7f61e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.711+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f61e808dab0 con 0x7f61f8101ce0 2024-05-25T06:38:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.711+0000 7f61f67fc700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61e406e140 0x7f61e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.712+0000 7f61f67fc700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61e406e140 0x7f61e40705f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61e000b3b0 tx=0x7f61e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:05.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.716+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f61e805aa50 con 0x7f61f8101ce0 2024-05-25T06:38:05.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.955+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f61f804bed0 con 0x7f61f8101ce0 2024-05-25T06:38:05.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.955+0000 7f61ef7fe700 1 -- 172.21.15.160:0/326218143 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 43 v43) v1 ==== 76+0+1973 (secure 0 0 0) 0x7f61e805a5e0 con 0x7f61f8101ce0 2024-05-25T06:38:05.955 INFO:teuthology.orchestra.run.smithi160.stdout:e43 2024-05-25T06:38:05.955 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-25T06:38:05.955 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-25T06:38:05.955 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 43 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-25T06:08:21.960553+0000 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-05-25T06:36:23.081091+0000 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-05-25T06:38:05.956 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 60 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 81 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 2 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=34388,1=34416} 2024-05-25T06:38:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 1 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 34388 members: 34388,34416 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.rrhzig{0:34388} state up:active seq 244 join_fscid=1 addr [v2:172.21.15.160:6826/1526023017,v1:172.21.15.160:6827/1526023017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.yltymd{0:44331} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.203:6824/1855238749,v1:172.21.15.203:6825/1855238749] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi160.tlctof{1:34416} state up:active seq 239 join_fscid=1 addr [v2:172.21.15.160:6828/2801259285,v1:172.21.15.160:6829/2801259285] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.cephfs.smithi203.tndnrb{1:44335} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.203:6826/370625010,v1:172.21.15.203:6827/370625010] compat {c=[1],r=[1],i=[1fff]}] 2024-05-25T06:38:05.958 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:05.959 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:05.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61e406e140 msgr2=0x7f61e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:05.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61e406e140 0x7f61e40705f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61e000b3b0 tx=0x7f61e0009e30 comp rx=0 tx=0).stop 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 msgr2=0x7f61f8113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 0x7f61f8113b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f61e800b790 tx=0x7f61e8009250 comp rx=0 tx=0).stop 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 shutdown_connections 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61f8101ce0 0x7f61f8113b80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61e406e140 0x7f61e40705f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 --2- 172.21.15.160:0/326218143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f8102680 0x7f61f81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 >> 172.21.15.160:0/326218143 conn(0x7f61f80f7850 msgr2=0x7f61f80f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 shutdown_connections 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:05.957+0000 7f61fd695700 1 -- 172.21.15.160:0/326218143 wait complete. 2024-05-25T06:38:05.960 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 43 2024-05-25T06:38:06.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:06 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/326218143' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:38:06.610 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-25T06:38:06.621 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-25T06:38:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:06 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/326218143' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 -- 172.21.15.160:0/364860962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 msgr2=0x7f08c01025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/364860962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c01025c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009c50 tx=0x7f08b0004d10 comp rx=0 tx=0).stop 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 -- 172.21.15.160:0/364860962 shutdown_connections 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/364860962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08c0102b90 0x7f08c0064350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/364860962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c01025c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 -- 172.21.15.160:0/364860962 >> 172.21.15.160:0/364860962 conn(0x7f08c00711c0 msgr2=0x7f08c00715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 -- 172.21.15.160:0/364860962 shutdown_connections 2024-05-25T06:38:06.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 -- 172.21.15.160:0/364860962 wait complete. 2024-05-25T06:38:06.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 Processor -- start 2024-05-25T06:38:06.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.720+0000 7f08c5a2e640 1 -- start start 2024-05-25T06:38:06.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08c5a2e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c0112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:06.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08c5a2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08c0102b90 0x7f08c01132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:06.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08c5a2e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01138a0 con 0x7f08c01021f0 2024-05-25T06:38:06.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08c5a2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c010cef0 con 0x7f08c0102b90 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c0112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c0112d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40910/0 (socket says 172.21.15.160:40910) 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 -- 172.21.15.160:0/2560544735 learned_addr learned my addr 172.21.15.160:0/2560544735 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08be7fc640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08c0102b90 0x7f08c01132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 -- 172.21.15.160:0/2560544735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08c0102b90 msgr2=0x7f08c01132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08c0102b90 0x7f08c01132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 -- 172.21.15.160:0/2560544735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b00098b0 con 0x7f08c01021f0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08beffd640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c0112d60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004a30 tx=0x7f08b0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08b003a070 con 0x7f08c01021f0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c010d170 con 0x7f08c01021f0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08b0035b20 con 0x7f08c01021f0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.721+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08b003ea70 con 0x7f08c01021f0 2024-05-25T06:38:06.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.722+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c010d690 con 0x7f08c01021f0 2024-05-25T06:38:06.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.723+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c00fd990 con 0x7f08c01021f0 2024-05-25T06:38:06.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.723+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f08b0032070 con 0x7f08c01021f0 2024-05-25T06:38:06.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.723+0000 7f08a77fe640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f08a806e0b0 0x7f08a8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:06.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.724+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f08b00afc50 con 0x7f08c01021f0 2024-05-25T06:38:06.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.724+0000 7f08be7fc640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f08a806e0b0 0x7f08a8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:06.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.724+0000 7f08be7fc640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f08a806e0b0 0x7f08a8070560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f08c010e620 tx=0x7f08b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:06.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.728+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f08b0079030 con 0x7f08c01021f0 2024-05-25T06:38:06.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.906+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08c010e3d0 con 0x7f08c01021f0 2024-05-25T06:38:06.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.908+0000 7f08a77fe640 1 -- 172.21.15.160:0/2560544735 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16719 (secure 0 0 0) 0x7f08b007c6f0 con 0x7f08c01021f0 2024-05-25T06:38:06.908 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:06.908 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":85,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:21:08.124903+0000","last_up_change":"2024-05-25T06:20:08.969929+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-25T06:08:20.895353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-25T06:08:21.186881+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6803","nonce":793926185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6805","nonce":793926185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6809","nonce":793926185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6807","nonce":793926185}]},"public_addr":"172.21.15.160:6803/793926185","cluster_addr":"172.21.15.160:6805/793926185","heartbeat_back_addr":"172.21.15.160:6809/793926185","heartbeat_front_addr":"172.21.15.160:6807/793926185","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6811","nonce":3648042119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6813","nonce":3648042119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6817","nonce":3648042119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6815","nonce":3648042119}]},"public_addr":"172.21.15.160:6811/3648042119","cluster_addr":"172.21.15.160:6813/3648042119","heartbeat_back_addr":"172.21.15.160:6817/3648042119","heartbeat_front_addr":"172.21.15.160:6815/3648042119","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6819","nonce":1307550666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6821","nonce":1307550666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6825","nonce":1307550666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6823","nonce":1307550666}]},"public_addr":"172.21.15.160:6819/1307550666","cluster_addr":"172.21.15.160:6821/1307550666","heartbeat_back_addr":"172.21.15.160:6825/1307550666","heartbeat_front_addr":"172.21.15.160:6823/1307550666","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6801","nonce":316097287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6803","nonce":316097287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6807","nonce":316097287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6805","nonce":316097287}]},"public_addr":"172.21.15.203:6801/316097287","cluster_addr":"172.21.15.203:6803/316097287","heartbeat_back_addr":"172.21.15.203:6807/316097287","heartbeat_front_addr":"172.21.15.203:6805/316097287","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6809","nonce":2420151889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6811","nonce":2420151889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6815","nonce":2420151889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6813","nonce":2420151889}]},"public_addr":"172.21.15.203:6809/2420151889","cluster_addr":"172.21.15.203:6811/2420151889","heartbeat_back_addr":"172.21.15.203:6815/2420151889","heartbeat_front_addr":"172.21.15.203:6813/2420151889","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6817","nonce":2019898829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6819","nonce":2019898829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6823","nonce":2019898829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6821","nonce":2019898829}]},"public_addr":"172.21.15.203:6817/2019898829","cluster_addr":"172.21.15.203:6819/2019898829","heartbeat_back_addr":"172.21.15.203:6823/2019898829","heartbeat_front_addr":"172.21.15.203:6821/2019898829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6829/3783773637":"2024-05-26T06:20:11.153443+0000","172.21.15.160:6828/3783773637":"2024-05-26T06:20:11.153443+0000","172.21.15.160:6800/4089125744":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000","172.21.15.203:6824/3325193142":"2024-05-26T06:20:11.153460+0000","172.21.15.203:0/892364096":"2024-05-26T06:12:42.594272+0000","172.21.15.160:6826/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/2473068109":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.203:6825/3325193142":"2024-05-26T06:20:11.153460+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.203:0/3131222858":"2024-05-26T06:12:42.594272+0000","172.21.15.203:6828/2357665398":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/2019174179":"2024-05-26T06:11:04.538090+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1884499665":"2024-05-26T06:15:42.839541+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.203:0/2025926580":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/1559445112":"2024-05-26T06:15:42.839541+0000","172.21.15.160:6801/4089125744":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1251129035":"2024-05-26T06:11:04.538090+0000","172.21.15.203:0/1389978098":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1144786189":"2024-05-26T06:15:42.839541+0000","172.21.15.203:6829/2357665398":"2024-05-26T06:12:42.594272+0000","172.21.15.160:6827/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6800/1875516529":"2024-05-26T06:15:42.839541+0000","172.21.15.160:6801/1875516529":"2024-05-26T06:15:42.839541+0000","172.21.15.160:0/3623795045":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/3363261834":"2024-05-26T06:15:42.839541+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f08a806e0b0 msgr2=0x7f08a8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f08a806e0b0 0x7f08a8070560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f08c010e620 tx=0x7f08b400b040 comp rx=0 tx=0).stop 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 msgr2=0x7f08c0112d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c0112d60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004a30 tx=0x7f08b0004d10 comp rx=0 tx=0).stop 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 shutdown_connections 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f08a806e0b0 0x7f08a8070560 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08c0102b90 0x7f08c01132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 --2- 172.21.15.160:0/2560544735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08c01021f0 0x7f08c0112d60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 >> 172.21.15.160:0/2560544735 conn(0x7f08c00711c0 msgr2=0x7f08c00fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:06.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 shutdown_connections 2024-05-25T06:38:06.911 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:06.910+0000 7f08c5a2e640 1 -- 172.21.15.160:0/2560544735 wait complete. 2024-05-25T06:38:06.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-25T06:05:48.910750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-25T06:08:20.895353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-25T06:08:21.186881+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-25T06:38:06.922 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 -- 172.21.15.160:0/1284803982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88064380 msgr2=0x7f9a880647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/1284803982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88064380 0x7f9a880647b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9a780048d0 tx=0x7f9a78009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 -- 172.21.15.160:0/1284803982 shutdown_connections 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/1284803982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88064380 0x7f9a880647b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/1284803982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a88102210 0x7f9a881025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 -- 172.21.15.160:0/1284803982 >> 172.21.15.160:0/1284803982 conn(0x7f9a880710c0 msgr2=0x7f9a880714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 -- 172.21.15.160:0/1284803982 shutdown_connections 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.021+0000 7f9a8f26e640 1 -- 172.21.15.160:0/1284803982 wait complete. 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8f26e640 1 Processor -- start 2024-05-25T06:38:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8f26e640 1 -- start start 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8f26e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a88064380 0x7f9a88112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8f26e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 0x7f9a881131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8f26e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a881137f0 con 0x7f9a88102210 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8f26e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8810ce40 con 0x7f9a88064380 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 0x7f9a881131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 0x7f9a881131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40916/0 (socket says 172.21.15.160:40916) 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 -- 172.21.15.160:0/3242175697 learned_addr learned my addr 172.21.15.160:0/3242175697 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8d00a640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a88064380 0x7f9a88112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 -- 172.21.15.160:0/3242175697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a88064380 msgr2=0x7f9a88112cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a88064380 0x7f9a88112cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 -- 172.21.15.160:0/3242175697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78005040 con 0x7f9a88102210 2024-05-25T06:38:07.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a8c809640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 0x7f9a881131f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a78004930 tx=0x7f9a78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.022+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a780094c0 con 0x7f9a88102210 2024-05-25T06:38:07.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.023+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a78032e50 con 0x7f9a88102210 2024-05-25T06:38:07.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.023+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8810d0c0 con 0x7f9a88102210 2024-05-25T06:38:07.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.023+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a780433b0 con 0x7f9a88102210 2024-05-25T06:38:07.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.023+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8810d580 con 0x7f9a88102210 2024-05-25T06:38:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.024+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a880fd910 con 0x7f9a88102210 2024-05-25T06:38:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.024+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9a7802f070 con 0x7f9a88102210 2024-05-25T06:38:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.025+0000 7f9a7dffb640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9a7406e0b0 0x7f9a74070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.025+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a780afae0 con 0x7f9a88102210 2024-05-25T06:38:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.025+0000 7f9a8d00a640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9a7406e0b0 0x7f9a74070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.025+0000 7f9a8d00a640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9a7406e0b0 0x7f9a74070560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a840041d0 tx=0x7f9a84009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.029+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9a78078e10 con 0x7f9a88102210 2024-05-25T06:38:07.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.206+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a881092f0 con 0x7f9a88102210 2024-05-25T06:38:07.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.206+0000 7f9a7dffb640 1 -- 172.21.15.160:0/3242175697 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a7807c4d0 con 0x7f9a88102210 2024-05-25T06:38:07.207 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.208+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9a7406e0b0 msgr2=0x7f9a74070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.208+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9a7406e0b0 0x7f9a74070560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a840041d0 tx=0x7f9a84009040 comp rx=0 tx=0).stop 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.208+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 msgr2=0x7f9a881131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.208+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 0x7f9a881131f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a78004930 tx=0x7f9a78009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 shutdown_connections 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9a7406e0b0 0x7f9a74070560 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a88102210 0x7f9a881131f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 --2- 172.21.15.160:0/3242175697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a88064380 0x7f9a88112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 >> 172.21.15.160:0/3242175697 conn(0x7f9a880710c0 msgr2=0x7f9a880f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 shutdown_connections 2024-05-25T06:38:07.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.209+0000 7f9a8f26e640 1 -- 172.21.15.160:0/3242175697 wait complete. 2024-05-25T06:38:07.220 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-25T06:38:07.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- 172.21.15.160:0/1100892162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 msgr2=0x7feaa4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 --2- 172.21.15.160:0/1100892162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4102560 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fea940048d0 tx=0x7fea94009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- 172.21.15.160:0/1100892162 shutdown_connections 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 --2- 172.21.15.160:0/1100892162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 0x7feaa40fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 --2- 172.21.15.160:0/1100892162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4102560 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- 172.21.15.160:0/1100892162 >> 172.21.15.160:0/1100892162 conn(0x7feaa40f7da0 msgr2=0x7feaa40fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- 172.21.15.160:0/1100892162 shutdown_connections 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- 172.21.15.160:0/1100892162 wait complete. 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 Processor -- start 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- start start 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 0x7feaa4113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa4113800 con 0x7feaa4102190 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.320+0000 7feaaa95d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa410ce50 con 0x7feaa4102b30 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40928/0 (socket says 172.21.15.160:40928) 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 -- 172.21.15.160:0/3228056080 learned_addr learned my addr 172.21.15.160:0/3228056080 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7fea9bfff640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 0x7feaa4113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 -- 172.21.15.160:0/3228056080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 msgr2=0x7feaa4113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 0x7feaa4113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 -- 172.21.15.160:0/3228056080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94005040 con 0x7feaa4102190 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7fea9bfff640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 0x7feaa4113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa3fff640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4112cc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fea940047f0 tx=0x7fea94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea940094c0 con 0x7feaa4102190 2024-05-25T06:38:07.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa410d0d0 con 0x7feaa4102190 2024-05-25T06:38:07.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea94032e50 con 0x7feaa4102190 2024-05-25T06:38:07.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea940435f0 con 0x7feaa4102190 2024-05-25T06:38:07.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.321+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa410d590 con 0x7feaa4102190 2024-05-25T06:38:07.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.322+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa40fd900 con 0x7feaa4102190 2024-05-25T06:38:07.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.323+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fea9402f070 con 0x7feaa4102190 2024-05-25T06:38:07.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.323+0000 7feaa17fa640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea9006e0b0 0x7fea90070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.323+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fea940afac0 con 0x7feaa4102190 2024-05-25T06:38:07.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.323+0000 7fea9bfff640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea9006e0b0 0x7fea90070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.324+0000 7fea9bfff640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea9006e0b0 0x7fea90070560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feaa410e580 tx=0x7fea8c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.328+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fea94078df0 con 0x7feaa4102190 2024-05-25T06:38:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:07 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2560544735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:38:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:07 smithi160 ceph-mon[112094]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:07.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.505+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feaa410e330 con 0x7feaa4102190 2024-05-25T06:38:07.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.505+0000 7feaa17fa640 1 -- 172.21.15.160:0/3228056080 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fea9407c4b0 con 0x7feaa4102190 2024-05-25T06:38:07.505 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:38:07.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea9006e0b0 msgr2=0x7fea90070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea9006e0b0 0x7fea90070560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feaa410e580 tx=0x7fea8c009040 comp rx=0 tx=0).stop 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 msgr2=0x7feaa4112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4112cc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fea940047f0 tx=0x7fea94009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 shutdown_connections 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fea9006e0b0 0x7fea90070560 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4102b30 0x7feaa4113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 --2- 172.21.15.160:0/3228056080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaa4102190 0x7feaa4112cc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 >> 172.21.15.160:0/3228056080 conn(0x7feaa40f7da0 msgr2=0x7feaa40f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 shutdown_connections 2024-05-25T06:38:07.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.508+0000 7feaaa95d640 1 -- 172.21.15.160:0/3228056080 wait complete. 2024-05-25T06:38:07.519 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.618+0000 7f6683814640 1 -- 172.21.15.160:0/2773095915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102180 msgr2=0x7f667c102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.618+0000 7f6683814640 1 --2- 172.21.15.160:0/2773095915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102180 0x7f667c102550 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f66780048d0 tx=0x7f6678009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- 172.21.15.160:0/2773095915 shutdown_connections 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 --2- 172.21.15.160:0/2773095915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667c102b20 0x7f667c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 --2- 172.21.15.160:0/2773095915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102180 0x7f667c102550 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- 172.21.15.160:0/2773095915 >> 172.21.15.160:0/2773095915 conn(0x7f667c0f7db0 msgr2=0x7f667c0fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- 172.21.15.160:0/2773095915 shutdown_connections 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- 172.21.15.160:0/2773095915 wait complete. 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 Processor -- start 2024-05-25T06:38:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- start start 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667c102180 0x7f667c112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 0x7f667c113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c113800 con 0x7f667c102b20 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.619+0000 7f6683814640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c10ce50 con 0x7f667c102180 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 0x7f667c113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 0x7f667c113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40954/0 (socket says 172.21.15.160:40954) 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 -- 172.21.15.160:0/90340821 learned_addr learned my addr 172.21.15.160:0/90340821 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f66815b0640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667c102180 0x7f667c112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 -- 172.21.15.160:0/90340821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667c102180 msgr2=0x7f667c112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667c102180 0x7f667c112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 -- 172.21.15.160:0/90340821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678005040 con 0x7f667c102b20 2024-05-25T06:38:07.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6680daf640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 0x7f667c113200 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f666c002a50 tx=0x7f666c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c007c30 con 0x7f667c102b20 2024-05-25T06:38:07.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f666c00f040 con 0x7f667c102b20 2024-05-25T06:38:07.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c10d130 con 0x7f667c102b20 2024-05-25T06:38:07.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c00db50 con 0x7f667c102b20 2024-05-25T06:38:07.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.620+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c10d650 con 0x7f667c102b20 2024-05-25T06:38:07.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.621+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c0fd8f0 con 0x7f667c102b20 2024-05-25T06:38:07.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.622+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f666c01c400 con 0x7f667c102b20 2024-05-25T06:38:07.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.622+0000 7f6671ffb640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f666806e0b0 0x7f6668070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.622+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f666c08d3c0 con 0x7f667c102b20 2024-05-25T06:38:07.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.622+0000 7f66815b0640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f666806e0b0 0x7f6668070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.623+0000 7f66815b0640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f666806e0b0 0x7f6668070560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6678003080 tx=0x7f6678009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.627+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f666c0567a0 con 0x7f667c102b20 2024-05-25T06:38:07.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:07 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2560544735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:38:07.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:07 smithi203 ceph-mon[83507]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:07.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.803+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f667c1092e0 con 0x7f667c102b20 2024-05-25T06:38:07.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.803+0000 7f6671ffb640 1 -- 172.21.15.160:0/90340821 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f666c059e60 con 0x7f667c102b20 2024-05-25T06:38:07.803 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:38:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f666806e0b0 msgr2=0x7f6668070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f666806e0b0 0x7f6668070560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6678003080 tx=0x7f6678009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 msgr2=0x7f667c113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 0x7f667c113200 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f666c002a50 tx=0x7f666c004600 comp rx=0 tx=0).stop 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 shutdown_connections 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f666806e0b0 0x7f6668070560 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f667c102b20 0x7f667c113200 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 --2- 172.21.15.160:0/90340821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667c102180 0x7f667c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.805+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 >> 172.21.15.160:0/90340821 conn(0x7f667c0f7db0 msgr2=0x7f667c0f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.806+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 shutdown_connections 2024-05-25T06:38:07.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.806+0000 7f6683814640 1 -- 172.21.15.160:0/90340821 wait complete. 2024-05-25T06:38:07.816 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-25T06:38:07.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 -- 172.21.15.160:0/955146667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fd1a0 msgr2=0x7f04680fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 --2- 172.21.15.160:0/955146667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fd1a0 0x7f04680fd570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0464004900 tx=0x7f0464009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 -- 172.21.15.160:0/955146667 shutdown_connections 2024-05-25T06:38:07.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 --2- 172.21.15.160:0/955146667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04680fdb40 0x7f0468106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 --2- 172.21.15.160:0/955146667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fd1a0 0x7f04680fd570 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 -- 172.21.15.160:0/955146667 >> 172.21.15.160:0/955146667 conn(0x7f04680f6c20 msgr2=0x7f04680f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.914+0000 7f046f59d640 1 -- 172.21.15.160:0/955146667 shutdown_connections 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 -- 172.21.15.160:0/955146667 wait complete. 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 Processor -- start 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 -- start start 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04680fd1a0 0x7f0468112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 0x7f04681132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04681138a0 con 0x7f04680fdb40 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046f59d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046810cef0 con 0x7f04680fd1a0 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046cb38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 0x7f04681132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046cb38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 0x7f04681132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40974/0 (socket says 172.21.15.160:40974) 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046cb38640 1 -- 172.21.15.160:0/1659596537 learned_addr learned my addr 172.21.15.160:0/1659596537 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:07.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.915+0000 7f046d339640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04680fd1a0 0x7f0468112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f046cb38640 1 -- 172.21.15.160:0/1659596537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04680fd1a0 msgr2=0x7f0468112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f046cb38640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04680fd1a0 0x7f0468112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f046cb38640 1 -- 172.21.15.160:0/1659596537 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0464005040 con 0x7f04680fdb40 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f046cb38640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 0x7f04681132a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0458002a50 tx=0x7f0458004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0458007c30 con 0x7f04680fdb40 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f045800f040 con 0x7f04680fdb40 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046810d1d0 con 0x7f04680fdb40 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045800db60 con 0x7f04680fdb40 2024-05-25T06:38:07.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.916+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046810d6f0 con 0x7f04680fdb40 2024-05-25T06:38:07.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.917+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0468065d50 con 0x7f04680fdb40 2024-05-25T06:38:07.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.918+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f045800dcc0 con 0x7f04680fdb40 2024-05-25T06:38:07.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.918+0000 7f045dffb640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f045406e0b0 0x7f0454070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:07.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.918+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f045808d960 con 0x7f04680fdb40 2024-05-25T06:38:07.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.918+0000 7f046d339640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f045406e0b0 0x7f0454070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:07.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.918+0000 7f046d339640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f045406e0b0 0x7f0454070560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f0464009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:07.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:07.923+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0458056d10 con 0x7f04680fdb40 2024-05-25T06:38:08.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.139+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f046804bcc0 con 0x7f04680fdb40 2024-05-25T06:38:08.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.140+0000 7f045dffb640 1 -- 172.21.15.160:0/1659596537 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5257 (secure 0 0 0) 0x7f045805a3d0 con 0x7f04680fdb40 2024-05-25T06:38:08.140 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:08.140 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:36:23.081091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":38,"state":"up:active","state_seq":244,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34416":{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":1,"incarnation":38,"state":"up:active","state_seq":239,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44331":{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44335":{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34388,34416]},"id":1}]} 2024-05-25T06:38:08.140 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 43 2024-05-25T06:38:08.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.142+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f045406e0b0 msgr2=0x7f0454070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.142+0000 7f046f59d640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f045406e0b0 0x7f0454070560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f0464009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.142+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 msgr2=0x7f04681132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.142+0000 7f046f59d640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 0x7f04681132a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0458002a50 tx=0x7f0458004600 comp rx=0 tx=0).stop 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 shutdown_connections 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f045406e0b0 0x7f0454070560 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04680fdb40 0x7f04681132a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 --2- 172.21.15.160:0/1659596537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04680fd1a0 0x7f0468112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 >> 172.21.15.160:0/1659596537 conn(0x7f04680f6c20 msgr2=0x7f04680f8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 shutdown_connections 2024-05-25T06:38:08.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.143+0000 7f046f59d640 1 -- 172.21.15.160:0/1659596537 wait complete. 2024-05-25T06:38:08.153 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 22, 'max_mds': 2, 'flags': 50} 2024-05-25T06:38:08.154 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.254+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/2874244944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 msgr2=0x7ff7a40fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.254+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/2874244944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a40fb5c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff7980048d0 tx=0x7ff798009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.254+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/2874244944 shutdown_connections 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.254+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/2874244944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a40fb5c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.254+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/2874244944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101170 0x7ff7a4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.254+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/2874244944 >> 172.21.15.160:0/2874244944 conn(0x7ff7a4071fd0 msgr2=0x7ff7a40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/2874244944 shutdown_connections 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/2874244944 wait complete. 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 Processor -- start 2024-05-25T06:38:08.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 -- start start 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a4114f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101170 0x7ff7a4115460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4115af0 con 0x7ff7a40fb150 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a8d5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410f0b0 con 0x7ff7a4101170 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a4114f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a4114f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40996/0 (socket says 172.21.15.160:40996) 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.255+0000 7ff7a259c640 1 -- 172.21.15.160:0/1480320569 learned_addr learned my addr 172.21.15.160:0/1480320569 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a1d9b640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101170 0x7ff7a4115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a259c640 1 -- 172.21.15.160:0/1480320569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101170 msgr2=0x7ff7a4115460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a259c640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101170 0x7ff7a4115460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a259c640 1 -- 172.21.15.160:0/1480320569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798005040 con 0x7ff7a40fb150 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a259c640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a4114f20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b790 tx=0x7ff7940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:08.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79400c700 con 0x7ff7a40fb150 2024-05-25T06:38:08.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a410f390 con 0x7ff7a40fb150 2024-05-25T06:38:08.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff79400cd30 con 0x7ff7a40fb150 2024-05-25T06:38:08.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff794013370 con 0x7ff7a40fb150 2024-05-25T06:38:08.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.256+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a410f880 con 0x7ff7a40fb150 2024-05-25T06:38:08.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.257+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a40fc870 con 0x7ff7a40fb150 2024-05-25T06:38:08.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.258+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff794012070 con 0x7ff7a40fb150 2024-05-25T06:38:08.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.258+0000 7ff792ffd640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff78c06e0b0 0x7ff78c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.258+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff79408d5c0 con 0x7ff7a40fb150 2024-05-25T06:38:08.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.258+0000 7ff7a1d9b640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff78c06e0b0 0x7ff78c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.259+0000 7ff7a1d9b640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff78c06e0b0 0x7ff78c070560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40f93f0 tx=0x7ff798009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:08.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.263+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff7940568f0 con 0x7ff7a40fb150 2024-05-25T06:38:08.421 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3242175697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:38:08.422 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3228056080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:38:08.422 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/90340821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:38:08.422 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:08 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1659596537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-25T06:38:08.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.482+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7ff7a4063f00 con 0x7ff7a40fb150 2024-05-25T06:38:08.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.482+0000 7ff792ffd640 1 -- 172.21.15.160:0/1480320569 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v43) v1 ==== 107+0+4901 (secure 0 0 0) 0x7ff7a4063f00 con 0x7ff7a40fb150 2024-05-25T06:38:08.482 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:08.482 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:10:37.388089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi160.tlctof","rank":0,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6829/3783773637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3783773637},{"type":"v1","addr":"172.21.15.160:6829","nonce":3783773637}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6827/2201510143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2201510143},{"type":"v1","addr":"172.21.15.160:6827","nonce":2201510143}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi203.yltymd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6825/3325193142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3325193142},{"type":"v1","addr":"172.21.15.203:6825","nonce":3325193142}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24283":{"gid":24283,"name":"cephfs.smithi203.tndnrb","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.203:6827/1913747634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1913747634},{"type":"v1","addr":"172.21.15.203:6827","nonce":1913747634}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:08.483 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 23 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff78c06e0b0 msgr2=0x7ff78c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff78c06e0b0 0x7ff78c070560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40f93f0 tx=0x7ff798009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 msgr2=0x7ff7a4114f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a4114f20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b790 tx=0x7ff7940091d0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 shutdown_connections 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff78c06e0b0 0x7ff78c070560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101170 0x7ff7a4115460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 --2- 172.21.15.160:0/1480320569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7a40fb150 0x7ff7a4114f20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 >> 172.21.15.160:0/1480320569 conn(0x7ff7a4071fd0 msgr2=0x7ff7a4064690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 shutdown_connections 2024-05-25T06:38:08.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.485+0000 7ff7a8d5d640 1 -- 172.21.15.160:0/1480320569 wait complete. 2024-05-25T06:38:08.496 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-05-25T06:38:08.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 -- 172.21.15.160:0/3145233013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 msgr2=0x7f9ba8102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3145233013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8102640 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9b980048d0 tx=0x7f9b98009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 -- 172.21.15.160:0/3145233013 shutdown_connections 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3145233013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ba8064390 0x7f9ba8064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3145233013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8102640 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 -- 172.21.15.160:0/3145233013 >> 172.21.15.160:0/3145233013 conn(0x7f9ba8071fd0 msgr2=0x7f9ba80723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 -- 172.21.15.160:0/3145233013 shutdown_connections 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 -- 172.21.15.160:0/3145233013 wait complete. 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.597+0000 7f9badf9d640 1 Processor -- start 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9badf9d640 1 -- start start 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9badf9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ba8064390 0x7f9ba8112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9badf9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9badf9d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8113770 con 0x7f9ba8102270 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9badf9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba810cdc0 con 0x7f9ba8064390 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41008/0 (socket says 172.21.15.160:41008) 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 -- 172.21.15.160:0/3934706092 learned_addr learned my addr 172.21.15.160:0/3934706092 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba77fe640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ba8064390 0x7f9ba8112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 -- 172.21.15.160:0/3934706092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ba8064390 msgr2=0x7f9ba8112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ba8064390 0x7f9ba8112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 -- 172.21.15.160:0/3934706092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98005040 con 0x7f9ba8102270 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.598+0000 7f9ba6ffd640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8113170 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c002a50 tx=0x7f9b9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.599+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b9c007c30 con 0x7f9ba8102270 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.599+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba810d0a0 con 0x7f9ba8102270 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.599+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b9c00f040 con 0x7f9ba8102270 2024-05-25T06:38:08.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.599+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b9c00db50 con 0x7f9ba8102270 2024-05-25T06:38:08.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.599+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba810d5c0 con 0x7f9ba8102270 2024-05-25T06:38:08.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.600+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba80fd970 con 0x7f9ba8102270 2024-05-25T06:38:08.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.600+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9b9c01c400 con 0x7f9ba8102270 2024-05-25T06:38:08.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.601+0000 7f9b8ffff640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9b9006e0b0 0x7f9b90070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.601+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9b9c08d470 con 0x7f9ba8102270 2024-05-25T06:38:08.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.601+0000 7f9ba77fe640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9b9006e0b0 0x7f9b90070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.601+0000 7f9ba77fe640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9b9006e0b0 0x7f9b90070560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b98003080 tx=0x7f9b98009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:08.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.605+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9b9c0567a0 con 0x7f9ba8102270 2024-05-25T06:38:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3242175697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:38:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3228056080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:38:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/90340821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:38:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:08 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1659596537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-25T06:38:08.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.831+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f9ba8109370 con 0x7f9ba8102270 2024-05-25T06:38:08.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.832+0000 7f9b8ffff640 1 -- 172.21.15.160:0/3934706092 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v43) v1 ==== 107+0+4902 (secure 0 0 0) 0x7f9b9c059e60 con 0x7f9ba8102270 2024-05-25T06:38:08.832 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:08.832 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:12:18.728742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi160.tlctof","rank":0,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6829/3783773637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3783773637},{"type":"v1","addr":"172.21.15.160:6829","nonce":3783773637}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6827/2201510143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2201510143},{"type":"v1","addr":"172.21.15.160:6827","nonce":2201510143}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi203.yltymd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6825/3325193142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3325193142},{"type":"v1","addr":"172.21.15.203:6825","nonce":3325193142}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24283":{"gid":24283,"name":"cephfs.smithi203.tndnrb","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.203:6827/1913747634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1913747634},{"type":"v1","addr":"172.21.15.203:6827","nonce":1913747634}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:08.832 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 24 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9b9006e0b0 msgr2=0x7f9b90070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9b9006e0b0 0x7f9b90070560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b98003080 tx=0x7f9b98009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 msgr2=0x7f9ba8113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8113170 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c002a50 tx=0x7f9b9c004600 comp rx=0 tx=0).stop 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 shutdown_connections 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f9b9006e0b0 0x7f9b90070560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ba8102270 0x7f9ba8113170 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 --2- 172.21.15.160:0/3934706092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ba8064390 0x7f9ba8112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 >> 172.21.15.160:0/3934706092 conn(0x7f9ba8071fd0 msgr2=0x7f9ba80fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 shutdown_connections 2024-05-25T06:38:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.834+0000 7f9badf9d640 1 -- 172.21.15.160:0/3934706092 wait complete. 2024-05-25T06:38:08.845 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-05-25T06:38:08.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/2614591162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d40fb150 msgr2=0x7ff5d40fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/2614591162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d40fb150 0x7ff5d40fb5c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80048d0 tx=0x7ff5c8009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/2614591162 shutdown_connections 2024-05-25T06:38:08.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/2614591162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d40fb150 0x7ff5d40fb5c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/2614591162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d4101170 0x7ff5d4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/2614591162 >> 172.21.15.160:0/2614591162 conn(0x7ff5d4071fd0 msgr2=0x7ff5d40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:08.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/2614591162 shutdown_connections 2024-05-25T06:38:08.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/2614591162 wait complete. 2024-05-25T06:38:08.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.944+0000 7ff5d8a2e640 1 Processor -- start 2024-05-25T06:38:08.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d8a2e640 1 -- start start 2024-05-25T06:38:08.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d8a2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d40fb150 0x7ff5d4101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d8a2e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 0x7ff5d41024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d8a2e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4103e60 con 0x7ff5d4101170 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d8a2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4102a00 con 0x7ff5d40fb150 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 0x7ff5d41024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d40fb150 0x7ff5d4101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 0x7ff5d41024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41024/0 (socket says 172.21.15.160:41024) 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.945+0000 7ff5d1d9b640 1 -- 172.21.15.160:0/176220500 learned_addr learned my addr 172.21.15.160:0/176220500 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5d1d9b640 1 -- 172.21.15.160:0/176220500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d40fb150 msgr2=0x7ff5d4101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5d1d9b640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d40fb150 0x7ff5d4101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5d1d9b640 1 -- 172.21.15.160:0/176220500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8005040 con 0x7ff5d4101170 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5d1d9b640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 0x7ff5d41024c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8004930 tx=0x7ff5c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c80094c0 con 0x7ff5d4101170 2024-05-25T06:38:08.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5c8032e50 con 0x7ff5d4101170 2024-05-25T06:38:08.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c80433b0 con 0x7ff5d4101170 2024-05-25T06:38:08.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d4102c80 con 0x7ff5d4101170 2024-05-25T06:38:08.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.946+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4119340 con 0x7ff5d4101170 2024-05-25T06:38:08.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.947+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d40fc870 con 0x7ff5d4101170 2024-05-25T06:38:08.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.948+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff5c802f070 con 0x7ff5d4101170 2024-05-25T06:38:08.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.948+0000 7ff5c2ffd640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff5bc06e0b0 0x7ff5bc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:08.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.948+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff5c80af990 con 0x7ff5d4101170 2024-05-25T06:38:08.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.948+0000 7ff5d259c640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff5bc06e0b0 0x7ff5bc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:08.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.949+0000 7ff5d259c640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff5bc06e0b0 0x7ff5bc070560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4004ed0 tx=0x7ff5c400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:08.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:08.957+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff5c8078d70 con 0x7ff5d4101170 2024-05-25T06:38:09.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.181+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7ff5d41082a0 con 0x7ff5d4101170 2024-05-25T06:38:09.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.181+0000 7ff5c2ffd640 1 -- 172.21.15.160:0/176220500 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v43) v1 ==== 107+0+4916 (secure 0 0 0) 0x7ff5c807c430 con 0x7ff5d4101170 2024-05-25T06:38:09.181 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:09.181 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:15:43.843616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi160.tlctof","rank":0,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6829/3783773637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3783773637},{"type":"v1","addr":"172.21.15.160:6829","nonce":3783773637}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6827/2201510143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2201510143},{"type":"v1","addr":"172.21.15.160:6827","nonce":2201510143}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi203.yltymd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6825/3325193142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3325193142},{"type":"v1","addr":"172.21.15.203:6825","nonce":3325193142}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24283":{"gid":24283,"name":"cephfs.smithi203.tndnrb","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.203:6827/1913747634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1913747634},{"type":"v1","addr":"172.21.15.203:6827","nonce":1913747634}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[14480,24271]},"id":1}]} 2024-05-25T06:38:09.182 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 25 2024-05-25T06:38:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.183+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff5bc06e0b0 msgr2=0x7ff5bc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.183+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff5bc06e0b0 0x7ff5bc070560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4004ed0 tx=0x7ff5c400b040 comp rx=0 tx=0).stop 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.183+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 msgr2=0x7ff5d41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.183+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 0x7ff5d41024c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8004930 tx=0x7ff5c8009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 shutdown_connections 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff5bc06e0b0 0x7ff5bc070560 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5d4101170 0x7ff5d41024c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 --2- 172.21.15.160:0/176220500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d40fb150 0x7ff5d4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 >> 172.21.15.160:0/176220500 conn(0x7ff5d4071fd0 msgr2=0x7ff5d40646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 shutdown_connections 2024-05-25T06:38:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.184+0000 7ff5d8a2e640 1 -- 172.21.15.160:0/176220500 wait complete. 2024-05-25T06:38:09.195 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-05-25T06:38:09.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.295+0000 7f4c726ae640 1 -- 172.21.15.160:0/3416703185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 msgr2=0x7f4c6c0fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.295+0000 7f4c726ae640 1 --2- 172.21.15.160:0/3416703185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c0fb5c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c540048d0 tx=0x7f4c54009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.295+0000 7f4c726ae640 1 -- 172.21.15.160:0/3416703185 shutdown_connections 2024-05-25T06:38:09.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.295+0000 7f4c726ae640 1 --2- 172.21.15.160:0/3416703185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c0fb5c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.295+0000 7f4c726ae640 1 --2- 172.21.15.160:0/3416703185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c101170 0x7f4c6c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.295+0000 7f4c726ae640 1 -- 172.21.15.160:0/3416703185 >> 172.21.15.160:0/3416703185 conn(0x7f4c6c071fd0 msgr2=0x7f4c6c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 -- 172.21.15.160:0/3416703185 shutdown_connections 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 -- 172.21.15.160:0/3416703185 wait complete. 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 Processor -- start 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 -- start start 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c101170 0x7f4c6c1120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c1126c0 con 0x7f4c6c0fb150 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.296+0000 7f4c726ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c10bd10 con 0x7f4c6c101170 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c111b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41032/0 (socket says 172.21.15.160:41032) 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 -- 172.21.15.160:0/1936769639 learned_addr learned my addr 172.21.15.160:0/1936769639 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6b7fe640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c101170 0x7f4c6c1120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 -- 172.21.15.160:0/1936769639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c101170 msgr2=0x7f4c6c1120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c101170 0x7f4c6c1120c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 -- 172.21.15.160:0/1936769639 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54005040 con 0x7f4c6c0fb150 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c6bfff640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c111b80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c002a50 tx=0x7f4c5c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c007c30 con 0x7f4c6c0fb150 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c10bff0 con 0x7f4c6c0fb150 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c5c00f040 con 0x7f4c6c0fb150 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c00db50 con 0x7f4c6c0fb150 2024-05-25T06:38:09.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.297+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c10c4e0 con 0x7f4c6c0fb150 2024-05-25T06:38:09.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.298+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c0fc870 con 0x7f4c6c0fb150 2024-05-25T06:38:09.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.299+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4c5c01c400 con 0x7f4c6c0fb150 2024-05-25T06:38:09.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.299+0000 7f4c68ff9640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c5806e0b0 0x7f4c58070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.299+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4c5c08d3f0 con 0x7f4c6c0fb150 2024-05-25T06:38:09.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.299+0000 7f4c6b7fe640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c5806e0b0 0x7f4c58070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.300+0000 7f4c6b7fe640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c5806e0b0 0x7f4c58070560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0f9580 tx=0x7f4c54009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:09.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.304+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4c5c056840 con 0x7f4c6c0fb150 2024-05-25T06:38:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:09 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1480320569' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-25T06:38:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:09 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3934706092' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-25T06:38:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:09 smithi160 ceph-mon[112094]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:09 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/176220500' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-25T06:38:09.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.522+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f4c6c0fe3a0 con 0x7f4c6c0fb150 2024-05-25T06:38:09.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.523+0000 7f4c68ff9640 1 -- 172.21.15.160:0/1936769639 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v43) v1 ==== 107+0+4915 (secure 0 0 0) 0x7f4c5c059f00 con 0x7f4c6c0fb150 2024-05-25T06:38:09.523 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:09.523 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:15:56.474429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi160.tlctof","rank":0,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6829/3783773637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3783773637},{"type":"v1","addr":"172.21.15.160:6829","nonce":3783773637}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6827/2201510143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2201510143},{"type":"v1","addr":"172.21.15.160:6827","nonce":2201510143}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi203.yltymd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6825/3325193142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3325193142},{"type":"v1","addr":"172.21.15.203:6825","nonce":3325193142}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24283":{"gid":24283,"name":"cephfs.smithi203.tndnrb","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.203:6827/1913747634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1913747634},{"type":"v1","addr":"172.21.15.203:6827","nonce":1913747634}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[24271,14480]},"id":1}]} 2024-05-25T06:38:09.523 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 26 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c5806e0b0 msgr2=0x7f4c58070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c5806e0b0 0x7f4c58070560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0f9580 tx=0x7f4c54009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 msgr2=0x7f4c6c111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c111b80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c002a50 tx=0x7f4c5c004600 comp rx=0 tx=0).stop 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 shutdown_connections 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c5806e0b0 0x7f4c58070560 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c101170 0x7f4c6c1120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 --2- 172.21.15.160:0/1936769639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c6c0fb150 0x7f4c6c111b80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 >> 172.21.15.160:0/1936769639 conn(0x7f4c6c071fd0 msgr2=0x7f4c6c0fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:09.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 shutdown_connections 2024-05-25T06:38:09.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.525+0000 7f4c726ae640 1 -- 172.21.15.160:0/1936769639 wait complete. 2024-05-25T06:38:09.536 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 -- 172.21.15.160:0/1227890374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b0102450 msgr2=0x7f23b0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 --2- 172.21.15.160:0/1227890374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b0102450 0x7f23b0102820 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f23a00048d0 tx=0x7f23a0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 -- 172.21.15.160:0/1227890374 shutdown_connections 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 --2- 172.21.15.160:0/1227890374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23b00faa50 0x7f23b00faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 --2- 172.21.15.160:0/1227890374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b0102450 0x7f23b0102820 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 -- 172.21.15.160:0/1227890374 >> 172.21.15.160:0/1227890374 conn(0x7f23b0071990 msgr2=0x7f23b0071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 -- 172.21.15.160:0/1227890374 shutdown_connections 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.634+0000 7f23b595d640 1 -- 172.21.15.160:0/1227890374 wait complete. 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23b595d640 1 Processor -- start 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23b595d640 1 -- start start 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23b595d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 0x7f23b0112f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23b595d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23b0102450 0x7f23b01134b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23b595d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0113ab0 con 0x7f23b00faa50 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23b595d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b010d100 con 0x7f23b0102450 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 0x7f23b0112f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 0x7f23b0112f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41040/0 (socket says 172.21.15.160:41040) 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 -- 172.21.15.160:0/2861242838 learned_addr learned my addr 172.21.15.160:0/2861242838 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23ae7fc640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23b0102450 0x7f23b01134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 -- 172.21.15.160:0/2861242838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23b0102450 msgr2=0x7f23b01134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23b0102450 0x7f23b01134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 -- 172.21.15.160:0/2861242838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0005040 con 0x7f23b00faa50 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.635+0000 7f23aeffd640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 0x7f23b0112f70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f23a0031720 tx=0x7f23a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.636+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23a00094c0 con 0x7f23b00faa50 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.636+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23a0032e50 con 0x7f23b00faa50 2024-05-25T06:38:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.636+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b010d380 con 0x7f23b00faa50 2024-05-25T06:38:09.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.636+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23a003b640 con 0x7f23b00faa50 2024-05-25T06:38:09.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.636+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b010d8a0 con 0x7f23b00faa50 2024-05-25T06:38:09.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.637+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b0065560 con 0x7f23b00faa50 2024-05-25T06:38:09.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.637+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f23a002f070 con 0x7f23b00faa50 2024-05-25T06:38:09.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.638+0000 7f23977fe640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f239806e0b0 0x7f2398070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.638+0000 7f23ae7fc640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f239806e0b0 0x7f2398070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.638+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f23a00b00a0 con 0x7f23b00faa50 2024-05-25T06:38:09.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.638+0000 7f23ae7fc640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f239806e0b0 0x7f2398070560 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23b010e830 tx=0x7f23a4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:09.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.642+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f23a00793d0 con 0x7f23b00faa50 2024-05-25T06:38:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:09 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1480320569' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-25T06:38:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:09 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3934706092' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-25T06:38:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:09 smithi203 ceph-mon[83507]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:09 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/176220500' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-25T06:38:09.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.856+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f23b00638a0 con 0x7f23b00faa50 2024-05-25T06:38:09.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.857+0000 7f23977fe640 1 -- 172.21.15.160:0/2861242838 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v43) v1 ==== 107+0+3337 (secure 0 0 0) 0x7f23a0036d20 con 0x7f23b00faa50 2024-05-25T06:38:09.857 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:09.857 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:09.995613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi160.tlctof","rank":0,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6829/3783773637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3783773637},{"type":"v1","addr":"172.21.15.160:6829","nonce":3783773637}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi203.yltymd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6825/3325193142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3325193142},{"type":"v1","addr":"172.21.15.203:6825","nonce":3325193142}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[14480,24271]},"id":1}]} 2024-05-25T06:38:09.857 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 27 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f239806e0b0 msgr2=0x7f2398070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f239806e0b0 0x7f2398070560 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23b010e830 tx=0x7f23a4009040 comp rx=0 tx=0).stop 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 msgr2=0x7f23b0112f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 0x7f23b0112f70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f23a0031720 tx=0x7f23a0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 shutdown_connections 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f239806e0b0 0x7f2398070560 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23b0102450 0x7f23b01134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 --2- 172.21.15.160:0/2861242838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23b00faa50 0x7f23b0112f70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 >> 172.21.15.160:0/2861242838 conn(0x7f23b0071990 msgr2=0x7f23b00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 shutdown_connections 2024-05-25T06:38:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.859+0000 7f23b595d640 1 -- 172.21.15.160:0/2861242838 wait complete. 2024-05-25T06:38:09.870 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-25T06:38:09.870 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-05-25T06:38:09.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 -- 172.21.15.160:0/1782745010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102bd0 msgr2=0x7f60540fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 --2- 172.21.15.160:0/1782745010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102bd0 0x7f60540fa8c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f60480048d0 tx=0x7f6048009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 -- 172.21.15.160:0/1782745010 shutdown_connections 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 --2- 172.21.15.160:0/1782745010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102bd0 0x7f60540fa8c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 --2- 172.21.15.160:0/1782745010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6054102230 0x7f6054102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 -- 172.21.15.160:0/1782745010 >> 172.21.15.160:0/1782745010 conn(0x7f60540717e0 msgr2=0x7f6054071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 -- 172.21.15.160:0/1782745010 shutdown_connections 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 -- 172.21.15.160:0/1782745010 wait complete. 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.967+0000 7f6059d9d640 1 Processor -- start 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f6059d9d640 1 -- start start 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f6059d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 0x7f6054112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f6059d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6054102bd0 0x7f6054113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f6059d9d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6054113870 con 0x7f6054102230 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f6059d9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605410cec0 con 0x7f6054102bd0 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 0x7f6054112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 0x7f6054112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41070/0 (socket says 172.21.15.160:41070) 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 -- 172.21.15.160:0/2970829905 learned_addr learned my addr 172.21.15.160:0/2970829905 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f6052ffd640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6054102bd0 0x7f6054113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 -- 172.21.15.160:0/2970829905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6054102bd0 msgr2=0x7f6054113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6054102bd0 0x7f6054113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 -- 172.21.15.160:0/2970829905 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048005040 con 0x7f6054102230 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f60537fe640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 0x7f6054112d30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f604400c830 tx=0x7f6044004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.968+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604400a850 con 0x7f6054102230 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.969+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605410d1a0 con 0x7f6054102230 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.969+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f604400ae80 con 0x7f6054102230 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.969+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6044012ea0 con 0x7f6054102230 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.969+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605410d690 con 0x7f6054102230 2024-05-25T06:38:09.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.969+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6054064350 con 0x7f6054102230 2024-05-25T06:38:09.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.970+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6044012400 con 0x7f6054102230 2024-05-25T06:38:09.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.970+0000 7f603bfff640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f603c06e0b0 0x7f603c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:09.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.970+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f604408dde0 con 0x7f6054102230 2024-05-25T06:38:09.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.971+0000 7f6052ffd640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f603c06e0b0 0x7f603c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:09.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.971+0000 7f6052ffd640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f603c06e0b0 0x7f603c070560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f60480048a0 tx=0x7f6048009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:09.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:09.975+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6044057190 con 0x7f6054102230 2024-05-25T06:38:10.190 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.190+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f6054109390 con 0x7f6054102230 2024-05-25T06:38:10.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.191+0000 7f603bfff640 1 -- 172.21.15.160:0/2970829905 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v43) v1 ==== 107+0+3301 (secure 0 0 0) 0x7f60440126b0 con 0x7f6054102230 2024-05-25T06:38:10.191 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:10.191 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34372,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/2550259626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2550259626},{"type":"v1","addr":"172.21.15.160:6827","nonce":2550259626}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:10.191 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 28 2024-05-25T06:38:10.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f603c06e0b0 msgr2=0x7f603c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f603c06e0b0 0x7f603c070560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f60480048a0 tx=0x7f6048009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 msgr2=0x7f6054112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 0x7f6054112d30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f604400c830 tx=0x7f6044004430 comp rx=0 tx=0).stop 2024-05-25T06:38:10.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 shutdown_connections 2024-05-25T06:38:10.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f603c06e0b0 0x7f603c070560 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6054102bd0 0x7f6054113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 --2- 172.21.15.160:0/2970829905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054102230 0x7f6054112d30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 >> 172.21.15.160:0/2970829905 conn(0x7f60540717e0 msgr2=0x7f60540fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:10.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 shutdown_connections 2024-05-25T06:38:10.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.193+0000 7f6059d9d640 1 -- 172.21.15.160:0/2970829905 wait complete. 2024-05-25T06:38:10.204 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:10.204 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-25T06:38:10.204 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-05-25T06:38:10.304 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.302+0000 7f191106e640 1 -- 172.21.15.160:0/2846871551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c102270 msgr2=0x7f190c102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.304 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.302+0000 7f191106e640 1 --2- 172.21.15.160:0/2846871551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c102270 0x7f190c102640 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009c50 tx=0x7f18fc004d10 comp rx=0 tx=0).stop 2024-05-25T06:38:10.304 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- 172.21.15.160:0/2846871551 shutdown_connections 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 --2- 172.21.15.160:0/2846871551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c064390 0x7f190c064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 --2- 172.21.15.160:0/2846871551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c102270 0x7f190c102640 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- 172.21.15.160:0/2846871551 >> 172.21.15.160:0/2846871551 conn(0x7f190c071fd0 msgr2=0x7f190c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- 172.21.15.160:0/2846871551 shutdown_connections 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- 172.21.15.160:0/2846871551 wait complete. 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 Processor -- start 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- start start 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 0x7f190c112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c102270 0x7f190c1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c113850 con 0x7f190c064390 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.303+0000 7f191106e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c10ce10 con 0x7f190c102270 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 0x7f190c112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 0x7f190c112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41086/0 (socket says 172.21.15.160:41086) 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 -- 172.21.15.160:0/1065981670 learned_addr learned my addr 172.21.15.160:0/1065981670 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190a59c640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c102270 0x7f190c1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 -- 172.21.15.160:0/1065981670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c102270 msgr2=0x7f190c1131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c102270 0x7f190c1131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 -- 172.21.15.160:0/1065981670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0098b0 con 0x7f190c064390 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f190ad9d640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 0x7f190c112c80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18fc004a30 tx=0x7f18fc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18fc03a070 con 0x7f190c064390 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18fc035b20 con 0x7f190c064390 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c10d090 con 0x7f190c064390 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18fc03eaa0 con 0x7f190c064390 2024-05-25T06:38:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.304+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c10d5b0 con 0x7f190c064390 2024-05-25T06:38:10.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.305+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c0fd970 con 0x7f190c064390 2024-05-25T06:38:10.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.306+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f18fc032070 con 0x7f190c064390 2024-05-25T06:38:10.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.306+0000 7f18f37fe640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18f406e0b0 0x7f18f4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.306+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f18fc0b03d0 con 0x7f190c064390 2024-05-25T06:38:10.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.306+0000 7f190a59c640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18f406e0b0 0x7f18f4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.306+0000 7f190a59c640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18f406e0b0 0x7f18f4070560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f190c10e540 tx=0x7f1900008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:10.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.311+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f18fc079700 con 0x7f190c064390 2024-05-25T06:38:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:10 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1936769639' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-25T06:38:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:10 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2861242838' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-25T06:38:10.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:10 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2970829905' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-25T06:38:10.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.522+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f190c063f00 con 0x7f190c064390 2024-05-25T06:38:10.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.522+0000 7f18f37fe640 1 -- 172.21.15.160:0/1065981670 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v43) v1 ==== 107+0+4865 (secure 0 0 0) 0x7f18fc07cdc0 con 0x7f190c064390 2024-05-25T06:38:10.522 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:10.522 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34372,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/2550259626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2550259626},{"type":"v1","addr":"172.21.15.160:6827","nonce":2550259626}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":34380,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/3633650351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3633650351},{"type":"v1","addr":"172.21.15.160:6829","nonce":3633650351}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44271,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1521635049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1521635049},{"type":"v1","addr":"172.21.15.203:6825","nonce":1521635049}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:10.522 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 29 2024-05-25T06:38:10.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18f406e0b0 msgr2=0x7f18f4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18f406e0b0 0x7f18f4070560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f190c10e540 tx=0x7f1900008040 comp rx=0 tx=0).stop 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 msgr2=0x7f190c112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 0x7f190c112c80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18fc004a30 tx=0x7f18fc004d10 comp rx=0 tx=0).stop 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 shutdown_connections 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f18f406e0b0 0x7f18f4070560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c102270 0x7f190c1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 --2- 172.21.15.160:0/1065981670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c064390 0x7f190c112c80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.524+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 >> 172.21.15.160:0/1065981670 conn(0x7f190c071fd0 msgr2=0x7f190c0fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.525+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 shutdown_connections 2024-05-25T06:38:10.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.525+0000 7f191106e640 1 -- 172.21.15.160:0/1065981670 wait complete. 2024-05-25T06:38:10.535 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:10.535 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-25T06:38:10.535 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-05-25T06:38:10.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 -- 172.21.15.160:0/3549529377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fd160 msgr2=0x7f63f40fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 --2- 172.21.15.160:0/3549529377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fd160 0x7f63f40fd530 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f63e4004900 tx=0x7f63e4009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 -- 172.21.15.160:0/3549529377 shutdown_connections 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 --2- 172.21.15.160:0/3549529377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f40fdb00 0x7f63f41061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 --2- 172.21.15.160:0/3549529377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fd160 0x7f63f40fd530 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 -- 172.21.15.160:0/3549529377 >> 172.21.15.160:0/3549529377 conn(0x7f63f40f6be0 msgr2=0x7f63f40f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 -- 172.21.15.160:0/3549529377 shutdown_connections 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.632+0000 7f63fa46e640 1 -- 172.21.15.160:0/3549529377 wait complete. 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63fa46e640 1 Processor -- start 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63fa46e640 1 -- start start 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63fa46e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f40fd160 0x7f63f4112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63fa46e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 0x7f63f4113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63fa46e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4113890 con 0x7f63f40fdb00 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63fa46e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f410cee0 con 0x7f63f40fd160 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f37fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 0x7f63f4113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f37fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 0x7f63f4113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41098/0 (socket says 172.21.15.160:41098) 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f37fe640 1 -- 172.21.15.160:0/80766699 learned_addr learned my addr 172.21.15.160:0/80766699 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f3fff640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f40fd160 0x7f63f4112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f37fe640 1 -- 172.21.15.160:0/80766699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f40fd160 msgr2=0x7f63f4112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f37fe640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f40fd160 0x7f63f4112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.633+0000 7f63f37fe640 1 -- 172.21.15.160:0/80766699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4005040 con 0x7f63f40fdb00 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.634+0000 7f63f37fe640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 0x7f63f4113290 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f63e8002a50 tx=0x7f63e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.634+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63e8007c30 con 0x7f63f40fdb00 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.634+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f410d1c0 con 0x7f63f40fdb00 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.634+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63e800f040 con 0x7f63f40fdb00 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.634+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63e800db50 con 0x7f63f40fdb00 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.634+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f410d6e0 con 0x7f63f40fdb00 2024-05-25T06:38:10.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.635+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f4065fe0 con 0x7f63f40fdb00 2024-05-25T06:38:10.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.635+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f63e801c400 con 0x7f63f40fdb00 2024-05-25T06:38:10.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.635+0000 7f63f0ff9640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63dc06e0b0 0x7f63dc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.636+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f63e808d3c0 con 0x7f63f40fdb00 2024-05-25T06:38:10.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.636+0000 7f63f3fff640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63dc06e0b0 0x7f63dc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.636+0000 7f63f3fff640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63dc06e0b0 0x7f63dc070560 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a30 tx=0x7f63e4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:10.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.640+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f63e80567a0 con 0x7f63f40fdb00 2024-05-25T06:38:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:10 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1936769639' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-25T06:38:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:10 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2861242838' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-25T06:38:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:10 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2970829905' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-25T06:38:10.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.849+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f63f41092f0 con 0x7f63f40fdb00 2024-05-25T06:38:10.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.849+0000 7f63f0ff9640 1 -- 172.21.15.160:0/80766699 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v43) v1 ==== 107+0+4083 (secure 0 0 0) 0x7f63e8059e60 con 0x7f63f40fdb00 2024-05-25T06:38:10.850 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:10.850 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/3633650351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3633650351},{"type":"v1","addr":"172.21.15.160:6829","nonce":3633650351}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44271,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1521635049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1521635049},{"type":"v1","addr":"172.21.15.203:6825","nonce":1521635049}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:10.850 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 30 2024-05-25T06:38:10.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63dc06e0b0 msgr2=0x7f63dc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63dc06e0b0 0x7f63dc070560 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a30 tx=0x7f63e4009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 msgr2=0x7f63f4113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 0x7f63f4113290 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f63e8002a50 tx=0x7f63e8004600 comp rx=0 tx=0).stop 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 shutdown_connections 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f63dc06e0b0 0x7f63dc070560 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63f40fdb00 0x7f63f4113290 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 --2- 172.21.15.160:0/80766699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f40fd160 0x7f63f4112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 >> 172.21.15.160:0/80766699 conn(0x7f63f40f6be0 msgr2=0x7f63f40f8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.851+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 shutdown_connections 2024-05-25T06:38:10.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.852+0000 7f63fa46e640 1 -- 172.21.15.160:0/80766699 wait complete. 2024-05-25T06:38:10.862 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:10.862 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-05-25T06:38:10.863 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-05-25T06:38:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1295853048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 msgr2=0x7f3b5c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1295853048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c101540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3b500048d0 tx=0x7f3b50009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1295853048 shutdown_connections 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1295853048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b5c0fb150 0x7f3b5c0fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1295853048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c101540 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1295853048 >> 172.21.15.160:0/1295853048 conn(0x7f3b5c071fd0 msgr2=0x7f3b5c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1295853048 shutdown_connections 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1295853048 wait complete. 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 Processor -- start 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.961+0000 7f3b62c6e640 1 -- start start 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b62c6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b5c0fb150 0x7f3b5c111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b62c6e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c1120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b62c6e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c1126f0 con 0x7f3b5c101170 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b62c6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c10bd40 con 0x7f3b5c0fb150 2024-05-25T06:38:10.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c1120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c1120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41116/0 (socket says 172.21.15.160:41116) 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 -- 172.21.15.160:0/1207355258 learned_addr learned my addr 172.21.15.160:0/1207355258 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b60a0a640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b5c0fb150 0x7f3b5c111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 -- 172.21.15.160:0/1207355258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b5c0fb150 msgr2=0x7f3b5c111bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b5c0fb150 0x7f3b5c111bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 -- 172.21.15.160:0/1207355258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b50005040 con 0x7f3b5c101170 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b5bfff640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c1120f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c790 tx=0x7f3b480043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4800a820 con 0x7f3b5c101170 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b4800ae50 con 0x7f3b5c101170 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c10c020 con 0x7f3b5c101170 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b480115e0 con 0x7f3b5c101170 2024-05-25T06:38:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.962+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c10c540 con 0x7f3b5c101170 2024-05-25T06:38:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.963+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c0fc870 con 0x7f3b5c101170 2024-05-25T06:38:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.964+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3b48012400 con 0x7f3b5c101170 2024-05-25T06:38:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.964+0000 7f3b597fa640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3b4c06e0b0 0x7f3b4c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.964+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3b4808dc80 con 0x7f3b5c101170 2024-05-25T06:38:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.964+0000 7f3b60a0a640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3b4c06e0b0 0x7f3b4c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.965+0000 7f3b60a0a640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3b4c06e0b0 0x7f3b4c070560 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b50003080 tx=0x7f3b50009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:10.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:10.969+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3b48057030 con 0x7f3b5c101170 2024-05-25T06:38:11.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.181+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f3b5c108270 con 0x7f3b5c101170 2024-05-25T06:38:11.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.181+0000 7f3b597fa640 1 -- 172.21.15.160:0/1207355258 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v43) v1 ==== 107+0+4933 (secure 0 0 0) 0x7f3b4805a6f0 con 0x7f3b5c101170 2024-05-25T06:38:11.181 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:11.181 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/3633650351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3633650351},{"type":"v1","addr":"172.21.15.160:6829","nonce":3633650351}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44271,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1521635049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1521635049},{"type":"v1","addr":"172.21.15.203:6825","nonce":1521635049}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:11.182 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 31 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3b4c06e0b0 msgr2=0x7f3b4c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3b4c06e0b0 0x7f3b4c070560 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b50003080 tx=0x7f3b50009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 msgr2=0x7f3b5c1120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c1120f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c790 tx=0x7f3b480043c0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 shutdown_connections 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3b4c06e0b0 0x7f3b4c070560 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b5c101170 0x7f3b5c1120f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.183+0000 7f3b62c6e640 1 --2- 172.21.15.160:0/1207355258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b5c0fb150 0x7f3b5c111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.184+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 >> 172.21.15.160:0/1207355258 conn(0x7f3b5c071fd0 msgr2=0x7f3b5c0fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.184+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 shutdown_connections 2024-05-25T06:38:11.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.184+0000 7f3b62c6e640 1 -- 172.21.15.160:0/1207355258 wait complete. 2024-05-25T06:38:11.194 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:11.194 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-05-25T06:38:11.194 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-05-25T06:38:11.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 -- 172.21.15.160:0/2425265397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101a30 msgr2=0x7fc7980fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 --2- 172.21.15.160:0/2425265397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101a30 0x7fc7980fb0c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc7880048d0 tx=0x7fc788009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 -- 172.21.15.160:0/2425265397 shutdown_connections 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 --2- 172.21.15.160:0/2425265397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101a30 0x7fc7980fb0c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 --2- 172.21.15.160:0/2425265397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101090 0x7fc798101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 -- 172.21.15.160:0/2425265397 >> 172.21.15.160:0/2425265397 conn(0x7fc7980f6c20 msgr2=0x7fc7980f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 -- 172.21.15.160:0/2425265397 shutdown_connections 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 -- 172.21.15.160:0/2425265397 wait complete. 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.294+0000 7fc79ea61640 1 Processor -- start 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc79ea61640 1 -- start start 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc79ea61640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 0x7fc798111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc79ea61640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101a30 0x7fc798112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc79ea61640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798112730 con 0x7fc798101090 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc79ea61640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981128a0 con 0x7fc798101a30 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 0x7fc798111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 0x7fc798111bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41128/0 (socket says 172.21.15.160:41128) 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 -- 172.21.15.160:0/3096315392 learned_addr learned my addr 172.21.15.160:0/3096315392 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc7977fe640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101a30 0x7fc798112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 -- 172.21.15.160:0/3096315392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101a30 msgr2=0x7fc798112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101a30 0x7fc798112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 -- 172.21.15.160:0/3096315392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788005040 con 0x7fc798101090 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc7977fe640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101a30 0x7fc798112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.295+0000 7fc797fff640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 0x7fc798111bf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc780002a50 tx=0x7fc780004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.296+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc780007c30 con 0x7fc798101090 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.296+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79810bf30 con 0x7fc798101090 2024-05-25T06:38:11.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.296+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc78000f040 con 0x7fc798101090 2024-05-25T06:38:11.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.296+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78000db50 con 0x7fc798101090 2024-05-25T06:38:11.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.296+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79810c450 con 0x7fc798101090 2024-05-25T06:38:11.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.297+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980fc800 con 0x7fc798101090 2024-05-25T06:38:11.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.297+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc78001c400 con 0x7fc798101090 2024-05-25T06:38:11.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.298+0000 7fc794ff9640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc78406e0b0 0x7fc784070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.298+0000 7fc7977fe640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc78406e0b0 0x7fc784070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.298+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc78008d470 con 0x7fc798101090 2024-05-25T06:38:11.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.298+0000 7fc7977fe640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc78406e0b0 0x7fc784070560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc788009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:11.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.302+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc7800567a0 con 0x7fc798101090 2024-05-25T06:38:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:11 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1065981670' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-25T06:38:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:11 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/80766699' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-25T06:38:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:11 smithi160 ceph-mon[112094]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:11 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1207355258' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-25T06:38:11.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.515+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fc798063670 con 0x7fc798101090 2024-05-25T06:38:11.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.516+0000 7fc794ff9640 1 -- 172.21.15.160:0/3096315392 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v43) v1 ==== 107+0+4151 (secure 0 0 0) 0x7fc780059e60 con 0x7fc798101090 2024-05-25T06:38:11.516 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:11.517 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44271,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1521635049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1521635049},{"type":"v1","addr":"172.21.15.203:6825","nonce":1521635049}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:11.517 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 32 2024-05-25T06:38:11.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc78406e0b0 msgr2=0x7fc784070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc78406e0b0 0x7fc784070560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc788009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 msgr2=0x7fc798111bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 0x7fc798111bf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc780002a50 tx=0x7fc780004600 comp rx=0 tx=0).stop 2024-05-25T06:38:11.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 shutdown_connections 2024-05-25T06:38:11.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc78406e0b0 0x7fc784070560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc798101a30 0x7fc798112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 --2- 172.21.15.160:0/3096315392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc798101090 0x7fc798111bf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 >> 172.21.15.160:0/3096315392 conn(0x7fc7980f6c20 msgr2=0x7fc7980f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:11.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 shutdown_connections 2024-05-25T06:38:11.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.518+0000 7fc79ea61640 1 -- 172.21.15.160:0/3096315392 wait complete. 2024-05-25T06:38:11.529 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:11.529 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-05-25T06:38:11.529 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-05-25T06:38:11.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 -- 172.21.15.160:0/3423451105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101070 msgr2=0x7f4c18101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/3423451105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101070 0x7f4c18101440 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4c100048d0 tx=0x7f4c10009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 -- 172.21.15.160:0/3423451105 shutdown_connections 2024-05-25T06:38:11.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/3423451105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101a10 0x7f4c180fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/3423451105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101070 0x7f4c18101440 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 -- 172.21.15.160:0/3423451105 >> 172.21.15.160:0/3423451105 conn(0x7f4c180f6be0 msgr2=0x7f4c180f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:11.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 -- 172.21.15.160:0/3423451105 shutdown_connections 2024-05-25T06:38:11.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 -- 172.21.15.160:0/3423451105 wait complete. 2024-05-25T06:38:11.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 Processor -- start 2024-05-25T06:38:11.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.627+0000 7f4c1f42e640 1 -- start start 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1f42e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101070 0x7f4c18111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1f42e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 0x7f4c181120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1f42e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c181126e0 con 0x7f4c18101a10 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1f42e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1810bd30 con 0x7f4c18101070 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 0x7f4c181120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 0x7f4c181120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41150/0 (socket says 172.21.15.160:41150) 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 -- 172.21.15.160:0/500967593 learned_addr learned my addr 172.21.15.160:0/500967593 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1d1ca640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101070 0x7f4c18111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 -- 172.21.15.160:0/500967593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101070 msgr2=0x7f4c18111ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101070 0x7f4c18111ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 -- 172.21.15.160:0/500967593 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10005040 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1d1ca640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101070 0x7f4c18111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c1c9c9640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 0x7f4c181120e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00ece0 tx=0x7f4c0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.628+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0c00ee40 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.629+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1810c010 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.629+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c0c00f040 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.629+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0c009750 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.629+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1810c530 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.630+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c180fc7e0 con 0x7f4c18101a10 2024-05-25T06:38:11.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.630+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4c0c007890 con 0x7f4c18101a10 2024-05-25T06:38:11.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.631+0000 7f4c15ffb640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c0006e0b0 0x7f4c00070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.631+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4c0c08e3e0 con 0x7f4c18101a10 2024-05-25T06:38:11.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.631+0000 7f4c1d1ca640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c0006e0b0 0x7f4c00070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.631+0000 7f4c1d1ca640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c0006e0b0 0x7f4c00070560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:11.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.635+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4c0c057710 con 0x7f4c18101a10 2024-05-25T06:38:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:11 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1065981670' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-25T06:38:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:11 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/80766699' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-25T06:38:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:11 smithi203 ceph-mon[83507]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:11 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1207355258' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-25T06:38:11.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.847+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f4c181081d0 con 0x7f4c18101a10 2024-05-25T06:38:11.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.848+0000 7f4c15ffb640 1 -- 172.21.15.160:0/500967593 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v43) v1 ==== 107+0+5001 (secure 0 0 0) 0x7f4c0c05add0 con 0x7f4c18101a10 2024-05-25T06:38:11.848 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:11.848 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44271,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1521635049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1521635049},{"type":"v1","addr":"172.21.15.203:6825","nonce":1521635049}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:11.848 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 33 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c0006e0b0 msgr2=0x7f4c00070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c0006e0b0 0x7f4c00070560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 msgr2=0x7f4c181120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 0x7f4c181120e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00ece0 tx=0x7f4c0c00b040 comp rx=0 tx=0).stop 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 shutdown_connections 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4c0006e0b0 0x7f4c00070560 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c18101a10 0x7f4c181120e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 --2- 172.21.15.160:0/500967593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c18101070 0x7f4c18111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 >> 172.21.15.160:0/500967593 conn(0x7f4c180f6be0 msgr2=0x7f4c180f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:11.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 shutdown_connections 2024-05-25T06:38:11.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.850+0000 7f4c1f42e640 1 -- 172.21.15.160:0/500967593 wait complete. 2024-05-25T06:38:11.861 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:11.861 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-05-25T06:38:11.861 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- 172.21.15.160:0/2295632609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 msgr2=0x7f0d5c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 --2- 172.21.15.160:0/2295632609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c101540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0d580048d0 tx=0x7f0d58009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- 172.21.15.160:0/2295632609 shutdown_connections 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 --2- 172.21.15.160:0/2295632609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d5c0fb150 0x7f0d5c0fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 --2- 172.21.15.160:0/2295632609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c101540 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- 172.21.15.160:0/2295632609 >> 172.21.15.160:0/2295632609 conn(0x7f0d5c071fd0 msgr2=0x7f0d5c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- 172.21.15.160:0/2295632609 shutdown_connections 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- 172.21.15.160:0/2295632609 wait complete. 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 Processor -- start 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- start start 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d5c0fb150 0x7f0d5c112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c113790 con 0x7f0d5c101170 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.959+0000 7f0d6426e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c10cde0 con 0x7f0d5c0fb150 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41176/0 (socket says 172.21.15.160:41176) 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 -- 172.21.15.160:0/1259274167 learned_addr learned my addr 172.21.15.160:0/1259274167 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d6200a640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d5c0fb150 0x7f0d5c112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 -- 172.21.15.160:0/1259274167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d5c0fb150 msgr2=0x7f0d5c112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d5c0fb150 0x7f0d5c112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 -- 172.21.15.160:0/1259274167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d58005040 con 0x7f0d5c101170 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d61809640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c113190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c002a50 tx=0x7f0d4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4c007c30 con 0x7f0d5c101170 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d4c00f040 con 0x7f0d5c101170 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c10d0c0 con 0x7f0d5c101170 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4c00dbe0 con 0x7f0d5c101170 2024-05-25T06:38:11.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.960+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c10d5e0 con 0x7f0d5c101170 2024-05-25T06:38:11.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.961+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c0fc870 con 0x7f0d5c101170 2024-05-25T06:38:11.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.962+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0d4c00dd40 con 0x7f0d5c101170 2024-05-25T06:38:11.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.962+0000 7f0d52ffd640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d4806e0b0 0x7f0d48070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:11.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.962+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0d4c08da70 con 0x7f0d5c101170 2024-05-25T06:38:11.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.962+0000 7f0d6200a640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d4806e0b0 0x7f0d48070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:11.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.963+0000 7f0d6200a640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d4806e0b0 0x7f0d48070560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0d58003080 tx=0x7f0d58009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:11.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:11.967+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0d4c056da0 con 0x7f0d5c101170 2024-05-25T06:38:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.179+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f0d5c1082b0 con 0x7f0d5c101170 2024-05-25T06:38:12.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.179+0000 7f0d52ffd640 1 -- 172.21.15.160:0/1259274167 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v43) v1 ==== 107+0+4219 (secure 0 0 0) 0x7f0d4c05a460 con 0x7f0d5c101170 2024-05-25T06:38:12.179 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:12.179 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:12.180 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 34 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d4806e0b0 msgr2=0x7f0d48070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d4806e0b0 0x7f0d48070560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0d58003080 tx=0x7f0d58009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 msgr2=0x7f0d5c113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c113190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c002a50 tx=0x7f0d4c004600 comp rx=0 tx=0).stop 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 shutdown_connections 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0d4806e0b0 0x7f0d48070560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d5c101170 0x7f0d5c113190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 --2- 172.21.15.160:0/1259274167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d5c0fb150 0x7f0d5c112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 >> 172.21.15.160:0/1259274167 conn(0x7f0d5c071fd0 msgr2=0x7f0d5c064650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 shutdown_connections 2024-05-25T06:38:12.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.182+0000 7f0d6426e640 1 -- 172.21.15.160:0/1259274167 wait complete. 2024-05-25T06:38:12.192 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:12.192 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-05-25T06:38:12.193 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 -- 172.21.15.160:0/4211783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 msgr2=0x7ff0281025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 --2- 172.21.15.160:0/4211783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff0281025b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff0240048d0 tx=0x7ff024009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 -- 172.21.15.160:0/4211783480 shutdown_connections 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 --2- 172.21.15.160:0/4211783480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff028102b80 0x7ff0280fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 --2- 172.21.15.160:0/4211783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff0281025b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 -- 172.21.15.160:0/4211783480 >> 172.21.15.160:0/4211783480 conn(0x7ff0280f7db0 msgr2=0x7ff0280fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:12.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.291+0000 7ff03066e640 1 -- 172.21.15.160:0/4211783480 shutdown_connections 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 -- 172.21.15.160:0/4211783480 wait complete. 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 Processor -- start 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 -- start start 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff028112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff028102b80 0x7ff0281131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0281137c0 con 0x7ff0281021e0 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff03066e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02810ce10 con 0x7ff028102b80 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff02e40a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff028112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff02e40a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff028112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41202/0 (socket says 172.21.15.160:41202) 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff02e40a640 1 -- 172.21.15.160:0/331361714 learned_addr learned my addr 172.21.15.160:0/331361714 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff02e40a640 1 -- 172.21.15.160:0/331361714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff028102b80 msgr2=0x7ff0281131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.292+0000 7ff02e40a640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff028102b80 0x7ff0281131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff02e40a640 1 -- 172.21.15.160:0/331361714 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff024005040 con 0x7ff0281021e0 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff02e40a640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff028112c80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff024004930 tx=0x7ff024009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0240094c0 con 0x7ff0281021e0 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff024032e50 con 0x7ff0281021e0 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02810d090 con 0x7ff0281021e0 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0240433b0 con 0x7ff0281021e0 2024-05-25T06:38:12.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.293+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02810d550 con 0x7ff0281021e0 2024-05-25T06:38:12.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.294+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0280fd950 con 0x7ff0281021e0 2024-05-25T06:38:12.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.294+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff02402f070 con 0x7ff0281021e0 2024-05-25T06:38:12.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.295+0000 7ff01effd640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff01406e0b0 0x7ff014070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.295+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff0240afb90 con 0x7ff0281021e0 2024-05-25T06:38:12.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.295+0000 7ff02dc09640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff01406e0b0 0x7ff014070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.295+0000 7ff02dc09640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff01406e0b0 0x7ff014070560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff02810e540 tx=0x7ff018007c30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:12.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.299+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff024078ec0 con 0x7ff0281021e0 2024-05-25T06:38:12.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:12 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3096315392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-25T06:38:12.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:12 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/500967593' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-25T06:38:12.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:12 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1259274167' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-25T06:38:12.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.511+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7ff028109320 con 0x7ff0281021e0 2024-05-25T06:38:12.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.512+0000 7ff01effd640 1 -- 172.21.15.160:0/331361714 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v43) v1 ==== 107+0+5069 (secure 0 0 0) 0x7ff02407c580 con 0x7ff0281021e0 2024-05-25T06:38:12.512 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:12.512 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":44263,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/2242985365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2242985365},{"type":"v1","addr":"172.21.15.203:6827","nonce":2242985365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:12.512 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 35 2024-05-25T06:38:12.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff01406e0b0 msgr2=0x7ff014070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff01406e0b0 0x7ff014070560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff02810e540 tx=0x7ff018007c30 comp rx=0 tx=0).stop 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 msgr2=0x7ff028112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff028112c80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff024004930 tx=0x7ff024009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 shutdown_connections 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7ff01406e0b0 0x7ff014070560 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff028102b80 0x7ff0281131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.514+0000 7ff03066e640 1 --2- 172.21.15.160:0/331361714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281021e0 0x7ff028112c80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.515+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 >> 172.21.15.160:0/331361714 conn(0x7ff0280f7db0 msgr2=0x7ff0280f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.515+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 shutdown_connections 2024-05-25T06:38:12.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.515+0000 7ff03066e640 1 -- 172.21.15.160:0/331361714 wait complete. 2024-05-25T06:38:12.525 DEBUG:tasks.fs:max_mds not reduced in epoch 35 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:12.525 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-05-25T06:38:12.526 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-05-25T06:38:12.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 -- 172.21.15.160:0/1386636854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 msgr2=0x7f8be00fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 --2- 172.21.15.160:0/1386636854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be00fb5c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00048d0 tx=0x7f8bd0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 -- 172.21.15.160:0/1386636854 shutdown_connections 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 --2- 172.21.15.160:0/1386636854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be00fb5c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 --2- 172.21.15.160:0/1386636854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0101170 0x7f8be0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 -- 172.21.15.160:0/1386636854 >> 172.21.15.160:0/1386636854 conn(0x7f8be0071fd0 msgr2=0x7f8be00723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.622+0000 7f8be6eae640 1 -- 172.21.15.160:0/1386636854 shutdown_connections 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 -- 172.21.15.160:0/1386636854 wait complete. 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 Processor -- start 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 -- start start 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0101170 0x7f8be010e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00652f0 con 0x7f8be00fb150 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be6eae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be010e8d0 con 0x7f8be0101170 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be4c4a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be4c4a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41226/0 (socket says 172.21.15.160:41226) 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.623+0000 7f8be4c4a640 1 -- 172.21.15.160:0/2696601102 learned_addr learned my addr 172.21.15.160:0/2696601102 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8bd7fff640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0101170 0x7f8be010e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8be4c4a640 1 -- 172.21.15.160:0/2696601102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0101170 msgr2=0x7f8be010e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8be4c4a640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0101170 0x7f8be010e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8be4c4a640 1 -- 172.21.15.160:0/2696601102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0005040 con 0x7f8be00fb150 2024-05-25T06:38:12.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8be4c4a640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be010de50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00c790 tx=0x7f8bdc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:12.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc00a820 con 0x7f8be00fb150 2024-05-25T06:38:12.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be010ebb0 con 0x7f8be00fb150 2024-05-25T06:38:12.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bdc00ae50 con 0x7f8be00fb150 2024-05-25T06:38:12.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc0115e0 con 0x7f8be00fb150 2024-05-25T06:38:12.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.624+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be0117170 con 0x7f8be00fb150 2024-05-25T06:38:12.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.625+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be00fc870 con 0x7f8be00fb150 2024-05-25T06:38:12.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.626+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8bdc012400 con 0x7f8be00fb150 2024-05-25T06:38:12.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.626+0000 7f8bd57fa640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc806e0b0 0x7f8bc8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.626+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8bdc08dd30 con 0x7f8be00fb150 2024-05-25T06:38:12.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.626+0000 7f8bd7fff640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc806e0b0 0x7f8bc8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.626+0000 7f8bd7fff640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc806e0b0 0x7f8bc8070560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0003080 tx=0x7f8bd0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:12.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.631+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8bdc0570e0 con 0x7f8be00fb150 2024-05-25T06:38:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:12 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3096315392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-25T06:38:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:12 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/500967593' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-25T06:38:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:12 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1259274167' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-25T06:38:12.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.841+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f8be00fe3a0 con 0x7f8be00fb150 2024-05-25T06:38:12.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.841+0000 7f8bd57fa640 1 -- 172.21.15.160:0/2696601102 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v43) v1 ==== 107+0+4287 (secure 0 0 0) 0x7f8bdc05e0d0 con 0x7f8be00fb150 2024-05-25T06:38:12.841 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:12.841 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:12.841 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 36 2024-05-25T06:38:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc806e0b0 msgr2=0x7f8bc8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc806e0b0 0x7f8bc8070560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0003080 tx=0x7f8bd0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 msgr2=0x7f8be010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be010de50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00c790 tx=0x7f8bdc0043c0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 shutdown_connections 2024-05-25T06:38:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f8bc806e0b0 0x7f8bc8070560 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0101170 0x7f8be010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 --2- 172.21.15.160:0/2696601102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8be00fb150 0x7f8be010de50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 >> 172.21.15.160:0/2696601102 conn(0x7f8be0071fd0 msgr2=0x7f8be00fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 shutdown_connections 2024-05-25T06:38:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.843+0000 7f8be6eae640 1 -- 172.21.15.160:0/2696601102 wait complete. 2024-05-25T06:38:12.854 DEBUG:tasks.fs:max_mds not reduced in epoch 36 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:12.854 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-05-25T06:38:12.855 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-05-25T06:38:12.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.953+0000 7f3433814640 1 -- 172.21.15.160:0/576059991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 msgr2=0x7f342c064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.953+0000 7f3433814640 1 --2- 172.21.15.160:0/576059991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c064800 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34200048d0 tx=0x7f3420009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.953+0000 7f3433814640 1 -- 172.21.15.160:0/576059991 shutdown_connections 2024-05-25T06:38:12.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.953+0000 7f3433814640 1 --2- 172.21.15.160:0/576059991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c064800 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.953+0000 7f3433814640 1 --2- 172.21.15.160:0/576059991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 0x7f342c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.953+0000 7f3433814640 1 -- 172.21.15.160:0/576059991 >> 172.21.15.160:0/576059991 conn(0x7f342c071fd0 msgr2=0x7f342c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 -- 172.21.15.160:0/576059991 shutdown_connections 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 -- 172.21.15.160:0/576059991 wait complete. 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 Processor -- start 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 -- start start 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c114e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 0x7f342c1153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1159b0 con 0x7f342c064390 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f3433814640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c10f000 con 0x7f342c102270 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f34315b0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c114e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f34315b0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c114e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41250/0 (socket says 172.21.15.160:41250) 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.954+0000 7f34315b0640 1 -- 172.21.15.160:0/1699871611 learned_addr learned my addr 172.21.15.160:0/1699871611 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f34315b0640 1 -- 172.21.15.160:0/1699871611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 msgr2=0x7f342c1153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:12.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3430daf640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 0x7f342c1153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f34315b0640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 0x7f342c1153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f34315b0640 1 -- 172.21.15.160:0/1699871611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3420005040 con 0x7f342c064390 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3430daf640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 0x7f342c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f34315b0640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c114e70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3424002a50 tx=0x7f3424004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3424007c30 con 0x7f342c064390 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f342400f040 con 0x7f342c064390 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c10f2e0 con 0x7f342c064390 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342400db50 con 0x7f342c064390 2024-05-25T06:38:12.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.955+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c10f800 con 0x7f342c064390 2024-05-25T06:38:12.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.956+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c0fd970 con 0x7f342c064390 2024-05-25T06:38:12.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.956+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f342400dcf0 con 0x7f342c064390 2024-05-25T06:38:12.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.957+0000 7f3429ffb640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f341406e0b0 0x7f3414070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:12.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.957+0000 7f3430daf640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f341406e0b0 0x7f3414070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:12.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.957+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f342408d3d0 con 0x7f342c064390 2024-05-25T06:38:12.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.957+0000 7f3430daf640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f341406e0b0 0x7f3414070560 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3420000c00 tx=0x7f3420009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:12.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:12.962+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3424056780 con 0x7f342c064390 2024-05-25T06:38:13.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.179+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f342c063f00 con 0x7f342c064390 2024-05-25T06:38:13.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.179+0000 7f3429ffb640 1 -- 172.21.15.160:0/1699871611 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v43) v1 ==== 107+0+5134 (secure 0 0 0) 0x7f3424059e40 con 0x7f342c064390 2024-05-25T06:38:13.179 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:13.180 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:20:11.153647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:13.180 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 37 2024-05-25T06:38:13.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f341406e0b0 msgr2=0x7f3414070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f341406e0b0 0x7f3414070560 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3420000c00 tx=0x7f3420009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 msgr2=0x7f342c114e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c114e70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3424002a50 tx=0x7f3424004600 comp rx=0 tx=0).stop 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 shutdown_connections 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f341406e0b0 0x7f3414070560 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c102270 0x7f342c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 --2- 172.21.15.160:0/1699871611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c064390 0x7f342c114e70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 >> 172.21.15.160:0/1699871611 conn(0x7f342c071fd0 msgr2=0x7f342c0fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 shutdown_connections 2024-05-25T06:38:13.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.181+0000 7f3433814640 1 -- 172.21.15.160:0/1699871611 wait complete. 2024-05-25T06:38:13.193 DEBUG:tasks.fs:max_mds not reduced in epoch 37 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-25T06:38:13.193 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-05-25T06:38:13.193 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-05-25T06:38:13.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.292+0000 7f8393b5d640 1 -- 172.21.15.160:0/2972167395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 msgr2=0x7f838c0fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.292+0000 7f8393b5d640 1 --2- 172.21.15.160:0/2972167395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c0fb5c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f83800048d0 tx=0x7f8380009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.292+0000 7f8393b5d640 1 -- 172.21.15.160:0/2972167395 shutdown_connections 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 --2- 172.21.15.160:0/2972167395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c0fb5c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 --2- 172.21.15.160:0/2972167395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838c101170 0x7f838c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 -- 172.21.15.160:0/2972167395 >> 172.21.15.160:0/2972167395 conn(0x7f838c071fd0 msgr2=0x7f838c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 -- 172.21.15.160:0/2972167395 shutdown_connections 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 -- 172.21.15.160:0/2972167395 wait complete. 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 Processor -- start 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 -- start start 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838c101170 0x7f838c112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c1127f0 con 0x7f838c0fb150 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f8393b5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c10bdb0 con 0x7f838c101170 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f83918f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f83918f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c111c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41264/0 (socket says 172.21.15.160:41264) 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.293+0000 7f83918f9640 1 -- 172.21.15.160:0/1824784774 learned_addr learned my addr 172.21.15.160:0/1824784774 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f83918f9640 1 -- 172.21.15.160:0/1824784774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838c101170 msgr2=0x7f838c112160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f83918f9640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838c101170 0x7f838c112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f83918f9640 1 -- 172.21.15.160:0/1824784774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380005040 con 0x7f838c0fb150 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f83918f9640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c111c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f837c00bbb0 tx=0x7f837c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837c00c9c0 con 0x7f838c0fb150 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f837c0092e0 con 0x7f838c0fb150 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c10c090 con 0x7f838c0fb150 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837c007640 con 0x7f838c0fb150 2024-05-25T06:38:13.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.294+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c10c5b0 con 0x7f838c0fb150 2024-05-25T06:38:13.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.295+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c0fc870 con 0x7f838c0fb150 2024-05-25T06:38:13.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.295+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f837c012070 con 0x7f838c0fb150 2024-05-25T06:38:13.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.296+0000 7f838a7fc640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f837406e0b0 0x7f8374070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.296+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f837c08dbf0 con 0x7f838c0fb150 2024-05-25T06:38:13.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.296+0000 7f83910f8640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f837406e0b0 0x7f8374070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.296+0000 7f83910f8640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f837406e0b0 0x7f8374070560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f83800043d0 tx=0x7f8380009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:13.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.301+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f837c094080 con 0x7f838c0fb150 2024-05-25T06:38:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/331361714' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-25T06:38:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2696601102' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-25T06:38:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:13 smithi160 ceph-mon[112094]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:13.471 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:13 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1699871611' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-25T06:38:13.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.514+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f838c063f00 con 0x7f838c0fb150 2024-05-25T06:38:13.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.515+0000 7f838a7fc640 1 -- 172.21.15.160:0/1824784774 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v43) v1 ==== 107+0+5224 (secure 0 0 0) 0x7f837c056f20 con 0x7f838c0fb150 2024-05-25T06:38:13.515 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:13.515 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:36:13.079474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34416":{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":1,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:13.515 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 38 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f837406e0b0 msgr2=0x7f8374070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f837406e0b0 0x7f8374070560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f83800043d0 tx=0x7f8380009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 msgr2=0x7f838c111c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c111c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f837c00bbb0 tx=0x7f837c009840 comp rx=0 tx=0).stop 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 shutdown_connections 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f837406e0b0 0x7f8374070560 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838c101170 0x7f838c112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 --2- 172.21.15.160:0/1824784774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f838c0fb150 0x7f838c111c20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 >> 172.21.15.160:0/1824784774 conn(0x7f838c071fd0 msgr2=0x7f838c0fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:13.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 shutdown_connections 2024-05-25T06:38:13.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.517+0000 7f8393b5d640 1 -- 172.21.15.160:0/1824784774 wait complete. 2024-05-25T06:38:13.528 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 38 2024-05-25T06:38:13.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2282897788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 msgr2=0x7f2bc80fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2282897788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fb0a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0048d0 tx=0x7f2bbc009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2282897788 shutdown_connections 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2282897788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fb0a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2282897788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 0x7f2bc8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2282897788 >> 172.21.15.160:0/2282897788 conn(0x7f2bc80f6be0 msgr2=0x7f2bc80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2282897788 shutdown_connections 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.626+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2282897788 wait complete. 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bcd31d640 1 Processor -- start 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bcd31d640 1 -- start start 2024-05-25T06:38:13.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bcd31d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 0x7f2bc80fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bcd31d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bcd31d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80febf0 con 0x7f2bc8101a10 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bcd31d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80fd790 con 0x7f2bc8101070 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc67fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc67fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41268/0 (socket says 172.21.15.160:41268) 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc67fc640 1 -- 172.21.15.160:0/2804589654 learned_addr learned my addr 172.21.15.160:0/2804589654 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc6ffd640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 0x7f2bc80fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc67fc640 1 -- 172.21.15.160:0/2804589654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 msgr2=0x7f2bc80fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc67fc640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 0x7f2bc80fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc67fc640 1 -- 172.21.15.160:0/2804589654 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc005040 con 0x7f2bc8101a10 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.627+0000 7f2bc6ffd640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 0x7f2bc80fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.628+0000 7f2bc67fc640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fd250 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc004930 tx=0x7f2bbc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:13.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.628+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bbc0094c0 con 0x7f2bc8101a10 2024-05-25T06:38:13.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.628+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc80fda10 con 0x7f2bc8101a10 2024-05-25T06:38:13.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.628+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bbc032e50 con 0x7f2bc8101a10 2024-05-25T06:38:13.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.628+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bbc0433b0 con 0x7f2bc8101a10 2024-05-25T06:38:13.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.628+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc8119370 con 0x7f2bc8101a10 2024-05-25T06:38:13.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.629+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc80fc7e0 con 0x7f2bc8101a10 2024-05-25T06:38:13.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.629+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2bbc02f070 con 0x7f2bc8101a10 2024-05-25T06:38:13.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.630+0000 7f2baf7fe640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2bb006e0b0 0x7f2bb0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.630+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2bbc0afb90 con 0x7f2bc8101a10 2024-05-25T06:38:13.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.630+0000 7f2bc6ffd640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2bb006e0b0 0x7f2bb0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.630+0000 7f2bc6ffd640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2bb006e0b0 0x7f2bb0070560 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8004c80 tx=0x7f2bb8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:13.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.634+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2bbc078ec0 con 0x7f2bc8101a10 2024-05-25T06:38:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/331361714' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-25T06:38:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2696601102' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-25T06:38:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:13 smithi203 ceph-mon[83507]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:13 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1699871611' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-25T06:38:13.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.847+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f2bc8064350 con 0x7f2bc8101a10 2024-05-25T06:38:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.848+0000 7f2baf7fe640 1 -- 172.21.15.160:0/2804589654 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v43) v1 ==== 107+0+5223 (secure 0 0 0) 0x7f2bbc07c580 con 0x7f2bc8101a10 2024-05-25T06:38:13.848 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:13.848 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:36:13.084659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34416":{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":1,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:13.848 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 39 2024-05-25T06:38:13.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2bb006e0b0 msgr2=0x7f2bb0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2bb006e0b0 0x7f2bb0070560 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8004c80 tx=0x7f2bb8009040 comp rx=0 tx=0).stop 2024-05-25T06:38:13.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 msgr2=0x7f2bc80fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fd250 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc004930 tx=0x7f2bbc009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 shutdown_connections 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f2bb006e0b0 0x7f2bb0070560 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc8101a10 0x7f2bc80fd250 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 --2- 172.21.15.160:0/2804589654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc8101070 0x7f2bc80fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 >> 172.21.15.160:0/2804589654 conn(0x7f2bc80f6be0 msgr2=0x7f2bc80f7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 shutdown_connections 2024-05-25T06:38:13.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.850+0000 7f2bcd31d640 1 -- 172.21.15.160:0/2804589654 wait complete. 2024-05-25T06:38:13.861 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 -- 172.21.15.160:0/1017401674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 msgr2=0x7f7150101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 --2- 172.21.15.160:0/1017401674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150101440 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f71400048d0 tx=0x7f7140009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 -- 172.21.15.160:0/1017401674 shutdown_connections 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 --2- 172.21.15.160:0/1017401674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7150101a10 0x7f71500fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 --2- 172.21.15.160:0/1017401674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150101440 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 -- 172.21.15.160:0/1017401674 >> 172.21.15.160:0/1017401674 conn(0x7f71500f6be0 msgr2=0x7f71500f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 -- 172.21.15.160:0/1017401674 shutdown_connections 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.959+0000 7f715542e640 1 -- 172.21.15.160:0/1017401674 wait complete. 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f715542e640 1 Processor -- start 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f715542e640 1 -- start start 2024-05-25T06:38:13.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f715542e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f715542e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7150101a10 0x7f71501120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f715542e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501126e0 con 0x7f7150101070 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f715542e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715010bd30 con 0x7f7150101a10 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f714effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f714effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150111ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41286/0 (socket says 172.21.15.160:41286) 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f714effd640 1 -- 172.21.15.160:0/372213723 learned_addr learned my addr 172.21.15.160:0/372213723 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f714effd640 1 -- 172.21.15.160:0/372213723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7150101a10 msgr2=0x7f71501120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f714effd640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7150101a10 0x7f71501120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.960+0000 7f714effd640 1 -- 172.21.15.160:0/372213723 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140005040 con 0x7f7150101070 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.961+0000 7f714effd640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150111ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7140004930 tx=0x7f7140009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.961+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71400094c0 con 0x7f7150101070 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.961+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7140032e50 con 0x7f7150101070 2024-05-25T06:38:13.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.961+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715010bfb0 con 0x7f7150101070 2024-05-25T06:38:13.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.961+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71400433b0 con 0x7f7150101070 2024-05-25T06:38:13.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.961+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715010c470 con 0x7f7150101070 2024-05-25T06:38:13.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.962+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71500fc7e0 con 0x7f7150101070 2024-05-25T06:38:13.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.962+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f714002f070 con 0x7f7150101070 2024-05-25T06:38:13.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.963+0000 7f71477fe640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f713c06e0b0 0x7f713c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:13.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.963+0000 7f7147fff640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f713c06e0b0 0x7f713c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:13.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.963+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f71400afa40 con 0x7f7150101070 2024-05-25T06:38:13.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.963+0000 7f7147fff640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f713c06e0b0 0x7f713c070560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f715010d460 tx=0x7f7138009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:13.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:13.968+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7140078d70 con 0x7f7150101070 2024-05-25T06:38:14.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.181+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f71501081d0 con 0x7f7150101070 2024-05-25T06:38:14.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.181+0000 7f71477fe640 1 -- 172.21.15.160:0/372213723 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v43) v1 ==== 107+0+5229 (secure 0 0 0) 0x7f714007c430 con 0x7f7150101070 2024-05-25T06:38:14.181 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:14.181 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:36:19.819400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":38,"state":"up:resolve","state_seq":241,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34416":{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":1,"incarnation":38,"state":"up:resolve","state_seq":236,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:14.182 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 40 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f713c06e0b0 msgr2=0x7f713c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f713c06e0b0 0x7f713c070560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f715010d460 tx=0x7f7138009c50 comp rx=0 tx=0).stop 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 msgr2=0x7f7150111ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150111ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7140004930 tx=0x7f7140009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 shutdown_connections 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f713c06e0b0 0x7f713c070560 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7150101a10 0x7f71501120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 --2- 172.21.15.160:0/372213723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7150101070 0x7f7150111ba0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 >> 172.21.15.160:0/372213723 conn(0x7f71500f6be0 msgr2=0x7f71500f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 shutdown_connections 2024-05-25T06:38:14.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.184+0000 7f715542e640 1 -- 172.21.15.160:0/372213723 wait complete. 2024-05-25T06:38:14.195 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-05-25T06:38:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 -- 172.21.15.160:0/4020916815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 msgr2=0x7f822c0fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 --2- 172.21.15.160:0/4020916815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c0fb0a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82200048d0 tx=0x7f8220009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 -- 172.21.15.160:0/4020916815 shutdown_connections 2024-05-25T06:38:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 --2- 172.21.15.160:0/4020916815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c0fb0a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 --2- 172.21.15.160:0/4020916815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 0x7f822c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 -- 172.21.15.160:0/4020916815 >> 172.21.15.160:0/4020916815 conn(0x7f822c0710c0 msgr2=0x7f822c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 -- 172.21.15.160:0/4020916815 shutdown_connections 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.293+0000 7f823166e640 1 -- 172.21.15.160:0/4020916815 wait complete. 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f823166e640 1 Processor -- start 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f823166e640 1 -- start start 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f823166e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 0x7f822c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f823166e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c10e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f823166e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0652f0 con 0x7f822c1019b0 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f823166e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c10e910 con 0x7f822c101010 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f822a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c10e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f822a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c10e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41302/0 (socket says 172.21.15.160:41302) 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.294+0000 7f822a7fc640 1 -- 172.21.15.160:0/3114362290 learned_addr learned my addr 172.21.15.160:0/3114362290 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f822a7fc640 1 -- 172.21.15.160:0/3114362290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 msgr2=0x7f822c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f822affd640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 0x7f822c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:14.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f822a7fc640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 0x7f822c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f822a7fc640 1 -- 172.21.15.160:0/3114362290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220005040 con 0x7f822c1019b0 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f822affd640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 0x7f822c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f822a7fc640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c10e3d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f822002bda0 tx=0x7f8220009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82200094c0 con 0x7f822c1019b0 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8220032e50 con 0x7f822c1019b0 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c10eb90 con 0x7f822c1019b0 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822003b640 con 0x7f822c1019b0 2024-05-25T06:38:14.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.295+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c117150 con 0x7f822c1019b0 2024-05-25T06:38:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.296+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c0fc790 con 0x7f822c1019b0 2024-05-25T06:38:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.297+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f822002f070 con 0x7f822c1019b0 2024-05-25T06:38:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.297+0000 7f82137fe640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f821406e0b0 0x7f8214070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.297+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f82200b0000 con 0x7f822c1019b0 2024-05-25T06:38:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.297+0000 7f822affd640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f821406e0b0 0x7f8214070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:14.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.298+0000 7f822affd640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f821406e0b0 0x7f8214070560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f821c004e80 tx=0x7f821c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:14.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.302+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8220079330 con 0x7f822c1019b0 2024-05-25T06:38:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:14 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1824784774' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-05-25T06:38:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:14 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2804589654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-05-25T06:38:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:14 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/372213723' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-05-25T06:38:14.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.517+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f822c04bcc0 con 0x7f822c1019b0 2024-05-25T06:38:14.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.517+0000 7f82137fe640 1 -- 172.21.15.160:0/3114362290 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v43) v1 ==== 107+0+5233 (secure 0 0 0) 0x7f822002d040 con 0x7f822c1019b0 2024-05-25T06:38:14.517 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:14.517 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:36:20.121214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":38,"state":"up:reconnect","state_seq":242,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34416":{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":1,"incarnation":38,"state":"up:reconnect","state_seq":237,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:14.518 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 41 2024-05-25T06:38:14.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f821406e0b0 msgr2=0x7f8214070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f821406e0b0 0x7f8214070560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f821c004e80 tx=0x7f821c009040 comp rx=0 tx=0).stop 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 msgr2=0x7f822c10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c10e3d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f822002bda0 tx=0x7f8220009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 shutdown_connections 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f821406e0b0 0x7f8214070560 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f822c1019b0 0x7f822c10e3d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 --2- 172.21.15.160:0/3114362290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c101010 0x7f822c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.519+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 >> 172.21.15.160:0/3114362290 conn(0x7f822c0710c0 msgr2=0x7f822c103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.520+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 shutdown_connections 2024-05-25T06:38:14.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.520+0000 7f823166e640 1 -- 172.21.15.160:0/3114362290 wait complete. 2024-05-25T06:38:14.531 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 42 2024-05-25T06:38:14.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.628+0000 7f0a55861640 1 -- 172.21.15.160:0/665785917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 msgr2=0x7f0a501061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.628+0000 7f0a55861640 1 --2- 172.21.15.160:0/665785917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a501061a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a440048d0 tx=0x7f0a44009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.628+0000 7f0a55861640 1 -- 172.21.15.160:0/665785917 shutdown_connections 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 --2- 172.21.15.160:0/665785917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a501061a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 --2- 172.21.15.160:0/665785917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 0x7f0a500fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 -- 172.21.15.160:0/665785917 >> 172.21.15.160:0/665785917 conn(0x7f0a500f6c50 msgr2=0x7f0a500f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 -- 172.21.15.160:0/665785917 shutdown_connections 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 -- 172.21.15.160:0/665785917 wait complete. 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 Processor -- start 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 -- start start 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 0x7f0a50112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a50113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a50113860 con 0x7f0a500fdb60 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.629+0000 7f0a55861640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5010ceb0 con 0x7f0a500fd1c0 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a50113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:14.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a50113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41316/0 (socket says 172.21.15.160:41316) 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 -- 172.21.15.160:0/1084117318 learned_addr learned my addr 172.21.15.160:0/1084117318 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4effd640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 0x7f0a50112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 -- 172.21.15.160:0/1084117318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 msgr2=0x7f0a50112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 0x7f0a50112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 -- 172.21.15.160:0/1084117318 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44005040 con 0x7f0a500fdb60 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4effd640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 0x7f0a50112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a4e7fc640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a50113260 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0a44004930 tx=0x7f0a44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a440094c0 con 0x7f0a500fdb60 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5010d130 con 0x7f0a500fdb60 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a44032e50 con 0x7f0a500fdb60 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a440433b0 con 0x7f0a500fdb60 2024-05-25T06:38:14.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.630+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5010d5f0 con 0x7f0a500fdb60 2024-05-25T06:38:14.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.631+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a50066040 con 0x7f0a500fdb60 2024-05-25T06:38:14.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.632+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0a4402f070 con 0x7f0a500fdb60 2024-05-25T06:38:14.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.632+0000 7f0a377fe640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0a3806e0b0 0x7f0a38070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:14.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.632+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0a440af9f0 con 0x7f0a500fdb60 2024-05-25T06:38:14.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.632+0000 7f0a4effd640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0a3806e0b0 0x7f0a38070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:14.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.633+0000 7f0a4effd640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0a3806e0b0 0x7f0a38070560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a40007fd0 tx=0x7f0a40009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:14.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.637+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0a44078d20 con 0x7f0a500fdb60 2024-05-25T06:38:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:14 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1824784774' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-05-25T06:38:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:14 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2804589654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-05-25T06:38:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:14 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/372213723' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-05-25T06:38:14.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.851+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7f0a50109310 con 0x7f0a500fdb60 2024-05-25T06:38:14.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.851+0000 7f0a377fe640 1 -- 172.21.15.160:0/1084117318 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v43) v1 ==== 107+0+5227 (secure 0 0 0) 0x7f0a4407c3e0 con 0x7f0a500fdb60 2024-05-25T06:38:14.851 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:14.851 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi203.yltymd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6825/1855238749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1855238749},{"type":"v1","addr":"172.21.15.203:6825","nonce":1855238749}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":44335,"name":"cephfs.smithi203.tndnrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/370625010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":370625010},{"type":"v1","addr":"172.21.15.203:6827","nonce":370625010}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-25T06:08:21.960553+0000","modified":"2024-05-25T06:36:21.097816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi160.rrhzig","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":243,"addr":"172.21.15.160:6827/1526023017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1526023017},{"type":"v1","addr":"172.21.15.160:6827","nonce":1526023017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34416":{"gid":34416,"name":"cephfs.smithi160.tlctof","rank":1,"incarnation":38,"state":"up:rejoin","state_seq":238,"addr":"172.21.15.160:6829/2801259285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2801259285},{"type":"v1","addr":"172.21.15.160:6829","nonce":2801259285}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-25T06:38:14.852 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 42 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0a3806e0b0 msgr2=0x7f0a38070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0a3806e0b0 0x7f0a38070560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a40007fd0 tx=0x7f0a40009e90 comp rx=0 tx=0).stop 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 msgr2=0x7f0a50113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a50113260 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0a44004930 tx=0x7f0a44009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 shutdown_connections 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f0a3806e0b0 0x7f0a38070560 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a500fdb60 0x7f0a50113260 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 --2- 172.21.15.160:0/1084117318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a500fd1c0 0x7f0a50112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 >> 172.21.15.160:0/1084117318 conn(0x7f0a500f6c50 msgr2=0x7f0a500f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:14.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 shutdown_connections 2024-05-25T06:38:14.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:14.854+0000 7f0a55861640 1 -- 172.21.15.160:0/1084117318 wait complete. 2024-05-25T06:38:14.865 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-25T06:38:14.875 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-25T06:38:14.876 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:38:14.876 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-25T06:38:14.892 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:38:14.893 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-25T06:38:14.950 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-25T06:38:15.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.085+0000 7fb5a30ae640 1 -- 172.21.15.160:0/4238252847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101100 msgr2=0x7fb59c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.085+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/4238252847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101100 0x7fb59c1014d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb5980048d0 tx=0x7fb598009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- 172.21.15.160:0/4238252847 shutdown_connections 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/4238252847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb59c101aa0 0x7fb59c0fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/4238252847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101100 0x7fb59c1014d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- 172.21.15.160:0/4238252847 >> 172.21.15.160:0/4238252847 conn(0x7fb59c0711c0 msgr2=0x7fb59c0715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- 172.21.15.160:0/4238252847 shutdown_connections 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- 172.21.15.160:0/4238252847 wait complete. 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 Processor -- start 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- start start 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb59c101100 0x7fb59c111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 0x7fb59c1120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1126f0 con 0x7fb59c101aa0 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.086+0000 7fb5a30ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c112860 con 0x7fb59c101100 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 0x7fb59c1120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 0x7fb59c1120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41340/0 (socket says 172.21.15.160:41340) 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 -- 172.21.15.160:0/3725102235 learned_addr learned my addr 172.21.15.160:0/3725102235 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb5a0e4a640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb59c101100 0x7fb59c111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 -- 172.21.15.160:0/3725102235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb59c101100 msgr2=0x7fb59c111bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb59c101100 0x7fb59c111bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 -- 172.21.15.160:0/3725102235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598005040 con 0x7fb59c101aa0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb593fff640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 0x7fb59c1120f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb58c002840 tx=0x7fb58c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c007c30 con 0x7fb59c101aa0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c10bf60 con 0x7fb59c101aa0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb58c00f040 con 0x7fb59c101aa0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c009570 con 0x7fb59c101aa0 2024-05-25T06:38:15.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.087+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c10c480 con 0x7fb59c101aa0 2024-05-25T06:38:15.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.088+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c0fc880 con 0x7fb59c101aa0 2024-05-25T06:38:15.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.089+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb58c00dc80 con 0x7fb59c101aa0 2024-05-25T06:38:15.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.089+0000 7fb5917fa640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb58406e0b0 0x7fb584070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.089+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb58c08d440 con 0x7fb59c101aa0 2024-05-25T06:38:15.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.089+0000 7fb5a0e4a640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb58406e0b0 0x7fb584070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.090+0000 7fb5a0e4a640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb58406e0b0 0x7fb584070560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb598031870 tx=0x7fb598009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.094+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb58c056770 con 0x7fb59c101aa0 2024-05-25T06:38:15.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.276+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb59c04bcc0 con 0x7fb59c101aa0 2024-05-25T06:38:15.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.277+0000 7fb5917fa640 1 -- 172.21.15.160:0/3725102235 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16719 (secure 0 0 0) 0x7fb58c059e30 con 0x7fb59c101aa0 2024-05-25T06:38:15.277 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":85,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:21:08.124903+0000","last_up_change":"2024-05-25T06:20:08.969929+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-25T06:08:20.895353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-25T06:08:21.186881+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6803","nonce":793926185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6805","nonce":793926185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6809","nonce":793926185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6807","nonce":793926185}]},"public_addr":"172.21.15.160:6803/793926185","cluster_addr":"172.21.15.160:6805/793926185","heartbeat_back_addr":"172.21.15.160:6809/793926185","heartbeat_front_addr":"172.21.15.160:6807/793926185","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6811","nonce":3648042119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6813","nonce":3648042119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6817","nonce":3648042119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6815","nonce":3648042119}]},"public_addr":"172.21.15.160:6811/3648042119","cluster_addr":"172.21.15.160:6813/3648042119","heartbeat_back_addr":"172.21.15.160:6817/3648042119","heartbeat_front_addr":"172.21.15.160:6815/3648042119","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6819","nonce":1307550666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6821","nonce":1307550666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6825","nonce":1307550666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6823","nonce":1307550666}]},"public_addr":"172.21.15.160:6819/1307550666","cluster_addr":"172.21.15.160:6821/1307550666","heartbeat_back_addr":"172.21.15.160:6825/1307550666","heartbeat_front_addr":"172.21.15.160:6823/1307550666","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6801","nonce":316097287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6803","nonce":316097287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6807","nonce":316097287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6805","nonce":316097287}]},"public_addr":"172.21.15.203:6801/316097287","cluster_addr":"172.21.15.203:6803/316097287","heartbeat_back_addr":"172.21.15.203:6807/316097287","heartbeat_front_addr":"172.21.15.203:6805/316097287","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6809","nonce":2420151889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6811","nonce":2420151889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6815","nonce":2420151889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6813","nonce":2420151889}]},"public_addr":"172.21.15.203:6809/2420151889","cluster_addr":"172.21.15.203:6811/2420151889","heartbeat_back_addr":"172.21.15.203:6815/2420151889","heartbeat_front_addr":"172.21.15.203:6813/2420151889","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6817","nonce":2019898829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6819","nonce":2019898829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6823","nonce":2019898829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6821","nonce":2019898829}]},"public_addr":"172.21.15.203:6817/2019898829","cluster_addr":"172.21.15.203:6819/2019898829","heartbeat_back_addr":"172.21.15.203:6823/2019898829","heartbeat_front_addr":"172.21.15.203:6821/2019898829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6829/3783773637":"2024-05-26T06:20:11.153443+0000","172.21.15.160:6828/3783773637":"2024-05-26T06:20:11.153443+0000","172.21.15.160:6800/4089125744":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000","172.21.15.203:6824/3325193142":"2024-05-26T06:20:11.153460+0000","172.21.15.203:0/892364096":"2024-05-26T06:12:42.594272+0000","172.21.15.160:6826/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/2473068109":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.203:6825/3325193142":"2024-05-26T06:20:11.153460+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.203:0/3131222858":"2024-05-26T06:12:42.594272+0000","172.21.15.203:6828/2357665398":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/2019174179":"2024-05-26T06:11:04.538090+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1884499665":"2024-05-26T06:15:42.839541+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.203:0/2025926580":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/1559445112":"2024-05-26T06:15:42.839541+0000","172.21.15.160:6801/4089125744":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1251129035":"2024-05-26T06:11:04.538090+0000","172.21.15.203:0/1389978098":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1144786189":"2024-05-26T06:15:42.839541+0000","172.21.15.203:6829/2357665398":"2024-05-26T06:12:42.594272+0000","172.21.15.160:6827/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6800/1875516529":"2024-05-26T06:15:42.839541+0000","172.21.15.160:6801/1875516529":"2024-05-26T06:15:42.839541+0000","172.21.15.160:0/3623795045":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/3363261834":"2024-05-26T06:15:42.839541+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:38:15.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.279+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb58406e0b0 msgr2=0x7fb584070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.279+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb58406e0b0 0x7fb584070560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb598031870 tx=0x7fb598009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.279+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 msgr2=0x7fb59c1120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.279+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 0x7fb59c1120f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb58c002840 tx=0x7fb58c004570 comp rx=0 tx=0).stop 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 shutdown_connections 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fb58406e0b0 0x7fb584070560 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb59c101aa0 0x7fb59c1120f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 --2- 172.21.15.160:0/3725102235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb59c101100 0x7fb59c111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 >> 172.21.15.160:0/3725102235 conn(0x7fb59c0711c0 msgr2=0x7fb59c0faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 shutdown_connections 2024-05-25T06:38:15.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.280+0000 7fb5a30ae640 1 -- 172.21.15.160:0/3725102235 wait complete. 2024-05-25T06:38:15.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-25T06:05:48.910750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-25T06:08:20.895353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-25T06:08:21.186881+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-25T06:38:15.291 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.390+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/673403974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fdb60 msgr2=0x7fd4d4106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.390+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/673403974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fdb60 0x7fd4d4106230 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc004900 tx=0x7fd4bc009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.390+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/673403974 shutdown_connections 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.390+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/673403974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fdb60 0x7fd4d4106230 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.390+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/673403974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d40fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.390+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/673403974 >> 172.21.15.160:0/673403974 conn(0x7fd4d40f6c60 msgr2=0x7fd4d40f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.391+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/673403974 shutdown_connections 2024-05-25T06:38:15.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.391+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/673403974 wait complete. 2024-05-25T06:38:15.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.391+0000 7fd4d9a2e640 1 Processor -- start 2024-05-25T06:38:15.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.391+0000 7fd4d9a2e640 1 -- start start 2024-05-25T06:38:15.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.391+0000 7fd4d9a2e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d41130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.391+0000 7fd4d9a2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d40fdb60 0x7fd4d41135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d9a2e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4113bf0 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d9a2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d410d290 con 0x7fd4d40fdb60 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d41130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d41130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41370/0 (socket says 172.21.15.160:41370) 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 -- 172.21.15.160:0/4168776602 learned_addr learned my addr 172.21.15.160:0/4168776602 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4ca7fc640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d40fdb60 0x7fd4d41135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 -- 172.21.15.160:0/4168776602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d40fdb60 msgr2=0x7fd4d41135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d40fdb60 0x7fd4d41135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 -- 172.21.15.160:0/4168776602 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc005040 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d2ffd640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d41130b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c830 tx=0x7fd4c4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4c400a850 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d410d570 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4c400ae80 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4c4012ea0 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.392+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d410da90 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.393+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4065d70 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.394+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd4c4011640 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.394+0000 7fd4cbfff640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd4c006e0b0 0x7fd4c0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.394+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd4c408ddf0 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.394+0000 7fd4ca7fc640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd4c006e0b0 0x7fd4c0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.395+0000 7fd4ca7fc640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd4c006e0b0 0x7fd4c0070560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.399+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd4c4057210 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.574+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4d410e720 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.575+0000 7fd4cbfff640 1 -- 172.21.15.160:0/4168776602 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4c405a8d0 con 0x7fd4d40fd1c0 2024-05-25T06:38:15.575 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.576+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd4c006e0b0 msgr2=0x7fd4c0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.576+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd4c006e0b0 0x7fd4c0070560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.576+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 msgr2=0x7fd4d41130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.576+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d41130b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c830 tx=0x7fd4c4004430 comp rx=0 tx=0).stop 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 shutdown_connections 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fd4c006e0b0 0x7fd4c0070560 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d40fdb60 0x7fd4d41135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 --2- 172.21.15.160:0/4168776602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4d40fd1c0 0x7fd4d41130b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 >> 172.21.15.160:0/4168776602 conn(0x7fd4d40f6c60 msgr2=0x7fd4d40f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 shutdown_connections 2024-05-25T06:38:15.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.577+0000 7fd4d9a2e640 1 -- 172.21.15.160:0/4168776602 wait complete. 2024-05-25T06:38:15.586 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:15 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3114362290' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-05-25T06:38:15.586 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:15 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1084117318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-05-25T06:38:15.586 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:15 smithi160 ceph-mon[112094]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:15.587 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-25T06:38:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:15 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3114362290' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-05-25T06:38:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:15 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1084117318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-05-25T06:38:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:15 smithi203 ceph-mon[83507]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 -- 172.21.15.160:0/2577628495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102190 msgr2=0x7fc710102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 --2- 172.21.15.160:0/2577628495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102190 0x7fc710102560 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0048d0 tx=0x7fc70c009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 -- 172.21.15.160:0/2577628495 shutdown_connections 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 --2- 172.21.15.160:0/2577628495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102b30 0x7fc7100fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 --2- 172.21.15.160:0/2577628495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102190 0x7fc710102560 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 -- 172.21.15.160:0/2577628495 >> 172.21.15.160:0/2577628495 conn(0x7fc7100f7da0 msgr2=0x7fc7100fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 -- 172.21.15.160:0/2577628495 shutdown_connections 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.687+0000 7fc71759d640 1 -- 172.21.15.160:0/2577628495 wait complete. 2024-05-25T06:38:15.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc71759d640 1 Processor -- start 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc71759d640 1 -- start start 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc71759d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102190 0x7fc710112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc71759d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 0x7fc710113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc71759d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc710113840 con 0x7fc710102b30 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc71759d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71010ce90 con 0x7fc710102190 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 0x7fc710113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 0x7fc710113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33832/0 (socket says 172.21.15.160:33832) 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 -- 172.21.15.160:0/2473718735 learned_addr learned my addr 172.21.15.160:0/2473718735 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc715339640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102190 0x7fc710112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 -- 172.21.15.160:0/2473718735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102190 msgr2=0x7fc710112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102190 0x7fc710112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 -- 172.21.15.160:0/2473718735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c005040 con 0x7fc710102b30 2024-05-25T06:38:15.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc715339640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102190 0x7fc710112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc714b38640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 0x7fc710113240 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc700002a50 tx=0x7fc700004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.688+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc700007c30 con 0x7fc710102b30 2024-05-25T06:38:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.689+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71010d170 con 0x7fc710102b30 2024-05-25T06:38:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.689+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc70000f040 con 0x7fc710102b30 2024-05-25T06:38:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.689+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc70000db50 con 0x7fc710102b30 2024-05-25T06:38:15.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.689+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71010d690 con 0x7fc710102b30 2024-05-25T06:38:15.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.690+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7100fd900 con 0x7fc710102b30 2024-05-25T06:38:15.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.690+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc70001c400 con 0x7fc710102b30 2024-05-25T06:38:15.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.691+0000 7fc705ffb640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc6fc06e0b0 0x7fc6fc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.691+0000 7fc715339640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc6fc06e0b0 0x7fc6fc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.691+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc70008d470 con 0x7fc710102b30 2024-05-25T06:38:15.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.691+0000 7fc715339640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc6fc06e0b0 0x7fc6fc070560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.695 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.695+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc7000567a0 con 0x7fc710102b30 2024-05-25T06:38:15.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.870+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc71010e370 con 0x7fc710102b30 2024-05-25T06:38:15.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.870+0000 7fc705ffb640 1 -- 172.21.15.160:0/2473718735 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc700059e60 con 0x7fc710102b30 2024-05-25T06:38:15.870 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:38:15.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc6fc06e0b0 msgr2=0x7fc6fc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc6fc06e0b0 0x7fc6fc070560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 msgr2=0x7fc710113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 0x7fc710113240 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc700002a50 tx=0x7fc700004600 comp rx=0 tx=0).stop 2024-05-25T06:38:15.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 shutdown_connections 2024-05-25T06:38:15.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fc6fc06e0b0 0x7fc6fc070560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc710102b30 0x7fc710113240 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 --2- 172.21.15.160:0/2473718735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710102190 0x7fc710112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 >> 172.21.15.160:0/2473718735 conn(0x7fc7100f7da0 msgr2=0x7fc7100f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 shutdown_connections 2024-05-25T06:38:15.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.872+0000 7fc71759d640 1 -- 172.21.15.160:0/2473718735 wait complete. 2024-05-25T06:38:15.883 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 -- 172.21.15.160:0/904364667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 msgr2=0x7f3c7c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 --2- 172.21.15.160:0/904364667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c102610 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c700048d0 tx=0x7f3c70009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 -- 172.21.15.160:0/904364667 shutdown_connections 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 --2- 172.21.15.160:0/904364667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c7c0fa840 0x7f3c7c0facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 --2- 172.21.15.160:0/904364667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c102610 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 -- 172.21.15.160:0/904364667 >> 172.21.15.160:0/904364667 conn(0x7f3c7c071700 msgr2=0x7f3c7c071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 -- 172.21.15.160:0/904364667 shutdown_connections 2024-05-25T06:38:15.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.983+0000 7f3c82aae640 1 -- 172.21.15.160:0/904364667 wait complete. 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c82aae640 1 Processor -- start 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c82aae640 1 -- start start 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c82aae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c7c0fa840 0x7f3c7c112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c82aae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c82aae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c1138c0 con 0x7f3c7c102240 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c82aae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c10cf10 con 0x7f3c7c0fa840 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c1132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c1132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33840/0 (socket says 172.21.15.160:33840) 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c7bfff640 1 -- 172.21.15.160:0/2221111658 learned_addr learned my addr 172.21.15.160:0/2221111658 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c7bfff640 1 -- 172.21.15.160:0/2221111658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c7c0fa840 msgr2=0x7f3c7c112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c8084a640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c7c0fa840 0x7f3c7c112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.984+0000 7f3c7bfff640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c7c0fa840 0x7f3c7c112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c7bfff640 1 -- 172.21.15.160:0/2221111658 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c70005040 con 0x7f3c7c102240 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c7bfff640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c1132c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3c68002a50 tx=0x7f3c68004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c68007c30 con 0x7f3c7c102240 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c6800f040 con 0x7f3c7c102240 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c10d1f0 con 0x7f3c7c102240 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c6800db50 con 0x7f3c7c102240 2024-05-25T06:38:15.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.985+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c10d710 con 0x7f3c7c102240 2024-05-25T06:38:15.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.986+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c0652b0 con 0x7f3c7c102240 2024-05-25T06:38:15.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.986+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3c6801c400 con 0x7f3c7c102240 2024-05-25T06:38:15.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.987+0000 7f3c797fa640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c6c06e0b0 0x7f3c6c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:15.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.987+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3c6808d470 con 0x7f3c7c102240 2024-05-25T06:38:15.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.987+0000 7f3c8084a640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c6c06e0b0 0x7f3c6c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:15.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.987+0000 7f3c8084a640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c6c06e0b0 0x7f3c6c070560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3c70003080 tx=0x7f3c70009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:15.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:15.991+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3c680567a0 con 0x7f3c7c102240 2024-05-25T06:38:16.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.169+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f3c7c109420 con 0x7f3c7c102240 2024-05-25T06:38:16.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.170+0000 7f3c797fa640 1 -- 172.21.15.160:0/2221111658 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3c68059e60 con 0x7f3c7c102240 2024-05-25T06:38:16.170 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c6c06e0b0 msgr2=0x7f3c6c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c6c06e0b0 0x7f3c6c070560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3c70003080 tx=0x7f3c70009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 msgr2=0x7f3c7c1132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c1132c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3c68002a50 tx=0x7f3c68004600 comp rx=0 tx=0).stop 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 shutdown_connections 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f3c6c06e0b0 0x7f3c6c070560 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c7c102240 0x7f3c7c1132c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 --2- 172.21.15.160:0/2221111658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c7c0fa840 0x7f3c7c112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 >> 172.21.15.160:0/2221111658 conn(0x7f3c7c071700 msgr2=0x7f3c7c0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:16.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 shutdown_connections 2024-05-25T06:38:16.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.172+0000 7f3c82aae640 1 -- 172.21.15.160:0/2221111658 wait complete. 2024-05-25T06:38:16.183 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 -- 172.21.15.160:0/3492127045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc102b50 msgr2=0x7fe7fc0fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 --2- 172.21.15.160:0/3492127045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc102b50 0x7fe7fc0fc1e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0048d0 tx=0x7fe7ec009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 -- 172.21.15.160:0/3492127045 shutdown_connections 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 --2- 172.21.15.160:0/3492127045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc102b50 0x7fe7fc0fc1e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 --2- 172.21.15.160:0/3492127045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 -- 172.21.15.160:0/3492127045 >> 172.21.15.160:0/3492127045 conn(0x7fe7fc0f7dc0 msgr2=0x7fe7fc0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 -- 172.21.15.160:0/3492127045 shutdown_connections 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.282+0000 7fe8044ae640 1 -- 172.21.15.160:0/3492127045 wait complete. 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe8044ae640 1 Processor -- start 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe8044ae640 1 -- start start 2024-05-25T06:38:16.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe8044ae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe8044ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc102b50 0x7fe7fc113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe8044ae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc113800 con 0x7fe7fc1021b0 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe8044ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc10ce50 con 0x7fe7fc102b50 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33862/0 (socket says 172.21.15.160:33862) 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 -- 172.21.15.160:0/1490304354 learned_addr learned my addr 172.21.15.160:0/1490304354 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 -- 172.21.15.160:0/1490304354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc102b50 msgr2=0x7fe7fc113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe801a49640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc102b50 0x7fe7fc113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc102b50 0x7fe7fc113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 -- 172.21.15.160:0/1490304354 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec005040 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe801a49640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc102b50 0x7fe7fc113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.283+0000 7fe80224a640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc112cc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c790 tx=0x7fe7f80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.284+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f800a820 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.284+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc10d130 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.284+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7f800ae50 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.284+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f80115e0 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.284+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc10d650 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.285+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc0fd920 con 0x7fe7fc1021b0 2024-05-25T06:38:16.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.285+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe7f8012400 con 0x7fe7fc1021b0 2024-05-25T06:38:16.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.285+0000 7fe7f2ffd640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe7e806e0b0 0x7fe7e8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.286+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe7f808dd30 con 0x7fe7fc1021b0 2024-05-25T06:38:16.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.286+0000 7fe801a49640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe7e806e0b0 0x7fe7e8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.286+0000 7fe801a49640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe7e806e0b0 0x7fe7e8070560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000c00 tx=0x7fe7ec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:16.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.290+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe7f80570e0 con 0x7fe7fc1021b0 2024-05-25T06:38:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:16 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3725102235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:38:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:16 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/4168776602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:38:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:16 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2473718735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:38:16.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:16 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/2221111658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:38:16.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.477+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe7fc109310 con 0x7fe7fc1021b0 2024-05-25T06:38:16.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.478+0000 7fe7f2ffd640 1 -- 172.21.15.160:0/1490304354 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v85) v1 ==== 81+0+2351 (secure 0 0 0) 0x7fe7f805a7a0 con 0x7fe7fc1021b0 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6829/3783773637 2024-05-26T06:20:11.153443+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6828/3783773637 2024-05-26T06:20:11.153443+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/4089125744 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3154277186 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6824/3325193142 2024-05-26T06:20:11.153460+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/892364096 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6826/2072396647 2024-05-26T06:08:36.175975+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/3983131949 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/2473068109 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/2875905800 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/4006145193 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6825/3325193142 2024-05-26T06:20:11.153460+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1552574721 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/3131222858 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6828/2357665398 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/500428502 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/2019174179 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/4006145193 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/3392624553 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3244490273 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/3983131949 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1884499665 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3435817331 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/2025926580 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:16.479 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1559445112 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/4089125744 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/846851479 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/3392624553 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1362855798 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1251129035 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/1389978098 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/650546693 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1144786189 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6829/2357665398 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6827/2072396647 2024-05-26T06:08:36.175975+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/1875516529 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/1875516529 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3623795045 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:16.480 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3363261834 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:16.481 INFO:teuthology.orchestra.run.smithi160.stderr:listed 39 entries 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe7e806e0b0 msgr2=0x7fe7e8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe7e806e0b0 0x7fe7e8070560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000c00 tx=0x7fe7ec009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 msgr2=0x7fe7fc112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc112cc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c790 tx=0x7fe7f80043c0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 shutdown_connections 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7fe7e806e0b0 0x7fe7e8070560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7fc102b50 0x7fe7fc113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 --2- 172.21.15.160:0/1490304354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7fc1021b0 0x7fe7fc112cc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 >> 172.21.15.160:0/1490304354 conn(0x7fe7fc0f7dc0 msgr2=0x7fe7fc0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 shutdown_connections 2024-05-25T06:38:16.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.480+0000 7fe8044ae640 1 -- 172.21.15.160:0/1490304354 wait complete. 2024-05-25T06:38:16.491 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-25T06:38:16.491 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-25T06:38:16.508 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.644+0000 7ff00d11d640 1 -- 172.21.15.160:0/1474266137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 msgr2=0x7ff0080fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.644+0000 7ff00d11d640 1 --2- 172.21.15.160:0/1474266137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0080fb160 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feff00048d0 tx=0x7feff0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- 172.21.15.160:0/1474266137 shutdown_connections 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 --2- 172.21.15.160:0/1474266137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0080fb160 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 --2- 172.21.15.160:0/1474266137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 0x7ff0081014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- 172.21.15.160:0/1474266137 >> 172.21.15.160:0/1474266137 conn(0x7ff0080711c0 msgr2=0x7ff0080715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- 172.21.15.160:0/1474266137 shutdown_connections 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- 172.21.15.160:0/1474266137 wait complete. 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 Processor -- start 2024-05-25T06:38:16.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- start start 2024-05-25T06:38:16.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 0x7ff008111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0081121e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081127e0 con 0x7ff008101aa0 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.645+0000 7ff00d11d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00810be30 con 0x7ff008101100 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0081121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0081121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33888/0 (socket says 172.21.15.160:33888) 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 -- 172.21.15.160:0/905161966 learned_addr learned my addr 172.21.15.160:0/905161966 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff006d9d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 0x7ff008111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 -- 172.21.15.160:0/905161966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 msgr2=0x7ff008111ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 0x7ff008111ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 -- 172.21.15.160:0/905161966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0005040 con 0x7ff008101aa0 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff006d9d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 0x7ff008111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00659c640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0081121e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7feff002bda0 tx=0x7feff0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:16.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feff00094c0 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feff0032e50 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00810c0b0 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feff0041410 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.646+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00810c570 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.647+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0080fc880 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.648+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7feff002f070 con 0x7ff008101aa0 2024-05-25T06:38:16.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.648+0000 7fefff7fe640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feff406e0b0 0x7feff4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:16 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3725102235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:38:16.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:16 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/4168776602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:38:16.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:16 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2473718735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:38:16.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:16 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/2221111658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:38:16.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.648+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7feff00b10a0 con 0x7ff008101aa0 2024-05-25T06:38:16.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.648+0000 7ff006d9d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feff406e0b0 0x7feff4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.649+0000 7ff006d9d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feff406e0b0 0x7feff4070560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feff8004d80 tx=0x7feff800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:16.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.653+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feff007a3d0 con 0x7ff008101aa0 2024-05-25T06:38:16.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.826+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff008108280 con 0x7ff008101aa0 2024-05-25T06:38:16.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.827+0000 7fefff7fe640 1 -- 172.21.15.160:0/905161966 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16719 (secure 0 0 0) 0x7feff0036030 con 0x7ff008101aa0 2024-05-25T06:38:16.827 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-25T06:38:16.827 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":85,"fsid":"0ed6924e-1a5c-11ef-bc9b-c7b262605968","created":"2024-05-25T06:01:28.264275+0000","modified":"2024-05-25T06:21:08.124903+0000","last_up_change":"2024-05-25T06:20:08.969929+0000","last_in_change":"2024-05-25T06:06:44.041625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-25T06:05:48.910750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-25T06:08:20.895353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-25T06:08:21.186881+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0838a204-c967-4d16-bf9c-3574b6649cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6803","nonce":793926185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6805","nonce":793926185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6809","nonce":793926185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":793926185},{"type":"v1","addr":"172.21.15.160:6807","nonce":793926185}]},"public_addr":"172.21.15.160:6803/793926185","cluster_addr":"172.21.15.160:6805/793926185","heartbeat_back_addr":"172.21.15.160:6809/793926185","heartbeat_front_addr":"172.21.15.160:6807/793926185","state":["exists","up"]},{"osd":1,"uuid":"b2c2a7e5-e6bf-48b6-82c3-b97abeacd8e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6811","nonce":3648042119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6813","nonce":3648042119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6817","nonce":3648042119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3648042119},{"type":"v1","addr":"172.21.15.160:6815","nonce":3648042119}]},"public_addr":"172.21.15.160:6811/3648042119","cluster_addr":"172.21.15.160:6813/3648042119","heartbeat_back_addr":"172.21.15.160:6817/3648042119","heartbeat_front_addr":"172.21.15.160:6815/3648042119","state":["exists","up"]},{"osd":2,"uuid":"63ed9e8f-565f-48c2-bcc6-4be1c0262eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6819","nonce":1307550666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6821","nonce":1307550666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6825","nonce":1307550666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1307550666},{"type":"v1","addr":"172.21.15.160:6823","nonce":1307550666}]},"public_addr":"172.21.15.160:6819/1307550666","cluster_addr":"172.21.15.160:6821/1307550666","heartbeat_back_addr":"172.21.15.160:6825/1307550666","heartbeat_front_addr":"172.21.15.160:6823/1307550666","state":["exists","up"]},{"osd":3,"uuid":"ee6a9eac-ed34-4283-9693-053d313f1524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6801","nonce":316097287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6803","nonce":316097287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6807","nonce":316097287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":316097287},{"type":"v1","addr":"172.21.15.203:6805","nonce":316097287}]},"public_addr":"172.21.15.203:6801/316097287","cluster_addr":"172.21.15.203:6803/316097287","heartbeat_back_addr":"172.21.15.203:6807/316097287","heartbeat_front_addr":"172.21.15.203:6805/316097287","state":["exists","up"]},{"osd":4,"uuid":"d4ecec6a-830e-456f-8e7a-429c2c16eec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6809","nonce":2420151889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6811","nonce":2420151889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6815","nonce":2420151889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2420151889},{"type":"v1","addr":"172.21.15.203:6813","nonce":2420151889}]},"public_addr":"172.21.15.203:6809/2420151889","cluster_addr":"172.21.15.203:6811/2420151889","heartbeat_back_addr":"172.21.15.203:6815/2420151889","heartbeat_front_addr":"172.21.15.203:6813/2420151889","state":["exists","up"]},{"osd":5,"uuid":"717a4698-b4f4-457e-840f-23de1aad2734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6817","nonce":2019898829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6819","nonce":2019898829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6823","nonce":2019898829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2019898829},{"type":"v1","addr":"172.21.15.203:6821","nonce":2019898829}]},"public_addr":"172.21.15.203:6817/2019898829","cluster_addr":"172.21.15.203:6819/2019898829","heartbeat_back_addr":"172.21.15.203:6823/2019898829","heartbeat_front_addr":"172.21.15.203:6821/2019898829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:04:50.884038+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:18.047314+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:05:46.696514+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:10.336918+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:06:35.486375+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-25T06:07:01.401378+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6829/3783773637":"2024-05-26T06:20:11.153443+0000","172.21.15.160:6828/3783773637":"2024-05-26T06:20:11.153443+0000","172.21.15.160:6800/4089125744":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/3154277186":"2024-05-26T06:02:33.125470+0000","172.21.15.203:6824/3325193142":"2024-05-26T06:20:11.153460+0000","172.21.15.203:0/892364096":"2024-05-26T06:12:42.594272+0000","172.21.15.160:6826/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6801/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/2473068109":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/2875905800":"2024-05-26T06:03:38.583980+0000","172.21.15.160:6800/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.203:6825/3325193142":"2024-05-26T06:20:11.153460+0000","172.21.15.160:0/1552574721":"2024-05-26T06:02:03.339352+0000","172.21.15.203:0/3131222858":"2024-05-26T06:12:42.594272+0000","172.21.15.203:6828/2357665398":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/500428502":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/2019174179":"2024-05-26T06:11:04.538090+0000","172.21.15.160:6801/4006145193":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6801/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/3244490273":"2024-05-26T06:02:03.339352+0000","172.21.15.160:6800/3983131949":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1884499665":"2024-05-26T06:15:42.839541+0000","172.21.15.160:0/3435817331":"2024-05-26T06:02:03.339352+0000","172.21.15.203:0/2025926580":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/1559445112":"2024-05-26T06:15:42.839541+0000","172.21.15.160:6801/4089125744":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/846851479":"2024-05-26T06:02:33.125470+0000","172.21.15.160:6800/3392624553":"2024-05-26T06:02:33.125470+0000","172.21.15.160:0/1362855798":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1251129035":"2024-05-26T06:11:04.538090+0000","172.21.15.203:0/1389978098":"2024-05-26T06:12:42.594272+0000","172.21.15.160:0/650546693":"2024-05-26T06:03:38.583980+0000","172.21.15.160:0/1144786189":"2024-05-26T06:15:42.839541+0000","172.21.15.203:6829/2357665398":"2024-05-26T06:12:42.594272+0000","172.21.15.160:6827/2072396647":"2024-05-26T06:08:36.175975+0000","172.21.15.160:6800/1875516529":"2024-05-26T06:15:42.839541+0000","172.21.15.160:6801/1875516529":"2024-05-26T06:15:42.839541+0000","172.21.15.160:0/3623795045":"2024-05-26T06:11:04.538090+0000","172.21.15.160:0/3363261834":"2024-05-26T06:15:42.839541+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-25T06:38:16.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feff406e0b0 msgr2=0x7feff4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feff406e0b0 0x7feff4070560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feff8004d80 tx=0x7feff800b040 comp rx=0 tx=0).stop 2024-05-25T06:38:16.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 msgr2=0x7ff0081121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0081121e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7feff002bda0 tx=0x7feff0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 shutdown_connections 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7feff406e0b0 0x7feff4070560 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff008101aa0 0x7ff0081121e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 --2- 172.21.15.160:0/905161966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008101100 0x7ff008111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 >> 172.21.15.160:0/905161966 conn(0x7ff0080711c0 msgr2=0x7ff0080faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 shutdown_connections 2024-05-25T06:38:16.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.829+0000 7ff00d11d640 1 -- 172.21.15.160:0/905161966 wait complete. 2024-05-25T06:38:16.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-25T06:05:48.910750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-25T06:08:20.895353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-25T06:08:21.186881+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-25T06:38:16.841 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 -- 172.21.15.160:0/4229869540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be81010b0 msgr2=0x7f4be8101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 --2- 172.21.15.160:0/4229869540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be81010b0 0x7f4be8101480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4be40048d0 tx=0x7f4be4009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 -- 172.21.15.160:0/4229869540 shutdown_connections 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 --2- 172.21.15.160:0/4229869540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be8101a50 0x7f4be80fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 --2- 172.21.15.160:0/4229869540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be81010b0 0x7f4be8101480 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 -- 172.21.15.160:0/4229869540 >> 172.21.15.160:0/4229869540 conn(0x7f4be80f6c60 msgr2=0x7f4be80f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 -- 172.21.15.160:0/4229869540 shutdown_connections 2024-05-25T06:38:16.942 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.940+0000 7f4bf066e640 1 -- 172.21.15.160:0/4229869540 wait complete. 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bf066e640 1 Processor -- start 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bf066e640 1 -- start start 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bf066e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be81010b0 0x7f4be8113040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bf066e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 0x7f4be8113580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bf066e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8113b80 con 0x7f4be8101a50 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bf066e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be810d1d0 con 0x7f4be81010b0 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bedc09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 0x7f4be8113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bedc09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 0x7f4be8113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33892/0 (socket says 172.21.15.160:33892) 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bedc09640 1 -- 172.21.15.160:0/99806800 learned_addr learned my addr 172.21.15.160:0/99806800 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bee40a640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be81010b0 0x7f4be8113040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bedc09640 1 -- 172.21.15.160:0/99806800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be81010b0 msgr2=0x7f4be8113040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bedc09640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be81010b0 0x7f4be8113040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bedc09640 1 -- 172.21.15.160:0/99806800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4005040 con 0x7f4be8101a50 2024-05-25T06:38:16.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.941+0000 7f4bee40a640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be81010b0 0x7f4be8113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-25T06:38:16.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.942+0000 7f4bedc09640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 0x7f4be8113580 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8002a50 tx=0x7f4bd8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:16.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.942+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd8007c30 con 0x7f4be8101a50 2024-05-25T06:38:16.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.942+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bd800f040 con 0x7f4be8101a50 2024-05-25T06:38:16.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.942+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd800db50 con 0x7f4be8101a50 2024-05-25T06:38:16.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.942+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be810d4b0 con 0x7f4be8101a50 2024-05-25T06:38:16.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.942+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be810d9d0 con 0x7f4be8101a50 2024-05-25T06:38:16.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.943+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be80fc820 con 0x7f4be8101a50 2024-05-25T06:38:16.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.943+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4bd801c400 con 0x7f4be8101a50 2024-05-25T06:38:16.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.944+0000 7f4bdeffd640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4bd406e0b0 0x7f4bd4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:16.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.944+0000 7f4bee40a640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4bd406e0b0 0x7f4bd4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:16.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.944+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4bd808d470 con 0x7f4be8101a50 2024-05-25T06:38:16.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.945+0000 7f4bee40a640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4bd406e0b0 0x7f4bd4070560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4be4000c00 tx=0x7f4be4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:16.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:16.949+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4bd80567a0 con 0x7f4be8101a50 2024-05-25T06:38:17.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.125+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4be8109310 con 0x7f4be8101a50 2024-05-25T06:38:17.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.125+0000 7f4bdeffd640 1 -- 172.21.15.160:0/99806800 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bd8059e60 con 0x7f4be8101a50 2024-05-25T06:38:17.125 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-25T06:38:17.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4bd406e0b0 msgr2=0x7f4bd4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4bd406e0b0 0x7f4bd4070560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4be4000c00 tx=0x7f4be4009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 msgr2=0x7f4be8113580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 0x7f4be8113580 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8002a50 tx=0x7f4bd8004600 comp rx=0 tx=0).stop 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 shutdown_connections 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f4bd406e0b0 0x7f4bd4070560 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4be8101a50 0x7f4be8113580 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 --2- 172.21.15.160:0/99806800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be81010b0 0x7f4be8113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 >> 172.21.15.160:0/99806800 conn(0x7f4be80f6c60 msgr2=0x7f4be80f7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 shutdown_connections 2024-05-25T06:38:17.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.127+0000 7f4bf066e640 1 -- 172.21.15.160:0/99806800 wait complete. 2024-05-25T06:38:17.138 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-25T06:38:17.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.240+0000 7f61ecb9d640 1 -- 172.21.15.160:0/4013028680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102180 msgr2=0x7f61e8102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.240+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/4013028680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102180 0x7f61e8102550 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f61e00048d0 tx=0x7f61e0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.240+0000 7f61ecb9d640 1 -- 172.21.15.160:0/4013028680 shutdown_connections 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.240+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/4013028680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61e8102b20 0x7f61e80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.240+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/4013028680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102180 0x7f61e8102550 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.240+0000 7f61ecb9d640 1 -- 172.21.15.160:0/4013028680 >> 172.21.15.160:0/4013028680 conn(0x7f61e80f7db0 msgr2=0x7f61e80fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 -- 172.21.15.160:0/4013028680 shutdown_connections 2024-05-25T06:38:17.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 -- 172.21.15.160:0/4013028680 wait complete. 2024-05-25T06:38:17.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 Processor -- start 2024-05-25T06:38:17.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 -- start start 2024-05-25T06:38:17.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61e8102180 0x7f61e806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 0x7f61e806fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80716e0 con 0x7f61e8102b20 2024-05-25T06:38:17.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61ecb9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80701f0 con 0x7f61e8102180 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61e5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 0x7f61e806fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61e5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 0x7f61e806fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33920/0 (socket says 172.21.15.160:33920) 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61e5d9b640 1 -- 172.21.15.160:0/3315264783 learned_addr learned my addr 172.21.15.160:0/3315264783 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.241+0000 7f61e659c640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61e8102180 0x7f61e806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61e5d9b640 1 -- 172.21.15.160:0/3315264783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61e8102180 msgr2=0x7f61e806f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61e5d9b640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61e8102180 0x7f61e806f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61e5d9b640 1 -- 172.21.15.160:0/3315264783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0005040 con 0x7f61e8102b20 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61e5d9b640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 0x7f61e806fcb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00bbb0 tx=0x7f61dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61dc00c940 con 0x7f61e8102b20 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61dc0092e0 con 0x7f61e8102b20 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61dc007640 con 0x7f61e8102b20 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e8070470 con 0x7f61e8102b20 2024-05-25T06:38:17.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.242+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e81194b0 con 0x7f61e8102b20 2024-05-25T06:38:17.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.243+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e80fd8f0 con 0x7f61e8102b20 2024-05-25T06:38:17.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.244+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f61dc012070 con 0x7f61e8102b20 2024-05-25T06:38:17.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.244+0000 7f61d6ffd640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61d006e0b0 0x7f61d0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.244+0000 7f61e659c640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61d006e0b0 0x7f61d0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.245+0000 7f61e659c640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61d006e0b0 0x7f61d0070560 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f61e0000c00 tx=0x7f61e0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:17.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.245+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f61dc05b230 con 0x7f61e8102b20 2024-05-25T06:38:17.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.250+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f61dc056dd0 con 0x7f61e8102b20 2024-05-25T06:38:17.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.424+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f61e804bcc0 con 0x7f61e8102b20 2024-05-25T06:38:17.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.425+0000 7f61d6ffd640 1 -- 172.21.15.160:0/3315264783 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f61dc05a490 con 0x7f61e8102b20 2024-05-25T06:38:17.425 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61d006e0b0 msgr2=0x7f61d0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61d006e0b0 0x7f61d0070560 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f61e0000c00 tx=0x7f61e0009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 msgr2=0x7f61e806fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 0x7f61e806fcb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00bbb0 tx=0x7f61dc009840 comp rx=0 tx=0).stop 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 shutdown_connections 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f61d006e0b0 0x7f61d0070560 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61e8102b20 0x7f61e806fcb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 --2- 172.21.15.160:0/3315264783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61e8102180 0x7f61e806f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 >> 172.21.15.160:0/3315264783 conn(0x7f61e80f7db0 msgr2=0x7f61e80f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 shutdown_connections 2024-05-25T06:38:17.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.427+0000 7f61ecb9d640 1 -- 172.21.15.160:0/3315264783 wait complete. 2024-05-25T06:38:17.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:17 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/1490304354' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-25T06:38:17.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:17 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/905161966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:38:17.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:17 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/99806800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:38:17.436 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:17 smithi160 ceph-mon[112094]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:17.438 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 -- 172.21.15.160:0/3890179063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101a30 msgr2=0x7f79640fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 --2- 172.21.15.160:0/3890179063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101a30 0x7f79640fb0c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f79580048d0 tx=0x7f7958009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 -- 172.21.15.160:0/3890179063 shutdown_connections 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 --2- 172.21.15.160:0/3890179063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101a30 0x7f79640fb0c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 --2- 172.21.15.160:0/3890179063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101090 0x7f7964101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 -- 172.21.15.160:0/3890179063 >> 172.21.15.160:0/3890179063 conn(0x7f79640f6c20 msgr2=0x7f79640f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 -- 172.21.15.160:0/3890179063 shutdown_connections 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 -- 172.21.15.160:0/3890179063 wait complete. 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 Processor -- start 2024-05-25T06:38:17.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.538+0000 7f796b86e640 1 -- start start 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796b86e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 0x7f796410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796b86e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101a30 0x7f796410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796b86e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640664b0 con 0x7f7964101090 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796b86e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796410e940 con 0x7f7964101a30 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 0x7f796410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 0x7f796410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33944/0 (socket says 172.21.15.160:33944) 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 -- 172.21.15.160:0/536108323 learned_addr learned my addr 172.21.15.160:0/536108323 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f7968e09640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101a30 0x7f796410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 -- 172.21.15.160:0/536108323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101a30 msgr2=0x7f796410e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101a30 0x7f796410e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 -- 172.21.15.160:0/536108323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958005040 con 0x7f7964101090 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f7968e09640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101a30 0x7f796410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:17.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796960a640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 0x7f796410de90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7960002a80 tx=0x7f7960004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:17.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7960007c30 con 0x7f7964101090 2024-05-25T06:38:17.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f796000f040 con 0x7f7964101090 2024-05-25T06:38:17.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796410ec20 con 0x7f7964101090 2024-05-25T06:38:17.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.539+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79600096b0 con 0x7f7964101090 2024-05-25T06:38:17.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.540+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79641171b0 con 0x7f7964101090 2024-05-25T06:38:17.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.541+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79640fc800 con 0x7f7964101090 2024-05-25T06:38:17.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.541+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f796000d6b0 con 0x7f7964101090 2024-05-25T06:38:17.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.541+0000 7f7955ffb640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f795006e0b0 0x7f7950070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.542+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f796008d570 con 0x7f7964101090 2024-05-25T06:38:17.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.542+0000 7f7968e09640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f795006e0b0 0x7f7950070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.542+0000 7f7968e09640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f795006e0b0 0x7f7950070560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f7958009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:17.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.546+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f79600568a0 con 0x7f7964101090 2024-05-25T06:38:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:17 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/1490304354' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-25T06:38:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:17 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/905161966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-25T06:38:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:17 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/99806800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-25T06:38:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:17 smithi203 ceph-mon[83507]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:17.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.719+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f796404bcc0 con 0x7f7964101090 2024-05-25T06:38:17.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.720+0000 7f7955ffb640 1 -- 172.21.15.160:0/536108323 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7960056030 con 0x7f7964101090 2024-05-25T06:38:17.720 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f795006e0b0 msgr2=0x7f7950070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f795006e0b0 0x7f7950070560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f7958009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 msgr2=0x7f796410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 0x7f796410de90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7960002a80 tx=0x7f7960004600 comp rx=0 tx=0).stop 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 shutdown_connections 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f795006e0b0 0x7f7950070560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7964101a30 0x7f796410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 --2- 172.21.15.160:0/536108323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7964101090 0x7f796410de90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 >> 172.21.15.160:0/536108323 conn(0x7f79640f6c20 msgr2=0x7f79640f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 shutdown_connections 2024-05-25T06:38:17.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.722+0000 7f796b86e640 1 -- 172.21.15.160:0/536108323 wait complete. 2024-05-25T06:38:17.732 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-25T06:38:17.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.834+0000 7f1d4c95d640 1 -- 172.21.15.160:0/3194506078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 msgr2=0x7f1d481014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.834+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/3194506078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d481014f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d380048d0 tx=0x7f1d38009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.834+0000 7f1d4c95d640 1 -- 172.21.15.160:0/3194506078 shutdown_connections 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.834+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/3194506078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 0x7f1d480fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.834+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/3194506078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d481014f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.834+0000 7f1d4c95d640 1 -- 172.21.15.160:0/3194506078 >> 172.21.15.160:0/3194506078 conn(0x7f1d480710c0 msgr2=0x7f1d480714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 -- 172.21.15.160:0/3194506078 shutdown_connections 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 -- 172.21.15.160:0/3194506078 wait complete. 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 Processor -- start 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 -- start start 2024-05-25T06:38:17.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 0x7f1d4810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d4810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d48065310 con 0x7f1d48101120 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4c95d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4810e970 con 0x7f1d480fb100 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d45d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d4810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d45d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d4810e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33954/0 (socket says 172.21.15.160:33954) 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d45d9b640 1 -- 172.21.15.160:0/497135582 learned_addr learned my addr 172.21.15.160:0/497135582 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.835+0000 7f1d4659c640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 0x7f1d4810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d45d9b640 1 -- 172.21.15.160:0/497135582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 msgr2=0x7f1d4810dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d45d9b640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 0x7f1d4810dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d45d9b640 1 -- 172.21.15.160:0/497135582 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d38005040 con 0x7f1d48101120 2024-05-25T06:38:17.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d4659c640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 0x7f1d4810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d45d9b640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d4810e400 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c002a50 tx=0x7f1d3c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d3c007c30 con 0x7f1d48101120 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4810ec50 con 0x7f1d48101120 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d3c00f040 con 0x7f1d48101120 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d3c00db50 con 0x7f1d48101120 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.836+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d481171e0 con 0x7f1d48101120 2024-05-25T06:38:17.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.837+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d480fc7e0 con 0x7f1d48101120 2024-05-25T06:38:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.837+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 40) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1d3c01c400 con 0x7f1d48101120 2024-05-25T06:38:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.838+0000 7f1d36ffd640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1d3006e0b0 0x7f1d30070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-25T06:38:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.838+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1d3c08d460 con 0x7f1d48101120 2024-05-25T06:38:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.838+0000 7f1d4659c640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1d3006e0b0 0x7f1d30070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-25T06:38:17.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.838+0000 7f1d4659c640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1d3006e0b0 0x7f1d30070560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1d38004540 tx=0x7f1d38009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-25T06:38:17.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:17.842+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1d3c056790 con 0x7f1d48101120 2024-05-25T06:38:18.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.032+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d4804bcc0 con 0x7f1d48101120 2024-05-25T06:38:18.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.033+0000 7f1d36ffd640 1 -- 172.21.15.160:0/497135582 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v85) v1 ==== 81+0+2351 (secure 0 0 0) 0x7f1d3c059e50 con 0x7f1d48101120 2024-05-25T06:38:18.033 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6829/3783773637 2024-05-26T06:20:11.153443+0000 2024-05-25T06:38:18.033 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6828/3783773637 2024-05-26T06:20:11.153443+0000 2024-05-25T06:38:18.033 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/4089125744 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3154277186 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6824/3325193142 2024-05-26T06:20:11.153460+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/892364096 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6826/2072396647 2024-05-26T06:08:36.175975+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/3983131949 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/2473068109 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/2875905800 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/4006145193 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6825/3325193142 2024-05-26T06:20:11.153460+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1552574721 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:18.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/3131222858 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6828/2357665398 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/500428502 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/2019174179 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/4006145193 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/3392624553 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3244490273 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/3983131949 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1884499665 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3435817331 2024-05-26T06:02:03.339352+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/2025926580 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1559445112 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/4089125744 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:18.035 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/846851479 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/3392624553 2024-05-26T06:02:33.125470+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1362855798 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1251129035 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:0/1389978098 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/650546693 2024-05-26T06:03:38.583980+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1144786189 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.203:6829/2357665398 2024-05-26T06:12:42.594272+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6827/2072396647 2024-05-26T06:08:36.175975+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/1875516529 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/1875516529 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3623795045 2024-05-26T06:11:04.538090+0000 2024-05-25T06:38:18.036 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3363261834 2024-05-26T06:15:42.839541+0000 2024-05-25T06:38:18.037 INFO:teuthology.orchestra.run.smithi160.stderr:listed 39 entries 2024-05-25T06:38:18.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1d3006e0b0 msgr2=0x7f1d30070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1d3006e0b0 0x7f1d30070560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1d38004540 tx=0x7f1d38009da0 comp rx=0 tx=0).stop 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 msgr2=0x7f1d4810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d4810e400 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c002a50 tx=0x7f1d3c004600 comp rx=0 tx=0).stop 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 shutdown_connections 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:6800/427616578,v1:172.21.15.160:6801/427616578] conn(0x7f1d3006e0b0 0x7f1d30070560 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d48101120 0x7f1d4810e400 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 --2- 172.21.15.160:0/497135582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d480fb100 0x7f1d4810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 >> 172.21.15.160:0/497135582 conn(0x7f1d480710c0 msgr2=0x7f1d481041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 shutdown_connections 2024-05-25T06:38:18.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-25T06:38:18.035+0000 7f1d4c95d640 1 -- 172.21.15.160:0/497135582 wait complete. 2024-05-25T06:38:18.046 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi160.front.sepia.ceph.com... 2024-05-25T06:38:18.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-25T06:38:18.046 DEBUG:teuthology.orchestra.run.smithi160:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-25T06:38:18.071 INFO:teuthology.orchestra.run:waiting for 300 2024-05-25T06:38:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:18 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/3315264783' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:38:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:18 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/536108323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:38:18.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:18 smithi160 ceph-mon[112094]: from='client.? 172.21.15.160:0/497135582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-25T06:38:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:18 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/3315264783' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-25T06:38:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:18 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/536108323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-25T06:38:18.650 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:18 smithi203 ceph-mon[83507]: from='client.? 172.21.15.160:0/497135582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-25T06:38:19.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:19 smithi203 ceph-mon[83507]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:19.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:19 smithi160 ceph-mon[112094]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:21 smithi160 ceph-mon[112094]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:21 smithi203 ceph-mon[83507]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:22.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:22 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:38:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:22 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:38:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:23 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:38:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:23 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:38:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:23 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:38:23.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:23 smithi160 ceph-mon[112094]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:23 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:38:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:23 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:38:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:23 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:38:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:23 smithi203 ceph-mon[83507]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:25 smithi160 ceph-mon[112094]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:25 smithi203 ceph-mon[83507]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:27.451 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:27 smithi203 ceph-mon[83507]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:27 smithi160 ceph-mon[112094]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:29.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:29 smithi160 ceph-mon[112094]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:29.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:29 smithi203 ceph-mon[83507]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:31.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:31 smithi160 ceph-mon[112094]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:31.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:31 smithi203 ceph-mon[83507]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:33.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:33 smithi160 ceph-mon[112094]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:33.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:33 smithi203 ceph-mon[83507]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:35.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:35 smithi160 ceph-mon[112094]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:35.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:35 smithi203 ceph-mon[83507]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:37.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:37 smithi160 ceph-mon[112094]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:37.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:37 smithi203 ceph-mon[83507]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:39 smithi160 ceph-mon[112094]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:39.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:39 smithi203 ceph-mon[83507]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:41 smithi160 ceph-mon[112094]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:41.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:41 smithi203 ceph-mon[83507]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:42 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:42 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:44.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:43 smithi203 ceph-mon[83507]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:44.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:43 smithi160 ceph-mon[112094]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:45 smithi160 ceph-mon[112094]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:45.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:45 smithi203 ceph-mon[83507]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:47 smithi160 ceph-mon[112094]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:47.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:47 smithi203 ceph-mon[83507]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:49 smithi160 ceph-mon[112094]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:49 smithi203 ceph-mon[83507]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:51 smithi160 ceph-mon[112094]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:51 smithi203 ceph-mon[83507]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:38:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:53 smithi160 ceph-mon[112094]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:53 smithi203 ceph-mon[83507]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:55 smithi160 ceph-mon[112094]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:55 smithi203 ceph-mon[83507]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:38:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:57 smithi160 ceph-mon[112094]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:57.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:57 smithi203 ceph-mon[83507]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:58.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:38:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:38:59 smithi160 ceph-mon[112094]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:38:59.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:38:59 smithi203 ceph-mon[83507]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:01.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:01 smithi160 ceph-mon[112094]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:01.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:01 smithi203 ceph-mon[83507]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:03.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:03 smithi160 ceph-mon[112094]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:03.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:03 smithi203 ceph-mon[83507]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:05 smithi160 ceph-mon[112094]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:05.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:05 smithi203 ceph-mon[83507]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:07 smithi160 ceph-mon[112094]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:07.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:07 smithi203 ceph-mon[83507]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:09 smithi160 ceph-mon[112094]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:09.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:09 smithi203 ceph-mon[83507]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:11 smithi160 ceph-mon[112094]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:11.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:11 smithi203 ceph-mon[83507]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:13.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:12 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:12 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:14.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:13 smithi203 ceph-mon[83507]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:14.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:13 smithi160 ceph-mon[112094]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:15 smithi160 ceph-mon[112094]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:15 smithi203 ceph-mon[83507]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:17 smithi160 ceph-mon[112094]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:17 smithi203 ceph-mon[83507]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:19 smithi160 ceph-mon[112094]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:19.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:19 smithi203 ceph-mon[83507]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:21 smithi160 ceph-mon[112094]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:21 smithi203 ceph-mon[83507]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:23.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:23 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:39:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:23 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:39:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:24 smithi203 ceph-mon[83507]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:24 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:39:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:24 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:39:24.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:24 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:39:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:24 smithi160 ceph-mon[112094]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:24 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:39:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:24 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:39:24.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:24 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:39:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:26 smithi160 ceph-mon[112094]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:26 smithi203 ceph-mon[83507]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:28 smithi160 ceph-mon[112094]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:28.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:28 smithi203 ceph-mon[83507]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:30.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:30 smithi160 ceph-mon[112094]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:30 smithi203 ceph-mon[83507]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:32.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:32 smithi160 ceph-mon[112094]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:32 smithi203 ceph-mon[83507]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:34 smithi203 ceph-mon[83507]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:34 smithi160 ceph-mon[112094]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:36 smithi203 ceph-mon[83507]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-05-25T06:39:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:36 smithi160 ceph-mon[112094]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-05-25T06:39:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:38 smithi203 ceph-mon[83507]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:38 smithi160 ceph-mon[112094]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:40 smithi203 ceph-mon[83507]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:40 smithi160 ceph-mon[112094]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:42.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:42 smithi203 ceph-mon[83507]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-25T06:39:42.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:42 smithi160 ceph-mon[112094]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-25T06:39:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:43.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:44 smithi203 ceph-mon[83507]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:44.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:44 smithi160 ceph-mon[112094]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:39:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:46 smithi203 ceph-mon[83507]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:46.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:46 smithi160 ceph-mon[112094]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:48 smithi203 ceph-mon[83507]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:48.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:48 smithi160 ceph-mon[112094]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:50 smithi203 ceph-mon[83507]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:50 smithi160 ceph-mon[112094]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:52 smithi203 ceph-mon[83507]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:52.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:52 smithi160 ceph-mon[112094]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:39:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:54 smithi203 ceph-mon[83507]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:54 smithi160 ceph-mon[112094]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:56 smithi203 ceph-mon[83507]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:56.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:56 smithi160 ceph-mon[112094]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:39:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:58 smithi203 ceph-mon[83507]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:39:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:39:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:58 smithi160 ceph-mon[112094]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:39:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:39:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:00 smithi203 ceph-mon[83507]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:00 smithi203 ceph-mon[83507]: overall HEALTH_OK 2024-05-25T06:40:00.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:00 smithi160 ceph-mon[112094]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:00.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:00 smithi160 ceph-mon[112094]: overall HEALTH_OK 2024-05-25T06:40:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:02 smithi203 ceph-mon[83507]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:02.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:02 smithi160 ceph-mon[112094]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:04 smithi203 ceph-mon[83507]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:04.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:04 smithi160 ceph-mon[112094]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:06 smithi203 ceph-mon[83507]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:06 smithi160 ceph-mon[112094]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:08 smithi203 ceph-mon[83507]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:08 smithi160 ceph-mon[112094]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:10 smithi203 ceph-mon[83507]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:10 smithi160 ceph-mon[112094]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:12 smithi203 ceph-mon[83507]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:12 smithi160 ceph-mon[112094]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:14 smithi203 ceph-mon[83507]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:14.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:14 smithi160 ceph-mon[112094]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:16 smithi203 ceph-mon[83507]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:16 smithi160 ceph-mon[112094]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:18 smithi203 ceph-mon[83507]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:18 smithi160 ceph-mon[112094]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:20 smithi203 ceph-mon[83507]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:20.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:20 smithi160 ceph-mon[112094]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:22 smithi203 ceph-mon[83507]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:22.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:22 smithi160 ceph-mon[112094]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:24 smithi203 ceph-mon[83507]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:24 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:40:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:24 smithi160 ceph-mon[112094]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:24 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:40:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:40:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:40:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:40:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:40:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:40:25.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:40:26.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:26 smithi203 ceph-mon[83507]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:26.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:26 smithi160 ceph-mon[112094]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:28.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:28 smithi203 ceph-mon[83507]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:28.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:28.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:28 smithi160 ceph-mon[112094]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:28.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:30.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:30 smithi203 ceph-mon[83507]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:30.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:30 smithi160 ceph-mon[112094]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:32.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:32 smithi203 ceph-mon[83507]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-25T06:40:32.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:32 smithi160 ceph-mon[112094]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-25T06:40:34.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:34 smithi203 ceph-mon[83507]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:34.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:34 smithi160 ceph-mon[112094]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:36.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:36 smithi203 ceph-mon[83507]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-05-25T06:40:36.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:36 smithi160 ceph-mon[112094]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-05-25T06:40:38.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:38 smithi203 ceph-mon[83507]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:38.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:38 smithi160 ceph-mon[112094]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-25T06:40:40.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:40 smithi203 ceph-mon[83507]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:40.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:40 smithi160 ceph-mon[112094]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:42.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:42 smithi203 ceph-mon[83507]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:42.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:42 smithi160 ceph-mon[112094]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:43.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:43.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:44.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:44 smithi203 ceph-mon[83507]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:44.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:44 smithi160 ceph-mon[112094]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:46.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:46 smithi203 ceph-mon[83507]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:46.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:46 smithi160 ceph-mon[112094]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:48.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:48 smithi203 ceph-mon[83507]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:48.924 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:48 smithi160 ceph-mon[112094]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:50.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:50 smithi203 ceph-mon[83507]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:50.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:50 smithi160 ceph-mon[112094]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:52.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:52 smithi203 ceph-mon[83507]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:52.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:52 smithi160 ceph-mon[112094]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:40:54.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:54 smithi203 ceph-mon[83507]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:54.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:54 smithi160 ceph-mon[112094]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:56.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:56 smithi203 ceph-mon[83507]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:56.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:56 smithi160 ceph-mon[112094]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:40:58.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:58 smithi203 ceph-mon[83507]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:58.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:40:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:40:58.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:58 smithi160 ceph-mon[112094]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:40:58.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:40:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:00.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:00 smithi203 ceph-mon[83507]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:00.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:00 smithi160 ceph-mon[112094]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:02.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:02 smithi203 ceph-mon[83507]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:02.969 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:02 smithi160 ceph-mon[112094]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:05.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:05 smithi203 ceph-mon[83507]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:05.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:05 smithi160 ceph-mon[112094]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:07.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:07 smithi203 ceph-mon[83507]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:07.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:07 smithi160 ceph-mon[112094]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:09.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:09 smithi203 ceph-mon[83507]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:09.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:09 smithi160 ceph-mon[112094]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:11.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:11 smithi203 ceph-mon[83507]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:11.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:11 smithi160 ceph-mon[112094]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:13.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:13 smithi160 ceph-mon[112094]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:13.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:13 smithi203 ceph-mon[83507]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:15.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:15 smithi160 ceph-mon[112094]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:15.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:15 smithi203 ceph-mon[83507]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:17 smithi160 ceph-mon[112094]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:17.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:17 smithi203 ceph-mon[83507]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:19.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:19 smithi160 ceph-mon[112094]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:19.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:19 smithi203 ceph-mon[83507]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:21.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:21 smithi160 ceph-mon[112094]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:21.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:21 smithi203 ceph-mon[83507]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:23.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:23 smithi160 ceph-mon[112094]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:23.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:23 smithi203 ceph-mon[83507]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:25 smithi160 ceph-mon[112094]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:25.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:25 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:41:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:25 smithi203 ceph-mon[83507]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:25.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:25 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:41:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:41:26.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:41:26.470 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:41:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:41:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:41:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:41:27.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:27 smithi160 ceph-mon[112094]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:27.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:27 smithi203 ceph-mon[83507]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:28 smithi160 ceph-mon[112094]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:28.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:28.550 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:28 smithi203 ceph-mon[83507]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:28.551 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:30 smithi203 ceph-mon[83507]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:30 smithi160 ceph-mon[112094]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:32 smithi203 ceph-mon[83507]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:32.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:32 smithi160 ceph-mon[112094]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:34 smithi203 ceph-mon[83507]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:34 smithi160 ceph-mon[112094]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:36 smithi203 ceph-mon[83507]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:36 smithi160 ceph-mon[112094]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:39.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:39 smithi203 ceph-mon[83507]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:39.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:39 smithi160 ceph-mon[112094]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:41.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:41 smithi203 ceph-mon[83507]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:41.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:41 smithi160 ceph-mon[112094]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:43 smithi203 ceph-mon[83507]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:43.399 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:43 smithi160 ceph-mon[112094]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:43.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:45.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:45 smithi160 ceph-mon[112094]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:45.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:45 smithi203 ceph-mon[83507]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:47.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:47 smithi160 ceph-mon[112094]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:47.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:47 smithi203 ceph-mon[83507]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:49.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:49 smithi160 ceph-mon[112094]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:49.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:49 smithi203 ceph-mon[83507]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:51.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:51 smithi160 ceph-mon[112094]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:51.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:51 smithi203 ceph-mon[83507]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:53.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:53 smithi160 ceph-mon[112094]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:53.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:53 smithi203 ceph-mon[83507]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:41:55.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:55 smithi160 ceph-mon[112094]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:55.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:55 smithi203 ceph-mon[83507]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:57.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:57 smithi160 ceph-mon[112094]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:57.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:57 smithi203 ceph-mon[83507]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:41:58.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:58.550 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:41:59.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:41:59 smithi160 ceph-mon[112094]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:41:59.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:41:59 smithi203 ceph-mon[83507]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:00.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:00 smithi160 ceph-mon[112094]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:00 smithi203 ceph-mon[83507]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:02 smithi203 ceph-mon[83507]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:02.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:02 smithi160 ceph-mon[112094]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:04 smithi203 ceph-mon[83507]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:04.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:04 smithi160 ceph-mon[112094]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:06 smithi203 ceph-mon[83507]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:06 smithi160 ceph-mon[112094]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:08 smithi203 ceph-mon[83507]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:08 smithi160 ceph-mon[112094]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:10 smithi203 ceph-mon[83507]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:10 smithi160 ceph-mon[112094]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:12 smithi203 ceph-mon[83507]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:12 smithi160 ceph-mon[112094]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:14 smithi203 ceph-mon[83507]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:14.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:14 smithi160 ceph-mon[112094]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:16 smithi203 ceph-mon[83507]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:16.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:16 smithi160 ceph-mon[112094]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:18.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:18 smithi203 ceph-mon[83507]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:18.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:18 smithi160 ceph-mon[112094]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:20.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:20 smithi203 ceph-mon[83507]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:20.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:20 smithi160 ceph-mon[112094]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:22.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:22 smithi203 ceph-mon[83507]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:22.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:22 smithi160 ceph-mon[112094]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:24.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:24 smithi203 ceph-mon[83507]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:24.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:24 smithi160 ceph-mon[112094]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:26 smithi203 ceph-mon[83507]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:42:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:42:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:42:26.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:26 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:42:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:26 smithi160 ceph-mon[112094]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-25T06:42:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-25T06:42:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-25T06:42:26.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:26 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' 2024-05-25T06:42:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:28 smithi203 ceph-mon[83507]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:28.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:28 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:28 smithi160 ceph-mon[112094]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:28.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:28 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:30.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:30 smithi203 ceph-mon[83507]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:30.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:30 smithi160 ceph-mon[112094]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:32.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:32 smithi203 ceph-mon[83507]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:32.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:32 smithi160 ceph-mon[112094]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:34.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:34 smithi203 ceph-mon[83507]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:34.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:34 smithi160 ceph-mon[112094]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:36.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:36 smithi203 ceph-mon[83507]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:36.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:36 smithi160 ceph-mon[112094]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:38.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:38 smithi203 ceph-mon[83507]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:38.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:38 smithi160 ceph-mon[112094]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:40.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:40 smithi203 ceph-mon[83507]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:40.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:40 smithi160 ceph-mon[112094]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:42.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:42 smithi203 ceph-mon[83507]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:42.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:42 smithi160 ceph-mon[112094]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:43.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:43 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:43.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:43 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:44.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:44 smithi203 ceph-mon[83507]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:44.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:44 smithi160 ceph-mon[112094]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:46.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:46 smithi203 ceph-mon[83507]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:46.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:46 smithi160 ceph-mon[112094]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:48.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:48 smithi203 ceph-mon[83507]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:48.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:48 smithi160 ceph-mon[112094]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:50.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:50 smithi203 ceph-mon[83507]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:50.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:50 smithi160 ceph-mon[112094]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:52.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:52 smithi203 ceph-mon[83507]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:52.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:52 smithi160 ceph-mon[112094]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:42:54.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:54 smithi203 ceph-mon[83507]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:54.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:54 smithi160 ceph-mon[112094]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:56.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:56 smithi203 ceph-mon[83507]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:56.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:56 smithi160 ceph-mon[112094]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:42:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:58 smithi203 ceph-mon[83507]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:58.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:42:58 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:42:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:58 smithi160 ceph-mon[112094]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:42:58.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:42:58 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:43:00.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:00 smithi203 ceph-mon[83507]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:00.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:00 smithi160 ceph-mon[112094]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:02.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:02 smithi203 ceph-mon[83507]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:43:02.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:02 smithi160 ceph-mon[112094]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:43:04.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:04 smithi203 ceph-mon[83507]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:04.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:04 smithi160 ceph-mon[112094]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:06.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:06 smithi203 ceph-mon[83507]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:43:06.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:06 smithi160 ceph-mon[112094]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:43:08.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:08 smithi203 ceph-mon[83507]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:08.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:08 smithi160 ceph-mon[112094]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:10.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:10 smithi203 ceph-mon[83507]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:10.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:10 smithi160 ceph-mon[112094]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:12.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:12 smithi203 ceph-mon[83507]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:43:12.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:12 smithi160 ceph-mon[112094]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-25T06:43:13.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:13 smithi203 ceph-mon[83507]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:43:13.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:13 smithi160 ceph-mon[112094]: from='mgr.34104 172.21.15.160:0/2838074499' entity='mgr.smithi160.nalzlg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-25T06:43:14.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:14 smithi203 ceph-mon[83507]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:14.719 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:14 smithi160 ceph-mon[112094]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-25T06:43:16.649 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:16 smithi203 ceph-mon[83507]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:43:16.731 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:16 smithi160 ceph-mon[112094]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-25T06:43:17.141 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-25T06:43:17.141 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-25T06:43:17.143 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-25T06:43:17.154 INFO:tasks.cephadm:Teardown begin 2024-05-25T06:43:17.154 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-25T06:43:17.156 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:43:17.189 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:43:17.224 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-25T06:43:17.224 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-25T06:43:17.247 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-25T06:43:17.282 INFO:tasks.cephadm:Stopping all daemons... 2024-05-25T06:43:17.282 INFO:tasks.cephadm.mon.smithi160:Stopping mon.smithi160... 2024-05-25T06:43:17.282 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160 2024-05-25T06:43:17.469 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:17 smithi160 systemd[1]: Stopping Ceph mon.smithi160 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:43:17.794+0000 7fd61f066700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi160 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:17 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160[112090]: 2024-05-25T06:43:17.794+0000 7fd61f066700 -1 mon.smithi160@0(leader) e3 *** Got Signal Terminated *** 2024-05-25T06:43:18.219 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:17 smithi160 podman[158229]: 2024-05-25 06:43:17.978224412 +0000 UTC m=+0.497090540 container died f6abc627de68ea95abdb82f6adbc13d0d578c05ef00174eca8685fc42e114ef9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:43:18.712 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:18 smithi160 podman[158229]: 2024-05-25 06:43:18.444915158 +0000 UTC m=+0.963781288 container cleanup f6abc627de68ea95abdb82f6adbc13d0d578c05ef00174eca8685fc42e114ef9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521) 2024-05-25T06:43:18.712 INFO:journalctl@ceph.mon.smithi160.smithi160.stdout:May 25 06:43:18 smithi160 bash[158229]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi160 2024-05-25T06:43:18.870 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi160.service' 2024-05-25T06:43:18.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:43:18.920 INFO:tasks.cephadm.mon.smithi160:Stopped mon.smithi160 2024-05-25T06:43:18.920 INFO:tasks.cephadm.mon.smithi203:Stopping mon.smithi203... 2024-05-25T06:43:18.920 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203 2024-05-25T06:43:19.239 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:18 smithi203 systemd[1]: Stopping Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:19.572 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:19 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203[83503]: 2024-05-25T06:43:19.238+0000 7f3073584700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi203 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:19.572 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:19 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203[83503]: 2024-05-25T06:43:19.238+0000 7f3073584700 -1 mon.smithi203@1(peon) e3 *** Got Signal Terminated *** 2024-05-25T06:43:19.572 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:19 smithi203 podman[116601]: 2024-05-25 06:43:19.311035472 +0000 UTC m=+0.185276693 container died bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-25T06:43:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:19 smithi203 podman[116601]: 2024-05-25 06:43:19.739388062 +0000 UTC m=+0.613629308 container cleanup bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-25T06:43:19.899 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:19 smithi203 bash[116601]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203 2024-05-25T06:43:20.165 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203.service' 2024-05-25T06:43:20.188 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:19 smithi203 podman[116614]: 2024-05-25 06:43:19.915886758 +0000 UTC m=+0.606079978 container remove bd246c6536a8c8f148aaec7b3636802a97fe63bff8a2f0f92c3a661c8cfbafdb (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-mon-smithi203, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-25T06:43:20.188 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:20 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203.service: Deactivated successfully. 2024-05-25T06:43:20.188 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:20 smithi203 systemd[1]: Stopped Ceph mon.smithi203 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968. 2024-05-25T06:43:20.188 INFO:journalctl@ceph.mon.smithi203.smithi203.stdout:May 25 06:43:20 smithi203 systemd[1]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@mon.smithi203.service: Consumed 11.035s CPU time. 2024-05-25T06:43:20.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:43:20.213 INFO:tasks.cephadm.mon.smithi203:Stopped mon.smithi203 2024-05-25T06:43:20.213 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-25T06:43:20.213 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.0 2024-05-25T06:43:20.632 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:20 smithi160 systemd[1]: Stopping Ceph osd.0 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:20.969 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:20 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[123594]: 2024-05-25T06:43:20.632+0000 7f90d6262700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:20.969 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:20 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[123594]: 2024-05-25T06:43:20.632+0000 7f90d6262700 -1 osd.0 85 *** Got signal Terminated *** 2024-05-25T06:43:20.970 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:20 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0[123594]: 2024-05-25T06:43:20.632+0000 7f90d6262700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:43:26.399 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:26 smithi160 podman[158361]: 2024-05-25 06:43:26.140678304 +0000 UTC m=+5.728179541 container died 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-25T06:43:26.719 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:26 smithi160 podman[158361]: 2024-05-25 06:43:26.4574712 +0000 UTC m=+6.044972429 container cleanup 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:43:26.719 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:26 smithi160 bash[158361]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0 2024-05-25T06:43:26.719 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:26 smithi160 podman[158376]: 2024-05-25 06:43:26.642010522 +0000 UTC m=+0.502951580 container remove 80bfe7bea4a32b49ad7c8563f8210564cfb418cd60b5726aec96a422ff56bf73 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-25T06:43:27.379 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:27 smithi160 podman[158467]: 2024-05-25 06:43:26.989758227 +0000 UTC m=+0.017618738 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:43:27.379 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:27 smithi160 podman[158467]: 2024-05-25 06:43:27.110119793 +0000 UTC m=+0.137980314 container create 9799ae4c916cc0dd7dd8e479a7f674681d278221d3accfb2f5217ccba95330bf (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-25T06:43:27.380 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:27 smithi160 podman[158467]: 2024-05-25 06:43:27.379537952 +0000 UTC m=+0.407398455 container init 9799ae4c916cc0dd7dd8e479a7f674681d278221d3accfb2f5217ccba95330bf (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-25T06:43:27.681 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:27 smithi160 podman[158467]: 2024-05-25 06:43:27.383204554 +0000 UTC m=+0.411065062 container start 9799ae4c916cc0dd7dd8e479a7f674681d278221d3accfb2f5217ccba95330bf (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-25T06:43:27.681 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:27 smithi160 podman[158467]: 2024-05-25 06:43:27.479449243 +0000 UTC m=+0.507309752 container attach 9799ae4c916cc0dd7dd8e479a7f674681d278221d3accfb2f5217ccba95330bf (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-25T06:43:27.934 INFO:journalctl@ceph.osd.0.smithi160.stdout:May 25 06:43:27 smithi160 podman[158467]: 2024-05-25 06:43:27.682937234 +0000 UTC m=+0.710797747 container died 9799ae4c916cc0dd7dd8e479a7f674681d278221d3accfb2f5217ccba95330bf (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:43:28.306 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-25T06:43:28.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:43:28.346 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-25T06:43:28.346 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-25T06:43:28.346 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.1 2024-05-25T06:43:28.579 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:28 smithi160 systemd[1]: Stopping Ceph osd.1 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:28.969 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:28 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[128988]: 2024-05-25T06:43:28.677+0000 7fcf6b39b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:28.969 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:28 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[128988]: 2024-05-25T06:43:28.677+0000 7fcf6b39b700 -1 osd.1 85 *** Got signal Terminated *** 2024-05-25T06:43:28.969 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:28 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1[128988]: 2024-05-25T06:43:28.677+0000 7fcf6b39b700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:43:34.495 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:34 smithi160 podman[158574]: 2024-05-25 06:43:34.220926889 +0000 UTC m=+5.633639271 container died 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c) 2024-05-25T06:43:34.903 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:34 smithi160 podman[158574]: 2024-05-25 06:43:34.603032336 +0000 UTC m=+6.015744697 container cleanup 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-25T06:43:34.903 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:34 smithi160 bash[158574]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1 2024-05-25T06:43:35.219 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:34 smithi160 podman[158603]: 2024-05-25 06:43:34.904391774 +0000 UTC m=+0.677654428 container remove 544e0dd60296e0dc972fbb72c936a2fe2a7491fff3c1399bb649f512af3e1494 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=squid-ea0684c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-25T06:43:35.524 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:35 smithi160 podman[158689]: 2024-05-25 06:43:35.171497471 +0000 UTC m=+0.017527497 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:43:35.524 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:35 smithi160 podman[158689]: 2024-05-25 06:43:35.288782752 +0000 UTC m=+0.134812765 container create 000bea9fbd1562cada2b2d999a1e271dabd3a09aa2b04d743a2de4e41a04cdc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-25T06:43:35.834 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:35 smithi160 podman[158689]: 2024-05-25 06:43:35.52454793 +0000 UTC m=+0.370577943 container init 000bea9fbd1562cada2b2d999a1e271dabd3a09aa2b04d743a2de4e41a04cdc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-25T06:43:35.834 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:35 smithi160 podman[158689]: 2024-05-25 06:43:35.528535099 +0000 UTC m=+0.374565120 container start 000bea9fbd1562cada2b2d999a1e271dabd3a09aa2b04d743a2de4e41a04cdc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-25T06:43:35.834 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:35 smithi160 podman[158689]: 2024-05-25 06:43:35.599396532 +0000 UTC m=+0.445426574 container attach 000bea9fbd1562cada2b2d999a1e271dabd3a09aa2b04d743a2de4e41a04cdc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-25T06:43:36.202 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 25 06:43:35 smithi160 podman[158689]: 2024-05-25 06:43:35.834854828 +0000 UTC m=+0.680884852 container died 000bea9fbd1562cada2b2d999a1e271dabd3a09aa2b04d743a2de4e41a04cdc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:43:36.525 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-25T06:43:36.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:43:36.567 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-25T06:43:36.567 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-25T06:43:36.567 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.2 2024-05-25T06:43:36.930 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:36 smithi160 systemd[1]: Stopping Ceph osd.2 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:37.219 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:36 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[134579]: 2024-05-25T06:43:36.929+0000 7f1a3308d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:37.219 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:36 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[134579]: 2024-05-25T06:43:36.929+0000 7f1a3308d700 -1 osd.2 85 *** Got signal Terminated *** 2024-05-25T06:43:37.219 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:36 smithi160 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2[134579]: 2024-05-25T06:43:36.929+0000 7f1a3308d700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:43:42.530 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:42 smithi160 podman[158794]: 2024-05-25 06:43:42.245962269 +0000 UTC m=+5.450786509 container died 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:43:42.530 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:42 smithi160 podman[158794]: 2024-05-25 06:43:42.530514657 +0000 UTC m=+5.735338879 container cleanup 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-25T06:43:42.969 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:42 smithi160 bash[158794]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2 2024-05-25T06:43:42.969 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:42 smithi160 podman[158818]: 2024-05-25 06:43:42.731944756 +0000 UTC m=+0.479277880 container remove 26bb5422342f7e12f8e8ae50d7cae9d7ec4893d1088afe979e029ee358c69ebc (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-25T06:43:43.418 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:43 smithi160 podman[158906]: 2024-05-25 06:43:43.057194142 +0000 UTC m=+0.017471831 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:43:43.418 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:43 smithi160 podman[158906]: 2024-05-25 06:43:43.183086386 +0000 UTC m=+0.143364057 container create 45f9a79fff572d4745cdee23bb291f472974c2e582d79eea8b44542834042687 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-25T06:43:43.418 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:43 smithi160 podman[158906]: 2024-05-25 06:43:43.41921178 +0000 UTC m=+0.379489468 container init 45f9a79fff572d4745cdee23bb291f472974c2e582d79eea8b44542834042687 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-ea0684c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0) 2024-05-25T06:43:43.719 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:43 smithi160 podman[158906]: 2024-05-25 06:43:43.422889243 +0000 UTC m=+0.383166923 container start 45f9a79fff572d4745cdee23bb291f472974c2e582d79eea8b44542834042687 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-25T06:43:43.719 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:43 smithi160 podman[158906]: 2024-05-25 06:43:43.543974812 +0000 UTC m=+0.504252488 container attach 45f9a79fff572d4745cdee23bb291f472974c2e582d79eea8b44542834042687 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=squid-ea0684c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-25T06:43:44.025 INFO:journalctl@ceph.osd.2.smithi160.stdout:May 25 06:43:43 smithi160 podman[158906]: 2024-05-25 06:43:43.724802735 +0000 UTC m=+0.685080414 container died 45f9a79fff572d4745cdee23bb291f472974c2e582d79eea8b44542834042687 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-25T06:43:44.345 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-25T06:43:44.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:43:44.388 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-25T06:43:44.388 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-25T06:43:44.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.3 2024-05-25T06:43:44.691 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:44 smithi203 systemd[1]: Stopping Ceph osd.3 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:44.948 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:44 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:43:44.691+0000 7fbb8d858700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:44.948 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:44 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:43:44.691+0000 7fbb8d858700 -1 osd.3 85 *** Got signal Terminated *** 2024-05-25T06:43:44.948 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:44 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:43:44.691+0000 7fbb8d858700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:43:48.399 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:48 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:43:48.094+0000 7fbb86645700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:22.642447+0000 front 2024-05-25T06:43:22.642531+0000 (oldest deadline 2024-05-25T06:43:47.942308+0000) 2024-05-25T06:43:49.399 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:49 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3[92779]: 2024-05-25T06:43:49.069+0000 7fbb86645700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:22.642447+0000 front 2024-05-25T06:43:22.642531+0000 (oldest deadline 2024-05-25T06:43:47.942308+0000) 2024-05-25T06:43:49.399 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:49 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:49.031+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:50.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:50 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:50.024+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:50.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:49 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:49.872+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:50.492 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:50 smithi203 podman[116722]: 2024-05-25 06:43:50.235109929 +0000 UTC m=+5.648426795 container died 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:43:50.759 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:50 smithi203 podman[116722]: 2024-05-25 06:43:50.58332224 +0000 UTC m=+5.996639108 container cleanup 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-25T06:43:50.759 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:50 smithi203 bash[116722]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3 2024-05-25T06:43:51.020 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:50 smithi203 podman[116749]: 2024-05-25 06:43:50.759737374 +0000 UTC m=+0.520206629 container remove 8e79c63db459415c7132504d1a8fd459b642612fd4e896a33aab3ef4ff9e5a36 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True) 2024-05-25T06:43:51.020 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:50 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:50.834+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:51.326 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:51 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:51.020+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:51.326 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:51 smithi203 podman[116842]: 2024-05-25 06:43:51.116420995 +0000 UTC m=+0.018007553 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:43:51.326 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:51 smithi203 podman[116842]: 2024-05-25 06:43:51.227415879 +0000 UTC m=+0.129002455 container create d31dadc314b97299fe7c37a55ac0b1281f52b78e12555ead81ee09f3fe7ac8f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-25T06:43:51.649 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:51 smithi203 podman[116842]: 2024-05-25 06:43:51.42142047 +0000 UTC m=+0.323007023 container init d31dadc314b97299fe7c37a55ac0b1281f52b78e12555ead81ee09f3fe7ac8f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-25T06:43:51.649 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:51 smithi203 podman[116842]: 2024-05-25 06:43:51.424775727 +0000 UTC m=+0.326362283 container start d31dadc314b97299fe7c37a55ac0b1281f52b78e12555ead81ee09f3fe7ac8f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-ea0684c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-25T06:43:51.649 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:51 smithi203 podman[116842]: 2024-05-25 06:43:51.513248899 +0000 UTC m=+0.414835462 container attach d31dadc314b97299fe7c37a55ac0b1281f52b78e12555ead81ee09f3fe7ac8f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-25T06:43:51.974 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:51 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:51.852+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:51.975 INFO:journalctl@ceph.osd.3.smithi203.stdout:May 25 06:43:51 smithi203 podman[116842]: 2024-05-25 06:43:51.719360362 +0000 UTC m=+0.620946920 container died d31dadc314b97299fe7c37a55ac0b1281f52b78e12555ead81ee09f3fe7ac8f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=squid-ea0684c) 2024-05-25T06:43:52.291 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:51 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:51.974+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:52.311 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-25T06:43:52.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:43:52.353 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-25T06:43:52.353 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-25T06:43:52.354 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.4 2024-05-25T06:43:52.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:52 smithi203 systemd[1]: Stopping Ceph osd.4 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:43:53.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:52 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:52.850+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:53.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:52 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:52.686+0000 7f59e5dca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:43:53.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:52 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:52.686+0000 7f59e5dca700 -1 osd.4 85 *** Got signal Terminated *** 2024-05-25T06:43:53.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:52 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:52.686+0000 7f59e5dca700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:43:53.150 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:52 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:52.925+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:54.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:53.964+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:54.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:53 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:53.848+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:55.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:54 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:54.963+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:55.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:54 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:54.963+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:32.806162+0000 front 2024-05-25T06:43:32.806224+0000 (oldest deadline 2024-05-25T06:43:54.506072+0000) 2024-05-25T06:43:55.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:54 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:54.866+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:55.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:54 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:54.866+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:43:56.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:55 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:55.981+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:56.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:55 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:55.981+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:32.806162+0000 front 2024-05-25T06:43:32.806224+0000 (oldest deadline 2024-05-25T06:43:54.506072+0000) 2024-05-25T06:43:56.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:55 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:55.871+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:56.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:55 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:55.871+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:43:57.149 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:56.943+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:24.605135+0000 front 2024-05-25T06:43:24.605251+0000 (oldest deadline 2024-05-25T06:43:48.705235+0000) 2024-05-25T06:43:57.150 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4[98057]: 2024-05-25T06:43:56.943+0000 7f59e2bbf700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:32.806162+0000 front 2024-05-25T06:43:32.806224+0000 (oldest deadline 2024-05-25T06:43:54.506072+0000) 2024-05-25T06:43:57.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:56.887+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:57.150 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:56 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:56.887+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:43:58.295 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:58 smithi203 podman[116950]: 2024-05-25 06:43:58.016409099 +0000 UTC m=+5.438133111 container died 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-ea0684c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-25T06:43:58.295 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:57 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:57.937+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:58.296 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:57 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:57.937+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:43:58.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:58 smithi203 podman[116950]: 2024-05-25 06:43:58.394942317 +0000 UTC m=+5.816666331 container cleanup 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-ea0684c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-25T06:43:58.649 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:58 smithi203 bash[116950]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4 2024-05-25T06:43:58.961 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:58 smithi203 podman[116965]: 2024-05-25 06:43:58.671606632 +0000 UTC m=+0.653616371 container remove 5331e6b134b3bde2d5ced213ddf5c9fbf91db01249402387ccc12a0b28bd78ac (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:43:59.282 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:58 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:58.962+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:43:59.283 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:58 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:58.962+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:43:59.283 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:59 smithi203 podman[117056]: 2024-05-25 06:43:58.936263409 +0000 UTC m=+0.017990889 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:43:59.283 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:59 smithi203 podman[117056]: 2024-05-25 06:43:59.05589747 +0000 UTC m=+0.137624997 container create 9c60663c9e7f86637bdae51b7197f58d6a9a98ec35f47c68e745e010538df3c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-25T06:43:59.283 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:59 smithi203 podman[117056]: 2024-05-25 06:43:59.283653036 +0000 UTC m=+0.365380518 container init 9c60663c9e7f86637bdae51b7197f58d6a9a98ec35f47c68e745e010538df3c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS) 2024-05-25T06:43:59.576 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:59 smithi203 podman[117056]: 2024-05-25 06:43:59.287373471 +0000 UTC m=+0.369100962 container start 9c60663c9e7f86637bdae51b7197f58d6a9a98ec35f47c68e745e010538df3c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.schema-version=1.0) 2024-05-25T06:43:59.576 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:59 smithi203 podman[117056]: 2024-05-25 06:43:59.383312454 +0000 UTC m=+0.465039964 container attach 9c60663c9e7f86637bdae51b7197f58d6a9a98ec35f47c68e745e010538df3c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c) 2024-05-25T06:43:59.860 INFO:journalctl@ceph.osd.4.smithi203.stdout:May 25 06:43:59 smithi203 podman[117056]: 2024-05-25 06:43:59.577539562 +0000 UTC m=+0.659267052 container died 9c60663c9e7f86637bdae51b7197f58d6a9a98ec35f47c68e745e010538df3c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-25T06:44:00.119 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:59 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:59.989+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:44:00.119 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:43:59 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:43:59.989+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:44:00.144 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-25T06:44:00.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:44:00.182 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-25T06:44:00.182 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-25T06:44:00.182 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.5 2024-05-25T06:44:00.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:00 smithi203 systemd[1]: Stopping Ceph osd.5 for 0ed6924e-1a5c-11ef-bc9b-c7b262605968... 2024-05-25T06:44:00.899 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:00.531+0000 7f09f71f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-25T06:44:00.899 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:00.531+0000 7f09f71f2700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-25T06:44:00.900 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:00 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:00.531+0000 7f09f71f2700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-25T06:44:01.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:01 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:01.014+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:44:01.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:01 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:01.014+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:44:02.362 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:02 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:02.019+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:44:02.362 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:02 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:02.019+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:44:03.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:03 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:03.030+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:44:03.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:03 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:03.030+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:44:03.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:03 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:03.030+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6822 osd.2 since back 2024-05-25T06:43:37.242149+0000 front 2024-05-25T06:43:37.242102+0000 (oldest deadline 2024-05-25T06:44:02.541925+0000) 2024-05-25T06:44:04.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:04 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:04.044+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:44:04.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:04 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:04.044+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:44:04.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:04 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:04.044+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6822 osd.2 since back 2024-05-25T06:43:37.242149+0000 front 2024-05-25T06:43:37.242102+0000 (oldest deadline 2024-05-25T06:44:02.541925+0000) 2024-05-25T06:44:05.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:05 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:05.000+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-05-25T06:43:23.740586+0000 front 2024-05-25T06:43:23.740691+0000 (oldest deadline 2024-05-25T06:43:49.040456+0000) 2024-05-25T06:44:05.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:05 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:05.000+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6814 osd.1 since back 2024-05-25T06:43:29.040963+0000 front 2024-05-25T06:43:29.040998+0000 (oldest deadline 2024-05-25T06:43:54.341008+0000) 2024-05-25T06:44:05.399 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:05 smithi203 ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5[103280]: 2024-05-25T06:44:05.000+0000 7f09f3fe7700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.160:6822 osd.2 since back 2024-05-25T06:43:37.242149+0000 front 2024-05-25T06:43:37.242102+0000 (oldest deadline 2024-05-25T06:44:02.541925+0000) 2024-05-25T06:44:06.149 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:05 smithi203 podman[117163]: 2024-05-25 06:44:05.890886231 +0000 UTC m=+5.487146933 container died 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, RELEASE=squid-ea0684c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:44:06.631 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:06 smithi203 podman[117163]: 2024-05-25 06:44:06.372959352 +0000 UTC m=+5.969220051 container cleanup 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-25T06:44:06.632 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:06 smithi203 bash[117163]: ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5 2024-05-25T06:44:06.899 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:06 smithi203 podman[117178]: 2024-05-25 06:44:06.632555868 +0000 UTC m=+0.735854987 container remove 7f2ea24e28eca66c485b6b687e24ca03d75cfa45144bd3c249782dfb6ceaebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-ea0684c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-25T06:44:07.246 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:06 smithi203 podman[117266]: 2024-05-25 06:44:06.898680743 +0000 UTC m=+0.017387042 image pull f9a9809cafd60a51ac0d352bb3d2d070ab25f705b0418cbd27290854c20186a2 quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee 2024-05-25T06:44:07.246 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:07 smithi203 podman[117266]: 2024-05-25 06:44:07.01666893 +0000 UTC m=+0.135375209 container create 577715d4f42ce25dc3d59895909bd85dfb79d4bd8c820d320c279f65ab313d5c (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-25T06:44:07.246 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:07 smithi203 podman[117266]: 2024-05-25 06:44:07.244080241 +0000 UTC m=+0.362786527 container init 577715d4f42ce25dc3d59895909bd85dfb79d4bd8c820d320c279f65ab313d5c (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-ea0684c) 2024-05-25T06:44:07.247 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:07 smithi203 podman[117266]: 2024-05-25 06:44:07.247417347 +0000 UTC m=+0.366123635 container start 577715d4f42ce25dc3d59895909bd85dfb79d4bd8c820d320c279f65ab313d5c (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-25T06:44:07.548 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:07 smithi203 podman[117266]: 2024-05-25 06:44:07.361145698 +0000 UTC m=+0.479851994 container attach 577715d4f42ce25dc3d59895909bd85dfb79d4bd8c820d320c279f65ab313d5c (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-25T06:44:07.822 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 25 06:44:07 smithi203 podman[117266]: 2024-05-25 06:44:07.54876634 +0000 UTC m=+0.667472626 container died 577715d4f42ce25dc3d59895909bd85dfb79d4bd8c820d320c279f65ab313d5c (image=quay.ceph.io/ceph-ci/ceph@sha256:87d6178d73fd8ae7a5c0bfa0c0b94e94dac0cc7c0eff96d5e0b3e0bddc5affee, name=ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-ea0684c, org.label-schema.vendor=CentOS) 2024-05-25T06:44:08.111 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ed6924e-1a5c-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-25T06:44:08.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-25T06:44:08.150 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-25T06:44:08.150 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-25T06:44:11.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:ceph-fuse[76154]: fuse finished with error 0 and tester_r 0 2024-05-25T06:44:28.608 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-25T06:44:42.141 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:44:42.174 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-25T06:44:42.202 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-25T06:44:42.204 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/remote/smithi160/crash 2024-05-25T06:44:42.205 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/crash -- . 2024-05-25T06:44:42.242 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-25T06:44:42.242 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-05-25T06:44:42.244 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/remote/smithi203/crash 2024-05-25T06:44:42.245 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/crash -- . 2024-05-25T06:44:42.268 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-25T06:44:42.269 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-05-25T06:44:42.269 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-25T06:44:42.270 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-25T06:44:42.319 INFO:teuthology.orchestra.run.smithi160.stdout:2024-05-25T06:20:00.000102+0000 mon.smithi160 (mon.0) 493 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-05-25T06:44:42.320 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-25T06:44:42.320 INFO:tasks.cephadm:Compressing logs... 2024-05-25T06:44:42.320 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-25T06:44:42.362 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-25T06:44:42.385 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/ceph-client.0.log 2024-05-25T06:44:42.385 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-25T06:44:42.385 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-25T06:44:42.386 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi160.rrhzig.log 2024-05-25T06:44:42.386 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log 2024-05-25T06:44:42.386 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-25T06:44:42.386 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi160.rrhzig.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-25T06:44:42.386 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mgr.smithi160.nalzlg.log 2024-05-25T06:44:42.387 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-25T06:44:42.393 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:: No such file or directory 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi203.tndnrb.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi203.tndnrb.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:gzip/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.5.log: -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.3.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mon.smithi203.log 2024-05-25T06:44:42.394 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi203.yltymd.log 2024-05-25T06:44:42.395 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mon.smithi203.log: 89.2% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-25T06:44:42.395 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-25T06:44:42.395 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi203.yltymd.log: 91.2% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-25T06:44:42.396 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mgr.smithi160.nalzlg.log: 89.1% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-25T06:44:42.396 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-25T06:44:42.396 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.1.log: 91.1% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-25T06:44:42.396 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mon.smithi160.log 2024-05-25T06:44:42.400 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-25T06:44:42.405 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mgr.smithi203.rotqex.log 2024-05-25T06:44:42.406 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-25T06:44:42.412 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mon.smithi160.log: gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-25T06:44:42.413 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-25T06:44:42.418 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-25T06:44:42.420 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi160.tlctof.log 2024-05-25T06:44:42.443 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi160.tlctof.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-25T06:44:42.470 INFO:teuthology.orchestra.run.smithi160.stderr: 91.1% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-25T06:44:42.478 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mgr.smithi203.rotqex.log: /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-25T06:44:42.489 INFO:teuthology.orchestra.run.smithi203.stderr: 90.0% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mgr.smithi203.rotqex.log.gz 2024-05-25T06:44:42.987 INFO:teuthology.orchestra.run.smithi160.stderr: 89.2% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mgr.smithi160.nalzlg.log.gz 2024-05-25T06:44:43.053 INFO:teuthology.orchestra.run.smithi203.stderr: 92.5% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mon.smithi203.log.gz 2024-05-25T06:44:44.248 INFO:teuthology.orchestra.run.smithi160.stderr: 90.3% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mon.smithi160.log.gz 2024-05-25T06:44:46.925 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-25T06:44:47.140 INFO:teuthology.orchestra.run.smithi160.stderr: 93.3% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-25T06:44:48.218 INFO:teuthology.orchestra.run.smithi203.stderr: 93.5% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-25T06:44:48.973 INFO:teuthology.orchestra.run.smithi160.stderr: 93.3% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-25T06:44:49.421 INFO:teuthology.orchestra.run.smithi160.stderr: 93.3% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-25T06:44:50.364 INFO:teuthology.orchestra.run.smithi203.stderr: 93.4% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-25T06:44:51.399 INFO:teuthology.orchestra.run.smithi203.stderr: 95.1% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi203.tndnrb.log.gz 2024-05-25T06:44:51.976 INFO:teuthology.orchestra.run.smithi160.stderr: 95.1% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi160.rrhzig.log.gz 2024-05-25T06:44:58.181 INFO:teuthology.orchestra.run.smithi203.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-25T06:44:58.181 INFO:teuthology.orchestra.run.smithi203.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-05-25T06:44:59.063 INFO:teuthology.orchestra.run.smithi160.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-25T06:45:29.669 INFO:teuthology.orchestra.run.smithi203.stderr: 93.4% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi203.yltymd.log.gz 2024-05-25T06:45:29.670 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-05-25T06:45:29.670 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m47.293s 2024-05-25T06:45:29.670 INFO:teuthology.orchestra.run.smithi203.stderr:user 1m28.253s 2024-05-25T06:45:29.670 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m2.756s 2024-05-25T06:45:29.711 INFO:teuthology.orchestra.run.smithi160.stderr: 93.3% -- replaced with /var/log/ceph/0ed6924e-1a5c-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi160.tlctof.log.gz 2024-05-25T06:45:29.713 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-25T06:45:29.713 INFO:teuthology.orchestra.run.smithi160.stderr:real 0m47.339s 2024-05-25T06:45:29.713 INFO:teuthology.orchestra.run.smithi160.stderr:user 1m30.706s 2024-05-25T06:45:29.713 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m2.961s 2024-05-25T06:45:29.714 INFO:tasks.cephadm:Archiving logs... 2024-05-25T06:45:29.714 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/remote/smithi160/log 2024-05-25T06:45:29.715 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-25T06:45:42.147 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/remote/smithi203/log 2024-05-25T06:45:42.148 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-25T06:45:54.112 INFO:tasks.cephadm:Removing cluster... 2024-05-25T06:45:54.112 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 --force 2024-05-25T06:45:54.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ed6924e-1a5c-11ef-bc9b-c7b262605968 --force 2024-05-25T06:45:55.322 INFO:tasks.cephadm:Removing cephadm ... 2024-05-25T06:45:55.323 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-25T06:45:55.339 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-25T06:45:55.355 INFO:tasks.cephadm:Teardown complete 2024-05-25T06:45:55.356 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-25T06:45:55.367 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-25T06:45:55.368 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-25T06:45:55.368 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-25T06:45:55.382 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-25T06:45:55.467 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-25T06:45:55.477 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-25T06:45:55.478 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-25T06:45:55.481 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-25T06:45:55.493 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-25T06:45:55.495 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-05-25T06:45:55.497 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-25T06:45:55.497 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-25T06:45:55.497 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +935us[ +935us] +/- 47ms 2024-05-25T06:45:55.497 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -636us[ -636us] +/- 72ms 2024-05-25T06:45:55.497 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -760us[ -760us] +/- 70ms 2024-05-25T06:45:55.497 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T06:45:55.500 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-25T06:45:55.500 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-05-25T06:45:55.500 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +940us[ +940us] +/- 47ms 2024-05-25T06:45:55.500 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -636us[ -636us] +/- 72ms 2024-05-25T06:45:55.500 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -762us[ -762us] +/- 70ms 2024-05-25T06:45:55.500 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-25T06:45:55.500 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-25T06:45:55.511 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-25T06:45:55.512 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-25T06:45:55.522 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-25T06:45:55.620 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-25T06:45:55.693 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-25T06:45:55.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:45:55.722 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-25T06:45:55.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:45:55.751 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-25T06:45:55.762 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-25T06:45:55.773 INFO:teuthology.task.internal:Duration was 3129.328726 seconds 2024-05-25T06:45:55.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-25T06:45:55.784 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-25T06:45:55.784 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-25T06:45:55.787 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-25T06:45:55.823 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-25T06:45:55.830 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-25T06:45:56.190 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-25T06:45:56.190 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-05-25T06:45:56.190 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-25T06:45:56.211 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-05-25T06:45:56.211 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-25T06:45:56.256 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-25T06:45:56.257 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-25T06:45:56.260 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-25T06:45:56.323 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-25T06:45:56.324 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-25T06:45:56.328 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-25T06:45:57.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-25T06:45:57.208 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-25T06:45:57.208 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-25T06:45:57.233 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-25T06:45:57.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-25T06:45:57.273 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-25T06:45:57.276 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-25T06:45:57.297 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-05-25T06:45:57.325 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-05-25T06:45:57.350 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-25T06:45:57.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:45:57.379 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-25T06:45:57.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:45:57.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-25T06:45:57.425 INFO:teuthology.task.internal:Transferring archived files... 2024-05-25T06:45:57.426 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/remote/smithi160 2024-05-25T06:45:57.426 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-25T06:45:57.478 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-24_21:08:02-orch-squid-distro-default-smithi/7725353/remote/smithi203 2024-05-25T06:45:57.479 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-25T06:45:57.518 INFO:teuthology.task.internal:Removing archive directory... 2024-05-25T06:45:57.519 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-25T06:45:57.521 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-25T06:45:57.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-25T06:45:57.585 INFO:teuthology.task.internal:Not uploading archives. 2024-05-25T06:45:57.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-25T06:45:57.597 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-25T06:45:57.597 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-25T06:45:57.600 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-25T06:45:57.613 INFO:teuthology.orchestra.run.smithi160.stdout: 262165 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 25 06:45 /home/ubuntu/cephtest 2024-05-25T06:45:57.613 INFO:teuthology.orchestra.run.smithi160.stdout: 655636 4 d--------- 2 ubuntu ubuntu 4096 May 25 06:08 /home/ubuntu/cephtest/mnt.0 2024-05-25T06:45:57.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-25T06:45:57.614 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-25T06:45:57.614 INFO:teuthology.orchestra.run.smithi160.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-25T06:45:57.614 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_17c123e524b17eb0cf38bdb0a0ca89d80b36c131/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi160 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-25T06:45:57.615 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-25T06:45:57.627 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-25T06:45:57.639 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-25T06:45:57.642 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3129.3287262916565 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-05-25T06:45:57.642 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-25T06:45:57.724 INFO:teuthology.run:FAIL